Lead Data Engineer / Data Platform Lead

Job Overview

The primary focus of this role will be to design, build, and maintain the EMFI Data Application Platform, a financial analytics application. The incumbent will work across the full stack to deliver reliable, performant, and secure data products.

Reports To

Director Data and AI, Embedded Finance

Responsibilities and Duties

• Collaborate with internal stakeholders to translate business and analytical requirements into well-defined backend features and data products.

• Design and deploy scalable microservices within the FastAPI ecosystem, establishing clear service boundaries for domain-specific services and ensuring loose coupling through well-defined API contracts.

• Design and develop Snowflake-native applications including Snowpark (Python), stored procedures, tasks, streams, and dynamic tables to support day-end processing engines that run natively within the Snowflake compute layer.

• Design and implement RESTful API endpoints using FastAPI ensuring data contracts are clearly defined, versioned, and documented via OpenAPI/Swagger.

• Write and optimize SQL queries against Snowflake and PostgreSQL for reporting, aggregation, and analytical use cases.

• Own end-to-end feature delivery: from requirements gathering and API contract definition through implementation, testing, and deployment.

• Ensure platform security by applying best practices around authentication, authorization (RBAC), input validation, and safe error handling, aligned with OWASP Top 10.

Technical Skills

• Experience designing and implementing microservices architectures, including service discovery, inter-service communication, API versioning, and distributed tracing/logging for observability.

• Proficiency in Python for backend development, including FastAPI async patterns, and Pydantic-based data validation.

• Hands-on experience with SQL and analytical databases for writing optimized queries, aggregations, and data transformations.

• Experience developing Snowflake-native applications: stored procedures (JavaScript/Python), Tasks, Streams, Dynamic Tables, and Snowpark for building backend processing engines.

• Understanding of REST API design, OpenAPI specifications, and API integration patterns in frontend applications.

• Knowledge of authentication and authorization patterns including JWT, OAuth2, and role-based access control.

• Experience with Docker and Docker Compose for local development and containerized deployments.

• Familiarity with CI/CD pipelines for automated testing and deployment.

• Experience with version control using Git, including branching strategies and code review workflows.

• Ability to work with cloud platforms (AWS, Azure, or GCP) for data storage, compute, and deployment.

Qualifications and Skills

• Bachelor’s degree (Master’s preferred) in Computer Science, Software Engineering, Information Systems, or a related discipline.

• 10+ years of professional full stack development experience, with demonstrated delivery of production-grade web applications.

• Experience in the Payments or Fintech industry would be a strong asset.

• Proven ability to work independently, manage multiple concurrent workstreams, and deliver with minimal oversight.

• Excellent communication and interpersonal skills, with the ability to engage both technical and non-technical audiences.

• Unquestionable personal and business ethics and integrity.