Data Engineer

Role & responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines to ingest, transform, and load data from diverse sources
• Write and optimize advanced SQL queries for data extraction, transformation, validation, and reporting
• Build and maintain data models, warehouses, and semantic layers to support analytics, reporting, and downstream consumption
• Work with cloud data platforms and services to enhance data processing, storage, and scalability
• Automate and orchestrate end-to-end data workflows using tools such as Airflow, dbt, Cloud Composer, or Data Factory
• Collaborate with cross-functional stakeholders Data Architects, Data Scientists, Analysts, and business teams to translate requirements into technical solutions
• Perform data validation, reconciliation, and quality checks to ensure accuracy, consistency, and governance across pipelines
• Document data pipelines, schemas, and technical processes to support knowledge sharing and maintainability
• Troubleshoot pipeline failures, performance bottlenecks, and data quality anomalies
• Mentor junior engineers and contribute to best practices, code reviews, and continuous improvement (scope matched to seniority)

Preferred candidate profile

• 410 years of relevant experience in Data Engineering, Data Pipeline Development, or a related field
• Advanced SQL complex joins, CTEs, window functions, and query performance optimization
• Hands-on proficiency in Python and/or PySpark for data transformation, automation, and processing
• Proven experience building and maintaining ETL/ELT pipelines end-to-end
• Hands-on expertise in at least one cloud data platform (AWS, GCP, or Azure) and its associated warehouse/lake service (Snowflake, BigQuery, Databricks, or Microsoft Fabric)
• Experience with workflow orchestration tools (Airflow, dbt, Cloud Composer, Data Factory, or similar)
• Strong problem-solving skills and the ability to collaborate effectively with technical and non-technical stakeholders

Good to have:

• Marketing/Media Data Engineering & MLOps: Data mart and harmonization design across disparate sources, MLOps and model deployment, model monitoring, cost optimization
• Unstructured Data & GenAI-Adjacent Engineering (GCP): Document AI, Vertex AI, embeddings and vector search, semantic data modeling, Agentic AI/LangChain exposure
• Cloud-Native Pipeline Engineering (AWS): S3, Lambda, SNS, Step Functions, NumPy/Pandas, independent end-to-end ownership as an individual contributor
• Databricks DataOps & BI Enablement: Databricks and PySpark at scale, data validation and reconciliation, BI tool exposure (Power BI, Tableau, or Looker), Git/CI-CD discipline
• Microsoft Fabric Platform Engineering: Data Factory/Dataflows Gen2, Medallion architecture (Bronze/Silver/Gold, Delta Lake, OneLake), Power BI DAX/Direct Lake, real-time streams (Eventstreams/KQL), Purview governance
• Team & Delivery Leadership: Mentoring, code reviews, sprint/delivery ownership, cross-functional and business stakeholder management

Qualifications

• Bachelor's or Master's degree in a quantitative field (Computer Science, Engineering, or related) or equivalent practical experience
• Strong analytical, problem-solving, and communication skills.