Senior Azure Data Engineer

Azure Data Engineer | Azure Databricks | PySpark | ADF | SQL

Role: Senior Azure Data Engineer

Experience: 5+ years of hands-on Azure Data Engineering experience

Location: Remote, India

Employment Type: Contract

About the Role

We are looking for experienced Azure Data Engineers to build, optimise and support scalable enterprise data pipelines and cloud-based data platforms. The ideal candidate will have strong hands-on experience with Azure Databricks, PySpark, Azure Data Factory, ADLS Gen2, Python and SQL, along with practical knowledge of modern Lakehouse architecture and production data engineering.

Candidates should be comfortable working with multiple data sources, troubleshooting pipeline issues, optimising performance and delivering reliable data solutions within enterprise environments.

Key Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, PySpark and SQL.
• Build batch and incremental ingestion frameworks to integrate data from databases, APIs, files and enterprise source systems.
• Develop data transformations using Python, PySpark, Spark SQL and Delta Lake, following Bronze, Silver and Gold Lakehouse patterns where appropriate.
• Work with ADLS Gen2, Azure SQL and Azure Synapse Analytics to support data storage, processing and analytics requirements.
• Implement data validation, reconciliation, error handling, monitoring and production support processes.
• Optimise Spark jobs, SQL queries and pipeline execution for performance, scalability and cost efficiency.
• Use Git, Azure DevOps and CI/CD practices to manage code, testing and deployments, while collaborating with distributed engineering teams.

Essential Technical Skills

• Strong hands-on experience with Azure Databricks, PySpark, Python and SQL.
• Practical experience developing pipelines using Azure Data Factory, including orchestration and incremental loading.
• Experience with ADLS Gen2, Delta Lake and Lakehouse or data lake architectures.
• Good understanding of ETL/ELT, data modelling, multi-source integration and data quality controls.
• Experience with SQL Server or Azure SQL, including complex queries and performance tuning.
• Familiarity with Git, Azure DevOps, deployment processes and production troubleshooting.

Preferred Skills

• Azure Synapse Analytics, dimensional modelling and enterprise data warehousing.
• Microsoft Fabric, Power BI and analytical datasets.
• CDC, streaming or near real time data ingestion.
• Azure Key Vault, managed identities, access controls and data governance.
• Automated testing, observability, infrastructure automation or API development.
• Experience in healthcare, pharmaceutical, financial services, retail or other enterprise data environments.
• Relevant Azure or Databricks certifications.

Candidate Profile

We are looking for engineers who can demonstrate real project experience building and supporting production grade Azure data pipelines. Candidates should be able to explain their architecture, write and optimise SQL/PySpark code, troubleshoot practical issues and describe their approach to data quality and deployment.

Contract: Long-term contract opportunity with strong potential for extension.

Please note: We review applications based on relevant skills, experience, and expertise. As applications are reviewed on a weekly basis, we appreciate your patience during the process. If your profile aligns with the requirements of the role, our team will be in touch to discuss the next steps.