Azure Data Factory (ADF) Engineer

Role & responsibilities

We are seeking an experienced Azure Data Factory (ADF) Engineer with a strong background in designing, developing, and maintaining enterprise-scale data integration solutions. The ideal candidate should possess hands-on expertise in Azure Data Factory (ADF), ETL/ELT development, pipeline orchestration, job scheduling, and data migration activities.

Candidates with prior experience in the Insurance, BFSI, or FinTech domain will be preferred due to the nature of our client engagements. Exposure to Azure Databricks (DBX) is desirable but not mandatory.

• Design, build, deploy, and maintain data integration solutions using Azure Data Factory (ADF).
• Develop and manage ETL/ELT pipelines for extracting, transforming, and loading data from multiple sources.
• Configure and maintain ADF triggers, schedules, and automated workflows.
• Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
• Integrate data from various sources including SQL databases, APIs, cloud storage, and third-party applications.
• Perform data validation, exception handling, and root cause analysis for data processing issues.
• Collaborate with Data Architects, Business Analysts, and Application Teams to understand business requirements.
• Create and maintain technical documentation, support documents, and process workflows.
• Participate in code reviews and adhere to data engineering best practices.
• Support production deployments and provide operational support when required.

Required Technical Skills

• Strong hands-on experience with Azure Data Factory (ADF).
• Expertise in creating and maintaining data pipelines and workflow orchestration.
• Experience in ETL/ELT design and implementation.
• Proficiency in SQL and relational database concepts.
• Experience with:
• Azure Data Lake Storage (ADLS)
• Azure Blob Storage
• Azure SQL Database
• SQL Server
• Good understanding of data warehousing concepts.
• Experience with pipeline scheduling, monitoring, troubleshooting, and optimization.
• Version control experience using Git or Azure DevOps.