Azure Data Engineer

AZURE DATA ENGINEER

Work Schedule: IST 1 PM TO 10 PM

Job Summary
We are looking for a highly experienced and motivated Azure Data Engineer with 7+ years of hands-on experience in Azure and Databricks architecture, migration, development, and data engineering.
The ideal candidate will have strong expertise in building scalable data platforms and pipelines, data migration and transformation, API integrations, performance optimization, and cloud-based data architectures. The role requires strong communication skills, ownership, self-management, and the ability to collaborate effectively with distributed teams while providing up to 6 hours of daily overlap with US CST working hours.
Key Responsibilities

• Design, develop, and implement advanced analytics and data engineering solutions using Azure and Databricks.
• Lead and execute data migration, transformation, and modernization initiatives using Databricks.
• Design, develop, and maintain scalable, reliable, and high-performance data pipelines.
• Build and maintain API integrations to support increasing data volumes, data sources, and business complexity.
• Work closely with data engineers, data scientists, architects, and business stakeholders to understand requirements and provide technical guidance.
• Design and implement robust data architectures, data models, ETL/ELT processes, and data integration solutions.
• Optimize and tune the performance of Databricks workloads, Spark jobs, data pipelines, and Azure-based solutions.
• Manage, monitor, and troubleshoot Databricks and Azure data environments to ensure reliability, scalability, availability, and performance.
• Lead the architecture and development of complex data integration and migration projects.
• Conduct code reviews and provide technical feedback while ensuring engineering standards and best practices are followed.
• Develop automation and reusable frameworks to improve data engineering efficiency and operational reliability.
• Troubleshoot complex data, pipeline, performance, and infrastructure issues and drive them through to resolution.
• Own and self-manage assigned deliverables, ensuring timely execution and high-quality outcomes.
• Collaborate with cross-functional teams and stakeholders in a distributed and remote working environment.
• Contribute to continuous improvement of data engineering architecture, processes, standards, and development practices.

Required Technical Skills

Mandatory Skills

• Bachelors or Masters degree in Computer Science, Information Technology, Engineering, or a related field.
• 7+ years of hands-on experience in Azure and Databricks architecture, migration, and development.
• Strong experience designing and implementing scalable data pipelines, data architectures, and data platforms.
• Strong proficiency in Python for data engineering, ETL/ELT, automation, and data processing.
• Hands-on experience with PySpark and Spark-based data processing.
• Strong SQL skills and experience working with relational databases.
• Experience with Databricks architecture, development, optimization, and administration.
• Experience with Azure data engineering services, including Azure Data Factory and Azure Synapse Analytics.
• Experience with data modeling, data warehousing, ETL/ELT processes, and data integration.
• Experience working with large-scale data and big data technologies such as Apache Spark and Hadoop.
• Experience with cloud infrastructure management, monitoring, troubleshooting, and performance optimization.
• Experience with Unix/Linux environments.
• Strong analytical, problem-solving, debugging, and troubleshooting skills.
• Excellent communication skills and ability to work independently in a remote team.
• Ability to work with up to 6 hours of overlap with the US CST time zone.

Good to Have

• Experience with Snowflake and modern cloud data warehouse architectures.
• Experience with NoSQL databases.
• Experience with Scala for data engineering.
• Familiarity with other cloud platforms such as AWS or Google Cloud.
• Experience developing and integrating REST APIs.
• Experience with CI/CD, DevOps, version control, and automated deployment practices.
• Azure Data Engineer or Databricks certification.

Key Technology Stack

Azure | Azure Data Factory (ADF) | Databricks | Apache Spark | PySpark | Snowflake | Azure Synapse Analytics | Python | Scala | SQL | NoSQL | Data Modeling | Data Warehousing | ETL / ELT | Data Integration | APIs | Hadoop | Unix / Linux | Cloud Infrastructure & Monitoring

Key Competencies
Azure Data Engineering | Databricks | Data Architecture | Data Migration | Data Transformation | Data Pipelines | ETL / ELT | PySpark | Apache Spark | Python | SQL | Data Modeling | Data Warehousing | API Integration | Performance Tuning | Data Integration | Cloud Data Platforms | Troubleshooting | Technical Leadership | Code Reviews | Remote Collaboration

Ideal Candidate
The ideal candidate is a hands-on senior Azure Data Engineer who can independently own complex data engineering initiatives from architecture and development through deployment, optimization, and production support.
The candidate should be comfortable working with Azure, Databricks, Spark, Python, SQL, and modern data platforms, while collaborating with technical and business stakeholders across a distributed team. Strong ownership, communication, problem-solving skills, and the ability to self-manage work are essential.