Azure Data Engineer - SQL Databricks

Hiring: Data Engineer Azure & Databricks

7- 10 years

Gurugram/Bangalore

We are looking for an experienced Data Engineer to join a high-performing data team working on large-scale, cloud-based data transformation initiatives. This role involves building robust, scalable data pipelines and enabling advanced analytics across enterprise data platforms.

Key Responsibilities:

Design, develop, and maintain end-to-end data pipelines using Azure Data Factory (ADF) and Azure Databricks

Build scalable ETL/ELT workflows to process high-volume structured and unstructured data

Develop and optimize PySpark-based data processing frameworks

Ingest data from multiple sources such as APIs, databases, and flat files into Azure Data Lake

Implement data transformation, cleansing, and validation logic to ensure high data quality

Work with Delta Lake architecture for efficient storage and processing

Collaborate with data analysts, data scientists, and business stakeholders to deliver data solutions

Monitor, troubleshoot, and optimize data pipelines for performance and cost efficiency

Ensure adherence to data governance, security, and compliance standards

Technical Skills Required:

Strong hands-on experience with Azure Data Factory, Azure Databricks

Expertise in PySpark, Python, SQL

Experience with Azure Data Lake (ADLS Gen2), Synapse Analytics

Knowledge of Delta Lake, Medallion Architecture (Bronze/Silver/Gold layers)

Familiarity with data warehousing concepts (Star Schema, Snowflake Schema)

Experience with CI/CD tools (Azure DevOps, Git)

Understanding of data orchestration and workflow scheduling

Exposure to performance tuning and optimization techniques

Good to Have:

Experience with streaming data pipelines (Kafka/Event Hub)

Knowledge of Power BI or other visualization tools

Exposure to data security and access control (RBAC, IAM)