Data Engineer

Experience:3-4 years

Job Location: Bangalore/Pune/Gurgaon/Kochi

Work Mode: Hybrid

Must Have Skills: Azure Databricks, Pyspark, SQL

Role Summary:

We are looking for a highly motivated Azure Databricks Developer to build, enhance and support modern cloud-native data platforms.

The ideal candidate will have hands-on experience developing scalable data pipelines using Azure Databricks, PySpark, SQL and Delta Lake, with strong understanding of data ingestion, transformation, performance optimization and production support.

The role involves working closely with Data Engineers, Architects, platform teams and business stakeholders to deliver scalable, secure and high-performance data solutions.

Key Responsibilities:

• Build and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Spark, and SQL for batch and near real-time data processing.
• Design and implement Delta Lake solutions leveraging Medallion Architecture (Bronze, Silver, Gold layers) to support enterprise data platforms.
• Perform data transformation, cleansing, enrichment, aggregation, validation, and reconciliation to ensure high-quality and reliable datasets.
• Develop and support data ingestion frameworks for structured and semi-structured data from multiple enterprise systems and cloud sources.
• Create and manage Azure Data Factory (ADF) pipelines for data ingestion, orchestration, automation, and metadata-driven processing.
• Integrate Azure Databricks with ADLS Gen2 and enterprise data sources, while implementing secure credential management, error handling, logging, and recovery mechanisms using Azure Key Vault and best practices.

Technical Experience:

• 3–4 years of Data Engineering/Analytics Engineering experience with strong expertise in Azure-based data platforms and modern data architecture.
• Hands-on experience with Azure Databricks, Apache Spark/PySpark, SQL, Delta Lake, Databricks Workflows, and Databricks SQL for large-scale data processing.
• Strong understanding of ETL/ELT frameworks, distributed computing, Lakehouse concepts, and Medallion Architecture.
• Experience with key Azure services including ADLS Gen2, Azure Data Factory (ADF), Azure Key Vault, along with Git and Azure DevOps.
• Ability to build reusable, metadata-driven data pipelines, integrate REST APIs and file-based sources, and troubleshoot production data workflows.
• Knowledge of Python, CI/CD practices, automated deployments, and exposure to enterprise-scale or regulated/life sciences environments is preferred.

Educational Qualification: BTech/BE/MTech/ME/MCA