Azure Databricks Engineer

We are looking for an experienced Azure Databricks Engineer to join our team on a full-time basis. The ideal candidate should have strong hands-on experience with Azure Databricks, data engineering, and building scalable data solutions on the Azure platform.

Role Details

• Position: Azure Databricks Engineer
• Experience: 7+ years of relevant experience
• Employment Type: Full-Time
• Joining: Immediate Joiners preferred
• Shift: Starting from 2:00 PM IST

Key Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Azure Databricks.
• Develop and optimize ETL/ELT pipelines for large-scale data processing.
• Work extensively with PySpark, Spark SQL, and Python.
• Build and manage data pipelines using Azure Data Factory (ADF) and other Azure data services.
• Implement data ingestion, transformation, cleansing, and aggregation processes.
• Optimize Spark jobs, Databricks notebooks, clusters, and SQL queries for performance and cost efficiency.
• Work with Azure Data Lake Storage (ADLS) and modern data lake/lakehouse architectures.
• Implement data quality, monitoring, error handling, and operational best practices.
• Collaborate with data architects, analysts, developers, and business stakeholders to deliver data solutions.
• Follow CI/CD, version control, and deployment best practices across data engineering workloads.

Required Skills

• 7+ years of relevant experience in Data Engineering / Big Data.
• Strong hands-on experience with Azure Databricks.
• Excellent knowledge of PySpark, Apache Spark, Python, and SQL.
• Strong experience with Azure Data Factory (ADF).
• Experience with Azure Data Lake Storage (ADLS).
• Good understanding of Delta Lake and Lakehouse architecture.
• Experience in performance tuning and optimization of Spark/Databricks workloads.
• Hands-on experience with data integration and ETL/ELT processes.
• Good understanding of cloud-based data engineering concepts.
• Experience with Git and CI/CD practices.

Preferred

• Experience with Databricks Workflows, Unity Catalog, and Delta Live Tables (DLT).
• Knowledge of Azure Synapse Analytics or other Azure data services.
• Experience working with enterprise-scale data platforms.
• Databricks or Azure certifications would be an advantage.

Who We're Looking For

We are particularly interested in candidates who can join immediately and are comfortable working in a shift starting at 2:00 PM IST.

Interested candidates can share their updated resume with us to explore this opportunity with Proso AI.