Databricks Lead Developer

Databricks Lead Developer

Location: Mississauga, Ontario, Canada

Employment Type: Permanent

Experience: 8+ Years

Job Summary

We are seeking an experienced Databricks Lead Developer to design, develop, and optimize scalable data engineering solutions using Azure Databricks. The ideal candidate will have strong hands-on expertise in PySpark, Python, SQL, and Azure data services, along with technical leadership experience.

Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Python.
• Implement Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
• Build and optimize data pipelines using Azure Data Factory and ADLS Gen2.
• Implement CI/CD pipelines using Azure DevOps and Git.
• Optimize data processing performance, scalability, and cost.
• Ensure data quality, governance, security, and operational excellence.
• Mentor developers and collaborate with cross-functional teams.

Mandatory Skills

• Azure Databricks, PySpark, Apache Spark, Python, and SQL
• Azure Data Factory (ADF) and ADLS Gen2
• Delta Lake, Delta Live Tables (DLT), and Unity Catalog
• Databricks Workflows
• CI/CD using Azure DevOps and Git
• Data Engineering, ETL/ELT, and Data Warehousing

Good-to-Have Skills

Scala, Azure Synapse Analytics, Snowflake, Airflow, Databricks Asset Bundles, Auto Loader, Terraform, Medallion Architecture, Data Governance, and GenAI/RAG integrations.