Databricks

Role & responsibilities

enior Databricks Developer

Position Description
We are seeking a skilled Databricks Developer with strong expertise in designing, developing, and optimizing data solutions on the Databricks platform. The ideal candidate should have hands-on experience in Azure Databricks, PySpark, Delta Lake, Data Engineering, and implementing CI/CD pipelines for automated deployment and release management.

Roles & Responsibilities
. Design, develop, and optimize data pipelines using Databricks, PySpark, and Python.
. Build scalable ETL/ELT processes to ingest, transform, and load data from multiple sources.
. Develop and maintain Delta Lake architectures and data lakehouse solutions.
. Implement data quality, validation, and monitoring frameworks.
. Collaborate with Data Analysts, Data Scientists, and business stakeholders to understand data requirements.
. Optimize Spark jobs for performance, scalability, and cost efficiency.
. Work with structured, semi structured, and unstructured data sources.
. Implement CI/CD practices and deployment automation for data engineering workloads.
. Ensure data security, governance, and compliance standards are followed.
. Troubleshoot and resolve data pipeline and processing issues.

Required Experience
Strong experience with Databricks
. Proficiency in Python
. Hands on experience with PySpark
. Strong SQL development skills
. Experience with Delta Lake
. Knowledge of Data Lake and Lakehouse architectures
. Experience building ETL/ELT pipelines
. Understanding of Spark optimization techniques
. Experience with Git/version control systems

Nice to Have
Kafka/Event Hub
Streaming Data Pipelines
Terraform
Docker
Kubernetes
Snowflake Integration
Keywords for Sourcing

Databricks, Azure Databricks, PySpark, Spark SQL, Delta Lake, Azure Data Factory, ADLS, Azure DevOps, CI/CD, GitHub Actions, YAML, Data Engineering, Data Pipelines, ETL, ELT, Unity Catalog, Terraform, Data Lake, Synapse Analytics.

Preferred candidate profile