Databricks Data Engineer

Mandatory Skills:

• Python, SQL, PySpark and Spark SQL
• Databricks and Delta Lake
• Lakehouse and Medallion Architecture
• Spark Performance Tuning and Troubleshooting
• Unity Catalog and Data Governance
• Data Modeling, Schema Evolution and Incremental Processing
• Batch Processing and Structured Streaming
• CI/CD and Cloud Platforms (Azure/AWS/GCP)

Preferred candidate profile

Key Responsibilities:

• Design, develop, and optimize scalable data pipelines.
• Build Lakehouse solutions using Medallion Architecture.
• Develop and optimize PySpark applications.
• Implement data ingestion, transformations, and data quality checks.
• Manage data access and governance using Unity Catalog.
• Support automated deployments and mentor junior team members as required.

Good to Have: Auto Loader, Lakeflow Connect, Lakeflow Declarative Pipelines, Databricks Asset Bundles, Delta Sharing and Lakehouse Federation.