Senior Databricks Data Engineer

Senior Databricks Data Engineer / DWH Developer

Location: Hybrid

Workload: 100% / Full-Time

Duration: 1 Year Contract (with extension options)

Languages: English

Position Summary We are seeking a Senior Databricks Data Engineer / DWH Developer for a 1-year engagement. In this role, you will design, build, and optimize enterprise-scale data pipelines and data warehousing solutions leveraging modern cloud technologies and Delta Lake.

You will work closely with cross-functional data teams to transform complex data streams into reliable, high-performance analytical assets for business decision-making.

Key Responsibilities & Deliverables

• ETL Pipeline Development: Architect and maintain robust, scalable ETL/ELT pipelines for batch and real-time processing using PySpark or Scala.
• Lakehouse Implementation: Design and optimize data storage and processing frameworks utilizing Databricks and Delta Lake architectures.
• Data Warehousing & Modeling: Build production-grade data models (star schema, snowflake, Data Vault) and optimize query performance across analytical platforms.
• Cloud Infrastructure Integration: Deploy and manage data solutions natively within a cloud environment (AWS, Azure, or GCP).
• Performance Tuning & Monitoring: Monitor pipeline stability, optimize Spark job performance, reduce compute costs, and resolve data quality issues.

Required Qualifications & Skills

• Databricks Expertise: Strong practical experience developing production data solutions on the Databricks platform.
• Programming Languages: Proficiency in PySpark and/or Scala, alongside strong advanced SQL skills.
• Data Lake & Storage: Deep understanding of Delta Lake transaction layer and storage optimizations (ACID transactions, time travel, Z-Ordering).
• Core Engineering: Proven background in traditional and modern Data Warehousing (DWH) and end-to-end ETL/ELT development.
• Cloud Platforms: Solid working knowledge of cloud-native data architectures (Azure Data Factory/ADLS, AWS Glue/S3, or GCP equivalents).

Preferred / Nice-to-Have

• Experience with CI/CD for Databricks (Databricks Asset Bundles, Terraform, GitHub Actions).
• Exposure to workflow orchestration tools (Airflow, Azure Data Factory, Databricks Workflows).
• Relevant Databricks or Cloud certifications (e.g., Databricks Certified Data Engineer Professional).