Senior Data Engineer – Databricks

Experience: 5+ Years

Employment Type: Fixed-Term / Contract

Role Summary

Design and build scalable data platforms on Databricks to enable reliable analytics, reporting, and AI-driven insights. Ensure data is secure, governed, and accessible across the organization.

Key Responsibilities

• Build and maintain data lakes/lakehouses, ETL/ELT pipelines, and integrations across enterprise systems.
• Develop data models and reusable datasets for analytics and AI use cases.
• Ensure data quality, governance, security, and compliance.
• Monitor, optimize, and automate data workflows for performance and cost efficiency.
• Collaborate with stakeholders and drive engineering best practices and improvements.

Requirements

• 5+ years in data engineering with strong expertise in SQL and Databricks (Spark, Delta Lake).
• Strong programming skills (Python/Scala) for data processing and automation.
• Hands-on experience with cloud platforms (Azure/AWS/GCP) and distributed data processing.
• Solid experience in building and optimizing ETL/ELT pipelines and orchestration tools (e.g., Airflow).
• Strong understanding of data modeling (dimensional/lakehouse) and data warehousing concepts.
• Experience with data governance, data quality frameworks, lineage, and security best practices.
• Familiarity with DevOps/DataOps practices (CI/CD, version control, testing, deployment).
• Experience integrating data from multiple enterprise systems (ERP, CRM, Finance, etc.).
• Strong troubleshooting skills across complex data pipelines and systems.
• Ability to design scalable, reliable, and cost-efficient data solutions.
• Good understanding of BI/analytics and AI/ML data consumption requirements.
• Strong communication skills with ability to work with cross-functional stakeholders.