Lead Data Engineer

• We are seeking an experienced Databricks Architect / Lead Data Engineer to lead the design, development, and delivery of enterprise-scale data platforms on Databricks and AWS.
• The role combines architecture leadership with hands-on engineering responsibilities, including building scalable Lakehouse solutions, modern data pipelines, and AI/ML workloads.

Key Responsibilities

• Design and own Databricks Lakehouse architecture on AWS. Lead multiple Databricks initiatives and mentor data engineers.
• Build and optimize data pipelines using Python, PySpark, SQL, and Delta Lake. Implement Medallion Architecture (Bronze, Silver, Gold layers).
• Develop batch, streaming, and near real-time data solutions. Enable AI/ML and Generative AI use cases on Databricks.
• Establish governance, security, Unity Catalog, RBAC, and operational controls. Integrate Databricks with AWS services including S3, IAM, networking, and data platforms.
• Drive CI/CD, automated testing, deployment automation, and observability.
• Troubleshoot and optimize Spark workloads for performance, reliability, and cost. Required Experience In Data Engineering experience.
• And hands-on Databricks experience. as a Databricks Architect or equivalent architecture role.
• delivering AI/ML or Generative AI solutions on Databricks. and AWS experience.

ELLIOTT MOSS CONSULTING PTE. LTD.

We at Elliott Moss Consultng inspire individuals and organisations to work more effectively and efficiently, and create greater choice in the work domain, for the benefit of all concerned. Our primary mission is to provide our clients with the best suitable employees who bring success to your organiza,on and allows us to grow together.