DataBricks - Technical Lead

DataBricks - Technical Lead (Exp: 8 Yrs to 12 Yrs)

We are seeking a talented and motivated DataBricks expert to join our team as a Technical Lead.

Required Skills Qualifications:

• 8 to 12 years of IT experience, with 3+ years focused on big data and Databricks.
• Strong hands-on experience with Databricks, PySpark, and Spark SQL.
• Experience with modern cloud data lakes (Azure ADLS, AWS S3).
• Hands-on experience with Azure Data Factory (ADF) or AWS Glue for orchestration.
• Strong experience with relational databases and SQL.
• Strong background in Agile delivery, CI/CD pipelines, and version control (Git).
• Experience with Delta Lake, Unity Catalog, and the medallion architecture.
• Experience with streaming data (Structured Streaming, Kafka, Event Hubs).
• Experience with Infrastructure-as-Code (Terraform).
• Experience in agile development processes using Jira and Confluence.
• Design scalable data engineering solutions and lead their delivery on Databricks.
• Understanding of the SDLC and Agile methodologies.
• Communication with the customer and producing the daily status report.
• Bachelors or Masters degree in Computer Science, Engineering, or a related field.
• Databricks Certified Data Engineer Professional certification.
• Microsoft Certified: Fabric Data Engineer Associate (DP-700) or AWS Certified Data Engineer - Associate.
• Should have good oral and written communication.
• Should be able to lead and mentor a team.
• Should be proactive and adaptive.

Key Responsibilities:

• Design scalable, high-performance data pipelines and Lakehouse workflows using Databricks and PySpark / Spark SQL.
• Build complex ETL/ELT pipelines on cloud platforms such as Azure or AWS.
• Guide development teams, perform code reviews, and enforce engineering best practices.
• Mentor engineers on Databricks, Spark, and data engineering design patterns.
• Work with product owners, business analysts, and architects to translate requirements into technical solutions.
• Monitor production workloads, fine-tune ETL/ELT performance, and ensure data reliability.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.