L2 Databricks Support Engineer

We are looking for an experienced L2 Databricks Support Engineer to provide production support for Databricks data platforms and workloads. The candidate should have strong experience in Databricks, Apache Spark, PySpark, SQL, Delta Lake, ETL/ELT, cloud platforms, and production incident management.

Key Responsibilities

• Provide Level-2 production support for Databricks workloads, notebooks, jobs, workflows, and clusters.
• Monitor platform health and proactively resolve incidents, alerts, and service disruptions.
• Troubleshoot Databricks job failures, data issues, SQL errors, and performance bottlenecks.
• Perform SQL analysis, debugging, validation, and query optimization.
• Support Delta Lake, ETL/ELT pipelines, and data integration workflows.
• Monitor scheduled jobs and investigate failed or delayed data pipelines.
• Perform Root Cause Analysis (RCA) and implement preventive measures.
• Manage incidents, problems, and changes following ITIL processes.
• Coordinate with Development, Cloud, Data Engineering, and Business teams during major incidents.
• Support production deployments and validate post-release activities.
• Maintain operational documentation, runbooks, and knowledge articles.
• Participate in 24x7 production support/on-call activities as required.

Technical Skills

• Strong experience with Databricks Workspace, Jobs, Workflows, Clusters, and Unity Catalog.
• Good knowledge of Apache Spark and PySpark.
• Strong SQL development, troubleshooting, and performance tuning skills.
• Hands-on experience with Delta Lake architecture and data management.
• Experience with Azure Data Lake Storage (ADLS), AWS S3, or equivalent cloud storage.
• Understanding of ETL/ELT processes and data warehouse concepts.
• Knowledge of Git, CI/CD, and deployment processes.
• Experience with monitoring tools, log analysis, and production troubleshooting.
• Understanding of cloud platforms such as Azure, AWS, or GCP.
• Knowledge of data pipeline monitoring and failure recovery processes.