Senior Databricks Solution Architect

We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions.

Key Responsibilities

• Design and architect Databricks Lakehouse solutions for large-scale enterprise data platforms

• Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization

• Define Databricks workspace architecture, cluster strategies, security models, and operational standards

• Design and develop scalable data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake

• Implement data ingestion frameworks supporting batch and real-time data processing

• Design Delta Lake architectures including:

• Delta tables

• Medallion architecture (Bronze/Silver/Gold layers)

• Data quality frameworks

• Schema evolution and optimization

• Perform Databricks performance tuning:

• Cluster optimization

• Spark job optimization

• Query tuning

• Cost optimization

• Implement Databricks governance and security using:

• Unity Catalog

• Access controls

• Data lineage

• Audit capabilities

• Lead migration projects from legacy data platforms to Databricks Lakehouse

• Define best practices for Databricks CI/CD, DevOps, and environment management

• Collaborate with data engineers, data scientists, analysts, and enterprise architects

• Provide technical guidance, architecture documentation, and design reviews

Skills & Experience

• Strong experience in data engineering, big data, or solution architecture

• Strong hands-on experience with Databricks Platform

• Expert knowledge of:

• Apache Spark

• PySpark / Scala

• Delta Lake

• SQL

• Data Lakehouse architecture

• ETL/ELT development

• Experience designing and delivering enterprise Databricks solutions

• Strong experience with cloud platforms:

• Azure Databricks (preferred)

• AWS Databricks

• Google Cloud Databricks

• Experience with:

• Databricks Workflows

• Jobs orchestration

• Notebooks

• Repos

• Cluster management

• Unity Catalog

Preferred Skills

• Databricks Certified Data Engineer Professional / Databricks Certified Architect

• Experience with:

• MLflow

• Databricks Machine Learning

• Feature Store

• Model deployment

• MLOps

• Knowledge of streaming technologies:

• Structured Streaming

• Kafka

• Experience with data governance, compliance, and enterprise security frameworks

Key Competencies

• Strong Databricks architecture and implementation expertise
• Ability to design scalable and cost-efficient Lakehouse solutions
• Strong troubleshooting and performance optimization skills
• Ability to lead technical discussions with enterprise stakeholders