Databricks Technical Lead

Key Responsibilities

Customer Architecture and Pre-sales

• Lead executive and technical discovery workshops; translate business outcomes and non-functional requirements into Data + AI roadmaps and target architectures.
• Architect end-to-end solutions covering ingestion, batch and streaming pipelines, lakehouse storage, governance, data warehousing, analytics, ML/GenAI and operational applications.
• Define phased migrations from legacy warehouses, Hadoop, cloud data platforms and ETL estates to Databricks.
• Lead technical solutioning for proposals, demonstrations, RFP/RFI responses, estimates and proofs of value; clearly articulate business value and architectural trade-offs.
• Collaborate with sales, delivery, cloud partners and Databricks account teams on joint pursuits and co-sell readiness.

Delivery Governance and Practice Building

• Provide architecture governance from pursuit through production, including design, security, performance, cost and operational-readiness reviews.
• Guide CI/CD, infrastructure as code, environment strategy, testing, observability, release management and production support.
• Define the practice roadmap, skills matrix, certification plan and hiring profile; mentor architects, data engineers and technical leads.
• Create reusable reference architectures, discovery and estimation templates, demos, accelerators and technical playbooks.

Databricks Platform Expertise

Core Lakehouse, Data Engineering and Governance

• Delta Lake and lakehouse design: ACID transactions, schema evolution, Change Data Feed, optimization and Medallion or domain data-product patterns.
• Unity Catalog: catalogs/schemas, access controls, managed and external locations, lineage, discovery, audit, federation and Delta Sharing.
• Lakeflow Connect, Lakeflow Spark Declarative Pipelines and Lakeflow Jobs for ingestion, batch/streaming transformation, data quality, orchestration and observability.
• Apache Spark, PySpark, SQL and Structured Streaming, including workload sizing, optimization and production troubleshooting.
• Databricks SQL, serverless warehouses, query optimization, AI/BI dashboards and Genie for governed natural-language analytics.

AI, Applications and Platform Engineering

• Lakebase architecture for PostgreSQL-compatible transactional workloads, application data and agent state, with appropriate integration to the lakehouse.
• Mosaic AI and MLflow: model lifecycle, evaluation, serving, Vector Search, RAG, AI Gateway, monitoring and governed agentic patterns. Familiarity with Agent Bricks is valuable.
• Databricks Apps and secure Data + AI application patterns, including the ability to separate operational, analytical and agentic workload requirements.
• Databricks Asset Bundles, Terraform, Git-based CI/CD, system tables, platform observability, serverless/compute policies and FinOps.
• Azure architecture is preferred; AWS or Google Cloud experience is valuable. Familiarity with Kafka, ADF, dbt, Snowflake, Synapse or Power BI is advantageous.

Required Experience, Qualifications and Certifications

• 10+ years in data engineering, data architecture, analytics or cloud data platforms, including at least 4 years of substantial Databricks experience.
• Demonstrated success architecting and delivering enterprise-scale Databricks solutions in productionnot only isolated notebooks or pipelines.
• Strong pre-sales capability across discovery, technical storytelling, estimates, proposals, demonstrations, proofs of value and senior customer presentations.
• Strong SQL and Python/PySpark skills, with the ability to review implementations, troubleshoot designs and guide engineering teams.
• Experience leading architecture governance and multidisciplinary teams; prior practice-building, hiring, mentoring or reusable-IP responsibility is strongly preferred.
• Professional-level Databricks certification is strongly preferred at joining. An otherwise strong candidate must complete an agreed professional certification within six months.
• Excellent communication, executive presence and structured problem-solving; able to explain technical and commercial trade-offs clearly.
• Experience in consulting, a systems integrator or a Databricks partner is preferred, particularly in BFSI, manufacturing, healthcare/life sciences or retail.
• Willingness to work from Palnis Hyderabad office and travel for customer or partner engagements when required.