Databricks Architect / Senior Databricks Architect

Role & responsibilities

We are looking for a hands-on Databricks Architect / Senior Architect to lead the design and implementation of scalable data platforms on Azure. You will leverage the modern Databricks ecosystem and cutting-edge Agentic AI tools to build performant, metadata-driven solutions.

Key Responsibilities

• Architect, design, and implement end-to-end data platform solutions on Azure Databricks.
• Build robust, metadata-driven frameworks for data ingestion, processing, and orchestration.
• Optimize PySpark scripts and SQL queries for high-throughput performance tuning and cost efficiency.
• Leverage Agentic AI tools (Claude Code, Copilot, Gemini Code, MCP tools) for high-efficiency code generation, context engineering, and quality validation.
• Establish data governance and access control using Databricks Unity Catalog and Azure Purview.
• Build continuous integration and deployment (CI/CD) pipelines using Azure DevOps or Git/Jenkins.

Key Skills & Qualifications

• Core Expertise: Deep knowledge of Azure Databricks, PySpark, Unity Catalog, Delta Live Tables (DLT), Workflows, and SQL Warehouse.
• Azure Suite: Hands-on experience with Azure Data Factory (ADF), ADLS Gen2, Azure Purview, Stream Analytics, and Microsoft Fabric.
• AI & Modern Engineering: Exposure to Agentic AI engineering tools (Claude Code, Copilot, MCP tools), context engineering, and developing LLM/GenAI-powered applications.
• Data Modeling: Strong experience with Dimensional Modeling and 3NF.
• Desirable: Vector databases (Neo4j, Elasticsearch) and legacy Big Data platforms (Cloudera/Hortonworks).
• Education: B.E./B.Tech in Computer Science, IT, or a related field.