Azure Databricks Developer (SJ-IQ)

Job highlights

• Required Qualifications
• Experience: 6-10 Years
• The ideal candidate will possess strong expertise in Azure Databricks,Unity Catalog,Delta Lake,Medallion Architecture,and hands-on experience with Databricks Genie AI for conversational analytics and AI-driven data consumption.
• Experience with Git,CI / CD,and DevOps practices.

Senior Developer - Azure Databricks (Unity Catalog Genie AI)
Job Title: Senior Developer - Azure Databricks, Unity Catalog Genie AI
Experience: 6-10 Years
Location: Bengaluru / Gurgaon/ Kochi
Employment Type: Full-Time

Role Overview
We are seeking an experienced Senior Developer - Azure Databricks to design, develop, and optimize enterprise-scale data and AI platforms on Azure. The ideal candidate will possess strong expertise in Azure Databricks, Unity Catalog, Delta Lake, Medallion Architecture, and hands-on experience with Databricks Genie AI for conversational analytics and AI-driven data consumption.
The role involves building modern data platforms, implementing governance frameworks, enabling self-service analytics, and supporting AI/GenAI initiatives across business functions.

Key Responsibilities
Data Engineering Platform Development

• Design and develop scalable data pipelines using Azure Databricks and Apache Spark.
• Implement ETL/ELT solutions for structured, semi-structured, and unstructured data.
• Develop and optimize Delta Lake-based data platforms following Medallion architecture (Bronze, Silver, Gold).
• Create reusable frameworks for ingestion, transformation, data quality, and monitoring.
• Optimize Spark workloads for performance, scalability, and cost efficiency.
  Unity Catalog Data Governance
• Design and implement enterprise data governance using Unity Catalog.
• Configure catalogs, schemas, tables, views, and external locations.
• Implement fine-grained access controls including row-level and column-level security.
• Establish data lineage, metadata management, and governance policies.
• Collaborate with data stewards and security teams to ensure compliance and auditability.
  Databricks Genie AI Self-Service Analytics
• Develop and configure Genie Spaces for conversational analytics.
• Enable natural language querying for business users across governed data domains.
• Create semantic layers, business definitions, and trusted datasets for Genie.
• Partner with business stakeholders to improve data discoverability and self-service adoption.
• Integrate Genie with enterprise AI and analytics use cases.
• Collaborate with Data Scientists and ML Engineers for model operationalization.

Required Qualifications
Technical Skills

• 6+ years of experience in Data Engineering.

• 4+ years of hands-on experience with Azure Databricks.

• Strong expertise in:
  Apache Spark (PySpark)
  Delta Lake
  Unity Catalog
  Databricks Workflows
  Databricks SQL

• Experience implementing Medallion Architecture and Lakehouse solutions.

• Strong understanding of data modelling principles and dimensional modelling.

• Experience with Azure ecosystem:
  Azure Data Factory
  Azure Storage (ADLS Gen2)
  Azure Key Vault
  Azure Entra ID (Azure AD)

• Experience with Git, CI/CD, and DevOps practices.
  Unity Catalog Expertise

• Catalog and schema management.

• Data lineage and metadata governance.

• Security and access management.

• External locations and storage credentials.

• Regulatory compliance and audit support.
  Genie AI Expertise

• Designing Genie Spaces and conversational experiences.

• Creating trusted semantic datasets.

• Natural language query optimization.

• Business glossary and metadata integration.

• User enablement and self-service analytics adoption.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.