Databricks Architect

Databricks Architect
Chennai
12+Years

Role Overview

Serves as the core technical authority for designing, modernizing, and optimizing the Databricks Lakehouse architecture, code refactoring, and data execution pipelines.

Roles & Responsibilities

Refactor legacy Synapse T-SQL stored procedures, scripts, and views into native PySpark, Databricks SQL, or Delta Live Tables (DLT)

Transition legacy Synapse distribution schemas (HASH, ROUND_ROBIN, REPLICATE) to Delta Lake Liquid Clustering (CLUSTER BY) for optimal query performance and maintenance

Design efficient batch and streaming ingestion patterns using Auto Loader and Delta Lake Lakehouse mechanisms

Conduct performance tuning (OPTIMIZE, Z-ORDER, compute cluster sizing, Serverless warehouse setup)

Key Skills Required

Designing end-to-end Target State Architectures Blueprint incorporating Databricks Lakehouse components (Unity Catalog, Delta Live Tables)

Articulating design decisions, evaluating build vs. buy, and balancing performance with licensing costs

Databricks: Databricks Runtime, Delta Lake, Delta Live Tables (DLT)

Performance & Storage: Liquid Clustering, Partitioning Strategies, Auto Loader, DB SQL Serverless

Languages: Python, PySpark, Advanced SQL, Scala (optional)

CI/CD & Tools: Databricks Asset Bundles (DABs), dbx, Git integration, MLflow

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.