Databricks Architect

Greetings from Maneva!

Job Description

Job Title - Databricks Architect

Experience - 15 - 20 Years

Location - Chennai (Hybrid)

Notice - Immediate to 30 days

Requirements:-

• Refactor legacy Synapse T-SQL stored procedures, scripts, and views into native PySpark, Databricks SQL, or Delta Live Tables (DLT).
• Transition legacy Synapse distribution schemas (HASH, ROUND_ROBIN, REPLICATE) to Delta Lake Liquid Clustering (CLUSTER BY) for optimal query performance and maintenance.
• Design efficient batch and streaming ingestion patterns using Autoloader and Delta Lake Lakehouse mechanisms.
• Conduct performance tuning using OPTIMIZE, Z-ORDER, compute cluster sizing, and Serverless Warehouse setup.

Key Skills

• Target State Architecture: Designing end-to-end Target State Architecture Blueprints incorporating Databricks Lakehouse components, Unity Catalog, and Delta Live Tables (DLT).
• Databricks: Databricks Runtime, Delta Lake, Delta Live Tables (DLT).
• Architecture & Cost Optimization: Articulating design decisions, evaluating build vs. buy, and balancing performance with licensing costs.
• Performance & Storage: Liquid Clustering, Partitioning Strategies, Auto Loader, Databricks SQL Serverless.
• Languages: Python, PySpark, Advanced SQL, Scala (Optional).
• CI/CD & Tools: Databricks Asset Bundles (DABs), dbx, Git Integration, MLflow.