Databrick Architect

JD

Mandatory

Former Databricks Employee (Non-Negotiable Requirement)

12+ Years of IT Experience

5+ Years of Databricks Architecture Experience

Strong Lakehouse Architecture Expertise

Hands-on Spark/PySpark Experience

Enterprise Cloud Data Platform Design Experience

Data Governance & Security Expertise

Stakeholder Management and Leadership Experience

Preferred

BFSI Domain Experience

GenAI / AI Platform Architecture Experience

Large-scale Cloud Migration Experience

Pre-sales / Solution Consulting Experience

Databricks Technologies

• Databricks Lakehouse Platform

• Client Lake

• Photon Engine

• Unity Catalog

• Client Live Tables (DLT)

• Databricks Workflows

• MLflow

• Auto Loader

• Structured Streaming

• Databricks SQL

• Databricks Asset Bundles

Data Engineering

• Apache Spark

• PySpark

• Spark SQL

• Scala (preferred)

• Python

• SQL Optimization

• Data Modeling

• Data Warehousing

Cloud Platforms

• Microsoft Azure (Preferred)

• Azure Data Factory

• Azure Synapse Analytics

• Azure Storage

• Azure Key Vault

• Azure Entra ID (Azure AD)

OR

• AWS Databricks Ecosystem

• Client Cloud Databricks Ecosystem

DevOps & Automation

• Terraform

• GitHub

• Azure DevOps

• CI/CD Pipelines

• Docker

• Kubernetes

Data Governance & Security

• Unity Catalog

• Apache Ranger

• Data Lineage Tools

• Data Quality Frameworks

• Data Classification & Governance

________________________________________

Domain Experience (Preferred)

• Banking & Financial Services (BFSI)

• Client Markets

• Wealth Management

• Insurance

• Lending & Consumer Finance

• Regulatory Reporting and Compliance

________________________________________

Education

• Bachelor's Degree in Computer Science, Engineering, Information Systems, or related field.

• Master's Degree preferred.

________________________________________

Certifications (Preferred)

• Databricks Certified Data Engineer Professional

• Databricks Certified Solution Architect

• Databricks Certified Machine Learning Professional

• Azure Solutions Architect Expert

• Azure Data Engineer Associate

Roles and Responsibilities:

Architecture & Strategy

• Design and implement enterprise-scale Databricks Lakehouse platforms.

• Define end-to-end data architecture for batch, streaming, and real-time analytics solutions.

• Develop cloud-native data platform strategies aligned with business objectives.

• Establish architectural standards, governance frameworks, and best practices.

• Lead modernization initiatives involving legacy data warehouse migration to Databricks.

Data Engineering & Integration

• Design scalable ETL/ELT pipelines using Databricks, Apache Spark, and Client Lake.

• Build medallion architecture (Bronze, Silver, Gold layers) for enterprise data platforms.

• Integrate data from multiple sources including databases, APIs, SaaS platforms, and streaming systems.

• Implement data quality, lineage, observability, and monitoring frameworks.

• Optimize data ingestion and transformation processes for performance and scalability.

Cloud & Platform Engineering

• Architect Databricks solutions on Azure, AWS, or Client Cloud Platform.

• Design secure networking, storage, identity management, and access controls.

• Implement Infrastructure as Code (Terraform, ARM, CloudFormation).

• Optimize platform usage and cloud costs through governance and monitoring.

• Lead Databricks workspace design and multi-environment deployment strategies.

AI, ML & Advanced Analytics

• Architect Databricks solutions supporting Machine Learning and Generative AI workloads.

• Design ML lifecycle management using MLflow.

• Enable enterprise AI capabilities leveraging Databricks Mosaic AI and GenAI frameworks.

• Collaborate with Data Scientists and AI teams to operationalize ML models.

• Establish MLOps best practices and governance standards.

Security & Governance

• Implement Unity Catalog for enterprise-wide governance.

• Define data security controls including RBAC, ABAC, encryption, and access management.

• Ensure compliance with industry standards including SOX, PCI-DSS, HIPAA, GDPR, and BFSI regulatory requirements.

• Establish data governance, metadata management, and audit frameworks.

Leadership & Stakeholder Management

• Act as the primary Databricks subject matter expert.

• Engage with executive stakeholders to define data transformation roadmaps.

• Conduct architecture reviews and provide technical leadership.

• Mentor data engineers, platform engineers, and architects.

• Collaborate with business, infrastructure, security, and application teams.