Azure Databricks Architect

Prodapt is the largest specialized player in the Connectedness industry. As an AI-first strategic technology partner, we serve telecom and tech companies that are building networks and shaping the digital experiences of tomorrow. We deliver consulting, managed services, and business reengineering. Our strength comes from being both AI-first and Telco-native. Gartner recognizes Prodapt as a Large, Telecom-Native Regional IT Service Provider.

Job Summary: We are seeking an experienced Azure Databricks Architect to lead a large-scale Palantir Foundry to Databricks Lakehouse migration. The ideal candidate will own the end-to-end architecture, design, migration strategy, technical governance, and delivery of enterprise data, AI/ML, analytics, and reporting solutions on Databricks.

Interview Levels/Mode: Two rounds - Video interview & In-person (Final discussion)

Key Responsibilities:

• Define and own the target-state Databricks Lakehouse Architecture, including Delta Lake, Unity Catalog, Medallion (Bronze/Silver/Gold) architecture, security, governance, and integration patterns.
• Assess existing Palantir Foundry applications and map capabilities to Databricks solutions.
• Design and implement scalable data ingestion, transformation, batch/streaming, and analytics pipelines using Databricks, Spark, Delta Lake, and Python.
• Architect AI/GenAI solutions including RAG, LLMs, document processing, chatbot, and information extraction capabilities.
• Lead migration planning, execution, validation, cutover, and optimization activities.
• Establish CI/CD, Infrastructure as Code, deployment standards, and best practices using Databricks Asset Bundles, Git, and Terraform.
• Drive performance optimization, cost management, data quality, observability, and governance.
• Collaborate with business and technology stakeholders, translate requirements into scalable solutions, and mentor engineering teams.

Required Skills & Experience:

• 12+ years of experience in Data Engineering, Data Architecture, or Data Platforms.
• 5+ years of hands-on Databricks experience designing enterprise-scale solutions.
• Strong expertise in Databricks, Apache Spark (PySpark), SQL, Python, Delta Lake, Unity Catalog, Databricks Workflows, and Databricks SQL.
• Experience leading large-scale cloud data platform migrations and modernization programs.
• Hands-on experience with AI/ML, GenAI, RAG, Vector Search, and LLM-based applications.
• Strong architecture, stakeholder management, and technical leadership skills.
• Experience creating HLD/LLD, conducting design reviews, and leading distributed engineering teams.
• Bachelor's or Master's degree in Computer Science, Engineering, IT, or related field.

Preferred Experience:

• Prior experience migrating from Palantir Foundry or other enterprise data platforms (Teradata, Oracle, Hadoop, Informatica, SQL Server, etc.) to Databricks.
• Experience working directly with US customers and leading enterprise transformation initiatives.

Best regards,

Raja Kumar