Senior Consultant - Digital Engineering

Job Summary

Senior Consultant-National-Tech Consulting-CNS - TC - Digital Engineering - New Delhi.

National comprises of sector agnostic teams working across industries for a well rounded experience.

Responsibilities

• We are seeking a Data Engineering Architect to design and lead the delivery of scalable, secure, and governed data platforms on Azure and Databricks. The role owns end-to-end architecture from source ingestion through the medallion layers to consumption and drives engineering standards across projects.
• Design Lakehouse architectures on Azure Databricks using Delta Lake and Unity Catalog.
• Define medallion (Bronze/Silver/Gold) patterns, data models, and enterprise data products.
• Architect ingestion frameworks (batch, streaming, CDC) using Auto Loader, ADF, Event Hubs, Kafka.
• Establish governance, security, and RBAC via Unity Catalog, Key Vault, and service principals.
• Optimize performance and cost (cluster sizing, Photon, Liquid Clustering, Z-Order, caching).
• Lead CI/CD with Databricks Asset Bundles, Git, and Azure DevOps / GitHub Actions.
• Guide teams on DLT, MLflow, Delta Sharing, and Lakehouse Monitoring.
• Partner with Cyber, Cloud, and IAM teams on approvals, allow-listing, and compliance.
• Mentor data engineers and review designs, code, and production readiness.

Must-Have Skills

• Azure Databricks
• Delta Lake
• Unity Catalog
• Spark (PySpark/SQL)
• ADLS Gen2
• ADF
• Event Hubs
• Key Vault
• Data modeling

What we look for

People with the ability to work in a collaborative manner to provide services across multiple client departments while following the commercial and legal requirements. You will need a practical approach to solving issues and complex problems with the ability to deliver insightful and practical solutions. We look for people who are agile, curious, mindful, and able to sustain positive energy, while being adaptable and creative in their approach.