Analytics Consultant

Role Overview

We are looking for experienced Azure Databricks Data Engineers and Technical Leads/Architects to design, develop, and maintain scalable data engineering solutions on Azure. The ideal candidates should have strong hands-on expertise in Databricks, PySpark, Delta Lake, Python, and Advanced SQL, along with experience in building enterprise-grade data pipelines and implementing Lakehouse architecture.

Key Responsibilities

• Design, develop, and maintain reliable, scalable ETL/ELT data pipelines for structured and semi-structured data.
• Build and optimize data processing solutions using Azure Databricks, PySpark, Delta Lake, Python, and Advanced SQL.
• Implement dimensional data modelling and efficient data transformation strategies.
• Optimize Spark jobs, SQL queries, and data pipelines for performance and cost efficiency.
• Ensure data quality, validation, consistency, and Change Data Capture (CDC) implementation.
• Work with Azure services, including ADLS Gen2, Azure Data Factory (ADF), and Azure Key Vault.
• Collaborate with architects, product owners, and business stakeholders to deliver effective data solutions.

Additional Responsibilities

• Own end-to-end solution design, Lakehouse architecture, technical standards, and production support.
• Establish data governance, security, and access control practices using Unity Catalog.
• Drive performance tuning and cost optimization across Databricks, Spark, SQL, and PostgreSQL workloads.
• Conduct code reviews, mentor team members, and promote engineering best practices.
• Lead technical decisions and ensure scalable, secure, and maintainable enterprise data platforms.

Mandatory Skills

• 6–12 years of overall IT experience, including 5+ years of hands-on Databricks experience.
• Strong expertise in Azure Databricks, PySpark, Delta Lake, Python, and Advanced SQL.
• Hands-on experience developing ETL/ELT pipelines and implementing dimensional data modelling.
• Good understanding of Azure data services: ADLS Gen2, ADF, and Key Vault.
• Strong knowledge of Spark and SQL performance tuning.
• Experience with data quality, validation, and CDC.
• For Lead/Architect candidates: Lakehouse architecture, Unity Catalog, end-to-end technical ownership, and data governance.

Good to Have

• Experience with Delta Live Tables, Databricks Workflows, and Unity Catalog.
• Knowledge of CI/CD practices, Git, and automated deployment.
• Exposure to dashboarding tools such as Power BI or Apache Superset.
• Experience with PostgreSQL performance optimization.
• Agile delivery experience.
• Databricks Certified Data Engineer Associate certification.