Lead Engineer / Associate Architect - Microsoft Fabric

Description

We are looking for an experienced Microsoft Fabric Engineer to architect and deliver enterprise-scale data platforms using the Microsoft Fabric ecosystem. The role involves designing OneLake, Lakehouse, data pipelines, and data transformation solutions while modernising legacy data workloads. The candidate will work closely with data engineers, analysts, data scientists, and business stakeholders to deliver scalable data solutions. Strong expertise in Microsoft Fabric, Azure Data Factory, PySpark, SQL, and Azure data platforms is required.

Responsibilities

• Lead the design and implementation of end-to-end Microsoft Fabric data solutions, including Lakehouse, Data Warehouse, Data Factory, and Real-Time Analytics.
• Architect and govern OneLake structures, workspace organisation, data domains, access controls, and platform standards.
• Build and maintain scalable data pipelines using Azure Data Factory and Fabric Data Pipelines.
• Develop and optimise PySpark transformation logic using Fabric Notebooks and Spark workloads.
• Design and optimise Delta/Parquet-based Lakehouse architectures, including medallion architecture and DirectLake integration with Power BI.
• Assess, migrate, and modernise legacy SSIS packages and SSRS reports using Fabric, ADF, and Power BI.
• Implement CI/CD, monitoring, governance, security, and infrastructure automation using Azure DevOps/GitHub Actions, Bicep, or Terraform.
• Lead technical discussions and reviews, mentor data engineers, and collaborate with analysts, data scientists, and business stakeholders.

Requirements

• 8+ years of experience in Data Engineering, with at least 2 years of hands-on experience with Microsoft Fabric or closely related Azure data platforms.
• Strong hands-on expertise across Microsoft Fabric workloads, including Lakehouse, Data Warehouse, Data Factory, OneLake, and Real-Time Analytics.
• Deep experience with Azure Data Factory, including pipeline design, orchestration, linked services, triggers, parameterisation, monitoring, and migration to Fabric.
• Strong PySpark, Python, and SQL skills with experience in Spark optimisation, partitioning, caching, broadcast joins, and shuffle optimisation.
• Proven experience designing OneLake/Lakehouse architectures, medallion architecture, Delta/Parquet, data domain management, and DirectLake integration.
• Hands-on experience with SSIS and SSRS, including ETL development, report development, and migration to modern Fabric/Power BI solutions.
• Strong Azure fundamentals including ADLS Gen2, Azure Synapse, Key Vault, RBAC, CI/CD, and exposure to Terraform/Bicep.
• Strong technical leadership and communication skills; DP-600, DP-203, or equivalent certifications and experience with Purview, Eventstream, Power BI Premium, or data mesh are preferred.