MS Fabric Data Engineer

Location : Sheffield

Start Date : Immediate

Work Pattern : Hybrid, client office 3 days p/w

About the Role

We are looking for an experienced Azure Data Engineer to support the design, development and delivery of modern data platform solutions, with a particular focus on Microsoft Fabric migration and implementation .

You will play a key role in delivering data engineering and migration activities, working closely with engineering teams and stakeholders to deliver high-quality, scalable and well-tested data solutions .

The role requires strong technical expertise across Azure data technologies, alongside the ability to provide technical guidance, communicate effectively and contribute to successful delivery.

Key Responsibilities

• Configure data ingestion processes and solutions.
• Lead the execution of Microsoft Fabric migration activities.
• Lead and guide migration engineers and QA resources.
• Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks.
• Create data integration test cases and support testing activities.
• Implement and amend data processing and worker pipelines.
• Implement data validation and curation rules.
• Translate data models and mappings into detailed technical data mappings.
• Design and implement data integrations.
• Support the implementation and execution of data migration activities.
• Perform data analysis and profiling.
• Configure Azure DevOps / GitHub environments and CI/CD processes for Azure Data Factory and Databricks code.

Essential Skills & Experience

We are looking for candidates with:

• Strong experience designing and delivering Azure-based data platform solutions.
• Experience with technologies including: Microsoft Fabric o Power Apps o Azure Synapse

o Azure Data Factory (ADF) o Databricks o Azure Functions o Azure App Service o Logic Apps

o Azure Kubernetes Service (AKS) o Web Apps

• Hands-on expertise in PySpark, SQL and Python.
• Good knowledge of real-time streaming applications, preferably including Kafka, Azure Functions or Azure Service Bus.
• Strong understanding of Spark processing and performance tuning.
• Experience working with file formats and partitioning, including Parquet, JSON, XML and CSV.
• Experience with Azure DevOps / GitHub and CI/CD.
• Strong programming experience in Python, with knowledge of other programming languages beneficial.
• Experience with both synchronous and asynchronous interface approaches.
• Knowledge of data modelling, including 3NF, dimensional modelling and Data Vault 2.0.
• Experience working within Agile delivery environments.
• Ability to produce comprehensive technical documentation.
• Ability to set and manage realistic expectations around timescales, costs, benefits and measures of success.
• Power BI experience is desirable.