Data Engineer – Microsoft Fabric Lead / Architect

Data Engineer – Microsoft Fabric Lead / Architect

Experience: 8-15 Years

Work locations Bangalore/ Hyderabad

Hybrid work mode ( 2 days work from office)

About the Role

We are looking for a hands-on Microsoft Fabric Lead Data Engineer with strong Data Engineering fundamentals and proven Microsoft Fabric implementation experience. The ideal candidate should have worked on end-to-end data engineering projects involving Azure Data Factory, Azure Synapse Analytics, Microsoft Fabric, PySpark, SQL, and enterprise-scale data platforms.

The role requires professionals who can design data ingestion frameworks, build scalable ETL pipelines, develop PySpark notebooks, handle large-volume data processing, optimize performance, and modernize enterprise data platforms using Microsoft Fabric. Candidates should be able to clearly explain the architecture, transformations, and business logic implemented in their projects.

Key Responsibilities

Microsoft Fabric & Data Engineering

• Design and implement enterprise data solutions using Microsoft Fabric.
• Build and support Lakehouse, Warehouse, OneLake, Shortcuts, and Medallion Architecture.
• Develop data ingestion frameworks from ERP and enterprise applications such as SAP, Dynamics, and other business systems.
• Build scalable batch and incremental ETL/ELT pipelines.

PySpark & Notebook Development

• Develop complex data transformation logic using PySpark.
• Build Fabric Notebooks for ingestion, cleansing, validation, and transformation.
• Implement reusable frameworks for: Data Quality, Error Handling, Logging, Incremental Processing
• Develop business logic to process data from multiple sources and formats.

Data Processing & Transformation

• Work with structured, semi-structured, and unstructured data.
• Transform and standardize enterprise data for analytics consumption.
• Implement Lakehouse and data warehousing best practices.

Performance Optimization

• Optimize large-volume data processing workloads.
• Improve PySpark, SQL, and pipeline performance.
• Handle scalability, partitioning, data volume, and processing bottlenecks.

Migration & Modernization

• Support Azure Synapse to Microsoft Fabric migration initiatives.
• Assess existing Azure data platforms and recommend Fabric-based target architectures.

Must-Have Skills

• Strong hands-on Data Engineer with implementation experience.
• Can explain end-to-end project architecture and transformation design.
• Strong understanding of Fabric concepts.
• Experienced in building and optimizing ETL pipelines for enterprise-scale datasets.
• Comfortable handling structured and unstructured data.
• Strong problem-solving mindset with a focus on development rather than support activities.
• Able to independently design, develop, optimize, and troubleshoot Fabric solutions.
• Should have handled large data set data processing
• 1-2+ Microsoft Fabric implementation projects.
• Deep understanding of Fabric architecture and core concepts.
• Worked with any of the ERP system like SAP, Salesforce for building data pipeline
• Azure Data Factory (Mandatory)
• Fabric Notebooks
• PySpark (Mandatory)
• Azure Synapse Analytics
• Enterprise Data Platform Modernization

Good to have Skills

Microsoft Fabric :

• Lakehouse
• OneLake
• Data Pipelines
• Warehouse
• Shortcuts
• Fabric Mirroring
• Fabric Warehouse
• Synapse to Fabric Migration Experience
• Medallion Architecture

Data Engineering

• PySpark (Mandatory)
• Python
• ETL/ELT Development
• Data Transformation Development
• Data Quality & Validation
• Error Handling Frameworks
• Incremental Processing

Azure Technologies

• ADLS
• Cloud Data Engineering Fundamentals

Enterprise Integration

• ERP connectivity and data ingestion.
• SAP, Dynamics, or similar enterprise application integrations.

Performance Engineering

• Large-scale data processing.
• Performance tuning.
• Optimization of notebooks, transformations, and pipelines.

Preferred Certifications

• DP-600: Fabric Analytics Engineer Associate
• DP-700: Microsoft Fabric Data Engineer Associate