Principal Microsoft Fabric Data Engineer

Job Description: Lead / Principal Microsoft Fabric Data Engineer

Location: Bangalore (On-site, with flexibility for 12 remote days)

Job Type: Permeant

Experience Level: 8+ years in data engineering overall, with 2+ years in a leadership/architect capacity

About the Role

We are seeking an experienced Lead Microsoft Fabric Data Engineer to drive our initiative to modernize our legacy BI platform onto Microsoft Fabric. This is a hands-on technical leadership role: you will architect the core Fabric solution yourself, set practical engineering standards, and guide a small team through a phased. We expect most candidates to bring strong, deep Azure Synapse/ADF/Databricks experience plus 1.52+ years of practical, hands-on Microsoft Fabric work.

Key Responsibilities

1. Technical Leadership & Hands-On Architecture

• Own the end-to-end technical design of the Fabric Lakehouse/Warehouse solution, and remain hands-on in build for the first 2-3 quarters
• Set pragmatic engineering standards and coding conventions for the Fabric platform (naming, folder structure, notebook/pipeline patterns)
• Act as the team's go-to person for Fabric - evaluate new features/capacity options and recommend what's actually useful for our workloads
• Mentor 2-4 mid-level data engineers day to day on Fabric and Spark/SQL best practices

2. Lakehouse & Warehouse Design

• Design the Lakehouse/Data Warehouse architecture in Fabric, including medallion (Bronze/Silver/Gold) layout
• Define workspace and domain structure, full enterprise data-mesh patterns are a plus, not a prerequisite
• Right-size Fabric capacity (F-SKU) and monitor/optimize compute and storage cost
• Choose sensibly between Direct Lake, Import, and Direct Query per use case

3. Legacy Platform Migration

• Assess our current BI/DW stack and produce a practical, phased migration plan
• Lead pilot migrations of 1-2 priority workloads, validate the approach, then scale to the rest
• Prior experience migrating any legacy platform (SQL Server, Synapse, Teradata, Oracle, Cognos, etc.) to a modern cloud/Fabric target is valued
• Define simple, trackable success metrics for migration progress and data quality

4. Core Data Engineering

• Build and review data pipelines using Fabric Data Factory, Dataflows Gen2, and Notebooks (PySpark/Spark SQL)
• Implement incremental loads, CDC, and SCD patterns
• Where relevant, build real-time ingestion with Event streams and KQL Database - deep Real-Time Intelligence expertise is a plus, not a must-have.
• Apply core performance practices: partition design, file/table optimization, query tuning
• Stand up basic automated data-quality checks (schema, null/volume checks)

5. Security & Governance

• Implement workspace security, item-level permissions, and row-level security (RLS) for the workloads we own
• Apply sensible data classification
• Follow existing company security/compliance policy; flag gaps to the security/compliance team

6. DevOps & Automation

• Set up Git integration and a working branching strategy for Fabric development
• Build CI/CD for Fabric deployments using Azure DevOps or GitHub Actions and the Fabric REST APIs
• Script routine deployment/admin tasks in PowerShell or Python
• Full infrastructure-as-code (Terraform/Bicep) and Kubernetes are nice-to-have.

7. Stakeholder Collaboration

• Work directly with business stakeholders and Power BI developers/analysts to translate requirements into working pipelines and models
• Report progress and blockers clearly to engineering leadership
• Run lightweight design reviews with the team rather than formal architecture review boards

8. Continuous Improvement

• Keep the team current on new Fabric features as they roll out
• Share learnings internally - a short write-up or demo after each major milestone is enough; formal thought-leadership/publishing is a bonus.

Required Skills & Experience

Microsoft Fabric (Must-Have)

• 1.52+ years of genuine hands-on Fabric experience (production or serious pilot work) - note this is close to the realistic ceiling given Fabric's Nov 2023 GA date
• Working knowledge of Lakehouse, Warehouse, Data Factory (Fabric), Dataflows Gen2, and Notebooks
• Practical understanding of OneLake, workspace management, and Fabric capacity basics
• Comfortable choosing between Direct Lake, Import, and DirectQuery for a given scenario

Azure & Data Platform (this is where most of the depth should come from)

• 8+ years in data engineering, including 4+ years hands-on with Azure data services (Synapse Analytics, ADF, ADLS, or Databricks)
• Strong PySpark, Spark SQL, Python, and T-SQL skills
• Solid grasp of Delta Lake and Parquet; distributed-computing fundamentals
• Most candidates will be Azure Synapse/ADF veterans who moved into Fabric over the last 1-2 years - that profile is exactly what we want

Architecture & Design

• Experience designing data warehouse/lakehouse solutions for a mid-to-large
• Solid dimensional modeling skills; data vault or data mesh exposure is a plus
• Basic FinOps awareness for cloud data cost control

Migration Experience

• Has led or been a key contributor on at least 1 legacy-to-cloud data platform migration (any source: on-prem SQL Server, Oracle, Teradata, Synapse, Cognos, Tableau, etc.)
• Comfortable doing lightweight current-state assessment and migration planning

Security & Governance

• Practical experience with RLS/OLS/CLS and workspace-level security in Power BI/Fabric or Synapse
• General familiarity with data governance concepts (classification, lineage) - deep Purview administration experience is a plus.

DevOps & Automation

• Experience with Azure DevOps or GitHub Actions for CI/CD
• Working knowledge of Fabric REST APIs, PowerShell, or Python for automation

Certifications

• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)
• Microsoft Certified: Fabric Data Engineer Associate (DP-700)
• Microsoft Certified: Azure Solutions Architect Expert (AZ-305)

Preferred Qualifications

• Experience in manufacturing, retail, or healthcare data environments
• Power BI administration and Premium/Fabric capacity management experience
• Exposure to AI/ML integration with Fabric (Azure ML, Copilot, OpenAI) a strong plus
• Comfort working in agile/Scrum delivery

Leadership & Soft Skills

• Able to lead and mentor a small team (2-4 engineers) by example, staying hands-on rather than purely directive
• Clear communicator who can explain technical trade-offs to both engineers and business stakeholders
• Comfortable operating with a fair amount of ambiguity - this is a build-as-we-go modernization, not a fully scoped enterprise program
• Pragmatic, delivery-focused - biased toward shipping a working pilot over designing the perfect long-term architecture upfront

Key Deliverables

• Current-state assessment and a phased migration plan for priority workloads
• Core Lakehouse/Warehouse architecture in Fabric, agreed with the team
• Workload migrated and validated end-to-end
• Baseline engineering standards, Git/CI-CD setup, and initial security model in place

Why Join Us?

This is a chance to build a modern Fabric-based data platform largely from scratch, with real ownership over the architecture and hands-on influence from day one. You'll work on one of Microsoft's newest and fastest-evolving data platforms, grow into a broader platform-leadership role as the team scales, and help shape how the organization does data engineering going forward.

Best Wishes,

Core Edge Solutions LLP
138,7th Cross,29th Main road
BTM 2nd Stage Bangalore 560076
www.coreedgesolutions.com