Senior Data Engineer

NO VISA CANDIDATES / NO AGENCIES / NO THIRD PARTIES

Senior Data Engineer (Microsoft Fabric, Dataverse & AI Data Platforms)

Position Summary

We are seeking a highly skilled Data Engineer to design, build, and govern enterprise data platforms that support operational applications, business analytics, and AI initiatives. This role will develop scalable data architectures using Microsoft Fabric tools, SQL Server, Dataverse, and Azure services to deliver trusted, AI-ready data assets across the organization.

The ideal candidate combines expertise in data engineering, dimensional and operational modeling, application integration, analytics enablement, and modern AI data architectures. This individual will partner closely with business stakeholders, application teams, architects, analysts, and AI initiatives to create reusable, governed, and scalable data products.

Key Responsibilities

Data Platform Engineering

• Design and implement enterprise data architectures using Microsoft Fabric, OneLake, Lakehouse, Warehouse, and SQL Server technologies.
• Build and maintain scalable ingestion frameworks from:
• Dataverse
• SQL Server
• REST APIs
• SaaS applications
• Flat files and SFTP sources
• Develop Fabric Data Pipelines, Dataflows Gen2, Notebooks, and Semantic Models.
• Implement bronze, silver, and gold data layer architectures.
• Optimize data pipelines for performance, reliability, scalability, and cost management.

Application Data Modeling

• Design Dataverse and operational application data models.
• Support application solution teams with:
• Data architecture
• Relationship design
• Master data management
• Integration patterns
• Establish reusable canonical business entities across systems.
• Design APIs and data contracts that support application interoperability.
• Partner with development teams to ensure applications generate high-quality analytical and AI-ready data.

Analytics Data Modeling

• Design dimensional, star-schema, and semantic models.
• Create business-friendly analytical structures supporting Power BI and self-service analytics.
• Build conformed dimensions and enterprise metrics.
• Develop data marts and analytical products that drive reporting consistency.
• Support Data Lake and enterprise semantic model strategies.

AI Data Engineering

• Design and prepare AI-ready datasets.
• Curate structured, semi-structured, and unstructured data assets for AI use cases.
• Build data pipelines that support:
• Retrieval-Augmented Generation (RAG)
• AI agents
• Semantic search
• Machine learning initiatives
• Implement metadata, lineage, and semantic enrichment patterns to improve AI effectiveness.
• Support AI model training, evaluation, and operationalization workflows.

Data Governance & Quality

• Implement data quality monitoring and remediation processes.
• Support enterprise data governance initiatives.
• Define and maintain:
• Data lineage
• Metadata
• Data cataloging
• Data ownership
• Data stewardship processes
• Ensure compliance with security, privacy, and regulatory standards.

Integration Engineering

• Design and implement integrations using:
• Dataverse
• Fabric
• Power Platform
• Azure Functions
• Logic Apps
• REST APIs
• Event-driven architectures
• Develop near real-time and batch integration patterns.
• Support change data capture (CDC) and data synchronization solutions.

Performance & Operational Excellence

• Monitor and optimize Fabric workloads.
• Tune SQL Server databases, queries, and data models.
• Establish DevOps and CI/CD practices for data assets.
• Develop monitoring, alerting, and observability processes.
• Support production operations and incident resolution.

Required Skills

Advanced

• Microsoft Fabric
• SQL Server
• T-SQL
• Dataverse
• Power BI Semantic Models
• Data Modeling
• ETL / ELT Design
• Data Warehousing
• Lakehouse Architecture

Strong

• Python
• Spark / PySpark
• Fabric Notebooks
• Data Pipelines
• Dataflows Gen2
• REST APIs
• Azure Functions
• Power Platform

Knowledge Of

• Medallion Architecture
• Delta Tables
• OneLake
• Master Data Management
• Microsoft Purview
• AI / LLM Architectures
• RAG Patterns
• Data Governance

Preferred Experience

• Microsoft Fabric implementation experience.
• Dataverse data architecture and integration experience.
• Healthcare, pharmacy, financial, or operational analytics experience.
• Building AI-ready data platforms.
• Designing enterprise-scale semantic and dimensional models.
• Working with both operational application data and analytical reporting environments.