Data Engineer – Microsoft Fabric & Cloud Data Platforms

Thinkgrid Labs designs and builds custom web, mobile, cloud, data, and AI solutions. Our team brings together software engineers, architects, data engineers, and UI/UX designers to deliver reliable systems for clients around the world.

We’re expanding our data practice and building a modern data platform for a US health insurer, with Microsoft Fabric as the primary platform. You’ll design, build, and operate reliable data pipelines—from source ingestion and transformation to analytics-ready lakehouse and warehouse models. Hands-on Microsoft Fabric experience is strongly preferred, and we welcome engineers with strong Databricks or comparable cloud data platform experience who are ready to apply their skills in Fabric.

Job Title: Data Engineer – Microsoft Fabric & Cloud Data Platforms
Location: Remote
Working Hours: 2 PM IST to 11 PM IST
Experience Required: 6–10 years in data engineering, including 2+ years working with cloud data platforms
Education: Bachelor’s or Master’s degree in Computer Science, Health Informatics, or Business
Preferred Certifications: Relevant Microsoft Fabric/Azure, Databricks, or equivalent data engineering certifications

Who are you?

• Strong in SQL and Data Modeling: You work confidently with complex relational databases and design practical data models for integration, reporting, and analytics.

• Experienced with Cloud Data Platforms: You’ve built production solutions on Microsoft Fabric, Databricks, or comparable platforms. Experience with Fabric Mirroring, OneLake, Lakehouse/Warehouse, and Fabric Data Factory is particularly valuable.

• A Reliable Pipeline Builder: You design ingestion and transformation pipelines that handle incremental loads, CDC, schema changes, and recovery from failures.

• Comfortable with Python and Spark: You use code and notebooks to process data, implement reusable workflows, and validate results.

• Focused on Data Quality: You build checks, reconciliation, and clear data contracts into your work so downstream teams can trust the data.

• Operationally Minded: You monitor pipeline health, troubleshoot production issues, document runbooks, and balance performance with cost.

• Security Conscious: You handle sensitive data responsibly, including PII/PHI, and apply appropriate access controls and governance practices.

  What will you be doing?

• Build Data Pipelines: Develop and maintain ingestion and ETL/ELT pipelines across relational databases, files, APIs, and other sources, primarily using Microsoft Fabric.

• Develop Lakehouse and Warehouse Solutions: Organize data across raw, refined, and curated layers, using OneLake and Fabric Lakehouse/Warehouse to support reliable downstream consumption.

• Transform and Model Data: Build reusable transformations and analytics-ready models that translate source data into useful, well-documented datasets.

• Manage Incremental Data and Change: Implement CDC, replication, or watermark-based loading as appropriate. Handle deletes, late-arriving data, backfills, and schema evolution safely, using Fabric Mirroring where suitable.

• Orchestrate Resilient Workflows: Manage dependencies, scheduling, retries, idempotency, and failure recovery using Fabric Data Factory, notebooks, and appropriate orchestration tools.

• Ensure Quality and Observability: Implement data validation, reconciliation, logging, metrics, lineage, and alerts to maintain data accuracy and pipeline reliability.

• Optimize Performance and Cost: Tune queries, Spark workloads, partitioning, file sizes, and compute usage to meet delivery expectations efficiently.

• Collaborate and Document: Work with platform architects, security teams, analysts, and business stakeholders on requirements, governance, and delivery. Maintain pipeline documentation, data definitions, SLAs, and operational runbooks.

Must-have skills

• Strong SQL and experience working with large, complex relational databases
• Production experience with Microsoft Fabric, Databricks, or a comparable cloud data platform
• Python and Apache Spark for data processing, transformation, and validation
• Experience building and operating ingestion and ETL/ELT pipelines
• Data modeling and lakehouse or data warehouse fundamentals
• CDC/incremental loading, schema evolution, and data quality practices
• Pipeline orchestration, monitoring, troubleshooting, and performance tuning
• Git-based development workflows and familiarity with CI/CD for data solutions

Benefits

• 5-day work week
• Health Insurance
• 100% remote setup with flexible work culture and international exposure
• Opportunity to work on mission-critical healthcare projects impacting providers and patients globally