Senior Data Engineer-Microsoft Fabric and/or Databricks- India

XCentium is looking for an experienced C#/Azure Integration Developer to develop integrations across enterprise applications and third-party solutions to support client business requirements.

The ideal candidate has strong experience with Azure integration technologies, enterprise application integration, C#, APIs, and data pipelines. You should be comfortable working across technical teams, following enterprise architecture standards, troubleshooting complex integration issues, and supporting solutions from development through production.

Key Responsibilities

• Coordinate with other teams to plan and roll out changes to production.

• Conform to enterprise architecture standards, data architecture standards, and governance frameworks.

• Maintain the technology stack, including commercial and open-source integration platforms.

• Understand and follow critical steps in the Software Development Life Cycle (SDLC) to maintain and improve enterprise-wide development and architecture standards.

• Assist with incident response, troubleshooting, and problem resolution.

• Communicate project status to Integration Leads and Project Managers.

• Develop and maintain middleware, ETL flows, and pipelines connecting enterprise applications, including real-time and batch data integrations.

• Maintain the integrity and smooth functioning of the company’s integration architecture.

• Work with Azure technologies including Azure Data Factory, Azure Logic Apps, Azure Function Apps, Azure Event Hub, API Management, Azure SQL Database, and Azure Cosmos DB.

• Work with clients to design, develop, and deploy end-to-end data integration solutions using Azure data and integration technologies.

• Collaborate with stakeholders to deliver effective data integration solutions.

Qualifications

• BA/BS degree preferred, with 7+ years of relevant work experience.

• 5+ years of experience implementing enterprise application integration (EAI) using technologies such as Azure Data Factory, MuleSoft, Dell Boomi, webMethods, or TIBCO.

• 3+ years of experience designing and developing solutions using Azure technologies.

• 3+ years of experience with C# and Azure Functions.

• Knowledge of enterprise data architecture and integration architecture standards.

• Knowledge of common system integration methods and technologies, including web services, SOAP, JSON, XML, business process automation, and orchestration tools and software.

• Experience working with APIs from commercial software applications such as CRM or ERP systems.

• Experience with continuous integration practices and tooling for automated build and delivery of commercial software.

Location

• Hybrid role based in the Dallas area.

How to Apply

Interested in joining XCentium? Please send your resume to [email protected] with C#/Azure Integration Developer in the subject line.

Senior Data Engineer — Microsoft Fabric & Databricks (India)

XCentium is looking for a Senior Data Engineer with deep hands-on expertise in Microsoft Fabric and/or Databricks to join our data practice.

The ideal candidate can own the end-to-end implementation of complex data pipelines and lakehouse components, translate architectural designs into production-grade solutions, and provide technical guidance to junior engineers. You will work closely with data architects and client stakeholders to build ingestion, transformation, data quality, and semantic modeling layers while leading performance optimization and engineering best practices across medallion architecture tiers.

Key Responsibilities

• Own the end-to-end implementation of complex data pipelines and lakehouse components.

• Translate architectural designs into scalable, production-grade data solutions.

• Work closely with data architects and client stakeholders to clarify requirements and technical design decisions.

• Build ingestion, transformation, data quality, and semantic modeling layers across enterprise data platforms.

• Implement raw ingestion at the Bronze layer, cleansing and validation at the Silver layer, and business-ready modeling at the Gold layer.

• Build star schemas in the Gold layer for BI and analytics consumption.

• Implement Change Data Capture (CDC) patterns within the medallion architecture framework.

• Build ingestion pipelines using Azure Data Factory, Dataflows Gen2, Databricks Workflows, or Autoloader.

• Develop solutions using PySpark and Spark Structured Streaming within Fabric and/or Databricks notebooks.

• Design and support real-time and batch ingestion paths using technologies such as Eventstream, Spark Structured Streaming, and scheduled pipelines.

• Optimize Spark and Delta workloads through performance tuning, partitioning, and other engineering best practices.

• Implement and design data quality frameworks and testing standards across medallion tiers.

• Support data catalog, lineage, and governance using Microsoft Purview or Unity Catalog.

• Support master data management integration patterns and related enterprise data requirements.

• Mentor and provide technical guidance to junior engineers.

• Troubleshoot complex pipeline, integration, and performance issues independently.

• Serve as a senior technical resource on client engagements, proof-of-concepts, accelerators, and enterprise-scale implementations.

Qualifications

• 6+ years of experience in data engineering or enterprise data platform roles.

• Strong hands-on experience with Microsoft Fabric, including OneLake, Lakehouse, Data Factory, Eventstream, and KQL Database, and/or Databricks, including Workflows, Unity Catalog, and Delta Live Tables.

• Strong proficiency with Delta Lake and its role across lakehouse architecture tiers.

• Strong proficiency with PySpark and Spark Structured Streaming.

• Experience supporting Power BI and DirectLake connectivity or other BI tools consuming Gold-layer data.

• Experience implementing medallion architecture patterns across Bronze, Silver, and Gold layers.

• Experience building star schemas for BI consumption.

• Experience implementing Change Data Capture (CDC) patterns.

• Experience building ingestion pipelines using Azure Data Factory, Dataflows Gen2, Databricks Workflows, or Autoloader.

• Familiarity with source systems such as Salesforce, Oracle Service Cloud, Profisee, or similar master data management platforms.

• Understanding of real-time and batch ingestion architecture.

• Experience with performance tuning, partitioning, and optimization of Spark and Delta workloads.

• Experience implementing data quality frameworks and testing standards.

• Familiarity with Microsoft Purview or Unity Catalog for data cataloging, lineage, and governance.

• Experience with master data management integration patterns.

• Ability to work directly with client stakeholders and architects to clarify requirements and design decisions.

• Ability to operate as the senior technical resource on client engagements.

• Ability to troubleshoot complex pipeline and performance issues independently.

Preferred Qualifications

• Microsoft Fabric, Azure Data Engineer, or Databricks certification, such as Databricks Certified Data Engineer Professional.

• Experience working across both Microsoft Fabric and Databricks platforms.

• Experience with CI/CD for data pipelines using Azure DevOps, GitHub Actions, or similar tools.

• Experience mentoring or providing technical guidance to junior engineers.

• Prior consulting experience across multiple client engagements.

• Exposure to Azure AI Foundry or AI/ML integration within the lakehouse.

Location

• India

Pay: ₹1,500,000.00 - ₹3,000,000.00 per year

Benefits:

• Commuter assistance
• Health insurance
• Paid sick time
• Paid time off
• Provident Fund
• Work from home

Work Location: Remote