Senior Data Engineer

Role & responsibilities

• Design, develop, and implement end-to-end data solutions using Microsoft Fabric components such as Lakehouse, Data Factory (Pipelines), Dataflows Gen2, Warehouse, and Notebooks.
• Build and maintain scalable data ingestion, transformation, and orchestration pipelines for structured and unstructured data.
• Develop and implement robust data models and semantic layers to support downstream analytics, including Power BI.
• Ensure data quality, reliability, and performance optimization across all Fabric workloads.
• Manage and optimize Delta Lake tables and Lakehouse architectures, including partitioning, indexing, and storage optimization.
• Collaborate with IT, analytics, and business teams to gather requirements, define architecture, and deliver enterprise data solutions.
• Implement and maintain data governance, lineage, security, and compliance frameworks using Fabric and Purview integration.
• Build and maintain real-time and near-real-time data processing solutions using Fabric Event Streams and streaming datasets.
• Design and manage data warehousing solutions within Fabric Warehouse for high-performance analytics.
• Develop and maintain documentation, including data architecture diagrams, data dictionaries, pipeline flows, and operational runbooks.
• Integrate data from various sources, including ERP systems, APIs, databases, files, and external platforms, into Fabric.
• Monitor and troubleshoot data pipelines, workloads, and capacity performance to ensure SLAs are met.
• Enable and support self-service analytics by preparing curated datasets and collaborating with Power BI developers.
• Implement CI/CD and DevOps practices for data pipelines and Fabric artifacts.
• Optimize storage and compute utilization to manage cost and capacity planning within Fabric environments.
• Work in Agile methodology to design, develop, test, and deploy data engineering solutions and enhancements.
• Continuously stay updated with Microsoft Fabric advancements, new features, and best practices.

Preferred candidate profile

• Bachelors degree in Computer Science, Information Technology, or a related field.
• 5+ years of experience in data engineering, data platforms, or enterprise analytics systems.
• 5+ years of experience in modern data platforms such as Azure Data Services, Microsoft Fabric, or equivalent.
• Strong experience with:
• Microsoft Fabric (Lakehouse, Pipelines, Warehouse, Notebooks)
• SQL and data modeling (dimensional and normalized models)
• PySpark / Spark (preferred within Fabric notebooks)
• Solid understanding of:
• Data warehousing concepts and architectures
• ETL/ELT frameworks and pipeline orchestration
• Distributed data processing systems
• Experience integrating and supporting Power BI semantic models and datasets.
• Strong knowledge of data governance, security, and access control models.
• Excellent problem-solving, analytical, and performance optimization skills.
• Strong communication and stakeholder management skills.
• Ability to work in fast-paced, cross-functional, and agile environments.