Senior Data Engineer

R Primary Responsibilities

• Design and develop scalable data pipelines using Azure Data Factory, Azure Databricks, and related services.
• Build and optimize data transformations using Python and PySpark.
• Develop and manage SQL-based data models for reporting and analytics.
• Integrate multiple data sources (ERP, Sales, External data feeds, etc.) into unified analytical models.
• Implement data quality checks, validation frameworks, and monitoring mechanisms.
• Support Power BI dashboards with optimized backend data models.
• Handle large-scale sales and inventory datasets with performance optimization.
• Collaborate with business stakeholders to understand sales and inventory KPIs.

Secondary Responsibilities

• Implement pipelines for data engineering workflows.
• Support data governance, access control, and security best practices.
• Optimize cost and performance across Azure data services.
• Provide technical documentation and support knowledge transfer.
• Work closely with BI developers and ML engineers for advanced analytics use cases.
• Support UAT, production deployment, and post-production monitoring.

Required Technical Skills

• Strong hands-on experience with Azure Fabric, Synopses, and Data Factory.
• Azure Databricks (PySpark) development expertise.
• Azure Data Lake (ADLS Gen2) knowledge.
• Proficiency in Python and PySpark.
• Strong SQL skills (performance tuning, query optimization).
• Experience working with large sales and inventory datasets.
• Data modeling for analytics and reporting (Star/Snowflake schema).
• Experience with Power BI backend data optimization.

Soft Skills

• Strong communication and stakeholder management skills.

• Ability to work in a fast-paced, client-facing environment.

• Strong problem-solving and analytical thinking capability.

• Ability to manage timelines and deliverables independently.

• Hands-on experience working with Microsoft Fabric is mandatory.