Data Engineer

Job Title: Data Engineer

Experience: 8+ Years

Location: Pan India

Notice Period: Immediate to 15 Days

Education: BE/B.Tech/ME/M.Tech/MCA/M.Sc graduates only

Job Summary We are looking for an experienced Data Engineer with strong hands-on expertise in Azure Services, Python, PySpark, ETL development, and SQL. The ideal candidate should have experience building and optimizing data pipelines, managing large datasets, and implementing data governance and data quality solutions using tools such as Soda and Alation.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure services, Python, and PySpark.
• Build and optimize ETL/ELT processes for data extraction, transformation, and loading.
• Write complex SQL queries and optimize database performance.
• Develop data processing solutions using PySpark for large-scale datasets.
• Implement data quality checks, monitoring, and validation using Soda.
• Support data cataloging, metadata management, data lineage, and governance using Alation.
• Ensure data accuracy, consistency, security, and compliance with governance standards.
• Troubleshoot data pipeline failures and improve performance and reliability.
• Collaborate with data analysts, architects, and business stakeholders to deliver reliable data solutions.

Essential Skills

• Azure Services
• Python programming
• PySpark
• ETL/ELT development
• SQL
• Data governance and data quality tools such as Soda and Alation