Data Engineer

Job Details:-

Location: Gurugram (Work From Office)
Experience: 6+ Years

Job Summary

We are seeking a highly skilled Data Engineer with 6+ years of experience and strong hands-on expertise in PySpark, AWS, and Python to build and maintain scalable, high-performance data pipelines. The role involves working on large-scale distributed data processing systems and delivering reliable data solutions to support analytics and business intelligence.

Key Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using Python and PySpark
• Build and maintain large-scale data processing frameworks on AWS
• Develop batch and near-real-time data pipelines
• Process structured and semi-structured data using Apache Spark
• Ingest data from multiple sources including APIs, databases, and streaming platforms
• Optimize Spark jobs for performance and cost efficiency
• Build and manage data lakes using AWS S3
• Implement data workflows using orchestration tools (Airflow, AWS Glue, Step Functions)
• Ensure data quality, validation, and reliability
• Monitor production pipelines and resolve failures
• Collaborate with data scientists, analysts, and business stakeholders
• Follow best practices for security, governance, and compliance on AWS

Required Skills & Qualifications

• 6+ years of experience as a Data Engineer

• Strong proficiency in Python and PySpark

• Hands-on experience with AWS services, including:

• S3, EC2, EMR, Glue, Lambda

• Redshift, Athena

• CloudWatch, IAM

• Strong experience with Apache Spark and distributed data processing

• Advanced SQL skills for querying and data transformation

• Experience with ETL tools and frameworks

• Experience with data warehousing and data modeling

• Strong understanding of data partitioning, performance tuning, and cost optimization

• Experience with Git and CI/CD pipelines

Good to Have / Preferred

• Experience with streaming technologies (Kafka, Kinesis)
• Knowledge of Delta Lake / Iceberg / Hudi
• Experience with Terraform or CloudFormation
• Exposure to BI tools (Power BI, Tableau, QuickSight)
• Experience supporting production data systems in enterprise environmentsRole & responsibilities