AWS Data Engineer

Job Title: Data Engineer – AWS | Python | PySpark | ETL/ELT

Experience: 4–10 Years

Location: Remote – India

Engagement: Contract

Duration: 6+ months, with possibility of extension

Job Overview

• We are looking for a skilled Data Engineer with hands-on experience in cloud-based data engineering, big data technologies, and large-scale data pipeline development. The ideal candidate will have strong proficiency in Python, SQL, PySpark, ETL/ELT processes, and cloud data services, with experience building, optimizing, and maintaining reliable data solutions for enterprise environments.
• Candidates should have practical experience with AWS data services and distributed data processing technologies. Experience with Databricks, data warehousing, workflow orchestration, and production data pipelines will be highly valued.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for data ingestion, transformation, processing, and integration.
• Build data processing solutions using Python, PySpark, SQL, and Apache Spark.
• Work with cloud data services, including AWS S3, Glue, EMR, and Redshift, to develop and maintain enterprise data solutions.
• Develop and optimize data workflows using Databricks and orchestration tools such as Apache Airflow.
• Integrate data from multiple sources, including databases, APIs, files, and other enterprise systems.
• Perform data cleansing, transformation, validation, and quality checks to ensure data accuracy and consistency.
• Design and maintain data models, data lakes, and data warehouse structures to support analytics and reporting.
• Troubleshoot production issues and optimize data pipelines for performance, scalability, reliability, and cost efficiency.
• Implement data security, governance, monitoring, and development best practices.
• Collaborate with cross-functional teams, including architects, analysts, and business stakeholders, to understand requirements and deliver effective data solutions.
• Follow software engineering best practices, including version control, testing, documentation, and CI/CD.

Required Skills and Qualifications

• Hands-on experience in Data Engineering, Big Data Engineering, or a related technical role.
• Strong proficiency in Python and SQL.
• Experience developing ETL/ELT pipelines and end-to-end data processing workflows.
• Hands-on experience with PySpark and Apache Spark for distributed data processing.
• Practical experience with AWS data services, particularly S3, AWS Glue, EMR, and/or Amazon Redshift.
• Understanding of data warehousing, data modeling, data lake architecture, and distributed processing concepts.
• Experience processing and transforming large volumes of structured and semi-structured data.
• Ability to troubleshoot data pipeline failures and optimize queries, jobs, and workflows.
• Experience working with production data pipelines in enterprise environments.