Aws Data Engineer (Blr/Chn/Hyd/Kochi/Kol/Pune )

We are seeking an experienced AWS Data Engineer with strong expertise in Python, PySpark, SQL, AWS Glue, Lambda, CloudFormation, and Data Engineering. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and cloud-native ETL solutions on AWS.

The role requires hands-on experience with AWS data services, data integration frameworks, real-time processing, and modern data engineering practices to build highly scalable and reliable data platforms.

Key Responsibilities

Data Engineering & ETL Development

• Design, develop, and optimize data ingestion, transformation, and integration pipelines using AWS services.
• Build scalable ETL/ELT frameworks using Python, PySpark, and AWS Glue.
• Develop data pipelines to process structured and semi-structured data from multiple sources including databases, files, APIs, and streaming platforms.
• Implement data quality checks and validation processes to ensure reliable data delivery.

AWS Cloud Development

• Develop and maintain cloud-based data solutions using:

• AWS Glue

• AWS Lambda

• AWS Step Functions

• AWS CloudFormation

• Amazon S3

• Amazon EC2

• AWS IAM

• Build highly available, scalable, and secure cloud-native data platforms.

• Automate infrastructure provisioning and deployment using Infrastructure as Code (IaC).

Data Modeling & Database Development

• Design and maintain data models supporting analytics and reporting solutions.
• Work with PostgreSQL, Redshift, and relational databases.
• Develop and optimize SQL queries, stored procedures, and transformations.
• Implement best practices for database performance tuning and scalability.

Real-Time & Pipeline Engineering

• Develop workflows for batch and real-time data processing.

• Build orchestration pipelines using:

• AWS Step Functions

• AWS Glue Workflows

• Apache Airflow

• Support event-driven architectures and streaming data use cases.

Agile Delivery & Collaboration

• Participate in Agile development processes and sprint-based delivery.
• Collaborate with architects, developers, product teams, and business stakeholders.
• Conduct peer reviews and ensure adherence to engineering best practices.
• Support client demos, testing activities, and production deployments.

Required Skills

• Minimum 6 years of experience in Data Engineering and Data Integration.

• 3+ years of hands-on experience in:

• AWS Glue

• AWS Lambda

• AWS CloudFormation

• PySpark

• Strong Python programming experience.

• Strong SQL development and optimization skills.

• Experience with:

• AWS Step Functions

• AWS IAM

• Amazon S3

• Amazon EC2

• Experience developing ETL/ELT pipelines and data integration frameworks.

• Strong understanding of Data Engineering concepts and best practices.

• Experience implementing data models and analytical platforms.

• Experience building scalable and high-performance workflows.

Preferred Skills

• Apache Airflow.
• Amazon Redshift.
• Kafka Streaming.
• Matillion.
• SSIS.
• Visual Studio.
• Real-time data processing frameworks.
• Enterprise Data Warehouse solutions.

Desired Candidate Profile

• Strong analytical and problem-solving skills.
• Excellent verbal and written communication abilities.
• Experience working on multiple concurrent projects.
• Ability to collaborate with technical and non-technical stakeholders.
• Strong ownership mindset and proactive approach.
• Ability to work effectively under tight timelines.