AWS Data Engineer

Key Skills: IAM, AWS, Python, AWS Lambda, Data Engineer, AWS S3, Spark

Roles and Responsibilities:

• Design, build, and maintain scalable data pipelines on AWS.
• Develop ETL/ELT workflows using AWS Glue, Lambda, EMR, and Step Functions.
• Develop data ingestion frameworks using Kinesis, Kafka, and SQS.
• Build and optimize data lakes using S3, Glue Data Catalog, and Lake Formation.
• Develop and maintain data warehousing solutions using Redshift and Snowflake.
• Implement data quality checks, monitoring, logging, and CI/CD deployments.
• Collaborate with analytics and business teams to deliver high-quality datasets.
• Design, develop, and deploy solutions using appropriate tools, design principles, and coding standards.
• Configure robotics processes and objects using core workflow principles, ensuring solutions are maintainable and easy to understand.
• Understand existing processes and manage changing requirements through structured change control processes.
• Troubleshoot day-to-day issues in data and automation processes and provide timely resolutions.
• Maintain proper documentation for solutions, test procedures, and scenarios throughout UAT and Production phases.
• Coordinate with process owners and business stakeholders to understand as-is processes and design appropriate automation workflows.

Skills Required:

• 6+ years of strong hands-on experience with AWS Glue, S3, Lambda, IAM, and Step Functions.
• Strong proficiency in Python, PySpark, SQL, and data transformation logic.
• Experience with Redshift, Athena, and Glue Data Catalog.
• Strong knowledge of Apache Spark and distributed data processing using EMR/Glue.
• Experience with data ingestion technologies such as Kinesis, Kafka, or SQS.
• Good understanding of data modeling, data lake architecture, and data engineering best practices.
• Experience implementing data quality, monitoring, logging, and CI/CD processes.
• Familiarity with Terraform or CloudFormation for Infrastructure as Code (IaC) is an added advantage.
• Strong problem-solving and analytical skills.
• Good communication and stakeholder management skills.

Education: Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.