Data Engineer-Data Platforms-AWS

Your Role and Responsibilities:

As a seasoned Data Engineer specializing in Data Platforms on AWS, you will advise on, develop, and maintain data engineering solutions on the AWS Cloud ecosystem. You will design, build, and operate batch and real-time data pipelines using various AWS services.

Your Primary Responsibilities Will Include:

• Design and Develop Data Pipelines: Design, build, and operate batch and real-time data pipelines using AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis, ensuring seamless integration and operation of data engineering solutions.
• Create Data Layers: Create data layers on AWS RedShift, Aurora, and DynamoDB, and migrate data using AWS DMS, demonstrating expertise in various AWS Data Platform components.
• Manage Data Services: Schedule and manage data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
• Develop Batch and Real-time Pipelines: Develop batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
• Ensure Seamless Integration: Ensure seamless integration and operation of data engineering solutions, leveraging expertise in open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform.

Required Education:

Bachelor's Degree

Preferred Education:

Master's Degree

Required Technical and Professional Expertise:

• Designing Data Engineering Solutions: Experience with designing, building, and operating batch and real-time data pipelines using AWS services such as AWS EMR, AWS Glue, Glue Catalog, and Kinesis.
• Developing Data Layers: Experience in creating data layers on AWS RedShift, Aurora, and DynamoDB, and migrating data using AWS DMS, with proficiency in various AWS Data Platform components.
• Managing Data Services: Experience in scheduling and managing data services on the AWS Platform, ensuring efficient operation of data engineering solutions.
• Developing Batch and Real-time Pipelines: Experience with developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka.
• Integrating Open Source Technologies: Experience in leveraging open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to ensure seamless integration and operation of data engineering solutions.

Preferred Technical and Professional Experience:

• Proficiency in Open Source: Experience with leveraging open source technologies like Apache Airflow and dbt, Spark / Python or Spark / Scala on AWS Platform to ensure seamless integration and operation of data engineering solutions.
• Expertise in AWS Services: Experience with various AWS Data Platform components, including S3, RedShift, RedShift Spectrum, AWS Glue with Spark, AWS Glue with Python, Lambda functions with Python, AWS Glue Catalog, and AWS Glue Databrew.
• Knowledge of Data Migration: Experience with migrating data using AWS DMS and creating data layers on AWS RedShift, Aurora, and DynamoDB.

****Years of Experience:****16 - 20