Data Engineer (ETL)

Skills Required: Python, ETL, AWS, Airflow/DBT, Pyspark , Spark.

Roles Responsibilities:

Mandatory skills (8+ Years of experience in ETL development with 4+ Years on AWS Pyspark scripting)

• Experience deploying and running AWS-based data solutions using services or products such as S3, Lambda, SNS, Cloud Step Functions.
• Person should be strong inPyspark
• Hands on and working knowledge in Python packages like NumPy, Pandas,Etc
• Experience deploying and running AWS-based data solutions using services or products such as S3, Lambda, SNS, Cloud Step Functions. Sound knowledge in AWS services is must.
• Person should work as Individual contributor
• Good to have familiar with metadata management, data lineage, and principles of data governance.

Good to have:

• Experience to process large set of data transformations both semi and structured data
• Experienceto builddata lake & configuration on delta tables.
• Good experiencewith computing & cost optimization.
• Understanding the environment and use case and ready to build holistic Data Integration frame works.
• Good experiencein MWAA (airflow orchestration)

Soft skill:

• Having good communicationto interact with IT-Stake holders and Business.
• Understand the pain point to deliver