Python Expert with AWS Data Engineering

We are seeking a Python Data Engineering expert with 8+ years of overall experience in Data Engineering and at least 8+ years of hands-on expertise in Python development. The ideal candidate should have strong proficiency in Python, RDBMS, and a solid understanding of AWS services.

Key Responsibilities:

• Python Expertise: Leverage 8+ years of hands-on experience with Python for developing scalable, high-performance data engineering solutions.
• RDBMS to NoSQL Conversion: Lead the effort in migrating large data sets from RDBMS to NoSQL databases, ensuring seamless and efficient data storage and retrieval.
• Data Lake Delta Tables: Design and build data lakes and manage configurations on Delta tables to optimize data processing and storage workflows.
• Cost Optimization: Implement computing strategies that minimize costs while maximizing data processing efficiency within cloud environments like AWS.
• Holistic Framework Development: Analyze current environments and use cases to develop holistic data engineering frameworks that address both short-term and long-term business needs.
• Airflow Framework: Utilize Apache Airflow to manage, schedule, and monitor workflows, ensuring data pipelines are running efficiently.
• AWS Service Integration: Work extensively with AWS services such as EMR, S3, Lambda, Step Functions, and Aurora (RDS) to build, manage, and optimize data solutions on the cloud.

Required Skills:

• 8+ Years of Data Engineering Experience: A strong background in Data Engineering, with 8+ years of Python development experience.
• RDBMS Expertise: Proficiency in RDBMS and SQL for managing large-scale relational databases.
• NoSQL: Familiarity with converting data from RDBMS to NoSQL databases to enable flexible, scalable data storage solutions.
• Data Lakes Delta Tables: Experience in building data lakes and working with Delta table configurations for optimized data storage and processing.
• AWS Services: Hands-on experience with AWS technologies such as EMR, S3, Lambda, Step Functions, and Aurora (RDS) for cloud-based data solutions.
• Apache Airflow: Strong understanding of Apache Airflow to automate, schedule, and manage data workflows.

Soft Skills:

• Team Collaboration: Demonstrated ability to work as part of a team, with a positive attitude and commitment to collaborative problem-solving.

Disclaimer: This job posting and Location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.