Data Engineer

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Python and PySpark.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
• Develop scalable data warehousing solutions on AWS cloud platform.
• Ensure data quality by implementing robust testing frameworks and monitoring dashboards.
• Participate in code reviews to ensure adherence to coding standards.

Desired Candidate Profile

• 4-9 years of experience in Data Engineering with expertise in Data Warehousing.
• Strong proficiency in Python programming language with hands-on experience working with PySpark.
• Experience designing and developing ETL processes for large datasets using AWS services such as S3, Lambda, Step Functions etc. .
• Knowledge of SQL concepts including query optimization techniques for performance tuning.