Data Engineer

Key Responsibilities

• Design, develop, and maintain data pipelines and ETL/ELT processes using AWS services.
• Develop and optimize SQL queries for data extraction, transformation, validation, and analysis.
• Use Python for data processing, automation, and pipeline development.
• Manage and process data using AWS S3 and AWS Redshift.
• Develop and maintain ETL workflows using AWS Glue.
• Configure and manage AWS IAM roles, users, and policies to support secure data access.
• Monitor data pipelines, jobs, and AWS resources using AWS CloudWatch.
• Perform data synchronization and ensure consistency across systems.
• Troubleshoot data pipeline failures, performance issues, and data quality concerns.
• Implement data validation and reconciliation processes to ensure accuracy and completeness.
• Collaborate with Data Analysts, Software Engineers, Business Analysts, and other stakeholders to understand data requirements.
• Maintain technical documentation for data pipelines, processes, and AWS environments.
• Follow data security, governance, and access control best practices.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
• 3+ years of experience in Data Engineering or a related role.
• Strong hands-on experience with SQL and Python.
• Experience working with AWS S3 and AWS Redshift.
• Hands-on experience with AWS Glue for ETL/data processing.
• Knowledge of AWS IAM, including Roles, Users, and Policies.
• Experience with AWS CloudWatch for monitoring and troubleshooting.
• Experience with data synchronization, ETL/ELT, and data pipeline development.
• Strong understanding of data processing, transformation, and validation.
• Strong analytical and problem-solving skills.
• Good communication and collaboration skills.

Preferred Skills

• Experience with additional AWS data services such as AWS Lambda, Athena, RDS, or Step Functions.
• Experience with Apache Spark/PySpark.
• Knowledge of data warehousing and dimensional modeling.
• Experience with CI/CD and Git.
• Knowledge of data quality and data governance concepts.
• Experience working in an Agile/Scrum environment.

Key Skills

Data Engineering | SQL | Python | AWS S3 | AWS Redshift | AWS Glue | AWS IAM | AWS CloudWatch | Data Synchronization | ETL/ELT | Data Pipelines | Data Warehousing | Data Processing | Data Validation | AWS