Data Engineer – Python & AWS

About YellowIpe

Our mission is to inspire the connection between technology and people. We nurture the best in our professionals through our expertise in finding and attracting top talent for the best projects. People Focus, Collaboration, and Commitment are the pillars that guide us on this journey.

Join the yellow team as our new Data Engineer – Python & AWS!

Responsibilities

• Design, develop, and maintain end-to-end data solutions within an AWS-based Big Data platform, ensuring scalability, reliability, and performance.
• Build, optimize, and maintain data pipelines and processing applications using Python and SQL.
• Collaborate with cross-functional teams to define technical architectures, data flows, and integration strategies.
• Develop and implement efficient solutions for data extraction, transformation, processing, and analysis.
• Work with relational and NoSQL databases, ensuring efficient data access, querying, and management.
• Support Data Science initiatives through exploratory data analysis, data preparation, and the integration of Machine Learning models into production environments.
• Contribute to the deployment, monitoring, and lifecycle management of Machine Learning models.
• Develop Proofs of Concept (PoCs) to evaluate new technologies, tools, and technical approaches.
• Apply software engineering and DevOps best practices, including automated testing, secure deployments, and continuous integration and delivery.
• Monitor, troubleshoot, and continuously improve existing data applications and production environments.
• Maintain technical documentation, contribute to technical debt management, and support infrastructure improvement initiatives.

Requirements

• Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or a related field.
• Up to 5 years of professional experience in Data Engineering, Software Engineering, or a similar technical role.
• Strong programming skills in Python , including hands-on experience with Pandas and NumPy .
• Solid knowledge of SQL , including the ability to develop and optimize complex queries.
• Experience designing, developing, or maintaining data pipelines and data processing solutions.
• Experience working with relational databases, such as PostgreSQL or MySQL, and NoSQL databases, such as MongoDB or DynamoDB.
• Good understanding of software development principles, debugging, testing, and code quality.
• Experience with Machine Learning model deployment and lifecycle management, preferably using tools such as MLflow or Kubeflow.
• Interest in Data Science, data analytics, and Machine Learning technologies.
• Familiarity with Agile methodologies and collaboration tools such as Jira and Confluence.
• Strong analytical and problem-solving skills, with the ability to work independently and collaborate with international teams.
• Excellent communication skills, including the ability to explain technical concepts to non-technical stakeholders.
• Fluent English is mandatory.

Nice to Have

• Hands-on experience with AWS services , including S3, Lambda, Glue, Athena, EC2, Step Functions, Batch, or Fargate.
• Experience with Infrastructure as Code (IaC) tools such as AWS CDK or CloudFormation.
• Knowledge of Apache Spark or PySpark for large-scale data processing.
• Experience with Git, Jupyter Notebooks, and collaborative development workflows.
• Experience developing, deploying, and maintaining production-grade applications.
• Previous experience in the Energy or Utilities sector.
• Knowledge of GDPR and data protection best practices.
• Portuguese language skills.

What You Can Expect from Us

• Integration into a collaborative and people-focused team.
• Opportunities to work on challenging projects involving Data Engineering, Cloud Technologies, and Data Science.
• A professional environment that encourages continuous learning, innovation, and knowledge sharing.
• Opportunities for professional growth and technical development.

Location: Lisbon or Porto

If you're passionate about building scalable data solutions and want to make an impact through technology, we'd love to hear from you!

Join the yellow team and take the next step in your career with YellowIpe!