Data Engineer

Requirements

• 5+ years of commercial experience in Data Engineering , ideally in senior-level positions.
• Strong hands-on experience with Microsoft Azure and Azure-based data solutions.
• Strong proficiency in Python .
• Solid experience with Apache Spark / PySpark or other distributed data processing frameworks.
• Strong understanding of Data Engineering best practices , principles, and patterns.
• In-depth knowledge of data concepts, data structures, and data modelling techniques .
• Experience designing and provisioning data to support different data consumption and business needs .
• Experience applying Test-Driven Development (TDD) practices.
• Strong understanding of the full data lifecycle , from ingestion and transformation to production and consumption.
• Good level of English , both written and spoken.
• Availability to work 1 day per week from the office in Matosinhos .

Responsibilities

• Design, develop, and maintain scalable and reliable data solutions using Azure and modern Data Engineering technologies.
• Deliver business value into Production , ensuring data solutions meet business and technical requirements.
• Develop and optimize data processing pipelines using Python and PySpark / Apache Spark .
• Apply Data Engineering best practices to ensure code quality, scalability, maintainability, and performance .
• Design appropriate data structures and data models according to business and technical requirements.
• Build and provision data solutions that support different consumption needs and use cases .
• Ensure data solutions are production-ready, reliable, and scalable .
• Apply TDD and automated testing practices throughout the development lifecycle.
• Collaborate with technical and business stakeholders to translate requirements into effective data solutions.