Data Engineer

Role Overview

The Data Engineer is responsible for building, supporting, and optimising scalable, repeatable, and secure data pipelines and analytical platforms. Operating as a core member of an agile engineering team, the role focuses on enabling business insights through the integration, transformation, and management of large-scale batch and real-time data solutions. The Data Engineer works across cloud and big data technologies to architect, develop, and support modern data platforms that enable analytics, reporting, automation, and data-driven decision-making.

Key Responsibilities

• Design, build, and maintain scalable data analytics frameworks and data platforms.
• Translate complex functional and technical requirements into scalable architecture and high-performing data solutions.
• Develop and support batch and real-time data processing solutions using modern big data technologies.
• Create and maintain secure and reliable data pipelines between on-premise systems and AWS cloud environments.
• Develop and maintain ETL processes and data transformation workflows using Talend or similar ETL tools.
• Manipulate, process, and analyse data using Python and related technologies.
• Develop and support Big Data and Business Intelligence solutions including automated testing and deployment practices.
• Process large-scale datasets using Hadoop paradigms and AWS EMR services.
• Support production data feeds and resolve operational issues through break-fix support activities.
• Contribute to database development, optimisation, and operational management.
• Participate in software development, DevOps, and data operations activities.
• Ensure alignment with policies, standards, procedures, business continuity, and disaster recovery practices.
• Research and evaluate emerging technologies, tools, and frameworks relevant to data engineering and analytics.
• Collaborate with business and technical stakeholders to understand data requirements and deliver effective solutions.

Essential Skills & Experience

• 5+ years’ experience in Data Engineering or Software Engineering roles.
• 2+ years’ experience working with Big Data technologies and platforms.
• 2+ years’ experience with ETL processes and tooling.
• 2+ years’ experience working with AWS cloud technologies.
• Strong experience with Python for data manipulation and processing.
• Hands-on experience with AWS services including EMR, EC2, and S3.
• Experience with PySpark, Spark, Hadoop, or distributed data processing frameworks.
• Strong understanding of data modelling, data structures, and scalable data architecture design.
• Experience designing highly scalable distributed systems using open-source technologies.
• Strong understanding of object-oriented design, coding standards, and testing practices.
• Experience working with large-scale data infrastructure and analytical platforms.
• Strong analytical, troubleshooting, and problem-solving skills.

Desirable Skills & Experience

• Experience with real-time streaming and event-driven data processing.
• Exposure to cloud-native data engineering architectures.
• Experience implementing automated deployment and CI/CD practices for data platforms.
• Exposure to business intelligence and analytics tooling.
• Experience working within agile software delivery environments.

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering, Information Technology, or equivalent practical experience.
• AWS certification would be advantageous.

Behavioural Competencies

• Strong analytical and problem-solving mindset.
• Excellent communication and stakeholder engagement skills.
• Strong attention to detail and commitment to quality.
• Ability to work effectively both independently and within collaborative agile teams.
• Adaptability and willingness to learn evolving technologies and tools.
• Proactive mindset with focus on innovation and continuous improvement.

What We Offer

• Competitive salary and benefits package.
• Opportunity to work on modern cloud and big data platforms.
• Exposure to large-scale data engineering and analytics projects.
• Collaborative and innovative engineering environment.
• Professional growth and learning opportunities.
• Opportunity to contribute to impactful data-driven solutions.