Senior Data Engineer

We are looking for a skilled Senior Data Engineer with expertise in data modelling and data pipeline development. As a Senior Data Engineer, you will work with the big data team, adopting best practices to develop and optimise data pipelines, reporting, and dashboards to meet business requirements. Proficiency in serverless cloud services such as AWS Glue, Aurora, Lambda, RDS, and Redshift is essential. You will also handle Management Information Systems (MIS) to generate reports and develop dashboards using business intelligence tools including SAP and Power BI.

⇒ Main Responsibilities

Data Pipeline Development

• Discuss with Solution Architect on designing data pipelines.
• Develop data pipelines with proposed solution.
• Manage and maintain data pipelines.

Data Modeling

• Create and maintain data models, including entity-relationship diagrams, to represent data structures and relationships.
• Design database schemas that optimize data storage, retrieval and query performance.

Reporting and Visualization

• Communicate effectively with stakeholders.
• Engage with business users to understand reporting needs.
• Deliver timely and accurate reports and dashboards.

BAU Tasks

• Understanding and translating business requirements into technical requirements.
• Develop/Enhance SAP reports based on business requirements.
• Develop/Enhance Dashboards based on business requirements.
• Fix bug whenever detected.
• Deploy changes into GitHub.

Documentation

• Create documentation for data pipelines, data models and challenges faced in BAU tasks.

⇒ Qualifications & Experience

• Bachelor's Degree in Computer Science, Engineering, or a related field; a Master's degree is preferred.
• 5+ years of hands-on experience in data integration and analytics in a corporate or consulting setting.
• Design, develop, and maintain efficient data pipelines to support data processing, transformation, and integration needs.
• Collaborate with cross-functional teams to understand data requirements and implement solutions that address business needs.
• Work with various data sources and formats, ensuring data quality and consistency.
• Monitor and troubleshoot data pipelines to ensure consistent data flow.
• Handle Management Information Systems (MIS), including data collection, analysis, and reporting for decision-making purposes.
• Able to build reports using tools such as SAP, Power BI, or other reputable BI tools.
• Hands-on experience in AWS services (AWS Redshift, AWS Glue, AWS S3, AWS Lambda, AWS RDS, AWS Aurora, AWS API Gateway).
• Proficient in Python, PySpark, SQL, and Linux.
• Strong ETL/ELT skills, including architecture modelling, data modelling, dashboard building, storytelling, and documentation.
• Strong analytical and problem-solving skills with the ability to translate business requirements into technical solutions.
• Excellent communication and stakeholder engagement skills.
• Proactive, self-driven, and able to work independently.
• Strong attention to data quality and consistency.
• Proficiency in Git and GitHub for version control.