Data Engineer (Public Sector)

Our client is a statutory body in Hong Kong. They are looking for experienced talent to implement a scalable cloud-based data platform to enable sustainable, reusable, secure, efficient ingestion, transformation, and storage of enterprise data across multiple sources. Th individual should possess knowledge and experience of near real-time analytics capabilities to support business intelligence, predictive modelling, and faster decision-making for strategic initiatives.

Responsibilities

• Optimize, operate at scale and enhance cloud-based data platform on Microsoft Azure (incl DataBricks)
• Involve in cloud data platform enhancement project lifecycle with external vendor including design, development, testing, deployment, and documentation.
• Work with Data Analyst to perform data preparation, cleansing, build optimized data models and visualization in Power BI
• Perform proof of concept for cloud data products
• Validate data workflows and integration points through rigorous testing (e.g., pipeline validation, schema checks, and performance benchmarking) to ensure solutions meet business and technical requirements.
• Create operational playbooks and automation scripts to streamline deployment, monitoring, and troubleshooting, enabling efficient handover and long-term maintainability of data solutions.
• Document data pipeline architecture, deployment processes, and operational runbooks to support effective troubleshooting and post-go-live maintenance by internal teams

Requirements:

• Bachelor’s degree in computer science, Information Technology, Data Science or related field
• 3 years of professional experience related to data platform
• Experience in Python, SQL, Spark, Azure Data Factory, Event Hub / Kafka / Event Messaging, Azure Data Lake Storage Gen2, Databricks, Unity Catalog, Power BI
• Understanding of data lifecycle and governance principles (e.g., data quality, lineage, security, and compliance
• Experience in software development lifecycle (SDLC)