Devops Engineer - Data Automation

Responsibilities:

• Design & Manage Platforms: Build secure, scalable, and high-performance Azure Databricks environments tailored for enterprise-wide insurance data transformation.
• Cross-Team Collaboration: Work closely with architects, engineers, and security teams to define infrastructure standards and ensure seamless integration with legacy systems, cloud sources, and third-party platforms.
• Infrastructure as Code: Use tools like Terraform to automate provisioning and configuration of Azure and Databricks resources, aligning with DevOps best practices.
• Automation & Monitoring: Implement GitHub Actions to automate environment deployment, monitoring, and incident response for greater consistency and efficiency.
• Performance Optimization: Monitor platform health and resource usage, anticipate scaling needs, and fine-tune performance for data pipelines and analytics workloads.
• Security & Compliance: Enforce enterprise and regulatory standards such as RBAC, managed identities, encryption, PDPO, GDPR, and insurance-specific policies.
• Data Governance Integration: Oversee integration of Informatica tools for metadata management, data lineage tracking, and compliance validation.
• Documentation: Maintain detailed records of infrastructure architecture, network topology, security settings, and operational procedures to support audits and team handovers.
• Troubleshooting & Reliability: Diagnose infrastructure issues, perform root cause analysis, and drive continuous improvements for platform stability.
• Innovation & Enhancement: Stay updated on Azure, Databricks, and DevOps advancements, recommending upgrades to improve platform capabilities and support business goals in Hong Kong’s life and general insurance sectors.

Requirements:

• Education: Bachelor’s degree in Computer Science, IT, or a related discipline.

• Experience: 5+ years in Azure cloud platform design and administration, with a focus on Azure Databricks for insurance data workloads.

• Technical Expertise:

  • Strong skills in provisioning and managing Databricks clusters and workspaces for various data types.
  • Experience integrating Azure Data Factory and Data Lake Storage Gen2 with Databricks.
  • Proficient in Terraform for infrastructure automation.
  • Familiarity with Informatica tools for data governance and cataloguing.
  • Deep understanding of security protocols (RBAC, NSG, managed identities, Key Vault), monitoring, and cost control in regulated environments.
  • Hands-on experience with GitHub Actions for CI/CD automation.
  • Proven ability in incident response and performance tuning for critical data platforms.

• Soft Skills: Excellent documentation, collaboration, and communication abilities to support both technical and business stakeholders.

**This position is offered through a strategic partnership between Don Nelson Technology Limited and NP Solution Limited.