Big Data Operations & Maintenance Engineer

About the role

Manage and develop automation tools and modules for system operations, deployment, monitoring and maintenance. Participate in requirement analysis, solution design, development, deployment and ongoing maintenance of the company's data operations platform and related components.

Key responsibilities

• Manage and develop automation tools and modules for system operations, deployment, monitoring and maintenance.
• Participate in requirement analysis, solution design, development, deployment and ongoing maintenance of the company's data operations platform and related components.
• Monitor data collection and processing workflows, identify data quality issues, and support the stability, accuracy and reliability of data pipelines.
• Perform daily operations and maintenance of production environments, including server deployment, configuration, monitoring, performance optimisation and troubleshooting.
• Monitor and maintain big data platforms and related components, respond to production incidents, and troubleshoot system or data processing issues in a timely manner.
• Support continuous improvement of system stability, operational efficiency and automated deployment processes.
• Prepare and maintain technical documentation, operational procedures and troubleshooting records.

About you

• Bachelor's degree or above in Computer Science, Information Technology or a related discipline, with at least 3 years of relevant experience.
• Strong knowledge of Linux systems, including system administration, configuration, performance optimisation and troubleshooting.
• Good understanding of commonly used open-source big data technologies such as Hadoop, Kafka, StarRocks, Flink and ClickHouse, including their architecture, configuration, deployment, optimisation and troubleshooting.
• Hands-on experience in the deployment, operation and troubleshooting of big data platforms.
• Proficient in at least one programming language such as Python, Rust, Golang or Java, with experience in development or scripting on Linux/Unix environments.
• Strong interest in technology and data, with a willingness to continuously learn, explore technical solutions and share knowledge within the team.
• Strong sense of responsibility with good problem-solving, communication and cross-team collaboration skills.
• Able to respond to production incidents and provide operational support outside normal working hours when required.