Big Data Development Enginee

ob Responsibilities

1. Participate in the design, development and implementation of highly available and scalable big data platforms and data processing solutions.
2. Work with big data architecture frameworks such as Lambda and Kappa, and support the implementation and optimisation of technical solutions.
3. Optimise large-scale data processing workloads, including Spark performance tuning, Flink optimisation, memory management, data processing efficiency and system stability improvements.
4. Develop, maintain and enhance core modules of the big data platform, and troubleshoot technical issues across data processing pipelines.
5. Collaborate with data analysts, algorithm engineers and other technical teams to provide reliable data interfaces, data services and supporting tools.
6. Participate in technical solution design, evaluate implementation approaches, and ensure solutions are scalable, stable and maintainable.
7. Prepare and maintain technical design documents, development documentation and implementation records to support successful project delivery.

Requirements

1. Bachelor's degree or above in Computer Science, Software Engineering, Information Technology or related disciplines.
2. 3+ years of hands-on experience in big data development, with experience in building, implementing or optimising large-scale data platforms.
3. Experience working with large-scale data processing scenarios, including data platforms or core module development involving 50TB+ data volumes is preferred.
4. Strong knowledge of the Hadoop ecosystem, including Spark and Flink architecture, development and performance optimisation.
5. Hands-on experience with streaming data technologies such as Kafka or Pulsar.
6. Strong programming and data processing skills with SQL, Hive and at least one programming language such as Python, Java or Scala.
7. Experience with analytical database technologies such as Doris or StarRocks, including configuration, SQL optimisation and performance tuning.
8. Experience in distributed system optimisation, such as Spark Shuffle optimisation, Flink backpressure handling, state management or resource optimisation is preferred.
9. Experience with data lake technologies such as Apache Iceberg, Delta Lake or Apache Paimon is a plus.
10. Experience with cloud-native big data environments, including EMR, CDH or Kubernetes-based resource scheduling and cluster operations is a plus.
11. Experience in logistics, e-commerce, telecommunications or other data-intensive industries is preferred.
12. Strong analytical thinking, problem-solving skills and ability to collaborate effectively with cross-functional teams.
13. Strong communication skills with the ability to work effectively in a multilingual and multicultural environment.