Big Data Operations Engineer (Cloud)

Job Responsibilities

• Manage and archive business data according to data lifecycle and retention policies, ensuring data is secure, accurate, and traceable.

• Deploy, monitor, and troubleshoot data synchronization and ETL processes to ensure data is transferred accurately and on time.

• Operate and maintain the GCP Big Data Platform, including:

  • BigQuery
  • Cloud Data Fusion
  • Cloud SQL
  • Cloud Storage (GCS)
  • Pub/Sub
  • Other related GCP services

• Manage BigQuery datasets, tables, permissions, and resources, while improving query performance and controlling storage costs.

• Manage Cloud SQL databases (MySQL/PostgreSQL), including deployment, backup/recovery, access management, performance tuning, and troubleshooting.

• Monitor and maintain ETL, data processing, and scheduled jobs, and resolve data or job failures.

• Develop and improve monitoring and alerting systems to improve platform stability and reduce downtime.

• Work closely with development, data, and business teams to support data-related requirements and operational needs.

• Create and maintain technical documentation, automation scripts, and standard operating procedures to improve operational efficiency.

Job Requirements

• Bachelor's degree or above in Computer Science, Software Engineering, Information Technology, or a related field.

• 3+ years of experience in big data operations, database administration, application operations, or cloud operations.

• Good knowledge of Linux and Shell scripting. Python or Go automation experience is a plus.

• Experience with at least one relational database, such as MySQL, PostgreSQL including:

  • Installation and configuration
  • Backup and recovery
  • Performance tuning
  • Troubleshooting

• Strong SQL skills, including SQL development and query performance optimization.

• Experience with data synchronization and ETL processes and at least one ETL/data integration tool.

• Experience in cloud operations with GCP, AWS, Azure, or Alibaba Cloud, with the ability to work with GCP data services.

• Experience with one or more of the following GCP services is preferred:

  • BigQuery
  • Cloud Data Fusion
  • Datastream
  • Cloud SQL
  • Cloud Storage (GCS)
  • Pub/Sub
  • Cloud Composer / Airflow
  • Dataflow
  • Cloud Monitoring
  • Cloud Logging

• Understanding of ITIL, DevOps, or SRE concepts is preferred.

• Strong troubleshooting, incident response, and problem-solving skills.

• Experience in big data or database performance optimization is preferred.

• Good communication and teamwork skills.

• English is required as a working language. Chinese communication/reading skills are an advantage.

• GCP, AWS, Azure, or other relevant cloud certifications are a plus.