Senior/Lead Data Engineer

Job Overview:

The Senior/Lead Data Engineer is responsible for designing, developing, and optimizing scalable data solutions on Alibaba Cloud’s big data platforms, including MaxCompute and Hologres. This role will take a hands-on approach in building robust data pipelines, ensuring data quality and integrity, and supporting the organization’s data-driven initiatives. The ideal candidate has deep experience with Alibaba Cloud’s big data ecosystem and is proficient in leading technical efforts in complex data warehousing and processing environments.

Responsibilities:

1. Data Pipeline Development and Optimization

• Design and implement efficient ETL/ELT pipelines on Alibaba Cloud to support ingestion, transformation, and loading of large-scale data
• Integrate data from multiple systems into a centralized data warehouse, ensuring consistency, accuracy, and accessibility for analytics and reporting
• Continuously optimize and monitor data pipelines to ensure high performance, scalability, and efficient resource usage within Alibaba Cloud environments

2. Data Warehouse Management

• Develop, manage, and optimize big data warehouses on Alibaba Cloud, using tools like Hologres, MaxCompute, and DataWorks
• Implement data modeling best practices to structure and organize data for optimal accessibility, flexibility, and query performance
• Establish data validation, monitoring, and quality assurance practices to maintain high data quality standards across the data warehouse

3. Collaboration and Team Management

• Work closely with data analysts, data scientists, and business stakeholders to understand data needs and provide tailored data solutions
• Translate business requirements into technical specifications for data pipelines, storage, and processing solutions
• Provide guidance on data engineering best practices and data warehouse solutions to team members and support their technical growth

4. Data Governance and Compliance

• Maintain clear and comprehensive documentation for data pipelines, data warehouse structures, and data governance practices
• Ensure compliance with data governance policies and standards for data security and privacy, working closely with security and governance teams

Requirements:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or related field
• 5+ years in data engineering, with strong hands-on experience in Alibaba Cloud, particularly Hologres, MaxCompute, and related big data tools
• Proficiency in big data technologies (e.g., Hologres, MaxCompute, DataWorks, Flink), Strong SQL, Python, or Spark skills for data manipulation and processing
• Ability to work independently and take initiative on complex data engineering tasks
• Capacity to lead technical efforts, mentor junior engineers, and set standards for data engineering best practices
• Team player with strong collaboration skills, able to work effectively with both internal and external teams