Sr. Associate Director, Software Engineering (Model Hosting Pod Tech Lead)

Some Careers Have More Impact Than Others. If you’re looking for a career where you can make a real impression, join HSBC and discover how valued you’ll be.

We are currently seeking an experienced professional to join our team in the role of Sr. Associate Director, Software Engineering (Model Hosting Pod Tech Lead) .

Business: CTO Platforms (AI Platform) Job ID ： 52424 Location: Shenzhen / Guangzhou / Xi'an Principal Responsibilities Pillar 1: Model Hosting & Inference Optimization

• Design, build, and maintain scalable, reliable model hosting systems to support large-scale deployment of AI models (LLMs, embedding models, STT/TTS, etc.) across diverse hardware platforms.
• Develop and implement inference optimization strategies to achieve low latency, high throughput, and cost-efficiency, including model quantization, KV cache optimization, and dynamic batch processing / Continuous Batching.
• Evaluate, integrate, and customize state-of-the-art inference frameworks (vLLM, TensorRT-LLM, SGLang, etc.) to optimize model performance on target hardware.
• Monitor and maintain inference system health, track key performance metrics (latency, throughput, TTFT, memory usage, availability), and troubleshoot performance bottlenecks or deployment issues.
• Collaborate with hardware teams to leverage hardware-specific and optimize hosting infrastructure for maximum resource utilization.
• Ensure model hosting systems comply with reliability, scalability, and security standards, supporting high-availability production workloads.

Pillar 2: Model Fine-Tuning Pipeline Development

• Design and build end-to-end, scalable model fine-tuning pipelines to support domain-specific adaptation of pre-trained using custom, domain-specified datasets.
• Collaborate with data scientists and domain experts to understand domain requirements, define fine-tuning objectives, and validate fine-tuned model performance against domain-specific metrics.
• Integrate fine-tuned models into the existing hosting and inference infrastructure, ensuring smooth handoff from fine-tuning to deployment.

Pillar 3: Team Management & Delivery Ownership

• Lead and develop a high-performing engineering team: hiring, coaching, performance management, and fostering an inclusive, collaborative culture.
• Own delivery outcomes end-to-end, including roadmap planning, prioritisation, stakeholder management, and execution against timelines and quality standards.

Knowledge & Experience/Qualifications

• Bachelor’s/Master’s/PhD degree in Machine Learning, Natural Language Processing, Computer Science, Data Science, Statistics, or related fields.
• 3+ years’ experience leading teams, including people management and delivery ownership.
• 5+ years of experience in AI, with proven expertise in both model inference optimization/hosting and model fine-tuning pipeline development; experience with LLM-related work is a strong plus.
• Proficiency in programming languages such as Python, CUDA; familiarity with GPU/CPU architecture and high-performance computing (HPC) principles.
• Deep understanding of AI model inference: KV cache management, batch processing, quantization (INT4/FP8/GPTQ/AWQ), operator optimization, and inference framework integration (vLLM, TensorRT-LLM, SGLang, etc.).
• Hands-on experience building and maintaining model hosting systems, including knowledge of containerization (Docker, Kubernetes) and cloud computing platforms (AWS, GCP, Azure) for scalable deployment.
• Expertise in designing end-to-end model fine-tuning pipelines: data preprocessing, distributed training, hyperparameter tuning, and integration with pre-trained models Familiarity with fine-tuning frameworks and tools (Hugging Face Transformers, Accelerate, LoRA, QLoRA) and experience optimizing fine-tuning workflows for efficiency.
• Experience with performance benchmarking, monitoring, and troubleshooting for both inference and fine-tuning systems.
• Demonstrates an AI-native mindset, applying AI-driven approaches to improve productivity, quality, and decision-making.
• Experienced in leveraging coding assistants (e.g., AI pair-programming tools) to accelerate software development, enhance code quality, and support engineering best practices.

/WX /51 /ZP /LP You’ll achieve more when you join HSBC. HSBC is an equal opportunity employer committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and, opportunities to grow within an inclusive and diverse environment. We encourage applications from all suitably qualified persons irrespective of, but not limited to, their gender or genetic information, sexual orientation, ethnicity, religion, social status, medical care leave requirements, political affiliation, people with disabilities, color, national origin, veteran status, etc., We consider all applications based on merit and suitability to the role.”

Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website.

***Issued By HSBC Software Development (GuangDong) Limited***