Inference & System Optimization Engineer （Experienced ）

Responsibilities / 岗位职责:

• End-to-end engineering from model to inference system, hardware platform, and edge-side deployment
• Continuous optimization of inference cost, latency, throughput, and stability
• Key technologies: quantization, pruning, distillation, KV Cache, Continuous Batching, Paged Attention, Speculative Decoding, CUDA/Ascend/PPU operator optimization, multi-GPU distributed inference, model compilation, and cloud-edge collaboration
• Design and build high-performance inference serving infrastructure, including request scheduling, dynamic batching, memory management, and fault tolerance
• Benchmark and profile model performance across diverse hardware (NVIDIA GPU, Huawei Ascend, edge devices), identify bottlenecks, and drive hardware-software co-optimization
• Collaborate with algorithm and research teams to ensure new models are deployment-ready from day one; provide early feedback on architecture decisions that affect inference efficiency
• Explore and implement next-generation inference paradigms: speculative decoding, mixture-of-experts routing, sparse attention, and model-hardware co-design

面向LLM/VLM的工程化落地与高效运行，负责打通从模型到推理系统、硬件平台和端侧部署的全链路，持续优化推理成本、时延、吞吐和稳定性。包括量化/剪枝/蒸馏/KV Cache、Continuous Batching、Paged Attention、Speculative Decoding、CUDA/昇腾/PPU算子优化、多卡并行分布式推理、模型编译端云协同等。

设计并构建高性能推理服务基础设施，涵盖请求调度、动态批处理、显存管理及容错机制。

跨多种硬件平台（NVIDIA GPU、华为昇腾、端侧设备）进行性能基准测试与Profiling，定位瓶颈并推动软硬件协同优化。

与算法及研究团队紧密协作，确保新模型从设计阶段即具备可部署性，对影响推理效率的架构决策提供前置反馈。

探索并落地新一代推理范式：投机解码、MoE路由、稀疏注意力、模型-硬件协同设计等。

Requirements / 岗位要求:

• Hands-on experience in large model inference efficiency optimization, with a proven track record of measurable improvements in latency, throughput, or cost
• Deep understanding of transformer architecture, attention mechanisms, and memory-bound vs. compute-bound workload characteristics
• Proficiency in CUDA programming and GPU kernel optimization; experience with Ascend CANN or other domestic AI chip stacks is a strong plus
• Solid experience with mainstream inference frameworks: vLLM, TensorRT-LLM, SGLang, DeepSpeed-Inference, or similar
• Familiarity with model compression techniques (quantization INT4/INT8/FP8, pruning, knowledge distillation) and their trade-offs in accuracy vs. efficiency
• Strong systems thinking: able to reason about end-to-end performance from model architecture through serving infrastructure to hardware constraints
• Experience deploying models on edge devices or heterogeneous computing environments is preferred
• Bachelor’s degree or above in Computer Science, Electronic Engineering, or related fields; 3+ years of relevant industry experience
• 有大模型推理效率优化相关经验，有可量化的时延、吞吐或成本优化成果。
• 深入理解Transformer架构、注意力机制，以及memory-bound与compute-bound负载的特征差异。
• 熟练掌握CUDA编程及GPU算子优化；有昇腾CANN或其他国产AI芯片栈经验者优先。
• 有主流推理框架的实际使用或二次开发经验：vLLM、TensorRT-LLM、SGLang、DeepSpeed-Inference等。
• 熟悉模型压缩技术（INT4/INT8/FP8量化、剪枝、知识蒸馏）及其在精度与效率之间的权衡。
• 具备系统性思维，能从模型架构到服务基础设施到硬件约束进行端到端性能分析。
• 有端侧设备部署或异构计算环境经验者优先。