Jean Baptiste Gidrol

- Leading the design and operation of RTBF’s sovereign, on-premise AI inference platform, serving LLMs, vision, image-generation, embedding and speech models to 15 internal teams through a shared LiteLLM gateway - Designing the serving architecture for a 24-GPU H100/L40S environment on Kubernetes and Ray, with heterogeneous replicas, fallback routing, shared observability and GPU capacity planning - Benchmarking and optimizing model serving across vLLM/CUDA versions, tensor-parallel layouts, FP8 weights and KV cache, FlashInfer, compiled CUDA graphs and speculative decoding (EAGLE, DFlash) - Tuned Qwen3.6 FP8 to ~5,000 output tokens/s on a single H100 at 75 concurrent requests, while maintaining low time-to-first-token and zero request errors - Improved prefix-cache hit rates to up to 99% through cache-aware routing and standardized client behavior, reducing warm agentic request startup from tens of seconds after model reloads to under one second - Built and operated multimodal AI pipelines for transcription, diarization, OCR, language detection, summarization and metadata extraction using WhisperX, Pyannote, PaddleOCR and self-hosted LLMs - Built production GenAI applications including RAG, editorial search and AI-assisted content workflows over more than 1.5M articles and years of video transcripts - Scaled recommendation systems serving 5M+ users and more than 2M daily recommendations across news, VOD, podcasts and live TV, operating a distributed architecture of 100+ Kubernetes microservices - Optimized graph-based workloads in Neo4j and improved observability across recommendation and AI services using centralized logging, Prometheus, Grafana and Langfuse Tech stack: Python, Kubernetes, Ray / Ray Serve, vLLM, LiteLLM, H100 / L40S, FlashInfer, PostgreSQL, Kafka, Neo4j, Redis, Prometheus, Grafana, Langfuse, WhisperX, Pyannote, PaddleOCR, AWS, Azure, Docker, Terraform

-Understanding and Running AI in the Real World — designing and delivering training sessions to upskill consultants, developers, and management, providing a clear mental model of AI systems and their operational realities -Leading the development of local/sovereign AI inference platforms to help companies become self-sufficient AI providers

- Developed cost efficient ETL pipelines : from the accounting and fleet management system to a postgres hosted on AWS RDS - Developed and implemented interactive dashboards : using Dash, leveraging years of delivery data to visualize client locations and extract valuable business insights. Hosted on AWS EC2, Dockerized and with a full CI/CD piepline. - Data-driven business development initiatives : automating data scraping processes to streamline lead generation efforts. Technical stack : AWS (EC2, S3, RDS), Docker, Dash, Streamlit, PostgreSQL, Selenium, Beautiful Soup, Github Actions, Pytest, Pandas, SQLAlchemy