Artificial Intelligence Engineer

Role & responsibilities

• Design, develop, and enhance AI/LLM applications using Python, LangChain, RAG, and Agentic AI frameworks.
• Build and optimize RAG pipelines covering document ingestion, chunking, embeddings, vector search, retrieval, reranking, and LLM response generation.
• Develop and integrate LLM/agent observability to monitor application health, performance, reliability, and AI quality.
• Implement tracing and monitoring across LLM calls, agent workflows, tools, retrieval pipelines, and APIs.
• Track key AI metrics such as latency, token usage, response quality, hallucination/error rates, retrieval accuracy, failures, and cost.
• Build dashboards, alerts, and monitoring mechanisms to identify LLM failures, degraded retrieval, latency issues, and production anomalies.
• Work with tools such as Azure Application Insights, Azure Monitor, OpenTelemetry, LangSmith, Azure AI Foundry/evaluation capabilities, or equivalent observability platforms.
• Implement LLM evaluation and RAG evaluation using appropriate metrics and feedback mechanisms.
• Monitor and troubleshoot multi-agent workflows, tool calls, prompt execution, and model responses.
• Integrate observability into FastAPI/microservices and cloud-native AI applications.
• Deploy and support AI workloads on Azure, with exposure to Docker/Kubernetes being an advantage.
• Establish logging, error handling, audit trails, and production monitoring for enterprise AI systems.
• Collaborate with AI engineers, data engineers, architects, and business stakeholders to improve AI reliability, performance, and user experience.
• Continuously identify opportunities to improve model performance, retrieval quality, scalability, latency, and operational cost.

Preferred candidate profile

Must-Have Skills

• Python strong hands-on development
• RAG – strong practical experience
• LangChain – strong hands-on experience
• LLMs / Generative AI
• Agentic AI / LangGraph
• Azure OpenAI
• Azure AI Search / Vector Search
• Embeddings, semantic search and document retrieval
• FastAPI / REST APIs
• Prompt engineering and LLM evaluation
• Production deployment of AI applications