Agentic AI Developer II CloudAngles

We are looking for an Agentic AI Developer with strong hands-on experience in Python, LLM application development, AI agents, evaluation frameworks, and modern GenAI technologies. The candidate will design and develop AI agents capable of reasoning, tool usage, workflow orchestration, retrieval, planning, and autonomous task execution. The role requires a strong engineering mindset with emphasis on reliability, evaluation, observability, and production readiness.

Key Responsibilities

• Design and develop AI agents and multi-agent workflows.
• Build LLM-powered applications using Python.
• Develop agentic workflows involving planning, reasoning, tool calling, memory, and orchestration.
• Integrate LLMs through APIs and model-serving frameworks.
• Develop RAG-based applications using vector databases and retrieval pipelines.
• Design tools and APIs that can be invoked by AI agents.
• Build and maintain agent state, memory, context, and workflow management.
• Implement structured outputs, function calling, guardrails, and validation mechanisms.
• Develop comprehensive LLM/AI evaluation frameworks.
• Create evaluation datasets, test cases, benchmarks, and quality metrics.
• Evaluate model outputs for accuracy, relevance, groundedness, safety, consistency, and task completion.
• Implement automated regression testing for AI applications.
• Monitor production AI systems and identify model/application degradation.
• Optimize prompts, workflows, models, latency, and cost.
• Collaborate with product, UX, backend, frontend, and QA teams.

Strong Python development experience.

• Strong understanding of LLMs and Generative AI.
• Hands-on experience building LLM-powered applications.
• Understanding of AI agents and agentic workflows.
• Experience with frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, CrewAI, AutoGen, or equivalent.

Experience with RAG architectures.

• Experience with vector databases such as Pinecone, Weaviate, FAISS, Chroma, Milvus, or equivalent.
• Experience with LLM APIs such as OpenAI, Anthropic, Gemini, or equivalent.
• Understanding of prompt engineering and structured outputs.
• Experience with LLM evaluations / Evals.
• Understanding of evaluation methodologies and metrics for AI applications.
• Experience building APIs using FastAPI, Flask, or equivalent.
• Strong Git and software engineering practices.

Preferred Skills

• Experience with LangSmith, Arize Phoenix, Ragas, DeepEval, Promptfoo, or equivalent evaluation/observability tools.
• Experience with multi-agent systems.
• Experience with MCP or similar tool-integration protocols.
• Experience with AI safety and guardrails.
• Experience deploying AI applications on AWS/Azure/GCP.
• Experience with Docker and Kubernetes.
• Knowledge of ML fundamentals and model evaluation.
• Experience building enterprise AI applications.

Interested candidate kindly share their profile at [email protected]