AI Harness Engineer

Company: AbilityEX
Location: Jaipur, Rajasthan
Job Type: Full-time
Experience: 5+ years
We are looking for an experienced AI Harness Engineer to design, build, and maintain reliable, production-grade AI agent systems.
This software engineering role focuses on building infrastructure around large language models (LLMs), including prompt and context engineering, tool calling, agent orchestration, memory, evaluation pipelines, observability, and human-in-the-loop workflows.
Key Responsibilities:

• Design and develop AI agent workflows for marketing, growth, reporting, SEO/AEO, lead intelligence, and reputation monitoring.

• Build production-grade LLM applications using prompt engineering, context engineering, tools, memory, structured outputs, guardrails, and evaluation systems.

• Develop agent orchestration workflows and integrate LLMs with APIs, databases, CRMs, advertising platforms, Slack, WhatsApp, and internal tools.

• Design systems for ingesting, cleaning, storing, and retrieving business context for AI agents.

• Build evaluation pipelines to test model performance, prompt changes, edge cases, regressions, and output quality.

• Implement model routing strategies to optimise response quality, latency, and cost.

• Develop reliable backend services with appropriate validation, retries, error handling, and deterministic workflows.

• Implement observability through logging, tracing, prompt versioning, failure analysis, and feedback loops.

• Collaborate with product and growth teams to translate ambiguous business requirements into well-defined technical solutions.

• Own the complete engineering lifecycle, including architecture, coding, testing, debugging, deployment, and maintenance.

Required Qualifications and Skills:

• 5+ years of professional software engineering experience.

• Strong backend development, system design, and software architecture fundamentals.

• Hands-on experience building LLM-powered applications, AI agents, RAG systems, tool-calling workflows, or agent frameworks.

• Proficiency in Python, Node.js, TypeScript, or a similar backend technology.

• Experience working with APIs, databases, queues, schedulers, version control, testing, and deployment.

• Familiarity with PostgreSQL, Redis, vector databases, and production services.

• Understanding of structured outputs, JSON schemas, validation, retries, error handling, and reliable workflow design.

• Ability to break ambiguous business problems into scalable and maintainable technical systems.

• Strong debugging, problem-solving, communication, and ownership skills.

• Willingness to work in a fast-paced, early-stage startup environment.

Preferred Qualifications:

• Experience with OpenAI, Anthropic, LangGraph, LangChain, CrewAI, LlamaIndex, MCP, or similar technologies.

• Experience building evaluation frameworks for AI applications and agents.

• Familiarity with agent infrastructure, workflow engines, tool servers, or automation platforms.

• Experience integrating marketing and growth platforms such as Meta Ads, Google Ads, GA4, HubSpot, Zoho, Shopify, Slack, WhatsApp, or other CRMs.

• Experience with cloud platforms such as AWS, GCP, Azure, Vercel, Render, or Railway.

• Knowledge of production monitoring, distributed tracing, logging, and system observability.

What You Can Expect

• The opportunity to build core AI agent infrastructure.

• Direct collaboration with founders and cross-functional teams.

• Ownership of technically challenging, production-grade AI systems.

• Exposure to emerging LLM technologies, agent architectures, and AI-powered business automation.

Application Process
Email your CV to [email protected] with:

• A system you've built: Describe a technically complex system you owned or contributed to, including the architecture, technologies, and your contribution.

• An AI workflow you'd improve: Choose an AI product or agent you have used and explain how you would improve its harness, context, tools, evaluation, or reliability.

Pay: ₹2,000,000.00 - ₹3,000,000.00 per year

Application Question(s):

• How many years of professional software engineering experience do you have?
• How many years of hands-on experience do you have building applications using LLMs, AI agents, or RAG systems?
• Have you built production-grade AI agent workflows involving tool calling, orchestration, memory, or structured outputs?
• Have you implemented evaluation pipelines, observability, or feedback loops to monitor and improve AI system performance?
• Are you currently based out of Jaipur?
• Are you available to join immediately?

Work Location: In person