LLM Systems Engineer /Prompt Architect

LLM Systems Engineer/Prompt Architect

Recruiter: Rain

Job Ref: CPT000225/ND

Date posted: Friday, September 18, 2026

Location: Cape Town, South Africa

SUMMARY: purpose We are seeking an LLM Systems Engineer to design, implement, and optimise our AI-powered customer service platform. This role requires expertise in prompt engineering, business logic translation, and production AI systems. You will work with our team to build multi-agent conversational AI systems that handle complex customer interactions across billing, technical support, and sales domains.

AI should be a force multiplier here. Use it to work faster and maintain high standards — but evaluate its output critically. We want engineers who apply judgment to amplify these tools, not avoid them or accept them blindly.

Key Responsibilities

• Design and maintain modular prompt architectures with clear separation of personas, rules, workflows, and tools.
• Translate complex business rules into executable AI behaviours and decision trees.
• Optimise prompts for accuracy, consistency, and token efficiency; implement RAG patterns for dynamic context assembly.
• Build production-grade AI systems using Python, FastAPI, Redis, and PostgreSQL.
• Design tool/function-calling integrations with backend APIs; implement caching, session management, and state persistence.
• Ensure observability through comprehensive logging, metrics, and tracing.
• Create behaviour-driven test plans for conversational AI; conduct regression testing and coordinate A/B testing.
• Debug inconsistent AI behaviours using conversation logs and tool execution traces.
• Write clear technical documentation; maintain structured, markdown-based knowledge bases and runbooks.

requirements

• Python expertise: 3+ years with FastAPI, async/await, decorators, type hints.
• Production experience with GPT-4, Claude, or similar LLMs via API.
• Advanced prompt engineering — context windows, token optimisation, tool-calling patterns.
• Strong system design fundamentals (Strategy, Registry, Factory, Decorator patterns).
• Experience with Redis (caching, sessions) and PostgreSQL.
• RESTful API design and WebSocket communication.
• Ability to translate complex business rules into AI workflows.
• Experience designing conversational flows with appropriate tone, empathy, and escalation logic.
• Systematic approach to testing emergent AI behaviour, not just code correctness.
• Experience deploying and monitoring AI systems in production.
• Exceptional, clear technical writing for multiple audiences.
• Comfortable with ambiguity and hypothesis-driven iteration.

Preferred experience:

• LangChain, LlamaIndex, or similar LLM frameworks.
• Workflow automation platforms (n8n, Zapier).
• Vector databases for semantic search.
• Background in customer service, telecom, fintech, or subscription billing.
• Compliance awareness (GDPR, audit trails, data security).
• Voice UX design experience.

You will thrive in this role if:

• You use AI as leverage on your judgment, not a substitute for it.

POSITION INFO: technology stack Python and FastAPI

• Redis and PostgreSQL
• GPT-4 / Claude via API, RAG, and tool calling
• production observability (logging, metrics, tracing)
• AI integrated into the engineering workflow.