Lead Software Engineer

Who you are

• Strong technical leadership: Proven experience leading technical initiatives or teams, capable of setting the technical vision for complex, ambiguous domains
• Deep software engineering fundamentals: Senior/advanced capability in designing secure components end-to-end, testing thoroughly, and reasoning heavily about system design, concurrency, and architecture tradeoffs. (Python preferred)
• Expertise in AI Evaluation and Safety: A highly critical thinker who understands the nuances of LLM behavior. You must be comfortable interrogating the quality of AI outputs and deeply experienced in building harnesses to measure it reproducibly
• Security and Governance mindset: Experience with threat modeling, authentication, red teaming, or building guardrails for internal systems, tooling, or platforms
• Cloud experience (AWS): Comfortable running internal services, owning reliability/scalability, and collaborating with platform/techops/security partners
• Excellent communication: Able to synthesize complex safety and eval metrics into actionable insights for both technical and non-technical stakeholders
• Experience with specific LLM evaluation and safety frameworks (e.g., Inspect AI, Ragas, OpenAI Evals, NeMo Guardrails)
• Django experience and strong backend engineering patterns (security, performance, maintainability)
• Experience with Datadog for complex observability, tracing, and monitoring in AI environments
• Familiarity with foundational AI engineering tooling like Pydantic AI, LiteLLM, or LangChain
• Studies have shown that some groups of people, like women, are less likely to apply to a role unless they meet 100% of the job requirements

What the job involves

• AI is a key investment area for Kraken Technologies as we look to expand our existing capabilities. A crucial part of this is broadening the foundational infrastructure to enable teams across the organisation to use AI effectively to accelerate our mission
• You’ll work in the Agent Safety/Evals team, a new sub-team within AI Foundations
• We build the shared platforms, harnesses, and guardrails that enable engineering and product teams to safely, reliably, and deterministically use machine learning and generative AI agents for internal systems and workflows across the business
• This is a delivery-focused team that sits at the intersection of engineering and delivery, focusing on empowering our internal users
• We’re hiring a Lead Software Engineer to head our newly formed Agent Safety/Evals team
• As AI agents take on more autonomous tasks across Kraken's internal workflows, this role is critical to ensuring they do so securely and predictably
• This is a leadership role focused on constraining our internally facing AI agents to an expected operation space and guaranteeing system reliability
• You will define the technical strategy for how we evaluate models, enforce safety guardrails, and govern AI behavior across internal platforms, skills, and harnesses
• You’ll work closely with the broader AI Foundations team and engineers across Kraken to ensure that our push for rapid AI adoption in internal tooling never compromises on security, determinism, or quality
• Lead the technical direction for internal agent safety: Design and implement systems focused on the reliability, security, and determinism of LLMs and autonomous agents, constraining them strictly to expected operational spaces within our internal ecosystems
• Build robust evaluation frameworks: Develop scalable harnesses and evals tailored to internal workflows and skills. You will be responsible for asking the right types of questions about the quality and reproducibility of our evals, while engineering the systems to measure them robustly
• Implement guardrails and governance: Create and enforce pre- and post-generation guardrails, managing the overarching governance of AI models operating within internal tools and platforms
• Drive AI security and observability: Build out dedicated auth/permissions for internal AI agents, establish deep monitoring/observability pipelines, and define incident response protocols for AI-specific anomalies
• Verification and Red Teaming: Lead continuous verification efforts and red teaming exercises to proactively identify vulnerabilities, prompt injections, or unpredictable behaviours in our internal AI implementations
• Operate in AWS: Deploy, run, and support high-throughput, low-latency safety services for internal use cases; make sensible architecture/cost tradeoffs; partner effectively with platform/techops/security stakeholders
• What Success Looks Like:
• Confidence in Internal Tooling: Engineers across Kraken can deploy new autonomous models, AI skills, and internal agents with total confidence, knowing they are bound by reliable safety guardrails and strict operational constraints within our internal ecosystems
• High-Quality Internal Evals: You have established a culture and infrastructure of robust, highly reproducible evaluation metrics that accurately reflect the real-world performance and safety of our internal-facing AI tooling and harnesses
• Resilient Internal Infrastructure: AI security, monitoring, and incident response are treated as first-class citizens, ensuring that any erratic agent behavior in our internal platforms is immediately caught and mitigated before it affects broader operations
• Technical Leadership: Strong collaboration across AI Foundations helps accelerate secure internal AI adoption; you are actively mentoring engineers and shaping the company's internal AI governance strategy

Benefits

• Equity option scheme: Everyone who works at Kraken owns a part of the business with shares in the Octopus Energy Group. So as the company grows we all benefit
• Pension scheme: We'll pay 5%. You decide on your contribution starting from 3%
• Holidays: We don't count your holiday days. We trust everyone to work hard and take as much time off as they need
• Flexible working: We're happy for you to work in a way that's best for you. Need to pick the kids up from school or work from home? That's not a problem. It's important to us that everyone has a positive work-life balance
• Free breakfast: Our offices are always stocked with fresh fruit, breakfast, snacks, great coffee and soft drinks.

Weekly on a Friday we all get together (including virtually) for company updates and drinks

• For parents: We can arrange access to tax free childcare, as well as maternity and paternity pay
• Free eye tests: We offer eye test vouchers for all colleagues who use Visual Display Units (VDU). Long spells of VDU work can be visually demanding so we want to make sure you're OK
• Car salary sacrifice: Save money on leasing an electric vehicle through our workplace salary sacrifice scheme
• Cycle to work scheme: Spread the cost of buying a bike, including accessories and safety equipment, up to a value of £2.5K. And save money on the repayments through the cycle to work scheme
• Get the right tech: Have a certain tech requirement? We’ll do our best to sort it for you
• Money coach: Get support with financial planning through our partnership with Octopus Money Coach
• Yoga + Headspace: Twice a week we have 45 Minute live yoga class open to all Kraken staff.

Need some calm in your life? We have Headspace licences available

• Team events and activities: Participate in team events and cohesion activities to build stronger connections
• Health Insurance: We’ll provide comprehensive coverage with Alan and access to Alan Mind.
• Transportation: If you cycle to the office we’ll support you by up to €600/year, if you use public transport we'll reimburse 75% of your Navigo pass. If you use both, we'll support you by up to €900/year
• Green energy discount: Enjoy a 32% discount on green energy compared to the regulated tariff
• Year-end gift voucher: Receive a €100 gift voucher at the end of each year
• Gymlib subscription: Get a subscription to Gymlib for access to a wide variety of fitness options
• Meal vouchers: Enjoy €10 Ticket Restaurant cards for meal expenses
• NGO support days: Benefit from three additional RTT days (Reduction of Working Time) to support NGO work, every year