AI Engineer (Software Developer Test/QA)

About the role

MetLife is offering an exciting opportunity to contribute to our digital and AI transformation journey. The AI Engineer (Software Developer Test/QA) has the responsibility of delivering high-quality software in an Agile environment. This includes the adoption of test automation within the Agile sprint, and the leverage of Gen-AI toolset to achieve high velocity of releases without compromising delivery quality. In this role, the SDET also operates as an AI Engineer, using GitHub Copilot and Copilot Agent Mode on our GitHub Enterprise platform, together with the Microsoft AI Foundry stack, to build AI agents that automate testing, and to design and run AI evaluations (AI evals) that measure the performance and reliability of those agents.

Key responsibilities

• Design, build and scale testing capabilities using AI agents, using GitHub Copilot Agent Mode on the GitHub Enterprise platform and the Microsoft AI Foundry stack, to automate testing tasks such as test case generation, test data creation, defect triage, and regression analysis.
• Develop and run AI evaluation (AI eval) frameworks and metrics to measure the accuracy, reliability, and performance of AI agents and Gen-AI-assisted testing workflows and iterate on prompts and agent design based on eval results.
• Write efficient test automation code using coding standards and best practices.
• Validate application functionality through the implementation of test strategies, automation frameworks, tools, and dashboard.
• Participate in planning and documentation of test strategies and acceptance criteria based on customer needs.
• Participate in test execution (automated and manual) (new builds and regression), analyze the reports, troubleshoot and investigate the failure.
• Maintain automation scripts and perform peer reviews of the automation codebase.
• Analyze and repair broken test cases and failures.
• Investigate, diagnose, and repair complex problems, test, and manage bug reports.
• Adoption of the Agile ways of working in the software delivery lifecycle.

About you

• Bachelor in Computer Science or Equivalent.
• At least 3+ years in automation testing preferably using AI in high tech and/or highly regulated industries.
• Hands-on experience with AI-assisted engineering tools such as GitHub Copilot (including Agent Mode) on GitHub Enterprise, and the Microsoft AI Foundry stack, including building AI agents and running AI evaluations to validate agent performance.
• Developing and implementing automation testing practices using modern technology stack and patterns.
• Strong scripting and programming skills in languages such as Java, Python, Bash, or PowerShell.
• Ability to design agent instructions/prompts, build and orchestrate AI agents (e.g. via GitHub Copilot Agent Mode and the Microsoft AI Foundry stack), and apply AI evaluation techniques to validate agent accuracy and performance within the test lifecycle.
• Hands-on in test automation using scripting and programming skills.
• Exposure to Behavior Driven Development (BDD).
• Continuous learning mindset, ability to acquire new skills and competencies independently and proactively.

About us

MetLife is a leading global life insurance and employee benefit company, with 90 million customers in over 60 countries. We are leading the global transformation of an industry we've long defined. At MetLife, you'll have accountability for end-to-end solutions, working with other talented engineers who share your passion for technology, distinctive digital experiences and customer-driven innovation. From implementing new platforms to migrating applications to the cloud, you'll be instrumental in creating technologies built to scale. We hire smart, creative, collaborative people who are kind to each other. We embrace diversity, equity and inclusion. This allows us to think bigger, experiment more, innovate faster, and win together.