Senior Test Engineer - Microsoft Power Platform AI

Job Title: Senior Test Engineer - Microsoft Power Platform & AI

Position Overview:

We are seeking a highly skilled Senior Test Engineer with strong expertise in the Microsoft Power Platform ecosystem and Artificial Intelligence / Machine Learning testing capabilities, and with strong understanding of the QA lifecycle and testing fundamentals. The ideal candidate will lead quality assurance for intelligent business solutions built using Power Apps, Power Automate, Power BI, Co-pilot Studio, AI Builder, and Azure AI Services, validating both traditional low-code solutions and AI-driven intelligent automation to ensure accuracy, reliability, and enterprise-grade quality. You will play a key role in establishing quality standards, mentoring the testing team, and contributing to overall delivery excellence.

Roles and Responsibilities:

• Test Strategy, Planning and Governance: Architect and own the end-to-end test strategy for Power Platform and AI-driven solutions across the project lifecycle, defining test plans, test scenarios, test cases, and acceptance criteria for Canvas and Model-Driven Apps, Power Automate flows, Power BI reports, Power Pages, Co-pilot Studio chatbots, AI Builder models, and Azure AI/Cognitive Services integrations. Establish quality gates, KPIs, and exit criteria for each sprint or release, perform risk-based test prioritization, and build and maintain a reusable test asset library of test cases, test data, and scripts.

• Power Platform Testing: Validate Power Apps UI/UX along with the underlying business rules, verify the accuracy of Power BI reports and dashboards, and test Co-pilot Studio generative answers, citation accuracy, and multi-channel (Teams/Web) deployments.

• AI and Machine Learning Testing: Conduct model accuracy testing (precision, recall) and perform data quality, drift, and hallucination testing by fact-checking AI-generated content against source data, and carry out adversarial and Responsible AI testing covering prompt injection, jailbreak attempts, and content safety. Execute AI performance, A/B comparison, regression, and explainability testing using SHAP values, feature importance, and confidence thresholds.

• Test Automation: Design automated test frameworks using Power Apps Test Engine, Playwright/Selenium, EasyRepro, Postman/Newman, and Power Automate Desktop, use Python (pytest/unittest) for AI model and data-quality validation, and build regression suites integrated into Azure DevOps CI/CD pipelines using data-driven and keyword-driven approaches.

• Performance, Security and Compliance: Performance-test Power Apps load times, flow execution and throttling, Power BI rendering, and AI model inference latency, while ensuring that security measures are implemented in accordance with industry standards.

• Reporting and Communication: Prepare test execution summaries, defect and root-cause analysis, and AI accuracy/drift reports through Power BI quality dashboards, and provide regular status updates to stakeholders and leadership while actively participating in Agile/Scrum ceremonies.

• Leadership and Collaboration: Mentor junior and mid-level testers, run knowledge-transfer sessions, and collaborate closely with developers, architects, business analysts, and data scientists to embed quality throughout the delivery lifecycle.

Qualifications:

• Experience: 4 - 6 years of total experience in software testing / QA.

• Power Platform: 2 - 3+ years of hands-on testing experience with Power Apps, Power Automate, Power BI, and Data verse.

• AI Testing: AI/ML or Generative AI testing exposure is a nice to have. Any experience on testing AI/ML models, NLP, Computer Vision, or Generative AI would be an advantage but not a must one.

• Automation: Hands-on experience with at least 23 automation tools or frameworks or at least the mindset and capability to learn automation.

• API Testing: Proficient in REST API testing using Postman, Newman, or equivalent tools.

• Azure DevOps: Hands-on experience with Boards, Test Plans, Repos, and Pipelines.

• SQL: Proficient in writing complex queries for data validation.

• Agile/Scrum and SDLC/STLC: Solid experience working in Agile teams with a deep understanding of SDLC and STLC.

• Business Analysis: Nice to have prior experience in a BA or hybrid BA/QA . Ability to learn a new domain quickly. Should be able to explain their domain clearly.

• Learnability and adaptability: These are more important with respect to new technologies.

• Defect Management: Expertise in defect lifecycle management, root-cause analysis, and triaging.

• Communication: Excellent verbal and written communication skills in English.

Good to Have:

• Familiarity with Responsible AI frameworks and the Power Platform CoE Starter Kit.

Education:

• Bachelor's degree in computer science, IT, Engineering, or a related field.

Soft Skills:

• Strong analytical and problem-solving abilities with high attention to detail. Excellent stakeholder communication, presentation, and documentation skills.

• Team leadership and mentoring capability, with adaptability to evolving AI and Power Platform technologies.