Specialist - Data QA

About the Role

We are looking for a Data Scientist & Data QA Engineer to join our team to ensure the quality, reliability, and performance of our data, AI, and machine learning solutions. This role is ideal for someone who enjoys working with data, solving complex problems, automating quality checks, and improving the reliability of AI-driven products.

Responsibilities:

• Data & QA: Design and execute data quality checks; validate data for accuracy, completeness, and consistency; perform source-to-target validation and reconciliation; use SQL and Python to investigate anomalies; develop automated testing frameworks; and support regression testing for production data and ML solutions.
• AI & Machine Learning: Evaluate AI/ML models against business requirements; analyze outputs for inconsistencies; and support testing for model performance, data drift, bias, and reliability alongside Data Scientists and ML Engineers.
• Generative AI, LLM & RAG Validation: Support the quality of GenAI, LLM, and RAG applications; test responses for accuracy and factuality; identify hallucinations and bias; perform prompt and scenario-based testing; and develop test data for continuous quality monitoring.
• Collaboration: Partner with Data Scientists, Data, ML, and Software Engineers, and Product teams to define quality standards, test strategies, and acceptance criteria while performing root-cause analysis on defects.

Profile Requirements:

• Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.
• Experience in one or more of the following: Data QA, Data Science, Data Engineering, Machine Learning, or AI/ML Testing.
• Strong Python and SQL programming skills.
• Experience with automated testing, preferably PyTest or similar frameworks.
• Solid understanding of data quality concepts and machine learning fundamentals.
• Strong knowledge of prompt engineering and experience designing and evaluating prompts for AI/LLM applications.
• Working knowledge of MCP (Model Context Protocol) servers and how they integrate with AI/LLM applications.
• Experience working with skills files / AI agent configuration and instruction files and understanding how they are used to guide AI workflows.
• Hands-on experience with Claude Code or similar AI-assisted development tools.
• Strong analytical, problem-solving, and communication skills.
• Preferred Qualifications
• Hands-on experience with Generative AI, LLM applications, RAG architecture, and AI evaluation frameworks such as Ragas, TruLens, or DeepEval.
• Familiarity with LLM platforms such as OpenAI, Anthropic, and Llama, as well as concepts such as data/model drift and bias monitoring.
• Experience with cloud platforms such as AWS, Azure, or GCP and CI/CD processes.