Senior QA/QC Data Analyst with AI/ML testing

Req ID: phone_number

Job Summary

We are currently seeking a Senior QA/QC Data Analyst with AI/ML testing to join our team in Bangalore, Karnataka, India.

Responsibilities

• Take responsibility for the project and work independently in a collaborative environment.
• Evaluate data across various dimensions, including accuracy, completeness, consistency, validity, timeliness, uniqueness, and integrity.
• Establish quality criteria for data accuracy, completeness, consistency, validity, and AI model performance.
• Test data ingestion, transformation (ETL/ELT), and outputs to ensure source-to-target integrity.
• Verify that datasets used for AI/ML are accurate, representative, unbiased, and fit for purpose.
• Validate model metrics against defined business and technical thresholds, including accuracy, precision, recall, and stability.
• Identify and assess bias, fairness issues, and unintended impacts in data and AI model outcomes.
• Implement automated tests for data validation, model regression, and pipeline checks within CI/CD and MLOps workflows.
• Log, prioritize, and track data and AI defects; perform root cause analysis and corrective actions.
• Validate adherence to data governance, privacy, security, and AI regulatory requirements.
• Track data drift, model drift, anomalies, and performance degradation post-deployment.
• Communicate quality status, risks, and recommendations clearly to stakeholders before and after releases.
• Design and develop testing plans, test cases, and test scripts to evaluate data quality.
• Identify, document, and report defects, inconsistencies, and inaccuracies.
• Maintain and update testing documentation and report on test outcomes.
• Document key data processes and transformations.
• Assist with quality improvement initiatives and recommend improvements to data quality processes and procedures.
• Assist developers, business stakeholders, and data governance teams in meeting data quality standards.
• Knowledge of the Agile/Waterfall approach and how quality assurance fits into it.
• Create automated testing solutions from scratch.
• Develop testing scenarios by analyzing feature requirements for the purpose of estimating testing effort.
• Track defects, analyze and communicate test results, and engage in daily QA activities.

Skills and Abilities

• Ability to take ownership of the project and work independently in a team environment.
• Ability to analyze problems, perform root cause analysis, and develop solutions essential for resolving data issues.
• Proficient in PyTest/unit test frameworks used for ML testing automation.
• Knowledge of programming languages such as C#, Python, or R for data process automation and analysis.
• Proficient in automated test suite development for ML models.
• Working knowledge of ML-specific API testing (e.g., inference endpoints).
• Working knowledge of automated metrics validation for model outputs.
• Expertise in the principles and testing procedures.
• Working knowledge of data validation frameworks such as Great Expectations.
• Working knowledge of ML-oriented data pipeline tools.
• Working knowledge of validation for massive, ML-ready datasets.
• Proficient in ML-specific QA strategy (model tests, feature tests, data contracts, drift tests).
• Working knowledge of CI/CD integration for ML (MLOps).
• Proficient in model-centric regression testing.
• Proficient in testing lineage, reproducibility, and experiment tracking.
• Proficient in developing and maintaining test plans, test scenarios, test cases, test defect tracking, summary reporting, and test scripts to perform thorough testing and validate the data, based on business requirements.
• Familiarity with Extract, Transform, Load (ETL) tools to move and transform data.
• Working knowledge of AI governance framework, ethical AI compliance, bias monitoring policies, and model traceability requirements.
• Working knowledge of debugging tools (Fiddler).
• Proficiency with version control practices using Git, including branching, collaboration, reviewing pull requests, and resolving merge conflicts.
• Knowledge of build servers (Azure CICD Pipeline, Jenkins, and Cruise Control).
• Working knowledge of source control systems (DevOps, GitHub, SVN, or TFS).
• Working experience in release management and bug tracking tools (e.g., Zephyr, etc.).
• Knowledge of HTML5, CSS, JavaScript, Angular Bootstrap, SQL, SharePoint, JSON, or XML objects.
• Knowledge of API testing using Postman and Swagger UI.
• Knowledge of Azure cloud infrastructure and its capabilities.
• Strong verbal and written communication skills are needed to explain findings to both technical and non-technical stakeholders.
• A willingness to learn new tools and adapt to evolving technologies is important for long-term career growth.