QA - Python testing

Position Overview

We are looking for a skilled QA Python Testing Engineer with strong experience in Python-based testing, Hadoop testing, PySpark/Spark, SQL, and Pandas. The ideal candidate will be responsible for designing and executing test strategies for data-intensive applications, validating large datasets, developing automation frameworks, and ensuring data quality and accuracy across distributed processing environments.

The role requires strong analytical and problem-solving skills, with the ability to work closely with development, data engineering, and business teams to identify and resolve defects.

Key Responsibilities

• Design, develop, and execute functional, regression, integration, and data validation test cases using Python.
• Develop and maintain Python-based test automation scripts and frameworks.
• Perform testing of Hadoop ecosystem components and validate large-scale data processing workflows.
• Validate data processing and transformation logic using PySpark/Spark.
• Perform database and data validation testing using SQL.
• Use Pandas for data manipulation, comparison, validation, and test-data analysis.
• Validate ETL/data pipelines and ensure data accuracy, completeness, consistency, and integrity.
• Perform source-to-target data validation across distributed data platforms.
• Analyze test results, identify defects, and work with development teams to troubleshoot and resolve issues.
• Create and maintain test scenarios, test cases, automation scripts, and test execution reports.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, reviews, and retrospectives.
• Collaborate with Data Engineers, Developers, Business Analysts, and other QA teams to ensure end-to-end quality.
• Identify opportunities to improve test automation, coverage, and overall QA processes.

Mandatory Skills

• Strong hands-on experience in Python Testing / Python Automation Testing.
• Strong experience in Hadoop Testing and validation of Hadoop-based data processing.
• Hands-on experience with PySpark or Apache Spark.
• Strong proficiency in SQL for data validation and database testing.
• Hands-on experience with Pandas for data analysis and validation.
• Good understanding of ETL/Data Pipeline Testing.
• Strong knowledge of Data Validation, Data Quality, and Reconciliation Testing.
• Experience in developing and executing automated test scripts using Python.
• Strong analytical and debugging skills.

Good to Have

• Experience with Hive, HDFS, Impala, or other Hadoop ecosystem components.
• Experience with Kafka or other data streaming technologies.
• Knowledge of CI/CD and test automation practices.
• Experience working in Agile/Scrum environments.
• Exposure to cloud-based data platforms such as AWS, Azure, or GCP.
• Familiarity with Git and other version-control tools.
• Experience with API testing and REST services.