Sr QA - Data & QA (Python + Azure Databricks)

Role Overview

We are seeking a QA Engineer with strong expertise in Python, CI/CD, and Azure data services to strengthen our data platform. The role involves building, testing, and validating data pipelines on Azure Databricks and Delta Live Tables, ensuring high-quality data delivery across the platform. The engineer will play a key role in ETL testing, data quality validations, and CI/CD automation to support reliable and scalable deployments.

Key Responsibilities

• Design, build, and maintain CI/CD pipelines for the data platform.
• Work extensively with Azure Databricks, Data Lakehouse (DLH), and Delta Live Tables (DLT).
• Implement pytest unit testing frameworks for pipeline validation and transformations.
• Conduct ETL testing and data quality validation (DQC) across Bronze, Silver, and Gold layers.
• Validate data stored in Azure SQL, ADB SQL, and S3 file systems.
• Collaborate with DevOps, Platform, and Data Engineering teams for seamless integration and deployment.
• Troubleshoot and resolve platform-related issues in Python-based workflows.
• Ensure platform governance, data quality, and compliance standards are met.

Required Skills

• Strong hands-on expertise in Python
• Automation Testing Framework for Data validation and BDD approach.
• Solid experience in pytest unit testing for ETL/data pipelines.
• Practical knowledge of CI/CD tools (Azure DevOps, GitHub Actions, or Jenkins).
• Proven experience in Azure Databricks, DLH, and DLT.
• Strong understanding of ETL testing methodologies.
• Proficiency in Azure SQL, ADB SQL, and S3 file validation.
• Expertise in data quality checks and governance validation.
• Strong analytical, debugging, and problem-solving skills.

Good to Have

• Knowledge of Medallion architecture (Bronze, Silver, Gold layers).
• Exposure to enterprise data platforms and cloud-native data ecosystems.
• Familiarity with Agile methodologies and DevOps best practices.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.