Data Engineer

We are seeking an experienced Data Engineer to join the Billing team in Hyderabad. Billing is a business-critical and highly sensitive area where data accuracy, reliability, traceability, and on-time delivery are essential. You will build and operate data pipelines, curated data assets, and integrations that support time-sensitive billing processes and downstream consumers.

You will work primarily in Microsoft Azure with Databricks, Python, PySpark, SQL, and ETL/ELT patterns, using ActiveBatch for orchestration and scheduling. The role includes integrations with internal systems and third-party providers through APIs, databases, and file-based interfaces, as well as ownership of data hub and curation processes that must be complete, accurate, and delivered within narrow processing windows.

The Billing team operates with a high quality bar and limited tolerance for delay. When production issues occur, the engineer is expected to investigate quickly, communicate clearly, implement safe corrective actions within the active processing window, and propose practical alternatives when the original approach is blocked. Strong judgment, prioritization, and time management are critical to success in this role.

Snowflake is expected to become part of the future-state ecosystem, so prior Snowflake exposure is useful but is not a requirement for the current role.

This role is based in Hyderabad and follows a hybrid work model. You will collaborate with colleagues across India and the United States, with reasonable working-hour overlap when needed for team ceremonies, planning, production support, and delivery.

Key Responsibilities

• Billing Data Engineering: Design, build, test, deploy, and maintain reliable ETL/ELT pipelines and reusable data components in Azure and Databricks using Python, PySpark, and SQL.
• Data Hub & Curation: Transform source data into trusted, billing-ready curated datasets. Maintain consistent business rules, source-to-target traceability, and accurate delivery across data hub and curation layers.
• Data Quality & Reconciliation: Implement rigorous automated validations and reconciliations for completeness, accuracy, consistency, duplicates, referential integrity, and expected volumes. Treat data quality defects as production-impacting issues and resolve them with urgency.
• Orchestration & Scheduling: Develop and support ActiveBatch schedules, dependencies, execution chains, reruns, and recovery procedures. Ensure pipelines complete within narrow billing windows and downstream delivery commitments.
• Third-Party & Internal Integrations: Build and operate integrations with third-party providers and internal systems using APIs, databases, secure file transfers, and other appropriate interfaces. Design for retries, recoverability, idempotency, and clear failure handling.
• Production Reliability & Incident Response: Monitor pipeline execution and data delivery; investigate failures, data anomalies, and missed dependencies immediately. Apply safe in-window fixes or workarounds when necessary, communicate impact and status, and follow through with root-cause and permanent corrective actions.
• Performance & Timeliness: Optimize jobs and workflows to meet strict processing deadlines. Identify bottlenecks early, manage competing priorities, and escalate risks before they threaten billing timelines.

Location: Hyderabad

Notice period : Immediate

Kindly share CV to 9364987225 or [email protected]