Snowflake Data Engineer with ADF

Key Responsibilities

Data Ingestion Engineering

• Develop and maintain ingestion pipelines from relational databases, files, and API-based sources into the Bronze layer.
• Build scalable and reusable ingestion frameworks to support multiple source systems.
• Develop Silver and Gold layer transformations using dbt, following defined data modelling and testing standards.
• Implement metadata-driven ingestion patterns across multiple source databases.
• Work with CDC and replication mechanisms to support near-real-time and incremental data ingestion.

Data Quality Operations

• Implement data quality checks, validation rules, error handling, and reprocessing mechanisms.
• Establish pipeline observability, monitoring, and alerting.
• Perform source data profiling, reconciliation, and defect triage.
• Troubleshoot production data pipeline issues and ensure timely resolution.
• Support operational activities for production pipelines.

Documentation Collaboration

• Prepare and maintain source-to-target mappings, pipeline documentation, and operational runbooks.
• Participate in code reviews and ensure adherence to development standards.
• Collaborate with architects, data engineers, analysts, and other distributed teams.
• Participate actively in Agile ceremonies, sprint planning, stand-ups, and retrospectives.
• Contribute to technical discussions and continuous improvement of data engineering practices.

Required Skills Experience

• Strong SQL skills with a minimum of 2 years of hands-on Snowflake development experience.
• Strong experience with Snowflake data engineering and development.
• Proficiency in Python for data engineering, automation, scripting, and API integration.
• Production-level experience with dbt (Core or Cloud).
• Working experience with at least one orchestration tool:
  • Apache Airflow
  • Dagster
  • Azure Data Factory (ADF)
  • Snowflake Tasks
• Understanding of CDC (Change Data Capture) and data replication patterns.
• Experience with one or more CDC/replication technologies such as:
  • Fivetran
  • Qlik Replicate
  • Debezium
  • Native Snowflake CDC capabilities
• Experience with Git-based version control and CI/CD fundamentals for data pipelines.
• Hands-on experience with Coalesce.
• Good understanding of data modelling, ETL/ELT concepts, and Medallion Architecture.
• Comfortable working in an Agile delivery environment with distributed teams.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.