Data Engineer with Spark, Scala +Kafka

Data Engineer with Scala , Kafka, GCP

1. Should know how to develop, deploy , monitor and optimize Etl workflows for both Batch and Streaming with Scala, Spark, Airflow, BQ , Kafka , Dataproc and GCP environments

2. Experience in Medallion architecture with proper Data quality checks and summarization activities

3. Experience in performance optimizations, Observability and runbook creations for continuous optimizations.

4. Good knowledge of Airflow orchestrations including dependency handing, idempotent execution etc.

5. Exposure to data governance and information security would be a plus

6. Through knowledge of handing streaming data including back pressure handling etc.

7. Minimum Experience - 6+ years

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.