GCP +Pyspark+Scala Professional

• We are looking for an experienced Data Engineer with 5+ years of experience in designing, developing, and maintaining scalable data engineering solutions on the Google Cloud Platform (GCP).
• The ideal candidate should have strong hands-on expertise in PySpark, Scala, Airflow, BigQuery (BQ), Dataproc, and ETL development.
• The candidate will be responsible for developing, deploying, monitoring, and optimizing batch ETL workflows in GCP environments.
• You should have experience building high-performance, reliable, and scalable data pipelines using PySpark and Scala, while leveraging Airflow for workflow orchestration, dependency management, scheduling, and idempotent execution.
• A strong understanding of Medallion Architecture (Bronze, Silver, Gold layers) is required, along with experience implementing data quality checks, data validation, summarization, and transformation processes.
• The role also requires expertise in performance tuning and optimization of ETL jobs, improving pipeline efficiency, and implementing observability solutions for proactive monitoring.
• Experience creating operational runbooks and supporting continuous optimization initiatives is highly desirable.
• The ideal candidate should be proficient in BigQuery, Dataproc, and other GCP services, with the ability to troubleshoot production issues and ensure high data reliability.
• Knowledge of data governance principles, metadata management, and best practices for secure and compliant data processing will be an added advantage.

Mandatory Skills:

• 5+ years of Data Engineering experience Google Cloud Platform (GCP) PySpark Scala Apache Airflow BigQuery (BQ) Dataproc ETL Development Medallion Architecture Data Quality & Validation Performance Optimization Observability Runbook Creation
• Good to Have: Data Governance Metadata Management Production Support and Monitoring

Location - Pan India,Mumbai, Delhi / NCR, Bengaluru , Kolkata, Chennai, Hyderabad, Ahmedabad, Pune