GCP Data engineer with Python/Pyspark,Airflow,Dataproc expertise

Job Description: Senior GCP Data Engineer

Job Title: Senior Data Engineer - GCP & Big Data Architecture
Experience Level: 6 to 9 Years of overall IT experience (with 3+ years of dedicated GCP)
Location: Bangalore / Chennai / Pune / Gurgram / Kolkata / Hyderabad

Role Overview

We are looking for a highly skilled Senior GCP Data Engineer with 6 to 9 years of experience to design, build, and optimize our next-generation enterprise data platform. In this role, you will lead the development of robust, scalable batch and real-time data pipelines. You will be the subject matter expert for our big data infrastructure, translating complex business needs into clean, production-ready architectures leveraging GCP, Python, Airflow, Dataproc, and PySpark.

Key Responsibilities

• Data Pipeline Development: Design, build, and maintain enterprise-grade ETL/ELT pipelines using Python and PySpark to process massive, unstructured, and structured datasets
• Orchestration & Workflow Management: Author, schedule, and monitor complex Directed Acyclic Graphs (DAGs) using Apache Airflow (Google Cloud Composer) to orchestrate modern data workflows
• Big Data Processing: Architect and tune scalable distributed computing environments using GCP Dataproc (Hadoop/Spark clusters) to optimize large-scale batch processing
• Data Warehousing: Manage and optimize data schemas, indexing, partitioning, and cost-efficient querying inside Google BigQuery.
• Data Architecture: Implement and scale modern frameworks like the Medallion Architecture (Bronze, Silver, Gold layers) or data lakes ensuring high performance

Required Technical Skills & Qualifications

Core Technical Stack (Must Haves):

• Google Cloud Platform (GCP): In-depth, hands-on experience with core data components including BigQuery, Cloud Storage (GCS), Pub/Sub, and Cloud Functions.
• Programming Language: Expert proficiency in Python for general programming and data engineering tasks.
• Big Data Processing: Deep technical expertise in PySpark / Apache Spark for distributed processing, including performance tuning and configurations on GCP Dataproc.
• Orchestration: Extensive experience designing, writing, and debugging Apache Airflow / Cloud Composer DAGs.
• Advanced SQL: Superior skills in writing, optimizing, and tuning highly complex SQL queries and analytic functions.

Preferred / Good to Have Skills:

• An active Google Certified Professional Data Engineer certification

Required Skills