Gcp Data Engineer

IntraEdge is hiring experienced Data Engineers with deep expertise in Google Cloud Platform (GCP) to join our Data Engineering team (current client/project- American Express) for Chennai location (Hybrid - Weekly 3 days in office). In this role, you will design, build, and optimize scalable data pipelines, data lakes, and data warehouses on GCP to support advanced analytics, machine learning, and business intelligence initiatives across the organization. This is a high-impact role for someone who thrives in designing end-to-end data architectures, optimizing performance, and enabling data-driven decision-making at scale.

Key Responsibilities
Data Engineering & Architecture

• Design and implement scalable and secure data pipelines using GCP-native services (e.g., BigQuery, Dataflow, Airflow).
• Develop ETL/ELT processes to ingest data from diverse sources including APIs, flat files, databases, and streaming sources.
• Build and maintain data lakes and data warehouses on GCP, ensuring schema optimization, partitioning, and cost-efficiency.
• Create and manage batch and real-time data workflows using Cloud Composer (Apache Airflow) and Dataflow (Apache Beam).

Data Management & Governance

• Ensure data quality, integrity, and reliability through robust validation and monitoring frameworks.
• Implement best practices for data lineage, metadata management, and data cataloging using tools like Data Catalog and Looker.
• Collaborate with data governance, security, and compliance teams to enforce data access controls and encryption.

Collaboration & Leadership

• Work closely with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
• Provide technical mentorship and code reviews for junior data engineers.
• Contribute to architecture reviews and technology evaluations for continuous improvement of the data platform.

Minimum Qualifications

• Bachelor's or Masters degree in Computer Science, Engineering, Information Systems, or a related field.
• 5+ years of experience in data engineering, with at least 3+ years working extensively on GCP.
• Strong proficiency with SQL, Python, PySpark and/or Java/Scala for data processing and scripting.
• Proven experience with GCP services such as:
• BigQuery
• Dataflow
• Pub/Sub
• Cloud Composer (Airflow)
• Experience in building large-scale ETL/ELT pipelines and optimizing query performance on cloud platforms.
• Solid understanding of data modeling, partitioning, and schema design for analytical workloads.
• Familiarity with CI/CD practices, Terraform, and version control (e.g., Git).