GCP Data EngineerIntermediate

• We are seeking a skilled GCP Data Engineer with 3 to 7 years of experience to join our growing data engineering team in Chennai/Hyderabad/Bangalore/Pune, focused on building and maintaining scalable cloud-based data pipelines and infrastructure on Google Cloud Platform.
• The ideal candidate will have hands-on expertise in GCP services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, and Dataproc
• Proficiency in leveraging Claude AI and other LLM-based tools for intelligent data transformation, automation of data workflows, and natural language query generation is highly desirable.
• Strong programming skills in Python are required for developing ETL/ELT pipelines, data ingestion scripts, and custom data transformation logic across batch and streaming workloads.
• Solid command of SQL is essential for writing complex queries, optimizing data models in BigQuery, and supporting analytics and reporting teams with structured data access layers.
• Candidates must have experience designing and implementing data lakehouse architectures, dimensional data models, and governance frameworks that ensure data quality, lineage, and security compliance.
• Experience working in agile development environments with CI/CD pipelines, version control using Git, and infrastructure-as-code tools such as Terraform for GCP resource provisioning is expected.
• Familiarity with data orchestration platforms such as Apache Airflow or Cloud Composer and monitoring tools like Cloud Logging and Cloud Monitoring will be considered a strong advantage.

Roles & Responsibilities

• Design, develop, and optimize end-to-end data pipelines on GCP using BigQuery, Dataflow, Cloud Composer, and Pub/Sub to support real-time and batch data processing requirements.
• Integrate multi-cloud data sources across GCP and AWS(nice to have) environments, ensuring seamless data movement, transformation, and availability for downstream analytics and machine learning teams.
• Utilize Claude AI and generative AI tools to automate data catalog creation, generate SQL queries from natural language inputs, and enhance the efficiency of data operations workflows.
• Write and maintain high-quality Python scripts for data ingestion, transformation, and pipeline orchestration, ensuring code is modular, well-documented, and production-ready.
• Develop and maintain complex SQL queries and stored procedures in BigQuery to support business intelligence dashboards, ad hoc analysis, and scheduled reporting pipelines.
• Collaborate with data architects, analysts, and ML engineers to translate business requirements into robust data models and ensure data accuracy, freshness, and consistency across the platform.
• Monitor and troubleshoot data pipeline failures, performance bottlenecks, and data quality issues using GCP-native observability tools and proactively implement preventive measures.
• Participate in code reviews, contribute to data engineering best practices, and mentor junior team members to uphold high engineering standards across the data platform.