DevOps Engineer-II

Platform Engineering | Google Cloud | Data & AI Enablement We are looking for a Senior Developer to help build and evolve the CI/CD frameworks used by our Data and AI Platform teams.

This role is focused on building reusable platform capabilities rather than one-off project-specific implementations. You will work closely with data engineers, AI/ML engineers, and platform teams to design reusable pipelines, automation frameworks, deployment patterns, and self-service engineering capabilities for modern data and AI workloads on Google Cloud Platform.

The right person for this role is hands-on, comfortable writing production-quality code, and has a strong understanding of both cloud services and CI/CD engineering practices. You should be able to take complex platform requirements and turn them into scalable, reusable frameworks that engineering teams can easily adopt.

Key Responsibilities Build Platform CI/CD Frameworks

• Design and develop reusable GitLab CI/CD frameworks for Data and AI platform teams.
• Create reusable pipeline templates, deployment workflows, validation steps, and automation standards that can be adopted across projects and environments.
• Build self-service deployment capabilities and golden-path engineering workflows for Data & AI teams.

Enable GCP Data Platform Deployments

• Build deployment patterns and automation for core GCP data services including:

• Dataproc

• BigQuery

• Dataform

• Cloud Composer

• Google Cloud Storage

• Dataflow

• Bigtable

• Cloud Spanner

• Enable teams to deploy, configure, and operate these services through reliable and repeatable CI/CD processes.

• Design frameworks supporting multi-environment and multi-tenant deployment patterns.

Support Data Pipeline Delivery

• Design CI/CD workflows for:

• Batch data pipelines

• Streaming workloads

• Transformation logic

• Orchestration workflows

• Environment-specific releases

• Work with engineering teams to improve how data pipelines are built, tested, promoted, and deployed across environments.

Develop Internal Tools and Automation

• Build platform tools, APIs, CLIs, scripts, and reusable libraries using Python and Go.
• Focus on reducing repetitive work, improving developer experience, and making platform capabilities easier for teams to adopt.
• Apply software engineering best practices including testing, modular design, versioning, and maintainability to platform tooling and frameworks.

Partner with Engineering Teams

• Work closely with data, AI, platform, and infrastructure teams to understand engineering challenges and convert them into scalable platform solutions.

• Help define practical standards for:

• Pipeline design

• Deployment automation

• Release management

• Operational readiness

• CI/CD governance

Must-Have Qualifications

• Strong experience designing and building CI/CD frameworks for platform or engineering teams.

• Expert-level knowledge of Google Cloud Platform and hands-on experience with GCP data services.

• Deep experience with:

• Dataproc

• BigQuery

• Dataform

• Cloud Composer

• GCS

• Dataflow

• Bigtable

• Spanner

• Strong experience with GitLab and GitLab CI/CD pipeline design.

• Experience designing CI/CD pipelines for data platforms, analytics workloads, or AI/ML platform teams.

• Strong understanding of data pipeline design including:

• Batch processing

• Streaming architectures

• Orchestration

• Transformation workflows

• Environment promotion

• Strong application development experience with Python.

• Hands-on development experience with Go.

• Ability to build reusable frameworks, deployment templates, automation utilities, and engineering standards.

• Experience working with cross-functional engineering teams in platform, DevOps, or cloud engineering environments.

Technical Skills Cloud Platform

• Google Cloud Platform (GCP)

GCP Data Services

• Dataproc
• BigQuery
• Dataform
• Cloud Composer
• GCS
• Dataflow
• Bigtable
• Spanner

CI/CD & Automation

• GitLab
• GitLab CI/CD
• Pipeline templates
• Deployment automation
• Environment promotion workflows

Programming

• Python
• Go

Data Engineering

• Data pipeline design
• Batch processing
• Streaming workloads
• Orchestration frameworks

Platform Engineering

• Reusable tooling
• Automation frameworks
• Developer enablement
• Self-service platforms

DevOps Practices

• Version control
• Release workflows
• Automated validation
• CI/CD governance