Senior Data Platform Engineer

We are seeking a Data Platform Engineer to manage and scale our core data infrastructure while empowering engineering and Data Science teams with reliable, streamlined processes. In this role, you will maintain distributed big data systems, drive operational excellence, and ensure robust data governance and observability across our production ecosystem.

This is a long-term, contractor position.

Working hours: Overlap until 11 am PST time zone. Responsibilities

• Manage, maintain, and scale our on-premise Kubernetes clusters, Airflow orchestrations, and distributed Spark infrastructure.
• Build and maintain robust CI/CD pipelines to streamline deployment, data governance, and authentication protocols.
• Perform routine file compaction, storage optimization, and maintenance across HDFS and cloud object stores.
• Maintain system observability, monitor health, and participate in an on-call rotation using Prometheus, Grafana, and OpenTelemetry.
• Apply core data engineering principles to optimize table formats (Iceberg/Hive), partitioning, schema design, and query performance.

Requirements

• Proven background in distributed systems with deep expertise in managing Kubernetes resources in production.
• Hands-on experience scaling and managing infrastructure for heavy Apache Spark workloads.
• Strong proficiency in modern CI/CD pipelines, Docker containerization, and data pipeline orchestration (Python/Airflow).
• Deep understanding of data engineering fundamentals, including schema design, partitioning, type safety, and modern table formats (Iceberg/Hive/Trino/Presto).
• Practical experience implementing authentication, access control, and data governance frameworks.
• Solid experience with observability, monitoring, and on-call practices using Prometheus, Grafana, and OpenTelemetry.

Nice to have

• Hands-on experience with Scala for big data processing systems.

We offer

• Opportunity to work on bleeding-edge projects
• Work with a highly motivated and dedicated team
• Competitive salary
• Flexible schedule
• Professional development opportunities

About Us Grid Dynamics (NASDAQ: GDYN) is a leading provider of technology consulting, platform and product engineering, AI, and advanced analytics services. Fusing technical vision with business acumen, we solve the most pressing technical challenges and enable positive business outcomes for enterprise companies undergoing business transformation. A key differentiator for Grid Dynamics is our 8 years of experience and leadership in enterprise AI, supported by profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization and customer experience. Founded in 2006, Grid Dynamics is headquartered in Silicon Valley with offices across the Americas, Europe, and India.