Senior Data Engineer (GCP - Python - Iceberg - Delta Lake - Kafka - Snowflake - Databricks)

For our client, we are seeking a Senior Data Engineer (GCP - Python - Iceberg - Delta Lake - Kafka - Snowflake - Databricks) to join the team of a leader in the Enterprise Technology space.

This role will focus on building and optimizing modern data infrastructure that supports reliable ingestion, sharing, and access across the enterprise. You will work across engineering and data teams to design scalable pipelines, strengthen data lineage, and enable efficient use of cloud-based analytics platforms. The position is well suited for someone who enjoys solving complex data architecture challenges and improving the way data moves and is consumed across the organization.

Location: Remote - US based candidates only, no visa sponsorship available

Compensation: $119,998 – $180,000 annually

Responsibilities

• Design and implement the UniForm write layer for dual metadata management
• Build ingestion pipelines for operational datasets between GCS and BigQuery
• Develop and implement data processes using Python and Spark
• Implement Kafka-based change data capture (CDC) patterns for real-time ingestion
• Develop data lineage and modular adapter code for reusability
• Configure Snowflake Horizon external tables for efficient access
• Collaborate with cross-functional teams to deliver comprehensive data features

Qualifications

• 6+ years of enterprise-level experience in Python and Spark
• Advanced expertise with GCP BigQuery
• Strong knowledge of Apache Iceberg and Delta Lake
• Experience with Delta Sharing and Kafka/CDC pipelines
• Hands-on experience with Snowflake Horizon and Databricks Unity Catalog
• Solid background in data lake architecture and ingestion pipeline design
• Excellent communication skills and ability to collaborate effectively

Benefits

• Collaborative work environment with cross-functional teams
• Opportunity to work on cutting-edge data engineering technologies
• Focus on building reusable, efficient data-sharing solutions
• Engagement in real-time data ingestion processes
• Potential exposure to enterprise SaaS and media/advertising sectors
• Use of AI-assisted development tools to enhance productivity

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.