Senior Data Engineer - Data Lakehouse

About the role
This is a hands-on, senior individual-contributor role with a strong passion on building a possible vendor neutral lakehouse. You will own data pipelines end to end and build out the governance and data-quality layer that a fast-growing platform now needs.

Our stack (so you know what you'd be working with)

• Ingestion / streaming: Debezium CDC → Apache Kafka
• Lakehouse: Apache Iceberg tables on S3, Apache Polaris (Iceberg REST catalog); Kafka Connect and Apache Flink write into it.
• Processing: Apache Flink (real-time upserts / streaming) and Apache Spark (batch silver/gold), both reading and writing Iceberg. Serving / query: ClickHouse
• Orchestration: Dagster.
• Platform / IaC: AWS, Amazon EKS (Kubernetes), Terraform, Helm; observability via Prometheus, Grafana, and Alertmanager.

We value experience with equivalent tools too — if you've done Delta/Hudi instead of Iceberg, Kinesis instead of Kafka, or Airflow instead of Dagster, that transfers.

Key responsibilities

• Own, operate and harden the streaming platform for reliability, correctness, and cost at scale.
• Build and tune the processing layers: batch transformations in Spark (silver/gold models) and real-time streaming jobs in Flink, including partitioning, compaction, and job/state tuning.
• Develop and optimize the serving layer (ClickHouse over Iceberg) so analysts and product teams get fast, reliable query access.
• Design data models and physical table layout (partitioning, sort order, compaction, retention) to balance write throughput, query performance, and storage cost across the lakehouse.
• Establish and enforce data governance: catalog organization, access control, data lineage, retention/compliance, and automated data-quality checks.
• Collaborate with data analysts, product teams, and backend engineers to ensure discoverable, well-documented, and usable data access.
• Stay current with the lakehouse ecosystem and propose targeted improvements.

Requirements

• 6+ years in data engineering or backend data-intensive roles.
• Deep, hands-on experience on data engineering practices.
• Production experience with an open table format / lakehouse.
• Strong distributed compute: Apache Spark and/or Apache Flink for batch and/or stream processing, including performance tuning is a plus.
• Expert SQL, dimensional/data modeling, and query performance tuning — including experience with an analytical/columnar store (e.g., ClickHouse, Redshift, Snowflake, BigQuery).
• Solid AWS fundamentals: S3, IAM, VPC/networking, and RDS.
• A production-reliability mindset: monitoring, alerting, on-call, and incident root-cause analysis.
• Proficiency in at least one of Python, Java, or Scala for pipeline and streaming code.
• Excellent communication and stakeholder-management skills.

Nice to have

• Real-time OLAP experience (ClickHouse, Apache Druid, Apache Pinot).
• Workflow orchestration (Dagster, Airflow, or equivalent).
• Multi-region and/or cross-account data platform experience.
• Data governance, lineage, and quality tooling (OpenLineage/Marquez, Great Expectations, dbt tests, catalog-based access control).
• Security and compliance frameworks (GDPR, HIPAA, SOC 2, etc.).
• Observability stacks (Prometheus, Grafana, Alertmanager).
• Cost optimization for cloud data/streaming platforms.