Data Engineering manager

Job Overview

We are looking for a Data Engineering Manager with 1015 years of experience to lead data engineering initiatives, build scalable data pipelines, manage data lake architecture, and enable real-time analytics and AI use cases. The role will work closely with application, ML, analytics, and business teams while mentoring data engineers and driving modern engineering practices.

Key Responsibilities

• Own the data engineering roadmap from streaming ingestion to analytics-ready data layers.
• Lead real-time pipelines using Kafka/MSK, Debezium CDC, and Apache Pinot.
• Design and maintain Java/Python-based frameworks for ingestion, validation, transformation, and replication.
• Design and manage data lakes using Apache Iceberg, AWS Glue, Athena, and S3.
• Define data modeling standards across Pinot, Redshift, and warehouse layers.
• Lead orchestration and workflow platforms using Apache Airflow and Superset.
• Drive cloud-native deployments using ECS/EKS, Docker, Terraform/CDK, and CI/CD.
• Support ML, analytics, product, and AI initiatives with reliable data solutions.
• Ensure data quality, observability, lineage, governance, and security across the data stack.
• Mentor data engineers and promote modern architecture and engineering best practices.

Required Skills

• Real-Time Data: Kafka/MSK, Debezium, Event-Driven Architecture, CDC
• Data Platforms: Apache Pinot, Redshift, RocksDB/NoSQL, Apache Iceberg, Athena, AWS Glue
• Programming: Java, Python
• Orchestration: Apache Airflow, Superset
• AWS: S3, IAM, Lambda, Glue, ECS/EKS, CloudWatch
• Data Architecture: Data Lakes, Streaming & Batch Processing, Data Modeling, Parquet, Avro
• DevOps: Docker, Terraform/CDK, CI/CD, GitHub Actions

Good to Have

• dbt, Trino/Presto, ClickHouse, or Druid.
• Data security, encryption at rest/in transit, GDPR/PCI compliance.
• CloudFormation and advanced DevOps practices.
• Experience with scalable cloud-native data architectures.