Kafka Data Engineer

Job Summary

We are looking for an experienced Kafka Data Engineer with strong hands-on expertise in Apache Kafka, Kafka Administration, Topic Migration, Cluster Migration, Data Streaming, and Distributed Systems. The candidate will be responsible for planning and executing Kafka migrations, cross-cluster replication, data validation, performance tuning, and ensuring minimal downtime.

Responsibilities

• The role requires hands-on experience with Kafka Topic Migration and cluster-to-cluster replication using MirrorMaker 2.0, Confluent Replicator, or Cluster Linking.
• Strong knowledge of Kafka Connect, Kafka Streams, Schema Registry, and Confluent Platform is mandatory.
• The candidate should understand Kafka brokers, partitions, replication, consumer groups, offsets, ACLs, consumer lag, throughput, and latency.
• Experience with Linux/Unix and proficiency in Python, Java, or Scala is required.
• Candidates should have a strong understanding of distributed systems, event-driven architecture, and messaging systems.
• Exposure to cloud platforms such as AWS MSK, Azure Event Hubs/Kafka, or GCP Pub/Sub is preferred.
• Knowledge of Terraform, CI/CD, Docker, Kubernetes, Databricks, Spark Streaming, or Flink will be an added advantage.

Skills & Competencies

• Apache Kafka
• MirrorMaker 2.0
• Confluent Replicator
• Cluster Linking
• Kafka Connect
• Kafka Streams
• Schema Registry
• Confluent Platform
• AWS MSK