Kafka Developer/Application Developer

We're Hiring: Kafka to AWS MSK Migration Engineer

Are you passionate about distributed messaging systems, cloud migration, and automation? Join our team to help drive large-scale Kafka modernization initiatives and migrate enterprise workloads from on-premises Kafka environments to AWS Managed Streaming for Apache Kafka (MSK).

Key Responsibilities

• Assess on-premises Kafka clusters and perform discovery across topics, partitions, consumer groups, retention policies, throughput patterns, and dependencies to evaluate migration readiness.
• Analyze workloads based on message volume, latency, ordering requirements, business criticality, and consumer group complexity.
• Plan, provision, and configure AWS MSK infrastructure, including broker sizing, storage, networking, and security controls.
• Execute UAT and production migrations using MirrorMaker 2 or similar replication tools.
• Validate migration outcomes, troubleshoot issues, and ensure successful cutovers.
• Perform performance tuning and optimization for scalability, reliability, and production readiness.
• Collaborate with application, infrastructure, and platform teams on migration planning, cutover strategies, validation criteria, and rollback procedures.
• Manage consumer group migrations, offset synchronization, and post-migration validation.
• Support decommissioning of legacy Kafka environments after successful migration.
• Develop automation frameworks and AI-assisted solutions to streamline and scale migration activities.
• Create comprehensive documentation covering procedures, risks, lessons learned, and best practices.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related field.
• 3-5+ years of hands-on experience with Apache Kafka and distributed messaging systems.
• Strong programming skills in Python, Java, or similar languages.
• Solid Unix/Linux administration and troubleshooting experience.
• Understanding of SDLC, CI/CD processes, change management, and Kubernetes-based deployments.

Technical Skills

• AWS Services: MSK, ECS, EKS, Lambda, CloudWatch, and related cloud services.
• Apache Kafka: Brokers, topics, partitions, consumer groups, replication, and cluster administration.
• AWS MSK: Provisioning, monitoring, security, performance tuning, and capacity planning.
• Migration Tools: MirrorMaker 2, Confluent Replicator, or equivalent technologies.
• Kafka Ecosystem: Kafka Connect, Schema Registry, Kafka Streams.
• Networking & Security: IAM, TLS/mTLS, SASL/SCRAM, VPC Peering, PrivateLink, and hybrid connectivity.
• Automation: Python, Java, Shell scripting, and automation frameworks.
• CI/CD & Version Control: GitLab, Jenkins, Maven, or similar tools.
• Observability: CloudWatch, Prometheus, Grafana, Splunk, or equivalent monitoring platforms.
• APIs & Integration: REST APIs and API Gateway concepts.

What We're Looking For

• Strong analytical, troubleshooting, and debugging capabilities.
• Ownership mindset with a focus on delivering high-quality outcomes.
• Excellent communication and stakeholder management skills.
• Ability to collaborate effectively across global engineering teams.
• Continuous improvement mindset with a passion for automation and innovation.
• Ability to work in a fast-paced environment and manage critical production migrations.