Kafka Platform Engineer

Onsite 4 days a week

Top 3 Must Have Skills:

• Experience with Confluent Kafka Platform
• Experience with both On-Prem and Cloud

Must have experience leveraging AI powered tools and building AI driven agents as as it relates to

• Design and implement secure Kafka architectures, including authentication, authorization, encryption, and network isolation.
• Define and support high availability, disaster recovery, and data replication strategies.
• Ensure alignment with enterprise security, compliance, and governance standards.

Data Streaming & Integration

• Advise on data ingestion, event streaming, batch and stream processing architectures.
• Provide guidance on data modeling and serialization standards (Avro, Protobuf, JSON Schema).
• Support enterprise integration use cases and resolve complex Kafka related integration challenges.

AI Development

• Design and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scale
• Leverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows aligned to AI-DLC vision.
• Design agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challenges
• Strong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integration to accelerate engineering
• Proven ability to leverage AI-assisted development (e.g., Copilot, code generation) and build AI-driven agents/workflows to improve team productivity and engineering efficiency

Automation & DevOps Enablement

• Drive automation first operations using Python, Ansible, and scripting.
• Build and maintain self-service operational tools to reduce day to day support overhead.
• Integrate Kafka platform operations with DevOps and CI/CD tooling (e.g., Azure, Jenkins, Puppet, Chef).

Incident Management & Problem Resolution

• Lead deep dive investigations into complex production issues and outages.
• Perform root cause analysis, document findings, and drive remediation to closure.
• Operate effectively in high pressure outage and incident management scenarios.

Advisory & Stakeholder Engagement

• Act as a trusted technical advisor to client engineering, operations, and leadership teams.
• Communicate complex technical concepts clearly to both technical and non technical stakeholders.
• Demonstrate strong customer focus, ownership, and onshore engagement.
• Collaborate effectively with cross functional teams and vendors.

Required Experience & Skills

• 10+ years of experience in distributed systems, messaging, and enterprise platform engineering.
• Extensive hands on experience with Confluent Kafka Platform (on prem and cloud).
• Strong background in Linux/Unix environments, networking, OS performance, and security.
• Proven experience with automation, DevOps, and operational tooling.
• Experience leveraging AI powered tools to accelerate development and improve team productivity and efficiency
• Excellent written and verbal communication skills.
• Strong organizational, relationship management, and teamwork skills.

Preferred Qualifications

• Confluent Kafka and/or Cloud certifications.
• Prior experience supporting large scale, regulated enterprise environments.