Sunil Kumar

Senior Kafka & DevOps Engineer | Kubernetes

Salary / Daily rate

Dubai - United Arab Emirates

Freelance, open to permanent contract

Skills

DevOpsKubernetesElasticsearch

Languages

EnglishHindi

About me

Intro

Senior DevOps/SRE Engineer with 15+ years in enterprise distributed systems and real-time data streaming. Expert in Kubernetes, Confluent Kafka, and ELK stack with proven success architecting multi-cloud banking platforms at Arab Bank and ADIB. I deliver production-grade infrastructure solutions that scale.

Links

Work experience

Senior Distributed Platform Lead

Arab Bank

Sep 2023 - Present

3 years 2 months

Dubai - United Arab Emirates

• Kubernetes/Site Reliability Engineer (SRE)/DevOps  Infrastructure & Automation o Designed and managed production-grade Kubernetes clusters across multi-cloud/hybrid environments o Automated provisioning using Helm, Kustomize, and Ansible o Implemented multi-zone/multi-region deployments for high availability o Configured HPA and Cluster Autoscaler for optimal resource utilization  Observability & Reliability o Established monitoring stacks with Prometheus, Grafana, ELK/EFK, and OpenTelemetry o Defined SLOs/SLIs, conducted RCAs, and improved MTTR through incident management o Implemented service mesh (Istio/Linkerd) for traffic management and mTLS  Security & Operations o Enforced security via RBAC, NetworkPolicies, and Vault integration o Built CI/CD pipelines (Jenkins, GitHub Actions, ArgoCD) for GitOps workflows o Configured disaster recovery with etcd backups and regular DR drills o Promoted DevOps culture across cross-functional teams • Confluent Kafka Administrator/SME  Platform Architecture o Architected multi-datacenter Kafka stretch cluster (v2.5 to v7.5.1) o Deployed core Confluent components: REST Proxy, Connect, KSQL, Schema Registry o Integrated digital banking systems for real-time streaming o Designed CDC solutions for DB2, Oracle, SQL Server using Informatica Power Exchange  Operations & Optimization o Secured infrastructure with SSL/mTLS across all components o Built high-throughput pipelines for critical banking operations o Managed connectors: HDFS, Snowflake, JDBC, CDC o Implemented monitoring (Grafana/Prometheus) for Kafka/Elasticsearch  Governance & Support o Established data governance and compliance standards o Provided 24/7 platform support and performance tuning o Developed operational best practices and naming conventions o Enabled real-time analytics through ESB-Kafka-MongoDB integration

Senior Consultant

ADIB - Abu Dhabi Islamic Bank

Nov 2021 - Oct 2023

2 years

United Arab Emirates · On-site

• Kafka Administrator / SME  Data Pipeline Architecture o Designed and maintained real-time and batch pipelines using Confluent Kafka & Hadoop ecosystem o Ensured data quality through robust validation and error handling o Architected data warehouse solutions for BI, reporting, and analytics o Optimized microservices performance with scalable data patterns  Kafka Integration & Development o Expert in Kafka connectors: HDFS Sink, CDC Source, ADLS Sink, MSSQL Sink/Source, Elastic Sink/Source, Snowflake Sink o Developed Kafka Producer/Consumer apps (Python) for efficient data streaming o Built real-time analytics dashboards using Kafka Streams, KSQL, and Power BI o Migrated Redis Streams → Kafka Streams for improved scalability o Integrated ESB with Kafka to stream logs into Azure Data Lake Storage o Automated ETL workflows in CDP NiFi using Kafka Producers/Consumers  Cluster Deployment & Documentation o Deployed high-availability Kafka clusters (multi-DC, 3-DC stretch configurations) o Translated business requirements (BRDs) into scalable real-time Kafka solutions o Delivered Solution Architecture (SAD) and Functional Specification (FSD) documents

Senior Application Analyst

Verifone

Jun 2019 - Dec 2021

2 years 7 months

Bengaluru, Karnataka, India

• Kafka Administrator / SME  Kafka Infrastructure & Architecture o Designed and deployed multi-datacenter Kafka clusters ensuring 100% high availability o Implemented Active-Active DR strategies for business continuity o Optimized Confluent Kafka and MongoDB clusters for performance at scale o Integrated Redis with Kafka using Kafka RedisSink connector  Streaming & Integration o Developed real-time streaming applications using KSQL and Kafka Connect o Built RESTful APIs (Python/Flask) for data access and analytics o Created Kafka-MongoDB integrations to expose data via APIs o Established database integrations (SQL/NoSQL) via Kafka Connect  Operations & Security o Provided production support for Kafka and Cassandra environments o Enhanced security with LDAP/RBAC integration for Kafka management o Authored architecture documentation (SAD/FSD/TDD/BDD) o Delivered enterprise-grade, maintainable solutions o Kong & Redis Administrator • Kong Gateway Management o Deployed and maintained Kong Gateway (OSS/Enterprise) across hybrid environments o Implemented security policies (TLS, OAuth2, JWT, ACLs) o Monitored performance using Prometheus/Grafana o Executed zero-downtime upgrades and scaling operations • Elasticsearch Operations o Managed Elasticsearch clusters (on-prem/cloud) o Optimized performance (sharding, JVM tuning) o Implemented backup/restore to S3/NFS o Secured clusters (TLS, RBAC, IP filtering) o Built log pipelines (Filebeat→Logstash)

Senior Technical Specialist

Zensar Technologies

Aug 2017 - Jul 2019

2 years

Bengaluru, Karnataka, India

• Kafka & Cassandra SME / Big Data Engineer  Data Pipeline & Infrastructure Management o Designed end-to-end data pipelines using Hadoop, Hive, Kafka, and Cassandra to meet business needs. o Managed high-availability infrastructure (DSE Cassandra, Solr, Linux/Unix, VMware). o Automated workflows (Python, Shell, PowerShell) to enhance operational efficiency. o Deployed and maintained multi-node Kafka clusters across Dev, QA, and Production. o Built scalable storage solutions (Cassandra + Solr) for analytics and lab environments.  Performance & Optimization o Conducted NoSQL/RDBMS benchmarking to select optimal database solutions. o Developed high-throughput ETL pipelines (Kafka, Python, Elasticsearch, Cassandra) processing 50M+ records/day. o Scaled VM orchestration & big data automation for improved infrastructure agility.  Leadership & Documentation o Led requirement analysis, solution design, and automation tooling for enterprise applications. o Delivered technical documentation and production-grade modules adhering to enterprise standards. o Performed large-scale data mining and generated executive-level insights. • ELK Administrator / DevOps Engineer  Elasticsearch Cluster Management o Installed, configured, and maintained Elasticsearch clusters (on-prem/cloud). o Optimized performance (sharding, indexing, JVM tuning) and scaled nodes dynamically. o Secured clusters (TLS, RBAC, audit logging, IP filtering). o Executed zero-downtime upgrades and backup/restore (S3/NFS snapshots).  Logging & Analytics Pipelines o Built log ingestion pipelines (Filebeat → Logstash) with parsing (Grok, JSON, mutations). o Integrated Kafka/RabbitMQ for buffering and reliability. o Designed Kibana dashboards for operational/business metrics. o Configured alerting (Elastic Alerting/Watcher) and managed multi-tenancy (spaces/saved objects).

Technical Consultant

Spineor Webservices Pvt. Ltd.

Nov 2016 - Aug 2017

10 months

Navi Mumbai, Maharashtra, India

• Cassandra & Big Data Expertise  Cluster Design & Management: o Deployed multi-data center DSE Cassandra clusters for high availability (HA) and disaster recovery (DR). o Managed multi-node Cassandra clusters, including zero-downtime operations (node scaling, upgrades, decommissions). o Automated backup, restore, and recovery to ensure data integrity and uptime. o Used DataStax OpsCenter for centralized monitoring and administration.  Performance & Optimization: o Tuned Cassandra clusters (nodetool operations: repair, flush, compact, snapshot). o Diagnosed and resolved performance bottlenecks (query optimization, data modeling). o Deep expertise in Cassandra’s read/write paths for architectural decisions.  Data Modeling & Integration: o Designed scalable schemas using DataStax Studio. o Built Python-based reporting tools for dynamic business insights. o Integrated Cassandra with Kafka & Python for distributed data solutions. o Developed Dockerized Cassandra for multi-node testing.  Capacity Planning & Analytics: o Led hardware sizing, capacity planning, and data distribution strategies. o Configured Elasticsearch & Kibana for search and analytics.

Technical Lead

Wipro Technologies

Sep 2011 - Dec 2016

5 years 4 months

Bengaluru, Karnataka, India

• Proficient in developing SQL queries, stored procedures, functions, and triggers using DB2, with hands-on experience working with relational and NoSQL databases such as MySQL and Cassandra. • Experienced in Apache Spark, Python, HDFS, and Hive for big data processing and analysis. • Built and configured Elasticsearch clusters to support scalable search and analytics use cases. • Designed and implemented ETL workflows using Sqoop, Flume, Hive, and Oozie Workflow Designer, and automated these processes using Oozie Coordinators. • Strong understanding of YARN architecture, including components like Resource Manager, Node Manager, Application Manager, and Containers. • Developed custom scripts to ingest data into HDFS from various structured and unstructured data sources. • Utilized Sqoop to import/export data between relational databases and NoSQL systems into the Hadoop ecosystem. • Created Python scripts to automate manual system tasks, improving operational efficiency. • Developed Python-Cassandra integration programs to generate dynamic business reports. • Solid understanding of the Hadoop ecosystem, including components like Job Tracker, Task Tracker, NameNode, and DataNode, with practical experience in managing and utilizing Hadoop clusters.

IBM

Jan 2005 - Oct 2011

6 years 10 months

Senior Software Engineer

Oct 2005 - Oct 2011

6 years 1 month

 Mainframe Expertise: Proficient in COBOL, DB2, JCL, and CICS for enterprise applications, including complex SQL query development.  SDLC & Agile: Hands-on experience with Agile/Scrum and Waterfall methodologies.  Client & Stakeholder Management: Skilled in requirement gathering, communication, and progress updates.  DB2 Database Administration: Utilized LOAD, COPY, UNLOAD, and REPAIR utilities for data migration between systems and flat files.  Production Support: Resolved bugs and issues to ensure system stability.  Technical Leadership: Led projects, governed deliverables via WSRs, and conducted PRMs with senior management.

Application Developer

Jan 2005 - Jan 2011

6 years 1 month


Education

Indira Gandhi National Open University

Master’s Degree

2001 - 2003

2 years 1 month

Magadh University

Bachelor’s Degree

1993 - 1996

3 years 1 month

Bihar Intermediate Education Council

High School

1990 - 1992

2 years 1 month

The best place to find your next move

© 2026. All Rights Reserved.