
Sunil Kumar
Senior Kafka & DevOps Engineer | Kubernetes
Salary / Daily rate
Dubai - United Arab Emirates
Freelance, open to permanent contract
Skills
Languages
About me
Intro
Senior DevOps/SRE Engineer with 15+ years in enterprise distributed systems and real-time data streaming. Expert in Kubernetes, Confluent Kafka, and ELK stack with proven success architecting multi-cloud banking platforms at Arab Bank and ADIB. I deliver production-grade infrastructure solutions that scale.
Work experience
Senior Distributed Platform Lead
Arab Bank
Sep 2023 - Present
3 years 2 months
Dubai - United Arab Emirates
• Kubernetes/Site Reliability Engineer (SRE)/DevOps Infrastructure & Automation o Designed and managed production-grade Kubernetes clusters across multi-cloud/hybrid environments o Automated provisioning using Helm, Kustomize, and Ansible o Implemented multi-zone/multi-region deployments for high availability o Configured HPA and Cluster Autoscaler for optimal resource utilization Observability & Reliability o Established monitoring stacks with Prometheus, Grafana, ELK/EFK, and OpenTelemetry o Defined SLOs/SLIs, conducted RCAs, and improved MTTR through incident management o Implemented service mesh (Istio/Linkerd) for traffic management and mTLS Security & Operations o Enforced security via RBAC, NetworkPolicies, and Vault integration o Built CI/CD pipelines (Jenkins, GitHub Actions, ArgoCD) for GitOps workflows o Configured disaster recovery with etcd backups and regular DR drills o Promoted DevOps culture across cross-functional teams • Confluent Kafka Administrator/SME Platform Architecture o Architected multi-datacenter Kafka stretch cluster (v2.5 to v7.5.1) o Deployed core Confluent components: REST Proxy, Connect, KSQL, Schema Registry o Integrated digital banking systems for real-time streaming o Designed CDC solutions for DB2, Oracle, SQL Server using Informatica Power Exchange Operations & Optimization o Secured infrastructure with SSL/mTLS across all components o Built high-throughput pipelines for critical banking operations o Managed connectors: HDFS, Snowflake, JDBC, CDC o Implemented monitoring (Grafana/Prometheus) for Kafka/Elasticsearch Governance & Support o Established data governance and compliance standards o Provided 24/7 platform support and performance tuning o Developed operational best practices and naming conventions o Enabled real-time analytics through ESB-Kafka-MongoDB integration
Senior Consultant
ADIB - Abu Dhabi Islamic Bank
Nov 2021 - Oct 2023
2 years
United Arab Emirates · On-site
• Kafka Administrator / SME Data Pipeline Architecture o Designed and maintained real-time and batch pipelines using Confluent Kafka & Hadoop ecosystem o Ensured data quality through robust validation and error handling o Architected data warehouse solutions for BI, reporting, and analytics o Optimized microservices performance with scalable data patterns Kafka Integration & Development o Expert in Kafka connectors: HDFS Sink, CDC Source, ADLS Sink, MSSQL Sink/Source, Elastic Sink/Source, Snowflake Sink o Developed Kafka Producer/Consumer apps (Python) for efficient data streaming o Built real-time analytics dashboards using Kafka Streams, KSQL, and Power BI o Migrated Redis Streams → Kafka Streams for improved scalability o Integrated ESB with Kafka to stream logs into Azure Data Lake Storage o Automated ETL workflows in CDP NiFi using Kafka Producers/Consumers Cluster Deployment & Documentation o Deployed high-availability Kafka clusters (multi-DC, 3-DC stretch configurations) o Translated business requirements (BRDs) into scalable real-time Kafka solutions o Delivered Solution Architecture (SAD) and Functional Specification (FSD) documents
Senior Application Analyst
Verifone
Jun 2019 - Dec 2021
2 years 7 months
Bengaluru, Karnataka, India
• Kafka Administrator / SME Kafka Infrastructure & Architecture o Designed and deployed multi-datacenter Kafka clusters ensuring 100% high availability o Implemented Active-Active DR strategies for business continuity o Optimized Confluent Kafka and MongoDB clusters for performance at scale o Integrated Redis with Kafka using Kafka RedisSink connector Streaming & Integration o Developed real-time streaming applications using KSQL and Kafka Connect o Built RESTful APIs (Python/Flask) for data access and analytics o Created Kafka-MongoDB integrations to expose data via APIs o Established database integrations (SQL/NoSQL) via Kafka Connect Operations & Security o Provided production support for Kafka and Cassandra environments o Enhanced security with LDAP/RBAC integration for Kafka management o Authored architecture documentation (SAD/FSD/TDD/BDD) o Delivered enterprise-grade, maintainable solutions o Kong & Redis Administrator • Kong Gateway Management o Deployed and maintained Kong Gateway (OSS/Enterprise) across hybrid environments o Implemented security policies (TLS, OAuth2, JWT, ACLs) o Monitored performance using Prometheus/Grafana o Executed zero-downtime upgrades and scaling operations • Elasticsearch Operations o Managed Elasticsearch clusters (on-prem/cloud) o Optimized performance (sharding, JVM tuning) o Implemented backup/restore to S3/NFS o Secured clusters (TLS, RBAC, IP filtering) o Built log pipelines (Filebeat→Logstash)
Senior Technical Specialist
Zensar Technologies
Aug 2017 - Jul 2019
2 years
Bengaluru, Karnataka, India
• Kafka & Cassandra SME / Big Data Engineer Data Pipeline & Infrastructure Management o Designed end-to-end data pipelines using Hadoop, Hive, Kafka, and Cassandra to meet business needs. o Managed high-availability infrastructure (DSE Cassandra, Solr, Linux/Unix, VMware). o Automated workflows (Python, Shell, PowerShell) to enhance operational efficiency. o Deployed and maintained multi-node Kafka clusters across Dev, QA, and Production. o Built scalable storage solutions (Cassandra + Solr) for analytics and lab environments. Performance & Optimization o Conducted NoSQL/RDBMS benchmarking to select optimal database solutions. o Developed high-throughput ETL pipelines (Kafka, Python, Elasticsearch, Cassandra) processing 50M+ records/day. o Scaled VM orchestration & big data automation for improved infrastructure agility. Leadership & Documentation o Led requirement analysis, solution design, and automation tooling for enterprise applications. o Delivered technical documentation and production-grade modules adhering to enterprise standards. o Performed large-scale data mining and generated executive-level insights. • ELK Administrator / DevOps Engineer Elasticsearch Cluster Management o Installed, configured, and maintained Elasticsearch clusters (on-prem/cloud). o Optimized performance (sharding, indexing, JVM tuning) and scaled nodes dynamically. o Secured clusters (TLS, RBAC, audit logging, IP filtering). o Executed zero-downtime upgrades and backup/restore (S3/NFS snapshots). Logging & Analytics Pipelines o Built log ingestion pipelines (Filebeat → Logstash) with parsing (Grok, JSON, mutations). o Integrated Kafka/RabbitMQ for buffering and reliability. o Designed Kibana dashboards for operational/business metrics. o Configured alerting (Elastic Alerting/Watcher) and managed multi-tenancy (spaces/saved objects).
Technical Consultant
Spineor Webservices Pvt. Ltd.
Nov 2016 - Aug 2017
10 months
Navi Mumbai, Maharashtra, India
• Cassandra & Big Data Expertise Cluster Design & Management: o Deployed multi-data center DSE Cassandra clusters for high availability (HA) and disaster recovery (DR). o Managed multi-node Cassandra clusters, including zero-downtime operations (node scaling, upgrades, decommissions). o Automated backup, restore, and recovery to ensure data integrity and uptime. o Used DataStax OpsCenter for centralized monitoring and administration. Performance & Optimization: o Tuned Cassandra clusters (nodetool operations: repair, flush, compact, snapshot). o Diagnosed and resolved performance bottlenecks (query optimization, data modeling). o Deep expertise in Cassandra’s read/write paths for architectural decisions. Data Modeling & Integration: o Designed scalable schemas using DataStax Studio. o Built Python-based reporting tools for dynamic business insights. o Integrated Cassandra with Kafka & Python for distributed data solutions. o Developed Dockerized Cassandra for multi-node testing. Capacity Planning & Analytics: o Led hardware sizing, capacity planning, and data distribution strategies. o Configured Elasticsearch & Kibana for search and analytics.
Technical Lead
Wipro Technologies
Sep 2011 - Dec 2016
5 years 4 months
Bengaluru, Karnataka, India
• Proficient in developing SQL queries, stored procedures, functions, and triggers using DB2, with hands-on experience working with relational and NoSQL databases such as MySQL and Cassandra. • Experienced in Apache Spark, Python, HDFS, and Hive for big data processing and analysis. • Built and configured Elasticsearch clusters to support scalable search and analytics use cases. • Designed and implemented ETL workflows using Sqoop, Flume, Hive, and Oozie Workflow Designer, and automated these processes using Oozie Coordinators. • Strong understanding of YARN architecture, including components like Resource Manager, Node Manager, Application Manager, and Containers. • Developed custom scripts to ingest data into HDFS from various structured and unstructured data sources. • Utilized Sqoop to import/export data between relational databases and NoSQL systems into the Hadoop ecosystem. • Created Python scripts to automate manual system tasks, improving operational efficiency. • Developed Python-Cassandra integration programs to generate dynamic business reports. • Solid understanding of the Hadoop ecosystem, including components like Job Tracker, Task Tracker, NameNode, and DataNode, with practical experience in managing and utilizing Hadoop clusters.
IBM
Jan 2005 - Oct 2011
6 years 10 months
Senior Software Engineer
Oct 2005 - Oct 2011
6 years 1 month
Mainframe Expertise: Proficient in COBOL, DB2, JCL, and CICS for enterprise applications, including complex SQL query development. SDLC & Agile: Hands-on experience with Agile/Scrum and Waterfall methodologies. Client & Stakeholder Management: Skilled in requirement gathering, communication, and progress updates. DB2 Database Administration: Utilized LOAD, COPY, UNLOAD, and REPAIR utilities for data migration between systems and flat files. Production Support: Resolved bugs and issues to ensure system stability. Technical Leadership: Led projects, governed deliverables via WSRs, and conducted PRMs with senior management.
Application Developer
Jan 2005 - Jan 2011
6 years 1 month
Education
Indira Gandhi National Open University
Master’s Degree
2001 - 2003
2 years 1 month
Magadh University
Bachelor’s Degree
1993 - 1996
3 years 1 month
Bihar Intermediate Education Council
High School
1990 - 1992
2 years 1 month