Real time data & No SQL

6 Days working from Office

Real time data & NoSQL

1. We are seeking a Real-Time Data & NoSQL Engineer to design, build and optimize high throughput, low latency data platform.

2. Focus on data streaming, caching layer and unstructured storage using Apache Kafka, Redis and MongoDB.

Core Responsibilities: • Event Streaming & Messaging (Apache Kafka)

• Design, implement and maintain high throughput Kafka topics, partitions and consumer groups.
• Ensure message delivery guarantees, manage schema evolution using Confluent Schema registry and tune broker/producer configurations for optimal performance. Distributed Caching & In-Memory Storage (Redis)
• Design and implement advanced caching patterns to reduce primary database load and latency.
• Utilize Redis data structures (Hashes, Lists, Sets, Sorted Sets, HyperLoglogs, Geospatial) to solve complex application logic at sub-millisecond speeds.
• Configure and manage Redis clusters, sentinel deployments, pub/sub messaging channels and persistent storage configurations.

NoSQL Document Modeling and Scaling(MongoDB)

• Architect, optimize and maintain large scale MongoDB databases including sharded clusters and replica sets.
• Design efficient BSON document schemas, avoiding anti patterns like unbounded arrays or excessive nesting.
• Perform advanced query optimization, build targeted compound/partial indexes and write complex Aggregation Framework pipelines.

Performance Tuning and System modernization

• Profile end to end data flows from Kafka producers through Redis caching layers down to MongoDB persistent storage.
• Troubleshoot bottleneck issues such as consumer lag, cache stampedes, slow queries and memory fragmentation.
• Implement modern data practises.

Soft Skills • Proven track record of executing at least 2 major enterprise-scale database modernization.

• Problem Solving - Exceptional engineering mindset focused on code efficiency.
• Collaboration- Strong ability to work alongside Application developers, DevOps Engineers etc. to coordinate seamless system cutovers.