Confluent Flink Engineer

Job Title: Confluent Flink Engineer

Experience: 6+ Years
Location: Remote
Shift Timings: 2:00 PM - 11:00 PM IST

Role Overview

We are seeking an experienced Confluent Flink Engineer with strong expertise in real-time data streaming, event-driven architectures, and large-scale distributed systems. The ideal candidate should have hands-on experience with Apache Flink, Confluent Kafka, Kafka Streams, and streaming ETL pipelines, along with a strong understanding of cloud-native data processing solutions.

The role involves designing, developing, and optimizing real-time streaming applications that process high-volume data efficiently while ensuring scalability, fault tolerance, and performance.

Key ResponsibilitiesStream Processing Data Engineering

• Design, develop, and maintain real-time data processing applications using Apache Flink.
• Build and manage large-scale streaming ETL pipelines.
• Develop event-driven architectures using Confluent Kafka and Kafka Streams.
• Implement real-time data transformation, aggregation, enrichment, and analytics workflows.

Kafka Confluent Platform

• Configure and manage Confluent Kafka environments.
• Work with Kafka Connect, Schema Registry, and other Confluent Platform components.
• Design efficient topic structures, partitioning strategies, and consumer group architectures.
• Ensure reliable event processing and message delivery.

Performance Scalability

• Optimize Flink jobs for performance, throughput, and resource utilization.
• Implement checkpointing, state management, and fault-tolerant processing strategies.
• Monitor and troubleshoot streaming applications and data pipelines.
• Ensure scalability and high availability of streaming platforms.

Cloud DevOps

• Deploy and manage streaming workloads on AWS, Azure, or GCP.
• Participate in CI/CD pipeline implementation and automation.
• Collaborate with DevOps teams to improve deployment, monitoring, and operational practices.

Collaboration

• Work closely with Data Engineers, Architects, Product Teams, and Business Stakeholders.
• Participate in architecture discussions and technical design reviews.
• Contribute to best practices, documentation, and knowledge sharing.

Required Skills Experience

• 6+ years of software engineering or data engineering experience.
• Strong hands-on experience with Apache Flink.
• Extensive experience with Confluent Kafka and Kafka Streams.
• Strong understanding of real-time streaming and event-driven architectures.
• Experience with distributed systems and large-scale data processing.
• Proficiency in Java, Scala, or Python.
• Experience building and managing streaming ETL pipelines.
• Strong knowledge of:
  • Kafka Connect
  • Schema Registry
  • Confluent Platform Components
• Strong understanding of:
  • Data Partitioning
  • State Management
  • Checkpointing
  • Fault Tolerance
• Experience with AWS, Azure, or GCP cloud platforms.
• Familiarity with CI/CD pipelines and DevOps practices.

Preferred Skills

• Experience with Kubernetes and containerized deployments.
• Knowledge of data lake and modern data platform architectures.
• Exposure to monitoring tools such as Prometheus, Grafana, or Datadog.
• Experience with Spark Streaming or other stream processing technologies.

Key Competencies

• Real-Time Data Streaming
• Apache Flink Development
• Confluent Kafka Kafka Streams
• Distributed Systems
• Event-Driven Architecture
• Streaming ETL Pipelines
• Cloud Data Engineering
• Performance Optimization
• CI/CD DevOps

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.