Data Engineer / Senior Data Streaming Engineer ( Apache Flink & Spark)

Required Technical Skills

• Apache Flink
• Strong hands-on experience in Apache Flink DataStream API.
• Expertise in Apache Flink Table API & SQL API.
• Experience designing and implementing: o Stateful stream processing o Event-time processing o Windowing operations o Checkpointing and fault tolerance o Watermarks and late event handling
• Knowledge of Flink deployment and operations in distributed environments.
• Apache Spark
• Strong proficiency in: o Spark Core o Spark SQL o Structured Streaming
• Experience developing high-volume ETL and data transformation pipelines.
• Performance tuning and optimization of Spark jobs.
• Knowledge of Spark execution architecture and resource management.

Responsibilities

• Design, develop, and maintain scalable real-time data processing pipelines.
• Build streaming applications using Apache Flink and Spark Structured Streaming.
• Develop batch and near real-time ETL workflows.
• Process and analyze large-scale datasets with low latency and high throughput.
• Optimize data pipelines for performance, reliability, and scalability.
• Collaborate with Data Engineers, Architects, and Business Stakeholders to deliver data solutions.
• Implement monitoring, logging, and troubleshooting for data processing applications.
• Ensure data quality, governance, and security standards are followed.

Location - Remote, Delhi NCR, Bangalore, Chennai, Pune, Kolkata, Ahmedabad, Mumbai, Hyderabad