Data Engineer – Scala / Spark / Kafka

Role: Data Engineer – Scala / Spark / Kafka

Location: India

Employment Type: Contract

Experience: 5+ Years

Duration: 6–12 Months

Start Date: ASAP

Job Description

We are looking for an experienced Data Engineer to design, build, and optimize scalable data platforms capable of processing large volumes of batch and real-time data.

The ideal candidate will have strong hands-on experience with Scala, Apache Spark, Kafka, and Big Data technologies . You will be responsible for developing robust data pipelines, designing scalable data models, troubleshooting performance issues, and collaborating with engineering and analytics teams to ensure reliable access to high-quality data.

The role involves extensive work with distributed processing technologies, streaming platforms, and modern cloud-based data infrastructure .

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for batch and real-time processing.
• Develop data processing applications using Scala and Apache Spark .
• Build and maintain real-time streaming pipelines using Kafka and Spark Structured Streaming .
• Design and implement scalable data models and data processing solutions.
• Work with Hive, SQL, S3, Presto, Delta Lake , and other Big Data technologies.
• Analyze and optimize Spark jobs and distributed data workloads.
• Troubleshoot data pipeline, performance, and production issues.
• Ensure data quality, reliability, scalability, and availability.
• Collaborate with engineering, analytics, and other technical teams to deliver data solutions.
• Contribute to the design and development of modern data lake and lakehouse architectures.

Required Skills

• 5+ years of professional experience in Data Engineering, Big Data Engineering, or a related field.
• Strong programming skills in Scala .
• Hands-on experience with Apache Spark for both batch and streaming workloads.
• Strong proficiency in Hive and SQL .
• Hands-on experience with Kafka and real-time event streaming.
• Strong understanding of Big Data technologies and ecosystems.
• Experience with S3, Hive, Presto, and Delta Lake .
• Experience designing and implementing scalable data models.
• Strong debugging and problem-solving skills.
• Experience with performance analysis and optimization of distributed data workloads.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.