DevOps Engineer - BigData, SRE, Devops, Apache Airflow, Nifi, Hadoop

Greetings from TCS!

TCS is hiring for DevOps Engineer - BigData, SRE, Devops, Apache Airflow, Nifi, Hadoop

Desired Experience Range: 6+ Years

Job Location: Pune

Experience and Skills

· Seeking an experienced Big Data Engineer with strong expertise in Spark, Scala, and Hadoop ecosystem technologies to design, develop, and optimize large-scale data processing solutions. · The candidate will be responsible for building high-performance data pipelines, managing distributed systems, and enabling scalable analytics platforms. · Required Skills: Apache Spark, Scala, Hadoop Ecosystem (HDFS, Hive, Yarn), SQL & Data Modeling, Distributed Computing Concepts, ETL/Data Pipeline Development · Preferred Skills: Apache NiFi, Apache Kafka, AWS Services, Airflow, CI/CD & DevOps

Roles and Responsibilities · Design and develop scalable Big Data applications using Spark and Scala. · Build and maintain data ingestion, transformation, and processing pipelines. · Work with Hadoop ecosystem components for distributed data storage and processing. · Develop batch and real-time data processing solutions. · Integrate multiple data sources using Apache NiFi and Kafka. · Optimize Spark jobs for improved performance and resource utilization. · Collaborate with business stakeholders, architects, and data consumers. · Implement data quality, governance, and security standards. · Support production deployments and troubleshoot complex data issues