Technical Lead

Role & responsibilities

• Develop and maintain scalable Big Data solutions using the Hadoop ecosystem.
• Work with HDFS, Hive, Spark, Sqoop, Oozie, and YARN.
• Develop and optimize Hive SQL queries and data transformations.
• Build batch-processing and ETL pipelines using Apache Spark.
• Perform data ingestion from relational databases and external sources using Sqoop or equivalent tools.
• Manage data storage, partitioning, bucketing, compression, and schema evolution.
• Monitor and troubleshoot Hadoop jobs and cluster performance.
• Work with Python / Scala / Java for data processing.
• Implement data quality, validation, and reconciliation processes.
• Collaborate with Data Architects, Analysts, and application teams to deliver data solutions.
• Optimize Spark/Hadoop jobs for performance and resource utilization.

Preferred candidate profile

• Hadoop
• HDFS
• Hive / HiveQL
• Apache Spark
• YARN
• SQL
• Big Data ETL
• Data Warehousing
• Python / Scala / Java
• Distributed computing concepts
• Performance tuning and troubleshooting