Big Data Engineer(8-16 years)

Role & responsibilities

Position: Big Data Developer Experience: 8 to 15 Years Location: PAN India Employment Type: Full-TimeAbout the RoleWe are looking for experienced Big Data Developers to design, build, and optimize large-scale data processing systems and analytics platforms. The ideal candidate will possess strong expertise in Big Data technologies, distributed computing, cloud platforms, and data engineering practices to support enterprise data-driven initiatives.Key Responsibilities

• Design, develop, and maintain scalable Big Data solutions for processing and analyzing large volumes of structured and unstructured data.
• Build and optimize data pipelines, ETL/ELT processes, and real-time streaming applications.
• Develop data ingestion frameworks from multiple sources including APIs, databases, files, and streaming platforms.
• Implement batch and real-time data processing solutions using Big Data technologies.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and stakeholders to understand business requirements.
• Ensure data quality, integrity, governance, and security across the data ecosystem.
• Optimize data processing performance and troubleshoot production issues.
• Participate in architecture reviews, code reviews, and technical discussions.
• Support cloud migration and modernization initiatives.
• Develop and maintain technical documentation and operational procedures.

Required SkillsBig Data Technologies

• Strong hands-on experience with:

• Hadoop Ecosystem

• Apache Spark (Core, SQL, Streaming)

• Hive

• HDFS

• YARN

• Experience developing distributed data processing applications.

Data Engineering

• Expertise in ETL/ELT development.
• Strong knowledge of data modeling and data warehousing concepts.
• Experience with large-scale data transformation and migration projects.
• Understanding of data governance and data quality frameworks.

Programming Skills

• Strong proficiency in:

• Python

• Scala

Streaming Technologies

• Experience with:

• Apache Kafka

• Experience with one or more cloud platforms:

• AWS

• Azure

• Google Cloud Platform (GCP)

Preferred candidate profile