BigData - L-3 Big Data Administrator

🌟 Why This Opportunity Stands Out

Join a high-performing engineering team supporting mission-critical enterprise platforms within a leading financial/enterprise environment.

This is a chance to work on large-scale, high-impact systems that directly support business operations, financial workflows, and digital transformation initiatives.

You will be part of a team that values clean engineering, scalability, and continuous improvement, while working with a modern Microsoft technology stack.

• Big Data Platform Engineer

  Key Responsibilities

  1. Support, maintain, and optimize Hadoop (HDP) clusters for batch processing, analytics, and regulatory reporting.
  2. Perform Hadoop cluster lifecycle management, including provisioning, scaling, patching, and decommissioning of nodes.
  3. Ensure high availability and resilience of production systems supporting critical business processes.
  4. Manage and optimize Apache Kafka environments for high-throughput, real-time data streaming.
  5. Ensure data consistency, reliability, and fault tolerance across data pipelines.
  6. Work with AWS EMR for scalable processing and data reconciliation workloads.
  7. Administer HDFS, ensuring optimal storage utilization, replication, and performance.
  8. Monitor and tune MapReduce and YARN workloads to improve system efficiency.
  9. Manage OpenSearch / Elasticsearch clusters for operational monitoring, search capabilities, and reporting.
  10. Develop and maintain Kibana dashboards and data ingestion pipelines.
  11. Optimize indexing and query performance to support near real-time analytics.
  12. Implement and maintain Kerberos-based authentication and security controls across the Hadoop ecosystem.
  13. Manage user provisioning and access governance across Linux and Hadoop platforms.
  14. Collaborate with cross-functional teams to support enterprise data platform initiatives and operational excellence.

  Required Skills & Experience

  1. 8-14 years of experience in Big Data, Data Engineering, or Data Platform Engineering within enterprise environments.
  2. Strong hands-on experience with the Hadoop ecosystem, including HDFS, YARN, Spark, MapReduce, Hive, HBase, Pig, Oozie, Flume, and Sqoop.
  3. Proven experience administering and supporting Apache Kafka environments.
  4. Strong knowledge of OpenSearch and/or Elasticsearch.
  5. Experience working with AWS EMR and AWS cloud services.
  6. Strong Linux system administration skills with scripting experience in Shell and/or Python.
  7. Experience with Kerberos authentication, data security, and access governance.
  8. Experience supporting high-volume, low-latency data platforms.
  9. Strong analytical, troubleshooting, and problem-solving skills.

  Interested?

  Interested candidates may submit their updated resume to:

  📧 [email protected] with Subject Line BigData - L-3 Big Data Administrator

Tamanna Bilandi
Direct Line: [8463 2870](tel:8463 2870)
EA License No: 91C2918
Personnel Registration Number: R2096241