Technical Lead - Data Engineering

Responsibilities

• Define client needs and oversee project milestones to ensure expectations, timelines, and budgets are met Define data platform architecture, design, and review code and make required changes Responsible for the overall quality of project deliverables and the successful implementation of defined solutions for the customer Build long-term, superior client relationships proactively manage client expectations, and ensure that change control is used when scope boundaries are exceeded Communicate a compelling and inspired vision to all customer levels, from CTOs/CIOs to engineering managers and programmers Leverage extensive knowledge of distributed systems, process flows, and procedures to aid analysis and recommendations for solution offerings Maintain a strong network and promote the organization at various meetings, forums, panels, publications, and conferences to establish thought leadership in the industry Responsible for POCs on any tech stacks introduced, or possible version upgrades

Requirements

• BE/B.Tech/MCA with at least 12+ years of experience in design and development using various ETL, database technologies with at least 5+ years associated Hadoop technology stack and programming languages
• Hands-on experience in 2 or more areas: Hadoop, HDFS, MR High Availability architecture and DR setup Spark Streaming, Spark SQL, Spark ML Kafka/Flume Apache NiFi Worked with Hortonworks Data Platform as Architect CDH (Cloudera Distribution for Hadoop) as developer/administrator Hive / Pig / Sqoop NoSQL Databases HBase/Cassandra/Neo4j/MongoDB Visualization and Reporting frameworks like D3.js, Zeppelin, Grafana, Kibana Tableau, Pentaho Scrapy for crawling websites ElasticSearch Google Analytics data streaming Data security (Kerberos/OpenLDAP/Knox/Ranger)
• Extensive knowledge of the current landscape and the ability to visualize technology and industry trends Working knowledge of Big Data Integration with Third-party/in-house built Metadata Management, Data Quality, Master Data Management solutions, structured/unstructured data
• Ability to successfully lead a team of big data engineers and big data developers through all phases of the development life cycle, including requirements definition, architecture, design, development, implementation, and testing Active involvement in the community in terms of articles,blogs or speaking engagements at conferences

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.