Lead Data Engineer - Spark/Python

Responsibilities for this position include :

• Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, NoSQL stores like Cassandra, HBase etc) across Fractal and contributes to open source Big Data technologies

• Write and tune complex Java, MapReduce, Pig and Hive jobs

• Adapt quickly to change in requirements and be willing to work with different technologies if required

• Experience leading a Backend/Distributed Data Systems team while remaining hands-on is very important

• Lead the effort to build, implement and support the data infrastructure

• Manage the business intelligence team and vendor partners, ensuring to prioritize projects according to customer and internal needs, and develops top-quality dashboards using industry best practices

• Manage team of data engineers (both full-time associates and/or third party resources)

• Own the majority of deliverables for the Big Data team from a delivery perspective

• Analyzes and confirms the integrity of source data to be evaluated

• Leads in deployment and auditing models and attributes for accuracy

Education for Lead Data Engineer :

Have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Statistics, Education, Technical, Information Technology, Information Systems, Mathematics, Computer Engineering, Management

Skills for Lead Data Engineer :

Desired skills for lead data engineer include :

• Python

• Spark

• Java

• Hive

• SQL

• Hadoop architecture

• Large scale search applications and building high volume data pipelines

• Message queuing

• NoSQL

• Scala

Desired experience for lead data engineer includes :

• Experience in development utilizing C# .NET 4.5

• Experience in managing a live service that customers depend

• Experience in coaching and managing other engineers

• Be a team player and enjoy collaboration with other engineers and teams

• Experience with software version management systems

• Experience with task/bug tracking software

Exp : 5-7yrs