Big Data Engineer

Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases

• Perform analysis of large data sets using components in AWS Cloud
• Own product features from the development, testing through to production deployment
• Evaluate big data technologies and prototype solutions to improve our data processing architecture
• Automate different pipelines

Required Qualification:

BS degree in computer science, computer engineering or equivalent

• 4-7 years of experience delivering enterprise software solutions
• Proficient with Java, Spark, Scala, Python, AWS Cloud technologies
• 2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala
• Flair for data, schema, data model, how to bring efficiency in big data related life cycle.
• Experience with Agile Development methodologies.
• Experience with data ingestion and transformation
• Have understanding for secure application development methodologies.
• Experience in with Airflow and Python will be preferred.
• Understanding of automated QA needs related to Big data technology.
• Strong object-oriented design and analysis skills
• Excellent written and verbal communication skills Responsibilities
• Utilize your software engineering skills including Spark, Python,

Scala to Analyze disparate, complex systems and collaboratively

design new products and services

• Integrate new data sources and tools
• Implement scalable and reliable distributed data replication strategies

Preferred Qualification:

• Knowledge and hands-on experience of using AI technologies for development, Testing and automation.
• Knowledge of Cloud Technologies is a must such as AWS and its services.