Big Data Engineer

Job Responsibilities:

Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases

• Perform analysis of large data sets using components in AWS Cloud

• Own product features from the development, testing through to production deployment

• Evaluate big data technologies and prototype solutions to improve our data processing architecture

• Automate different pipelines

Required Qualification:

BS degree in computer science, computer engineering or

equivalent

• 4-7 years of experience delivering enterprise software solutions

• Proficient with Java, Spark, Scala, Python, AWS Cloud technologies

• 2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala

• Flair for data, schema, data model, how to bring efficiency in big

data related life cycle.

• Experience with Agile Development methodologies.

• Experience with data ingestion and transformation

• Have understanding for secure application development

methodologies.

• Experience in with Airflow and Python will be preferred.

• Understanding of automated QA needs related to Big data

technology.

• Strong object-oriented design and analysis skills

• Excellent written and verbal communication skills

Responsibilities

• Utilize your software engineering skills including Spark, Python,

Scala to Analyze disparate, complex systems and collaboratively

design new products and services

• Integrate new data sources and tools

• Implement scalable and reliable distributed data replication

strategies

Preferred Qualification:

• Knowledge and hands-on experience of using AI technologies for development, Testing and automation.
• Knowledge of Cloud Technologies is a must such as AWS and its services.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.