Data Engineer -2

Role: Data Engineer - 2

Responsibilities:

Designing, developing, and maintaining data architecture, data pipelines, and data workflows.

Building and optimizing large-scale data processing systems for both structured and unstructured data.

Collecting, integrating, and transforming data from various sources, including databases, APIs, and file systems.

Collaborating with data scientists, analysts, and other stakeholders to understand data requirements and design effective data solutions.

Developing and implementing data quality and data governance processes to ensure data accuracy, consistency, and reliability.

Writing efficient and scalable code/scripts to automate data processing tasks and optimize data workflows.

Monitoring and troubleshooting data pipelines to ensure their performance, availability, and reliability.

Identifying and implementing improvements to enhance data infrastructure, including data storage, processing, and retrieval mechanisms.

Keeping up to date with the latest technologies, tools, and trends in the field of data engineering.

Mentoring and providing guidance to junior data engineers, as

needed.

Qualifications:

Bachelor's degree in computer science, data science, or a related field. Advanced degree preferred.

Proven experience as a data engineer or in a similar role, handling large-scale data processing and analytics projects.

Strong programming skills in languages like Python, Java, or Scala, with experience in data manipulation, scripting, and automation.

Proficiency in working with relational databases (SQL) and NoSQL databases.

Experience with big data technologies and frameworks, such as Hadoop, Spark, Kafka, or Hive.

Knowledge of data integration and ETL (Extract, Transform, Load) processes.

Familiarity with cloud platforms, such as AWS or GCP, and their data-related services.

Understanding of data modelling concepts and data warehousing= principles.

Strong problem-solving and analytical skills, with the ability to troubleshoot and optimize data processing workflows.

Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.