Data Engineer

Position Overview

We are seeking a skilled Data Engineer to design, build, and maintain robust data pipelines and systems that support both batch and real-time processing. This role involves developing ETL workflows, optimizing data infrastructure, and collaborating with cross-functional teams to enable advanced analytics and machine learning initiatives. You will play a critical role in ensuring data accuracy, security, and accessibility across the organization.

Key Responsibilities:

• Design, construct, and manage scalable data pipelines and systems for batch and real-time processing.

• Develop ETL procedures to extract, transform, and load data from diverse sources.

• Maintain and enhance data infrastructure to ensure reliable ingestion, transformation, and extraction.

• Automate workflows for data ingestion, aggregation, and ETL processing.

• Transform raw data into usable datasets for technical and non-technical stakeholders.

• Collaborate with data scientists and business leaders to implement machine learning models.

• Apply data governance controls to ensure privacy, security, compliance, and quality.

• Monitor system performance and implement optimization strategies.

• Document processes and apply quality control methods to maintain data integrity.

Qualifications:

Experience

• Minimum 2 years of relevant work experience in data engineering or related fields.

• Strong expertise in relational databases, database architecture, and advanced SQL.

• Proven ability to design and manage data pipelines and workflows.

• Experience developing and deploying machine learning models.

• Familiarity with Kubernetes and containerized environments.

• Prior experience with data streaming systems (e.g., Apache Kafka, MySQL, Apache Pinot).

• Experience with distributed computing and big data frameworks (e.g., Hadoop, Spark, Presto/Trino, Superset).

Technical Skills

• Proficiency in programming languages such as Python, Java, or Go.

• Strong mathematical, analytical, and problem-solving skills.

• Knowledge of workflow orchestration and pipeline management tools.

Soft Skills

• Excellent organizational and communication abilities.

• Ability to work independently and collaboratively in team environments.

• Strong problem-solving mindset with attention to detail.