Data Engineer

We are looking for an experienced ETL / Data Engineer to design and implement robust, end-to-end data pipelines on our lakehouse platform. You will work with modern Python-based frameworks and open-source tooling to deliver high-quality, trusted data for analytics and enterprise reporting.

---

Key Responsibilities

• Design and implement end-to-end data pipelines across a medallion architecture using Python-based frameworks.
• Develop scalable ETL/ELT solutions for ingestion, validation, cleansing, and transformation of large datasets.
• Ensure data quality, lineage, and auditability by integrating with metadata catalogs and validation frameworks.
• Optimize data workflows for performance, reliability, and maintainability using modern open-source lakehouse tools.
• Collaborate with DataOps, DevOps, and Analytics teams to ensure seamless integration of pipelines into enterprise data platforms.
• Develop and maintain technical documentation, including pipeline architecture, transformation logic, and validation rules.
• Monitor, troubleshoot, and continuously improve pipeline execution and data quality across environments.
• Mentor junior data engineers, promoting best practices in coding, testing, and data governance.
• Stay current with emerging technologies in data processing, orchestration, and lakehouse architectures, and propose continuous improvements.

---

Qualifications

• Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, or a related field.
• 3–5 years of experience in data engineering or ETL development.
• Strong Python programming skills, with the ability to design efficient, modular, and testable code.
• Proven experience building data pipelines with DataFrame processing frameworks (e.g., Polars, Pandas, Dask, or equivalent).
• Deep understanding of data lakehouse concepts, Parquet, and relational databases such as PostgreSQL and Oracle.
• Experience implementing orchestration workflows (e.g., Prefect, Airflow) and data validation frameworks (e.g., Great Expectations).
• Familiarity with metadata and lineage tools such as OpenMetadata or DataHub.
• Solid knowledge of SQL, schema design, and data modeling.
• Experience with CI/CD pipelines, version control (Git), and containerization (Docker/Kubernetes).
• Understanding of event-driven automation and alerting mechanisms (e.g., n8n, SMTP, API-based notifications).
• Strong analytical and problem-solving skills, with a focus on system performance and maintainability.
• Excellent communication and documentation abilities.
• Experience working in Agile/Scrum methodologies and collaborative development environments.