Data Engineer

Ümumi məlumat We are looking for a hands-on Leading / Senior Data Engineer to own critical ingestion and lakehouse workflows in a high-throughput, production-grade data platform.

Öhdəliklər

• Designing a data lake to process and store an array of (un)structured data loaded from raw sources. Development of the data model - properly store the data and access it as needed for business purposes
• Development of the integration process - development of integration with various systems so that they can have a single view of key indicators when making decisions
• Data preparation and ETL - development of a pipeline for extracting, transforming and loading data Developing and monitoring data pipelines to ensure data quality and integrity
• Automation and optimization of the data transformation process
• Working closely with other teams in the organization to ensure data is available and accessible for analytics and decision-making

Tələblər

• Degree in computer science, information science, engineering, mathematics, or related technical discipline
• 5+ years experience with SQL and NoSQL technologies
• Strong experience working with Oracle and PostgreSQL databases
• Extensive experience in ETL development, with strong skills in Python programming
• Advanced knowledge of PySpark for large-scale distributed data processing
• Proficient in Airflow for scheduling, orchestrating and monitoring complex ETL/ELT workflows
• Expertise in Kafka for event streaming and messaging pipelines
• Experience working with MinIO (S3-compatible storage), managing data in Parquet and Iceberg formats
• Familiarity with Lakehouse concepts
• Deep understanding of MPP architecture concepts

The Following Experience Will Be Considered As An Advantage

• Familiarity with Impala for interactive SQL querying on Big Data
• Experience with Greenplum
• Familiarity with containerization (Docker, Kubernetes)
• Experience with CDC tools like Debezium and Oracle GoldenGate for real-time data replication
• Experience with data visualization tools (e.g. Tableau, PowerBI, etc.)
• Understanding of DataOps and DevOps principles
• Knowledge of data modeling and database design
• Experience with data engineering best practices, such as data security, data access control and data governance
• Knowledge of Grafana for monitoring metrics
• Proficient with GitLab for version control
• Experience mentoring junior data engineers and collaborating with cross-functional teams (Data Scientists, Analysts, DevOps)