Senior Data Engineer - Abu Dhabi (On site)

Excellent opportunity to work on site in Abu Dhabi.

Contract - 6-12 Months, extendable
Employment Type: Full-Time
Benefits: Health insurance, Medical insurance, flight tickets etc will be taken care by the hiring company

Responsibilities:

• Design, develop, and maintain scalable data pipelines, ETL/ELT processes, data models, and schemas to process large-scale datasets for analytics and machine learning.
• Build and optimize high-performance data processing systems using technologies such as Apache Spark, Delta Lake, Kafka, and Hadoop.
• Develop real-time data pipelines using Apache Kafka, Azure Event Hub, or similar streaming technologies.
• Design and implement cloud-native data solutions on Azure ensuring scalability, reliability, performance, and cost optimization.
• Build and maintain data lakes, data warehouses, and lakehouse architectures, including data modeling, ETL processes, and data quality management.
• Implement data validation, monitoring, and testing frameworks using tools such as Great Expectations.
• Ensure data governance, security, access control, and compliance with industry standards for handling sensitive data, including Protected Health Information (PHI).
• Support healthcare data interoperability standards such as FHIR.
• Collaborate with Data Scientists, Data Analysts, Software Engineers, and Product teams to deliver reliable, high-quality data solutions.
• Continuously evaluate and adopt modern data engineering tools, technologies, and best practices.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.

• 5+ years of experience building and managing production-grade data engineering platforms and pipelines.

• Strong programming skills in Python and SQL.

• Hands-on experience with:

• Apache Spark

• Apache Kafka

• Delta Lake

• Hadoop ecosystem

• Presto/Trino

• Workflow orchestration tools (Airflow or similar)

• Docker and Kubernetes

• Experience designing scalable ETL/ELT pipelines, data models, and distributed data systems.

• Experience with cloud platforms such as Microsoft Azure

• Strong knowledge of data lakes, data warehouses, and lakehouse architecture.

• Experience with data governance, Master Data Management (MDM), data cataloging, data anonymization, and pseudonymization.

• Strong analytical, problem-solving, and communication skills.

• Ability to work effectively in a collaborative Agile environment.