Azure Data Engineer

Role Description:

The Data Engineer is responsible for building, managing, and optimizing data pipelines and ensuring data flow from source to destination. This role requires expertise in data architecture, data lakehouse, data warehousing, and data integration. You will work closely with data architects, data analysts, business system owners to deliver high-quality data ingestion solutions that meet business needs.

Key responsibilities:

• Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading (ETL) data from various sources.
• Automate data workflows to ensure data is consistently and reliably processed.
• Design and implement scalable and secure data solutions using Azure analytics services, Azure SQL Database and/or Databricks.
• Develop and maintain data storage solutions, including data warehouses, data lakes, and databases, ensuring optimal performance and cost efficiency.
• Implement and maintain data quality checks to ensure data accuracy & consistency.
• Implement security measures to protect sensitive data, including encryption &, access control.
• Identify, troubleshoot, and resolve issues related to data pipelines, storage, and processing.
• Monitor and optimize the performance of data pipelines, databases & queries.

Education: Bachelors degree in Computer Science, Information Technology, Data Engineering, or a related field.

Must Have:

• 5+ years of relevant work experience in data engineering.
• Proven hands-on experience data pipeline design & development on Azure cloud platform.
• 5+ years experience developing solutions using Databricks,
• Proficiency in at least one scripting languages such as Python, PowerShell, JavaScript or Scala.

Nice to have:

• Previous experience with Informatica IICS data governance or IDQ a strong plus.
• MS Power BI Development experience is good to have.
• Experience with Oracle Analytics Cloud a strong plus.