DE&A - Core - Advanced Data Engineer - Advanced Data Engineering

Job highlights

• Knowledge and experience with MLOps (E2E Data and Machine Learning Pipelines).
• Experience in data quality management and monitoring is a plus. .
• Working with cloud services (preferably with Azure and Databricks) .
• Experience in building APIs.
• Experience in data engineering using DevOps principles . .

In your role, you will participate together with the other Data Engineers in solution design discussions led by our product architect, where your knowledge and input will be highly valued. You will collaboratively work with IT and business SMEs to ensure delivery of high-quality end-to-end data pipelines.

• Minimum 5 years experience with building complex data pipelines and data solutions.
• Bachelors or Masters degree in Computer Science , IT, or equivalent.
• Hands-on experience with data modeling
• Hands-on experience with programming in Java and Scala (python is a plus).
• Hands-on experience with Streaming processing (e.g., Spark) and Streaming and/or batch storage (e.g. Kafka, Oracle)
• Experience in building APIs
• Experience in data engineering using DevOps principles
• Experience in building highly performant and secure data pipelines
• Data Schema s (e.g., JSON/XML/Avro)
• Storage formats (e.g., Azure Blob, SQL, NoSQL)
• Deploying services as containers (e.g., Docker)
• Experience in building highly performant and secure data pipelines
• Knowledge and experience with MLOps (E2E Data and Machine Learning Pipelines)
• Working with cloud services (preferably with Azure and Databricks)
• Experience in data quality management and monitoring is a plus.
• Experience with Predictive Maintenance model development is a plus.
• You are fluent in English.

Your responsibilities:

• You will be developing, testing, and documenting the data (collection and processing) pipelines for Predictive Maintenance solutions. The data collection consists of (complex) data pipelines from (IoT) sensors and low/high level control components to our data platform. When the data is in the cloud, it must be processed and made available as data products to our data scientists.
• Align implementation efforts with other back-end developers across multiple development teams.
• You will develop scalable data pipelines to transform and aggregate data for business use, following software engineering best practices. For these data pipelines you will make use of the best frameworks available for data processing like Spark. We are continuously improving on the solutions we use and encourage you to keep challenging the status quo.
• You develop our data services for customer sites towards a product, using (test & deployment) automation, componentization, templates, and standardization to reduce delivery time of our projects for customers. The product provides insights into the performance of our material handling systems at customers all around the globe.
• You will contribute to the design, build, and improve a CI/CD pipeline, including (integration) test automation for data pipelines. In this process you strive for an ever-increasing degree of automation.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.