Data Engineer 2

How is this team contributing to vision of Providence?

• Ensure a continual and seamless service for customers regardless of which services they access.

What will you be responsible for?

• Strong knowledge of data analysis, data integration.
• Design, develop, and maintain scalable data pipelines for ingesting structured, semi-structured, and unstructured data.
• Extract, parse, validate, and transform data from various file formats including CSV, JSON, XML, Excel, PDFs, and log files.
• Build and maintain ELT/ETL workflows using Azure Data Factory, Azure Databricks, and Snowflake.
• Working on Azure Services like Azure Data Factory, Azure Databricks, ADLS.
• Proficient in data visualization, PowerBI, writing complex SQL queries using Synapse/Azure Databricks/Snowflake.

What would your day look like?

• Work closely with teams to determine project engagement and priority, business data reporting and analysis requirements.
• Create notebooks/scripts in Databricks/Snowflake or use Azure Data Factory to ingest data into system. Design file parsing solutions to handle complex and unstructured data sources for analytics and reporting.
• Work with different teams to understand APIs. Integrating API data into our system for reporting purposes.
• Gather requirements, develop, and implement solutions across all endpoint technologies in the Microsoft stack.
• Support AI use cases by preparing datasets, enabling feature engineering, and integrating AI-powered data processing capabilities.
• Troubleshoot data pipeline failures, performance bottlenecks, and data quality issues.

Who are we looking for?

• 3-5 years of experience in Data Engineering, ETL/ELT development, or Analytics Engineering.
• 2-4 years of hands-on experience with Azure cloud technologies.
• Strong experience with Snowflake, Azure Data Factory, Azure Databricks, ADLS Gen2, and SQL.
• Experience building data ingestion frameworks for APIs, flat files, Excel, JSON, XML, PDFs, and other enterprise file formats.
• Strong SQL skills with experience in query optimization, data modelling, and data warehousing concepts.
• Familiarity with AI/ML concepts, GenAI use cases, vector search, embeddings, or AI-assisted data processing workflows.
• Experience with Git, Azure DevOps, CI/CD pipelines, and Agile methodologies.
• Excellent communication and stakeholder management skills.