Data Engineer

Role Overview

We are looking for a Data Engineer to design, build, and maintain our cloud data warehouse and ingestion pipelines. The role spans data ingestion from relational databases, APIs, and semi-structured sources via AWS DMS and custom ETL/ELT pipelines, through to data modeling and mart delivery for Spotfire and downstream business applications. The candidate will be responsible for data quality, platform performance, and scalability of the data layer.

Key Responsibilities

• Build and maintain scalable data pipelines and transformation layers within Snowflake.
• Design and develop Snowflake objects including tables, views, stages, tasks, streams, and stored procedures.
• Apply Snowflake best practices for warehouse sizing, clustering, caching, and query optimization.
• Ensure data quality, consistency, and availability across the warehouse layers.
• Support performance tuning and scalability improvements as data volumes grow.
• Design and develop ETL/ELT pipelines to ingest data from Oracle, PostgreSQL, and MySQL databases.
• Design, develop, and maintain ETL/ELT jobs and transformations using Pentaho Data Integration (PDI/Kettle) for loading data into Snowflake.
• Configure and manage AWS DMS tasks for Full Load and CDC-based data ingestion into Snowflake.
• Build ingestion pipelines for API-based sources including JSON parsing and semi-structured data processing.
• Handle ingestion from flat files and JSON files through Snowflake external stages and COPY commands.
• Monitor pipeline health, resolve replication lag, and maintain ingestion SLAs.
• Design data models using Star Schema and Fact/Dimension structures aligned to business reporting needs.
• Use AWS S3 for staging raw data files as part of the ingestion workflow.
• Configure IAM roles and policies to manage access between AWS services and Snowflake.
• Use AWS Lambda for lightweight data transformation or pipeline trigger automation where applicable.
• Coordinate with infrastructure teams on DMS instance sizing, networking, and security group configuration.

Required Skills & Experience

Core - Mandatory

• 3+ years hands-on development - pipelines, transformations, performance tuning, and warehouse management: Snowflake
• Strong proficiency in both; SQL for complex analytical queries and transformations, Python for pipeline scripting and automation: SQL and Python
• Hands-on experience designing and developing data ingestion and transformation pipelines: ETL/ELT Pipelines

[email protected] - Please share your resume if you are interested to apply