Data Engineer

NRMA - Data Engineer

Role Responsibilities

· Build and maintain scalable data extraction, transformation, and loading workflows using Informatica

· Manage tables, datasets, partitions, and clustering inside Google BigQuery for large-scale analytics.

· Write and tune advanced SQL scripts and queries to boost execution speed and lower compute costs.

· Use Unix shell scripting to automate file transfers, pre- and post-load routines, and batch job sequencing.

· Develop custom Python scripts to parse files, orchestrate data movement, and interact with cloud APIs.

· Schedule, monitor, and troubleshoot integrated workflows across Unix cron jobs and cloud schedulers.

· Move legacy on-premise data assets into cloud environments via hybrid ETL pipelines.

· Implement error-handling, logging, and data validation rules using SQL and Python checks.

· Diagnose bottlenecks in Informatica mappings, BigQuery queries, and shell scripts.

· Maintain technical mapping specs, system architecture diagrams, and partner with analysts or data

Key Skills & Experience

· 4–7 years of hands-on data engineering experience designing, developing, and supporting production data pipelines.

· Experience building ETL/ELT workflows using Informatica, including mapping development, workflow scheduling, monitoring, and troubleshooting.

· Proven capability working with Google BigQuery, including dataset management, table design, partitioning, clustering, and query performance optimisation.

· Advanced SQL skills, including complex joins, window functions, stored procedures, query tuning, and data validation routines.

· Practical experience with Unix shell scripting for file handling, batch orchestration, job sequencing, and automation of operational tasks. (desired)

· Proficiency in Python for data processing, file parsing, API integration, automation, and development of reusable data engineering utilities.

· Experience supporting cloud migration or hybrid data integration initiatives, including movement of legacy on-premise data assets to cloud platforms. (desired)

· Good understanding of data quality, reconciliation, error handling, logging, and operational monitoring practices across batch and scheduled workloads.

· Ability to diagnose and resolve technical issues across Informatica mappings, SQL queries, shell scripts, and cloud data pipelines.

· Experience maintaining technical documentation, mapping specifications, operational runbooks, and working closely with analysts, testers, and business stakeholders.

· Familiarity with Agile delivery practices, code version control, release processes, and collaborative delivery across onshore and offshore teams.

· Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related discipline; relevant cloud, data, or Informatica certifications are desirable.