Data Engineer

About DysrupIT

DysrupIT is a consulting-led technology firm. We help mid-market to enterprise businesses solve business problems through technology whether that's consulting, execution, managed services, or staff augmentation and we take accountability for the outcome. We are dedicated to making a positive impact in the communities we serve.

Company Culture

At DysrupIT, success isn't measured in headcount placed or hours billed — it's measured in outcomes delivered. We're a team that takes ownership of the work, stays curious about problems beyond our immediate scope, and builds relationships meant to grow, not just renew or end. We invest in our people with the training and support they need to grow their careers, and we back a culture where everyone, regardless of role, is encouraged to notice opportunities, ask one more question, and help lead the story for our clients, not just deliver it.

Job Summary

The role is focused on supporting, enhancing, and evolving critical data platforms that underpin Financial Crime Risk and Legal and Corporate Tech. You will contribute to the development, maintenance, and operational support of enterprise-scale data pipelines that ingest, transform, and manage data from multiple internal and external sources. Working closely with technology, platform, and business stakeholders, you will deliver reliable, scalable, and high-quality data solutions while ensuring the operational stability of production systems. This role requires a strong engineering mindset, a passion for solving complex data challenges, and a commitment to maintaining resilient production platforms.

Job Responsibilities

• Support and enhance data engineering solutions that enable Financial Crime Risk and/or Legal and Corporate Tech technology platforms.
• Design, develop, and maintain scalable and resilient data ingestion, transformation, and reconciliation pipelines.
• Integrate data from multiple internal and external providers into existing platforms and data stores.
• Transform and curate raw source data to align with target business and technical data models.
• Migrate and modernise existing data pipelines as enterprise ingestion and/or transformation frameworks, tooling, and standards evolve.
• Develop and optimise complex SQL transformations and large-scale distributed data processing workloads.
• Implement robust data quality controls, validation frameworks, reconciliation processes, and monitoring capabilities.
• Perform root cause analysis and troubleshoot complex data and platform issues across development, testing, and production environments.
• Participate in production support activities, operational incident management, and on-call support rotations.
• Improve platform reliability, observability, and operational processes through automation and engineering best practices.
• Collaborate with data architects, platform engineers, business stakeholders, and delivery teams to deliver high-quality outcomes.
• Contribute to source control, CI/CD pipelines, automated testing, and release management practices.
• Promote engineering excellence through knowledge sharing, continuous improvement, and technical leadership within the team.

Qualifications

Data Engineering

• Strong hands-on experience developing enterprise-scale data engineering solutions.
• Advanced SQL development skills with experience processing and analysing large datasets.
• Strong Python development skills for data processing, automation, and integration.
• Experience building and maintaining complex ETL and ELT pipelines.
• Strong understanding of data integration patterns and multi-source data processing.
• Experience working with structured, semi-structured, and large-volume datasets.
• Strong understanding of data modelling principles and data access patterns.

Cloud and Platform Technologies

• Experience working with AWS services, particularly: Amazon S3, Amazon RDS, EKS, EC2, Lambda, and IAM.
• Experience developing and supporting cloud-based data platforms.
• Experience working with source control systems and modern software development practices.

Data Quality and Operational Excellence

• Strong experience implementing data quality controls and validation frameworks.
• Experience designing reconciliation processes, control totals, and load assurance mechanisms.
• Experience troubleshooting production issues and supporting critical business systems.
• Experience participating in operational support and incident management processes.

Engineering Practices

• Experience with CI/CD pipelines and automated deployment practices.
• Experience working with Agile delivery methodologies.
• Strong stakeholder engagement and communication skills.
• Ability to balance delivery outcomes with operational stability and platform resilience.

Nice to Have

• Experience with Apache Spark or distributed data processing frameworks.
• Experience with orchestration technologies such as Apache Airflow, Argo Workflows, or Control-M (or equivalent).
• Experience with AWS Glue.
• Experience using Trino or Presto.
• Experience working with containerised workloads and Kubernetes platforms, preferably AWS EKS.
• Experience modernising or migrating enterprise data pipelines to new ingestion, orchestration, or cloud-based platforms.
• Experience with OpenSearch / Elasticsearch platforms.
• Experience working within banking, financial services, or regulated environments.
• Exposure to Financial Crime Risk technologies and business processes.
• Exposure to Actimize, ActOne, Quantexa, or similar financial crime platforms.

What We Offer

• Competitive compensation package commensurate with experience.
• Government-mandated benefits plus supplemental HMO coverage.
• Collaborative and professional work environment.
• Career growth opportunities within a growing technology organization.
• Hybrid/Remote work arrangement (where applicable).