Senior Data Integration Engineer

Senior Data Integration Engineer

About the Role

We are seeking a Senior Data Integration Engineer to join our growing platform team. In this role, you will lead the design and implementation of scalable, real-time data integration pipelines to connect and harmonize diverse Enterprise Resource Planning (ERP) systems across our organization. You will leverage event-driven architectures to ensure seamless, high-throughput, and low-latency data movement between enterprise platforms.

Responsibilities

• Pipeline Architecture & Development Design, build, and deploy robust, real-time, and batch data pipelines integrating disparate ERP with downstream data lakes and platforms.
• Event-Driven Systems Leverage Apache Kafka, Kafka Connect, and stream processing engines (Kafka Streams or Apache Flink) to process and transform high-volume change data capture (CDC) and transactional data.
• Service & API Engineering Build resilient microservices and integration layers using Java and Spring Boot to orchestrate complex data flows and enterprise APIs.
• Containerization & Orchestration Package applications into Docker containers and manage scalable deployments, service discovery, and container orchestration via Kubernetes.
• Data Modeling & Storage Optimize multi-database environments across relational systems including PostgreSQL, MySQL, SQL Server, and Oracle.
• DevOps & Automation Manage Infrastructure as Code (IaC) using Terraform or AWS CloudFormation and build automated CI/CD deployment pipelines using GitHub Actions.
• Observability & Reliability Set up end-to-end monitoring, APM, log management, and alerting using tools like DataDog, New Relic, or Splunk to ensure data pipeline health and transactional accuracy.

Qualifications

• Core Engineering 5+ years of software/data engineering experience with strong proficiency in Java (Spring Boot) and Python.
• Streaming & Messaging Hands-on experience building event-driven systems using Apache Kafka, Kafka Connect (source/sink connectors, CDC), and Kafka Streams or Apache Flink.
• Containerization Hands-on experience containerizing applications with Docker and managing workloads on Kubernetes.
• Monitoring & Observability Production experience setting up metrics, tracing, and logging dashboards using platforms like DataDog, New Relic, or Splunk.
• Database Expertise Deep expertise in relational database design, query optimization, and schema management across engines such as PostgreSQL, MySQL, SQL Server, or Oracle.
• Cloud Infrastructure Hands-on experience deploying scalable applications and data workloads on AWS.
• Infrastructure as Code (IaC) & CI/CD Demonstrated experience writing Terraform or AWS CloudFormation scripts and managing CI/CD workflows via GitHub Actions.
• Version Control Advanced proficiency with Git and standard collaborative workflows.

Required Skills

• Strong proficiency in Java (Spring Boot) and Python.
• Event-driven systems using Apache Kafka, Kafka Connect, Kafka Streams or Apache Flink.
• Containerization with Docker and orchestration with Kubernetes.
• Monitoring, metrics, tracing, and logging using DataDog, New Relic, or Splunk.
• Relational database design, query optimization, and schema management (PostgreSQL, MySQL, SQL Server, Oracle).
• Deploying scalable applications and data workloads on AWS.
• Infrastructure as Code with Terraform or AWS CloudFormation.
• CI/CD workflows via GitHub Actions.
• Advanced proficiency with Git.