Sr Backend Engineer

Job highlights

• High-volume data pipelines - 100K+ documents / records per batch
• Experience: 4-8 years
• 2+ years building data-intensive pipelines . Technical Focus: Own accelerators 6-n stage processing pipeline end-to-end: Ingestion to Output

About Aivar Innovations

Aivar is an AI-first technology partner where cutting-edge technology meets industry expertise to supercharge your projects.

Team: Accelerators

Experience: 4-8 years | 2+ years building data-intensive pipelines

Technical Focus: Own accelerators 6-n stage processing pipeline end-to-end: Ingestion to Output. Build the connectors, PII/PHI safety layer, compliance audit trail, and multi-tenant isolation. Pure platform engineering - async Python, event-driven architecture, enterprise-scale reliability.

Key Responsibilities:

• Design the core pipeline orchestration framework with async Python, supporting parallel/sequential stage execution with retry and dead-letter queues.
• Build the connector framework for 50+ enterprise data sources (APIs, databases, file systems, SaaS platforms).
• Implement safety validation - PII/PHI detection, data integrity checks, format compliance, input sanitisation.
• Build the reconstruction module - transforming extracted data into Markdown/vector representations and context maps.
• Design event-driven architecture with message queuing, backpressure handling, and graceful degradation.
• Implement immutable audit trail logging across all 6 stages - every decision, action, and transformation.
• Build multi-tenant data isolation, ensuring strict customer separation at every pipeline stage.

Must-Have Technical Skills:

• Distributed Python systems - asyncio, aiohttp, Celery; not scripting.
• Message queues/streaming - Kafka, SQS, RabbitMQ, or Redis Streams.
• High-volume data pipelines - 100K+ documents/records per batch.
• API design - REST/GraphQL, versioning, rate limiting, and auth.
• AWS services - Lambda, SQS/SNS, MSK, EKS, Step Functions, ECS/EKS, S3, DynamoDB, RDS.
• Multi-tenant SaaS architecture and customer data isolation patterns.
• Production habits - comprehensive testing, CI/CD, monitoring, and incident response.

Core Tech Stack:

Python, FastAPI, Celery, asyncio, Kafka/SQS/Redis Streams, PostgreSQL, DynamoDB, Redis, Elasticsearch, Terraform/CDK, Prometheus / Grafana / OpenTelemetry , GitHub Actions, Docker, AWS (Lambda, ECS, Step Functions, S3)

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.