Lead Backend Engineer - Data Engineering & AI

Job highlights

• Required skills Backend engineering . 5+ years designing,building,and maintaining production backend services at scale
• Experience adding AI capabilities to existing production systems. . Disclaimer: This job posting has been aggregated from external source

Design, build, and maintain the backend services, APIs, data-platform automation, and AI agents behind our data products. This is a backend-heavy role: distributed Python services, streaming LLM/agent runtimes on AWS, RAG pipelines, and the dbt/Airflow automation that powers our data products.

Responsibilities

• • Design, build, and maintain backend services and APIs in Python - RESTful and streaming (SSE) endpoints, agent runtimes on AWS Bedrock AgentCore / ECS Fargate, and event-driven Lambda handlers.
  • Architect service boundaries and data flows: define contracts between services, model persistence, and manage state, caching, and asynchronous/background processing.
  • Build RAG pipelines and tool-calling AI agents over our data products: retrieval, orchestration, grounding, and evaluation.
  • Design data access and storage layers: schema/data modeling, query performance, connection/session management, and integration with warehouses (Snowflake) and key-value stores (DynamoDB).
  • Implement auth and identity: OAuth/OIDC flows (per-user 3LO, token vaulting, session binding), least-privilege IAM, secrets management.
  • Build and maintain data-platform automation: dbt models, MWAA/Airflow orchestration, and tooling for discoverable, governed, consumable data products.
  • Own service reliability and delivery: Terraform, GitHub Actions CI/CD, container builds, structured logging, metrics/tracing, alerting, and cost controls.
  • Set technical direction: system and API design, code review, and mentoring.

Required skills Backend engineering

• • 5+ years designing, building, and maintaining production backend services at scale. 10+ years if Lead level engineering candidate.
  • Expert-level Python for server-side development; solid grasp of at least one web/async framework (eg aiohttp, FastAPI, Flask) and the WSGI/ASGI model.
  • Service and API design: REST (and/or gRPC), request/response and streaming patterns, pagination, versioning, idempotency, and backward-compatible contracts.
  • Data layer: SQL and data modeling, query optimization and indexing, transactions, connection pooling; experience with relational, warehouse (Snowflake), and NoSQL/key-value (DynamoDB) stores.
  • Server-side patterns: caching strategies, background jobs/workers, queues and event-driven processing, rate limiting, retries/backoff, and timeouts.
  • Performance & reliability: profiling, load handling, latency/throughput trade-offs, graceful degradation, and designing for failure.
  • Observability: structured logging, metrics, distributed tracing, and debugging live production issues.

Software engineering fundamentals

• • Object-oriented programming (required): encapsulation, abstraction, inheritance, composition, polymorphism; SOLID principles; design patterns applied pragmatically; strong domain modeling.
  • Solid data structures & algorithms; ability to reason about time/space complexity.
  • Concurrency and async programming (async/await, threading, event loops) and their failure modes.
  • Testing (unit, integration, end-to-end) and testable design; Git and PR-based workflows; disciplined code review.

Cloud & infrastructure

• • Production AWS: ECS/containers, Lambda, IAM, API Gateway, DynamoDB.
  • Infrastructure as code with Terraform; CI/CD (GitHub Actions or equivalent) and container builds.

Distributed systems

• • Building services that are horizontally scalable, resilient, and loosely coupled; handling consistency, retries, idempotency, and partial failure.

Security

• • OAuth/OIDC, authn/authz, token handling, least-privilege access, multi-tenant isolation, secrets management.

AI / LLM engineering

• • Building LLM applications in production (not research).
  • RAG: chunking, embeddings, vector search, hybrid search, reranking, grounding/citations, context-window management, retrieval evaluation.
  • Agents: prompt engineering, tool use/function calling, structured outputs, agent orchestration (single- and multi-step), prompt caching.
  • Integration with model providers - Anthropic/Claude, AWS Bedrock/AgentCore, Snowflake Cortex - and response streaming.
  • AI quality & ops: eval harnesses, guardrails, tracing/observability, token/latency/cost optimization.
  • AI security: prompt injection, data exfiltration, PII handling, per-user identity/RBAC enforcement.

Nice to have

• • dbt, Airflow/MWAA, Snowflake.
  • MCP (Model Context Protocol) and/or RAG
  • Slack platform (Bolt, Socket Mode, Block Kit) or other real-time/conversational backends.
  • Fine-tuning/adaptation, semantic caching, or model routing/fallback.
  • Experience adding AI capabilities to existing production systems.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.