E
Exa
Software Engineer (Distributed Data Systems)
Skills
Kafka
2 months ago
Share this opportunity
Share this opportunity to other talents of your network:
✓ Offer them a visibility boost with clients.
✓ Help your contacts find their next job.
Important information
Contract type:
Permanent
Salary:
$150,000 – $300,000 / year
Location:
San Francisco, CA, USA
Work mode:
Onsite
Published on:
29 August 2026
What they need
- As a Data Engineer, you’ll architect and build the data infrastructure that powers everything we do—from crawling billions of pages to training our embedding models to serving real-time search
- You’ll have enormous autonomy in designing systems that scale to hundreds of petabytes
- Design a lakehouse architecture that handles 100+ PB of web crawl data
- Build streaming pipelines that process billions of documents per day for real-time indexing
- Architect the data layer for our embedding training infrastructure on Ray
- Scale our ClickHouse deployment to handle analytical queries across petabytes of search logs- If you’ve ever wanted to build data pipelines at a scale that most companies only dream about, this is your chance
- An obsessive focus on reliability and building systems that don’t page you at 3am
- Familiarity with Ray, Spark, or ClickHouse at production scale
- Deep understanding of lakehouse architectures (Delta Lake, Iceberg, Hudi) and when to use them
- Hands-on experience with streaming data systems (Kafka, Flink, or similar)
- Experience building and operating large-scale distributed data processing pipelines
- Experience with Lance or other vector-native storage formats
- Background in GPU-accelerated data processing (RAPIDS, cuDF)
Other offers great for you!
These companies are also looking for great profiles
N
NGC x OPTIMA
Ingénieur Logiciel embarqué
Freelance
Cergy, France
Hybrid, Onsite
Skills
AUTOSARBSW
1 month ago
Quick apply
S
Salutech
Ingénieur Logiciel IA
Freelance
Paris, France
Onsite, Hybrid
Skills
DockerPythonTesting & QAKubernetesSoftware refactoringCI/CDObservabilityAsynchronous programming
5 days ago
Quick apply
F
Fakto
AI Software Engineer
€70k–€90k
Permanent contract
Neuilly-sur-Seine, France
Hybrid
Skills
LLMs / Large language modelsBackend DevelopmentDevOpsAWSTerraform
3 hours ago
Quick apply
No company information available.