Principal Production Engineer

• Own the reliability and scalability of Crusoe’s cloud infrastructure — compute, storage, and networking — defining SLOs, leading incident response, and driving systemic improvements that reduce toil and raise the bar across the platform
• Build and mature the observability and tooling layer — from network fabric telemetry and storage health to control plane instrumentation and on-call tooling — so the team can detect, diagnose, and resolve issues faster than customers notice them
• Drive platform reliability improvements across the full cloud stack, partnering closely with software, hardware, and network engineering teams to influence architecture decisions early, before they become operational debt
• Act as a trusted advisor to senior leadership, bringing perspective on observability trends and advocating for the right long-term technology investments
• Set the technical standards for how Crusoe’s production engineering organization builds, operates, and scales — defining on-call culture, incident frameworks, and reliability practices that grow with the company
• Mentor senior and staff engineers, elevate the team’s collective technical depth, and be the person others seek out when the problem is genuinely hard

Benefits

• Health & wellbeing: Comprehensive health benefits designed to support your overall wellness
• Time away: Paid time off for vacations, family bonding, and unexpected needs
• 401(k) match: Build your financial future with our 401(k) matching program
• Mental wellness: Resources and support for your emotional wellbeing and navigating life’s challenges- Strong systems fundamentals: Linux, distributed systems, storage, compute scheduling — you understand the full stack from hardware up
• 15+ years of experience in infrastructure, networking, or production engineering — with meaningful time at companies operating at internet scale (cloud providers, CDNs, large-scale social/media platforms, or similar)
• The ability to write code — not necessarily full-time, but enough to automate what shouldn’t be manual, instrument what isn’t observable, and build tooling your team will actually use
• Strong incident command: you lead calmly under pressure, communicate clearly during outages, and run blameless retrospectives that actually improve systems
• Hands-on data center experience: you’ve done physical infra, understand power and thermal constraints, and can reason about reliability at the facility level, not just the server level
• Excellent analytical and problem-solving skills, including the ability to synthesize ambiguous customer and system signals into clear problem statements and designs
• Deep networking expertise: BGP, OSPF, ECMP, load balancing, and low-latency network design in production — you can debug a routing issue and design a fabric, sometimes in the same incident
• Experience with HPC infrastructure: GPU cluster operations, job schedulers (Slurm, Kubernetes), high-bandwidth interconnects (InfiniBand, RoCE)
• Prior principal or staff IC role where you influenced org-level technical strategy, not just project-level execution
• Exposure to sustainability-focused or energy-constrained compute environments