Staff Applied AI Engineer

Who you are

• Applied AI is a broad discipline. We don't expect every candidate to have deep expertise in every part of the stack
• We particularly encourage candidates with substantial experience in one or more of the following areas:
• Agentic systems and context engineering: Agent orchestration patterns, planning and execution, tool-use architectures, persistent and transient memory, and context management
• Information retrieval and knowledge representation: Semantic and structured retrieval, information extraction, document understanding, entity and relationship modeling, and knowledge-driven tool design
• AI evaluation and reliability: Evaluation methodology, benchmark and dataset construction, human and model-based assessment, statistical analysis, and systematic failure characterization
• Model adaptation and inference optimization: Fine-tuning, model selection, inference serving, quantization, batching, and performance optimization
• We value candidates who bring genuine depth to at least one of these areas, along with the curiosity and engineering judgment to work effectively across disciplinary boundaries
• What will make you successful:
• Strong software engineering fundamentals. You write maintainable production code, reason about system architecture, and understand testing, debugging, and operational reliability
• Substantive applied AI experience. You've worked on AI systems beyond basic model API integrations and can explain their architecture, tradeoffs, limitations, and observed behavior
• Information modeling instincts. You can understand a new domain, identify its important entities and relationships, and design software abstractions that reflect how its information is actually used
• Experimental discipline. You test assumptions, measure results, investigate failure modes, and use evidence to guide technical decisions
• Independent technical judgment. You can evaluate alternative approaches, recognize unnecessary complexity, and make appropriate tradeoffs
• Clear communication. You can discuss technical decisions with both technical and non-technical stakeholders and domain experts
• We care more about demonstrated technical understanding and the quality of your engineering work than familiarity with any particular AI framework
• Our systems primarily use Python, LangGraph, BAML, AWS Bedrock, PostgreSQL/pgvector, and event-driven AWS infrastructure
• Experience with these technologies is helpful but not required. We're more interested in your understanding of the underlying concepts and ability to work effectively with unfamiliar tools
• Aviation, aerospace, or another technically complex, safety-critical domain
• Computational linguistics, NLP, information science, or knowledge representation
• Designing systems that reason over technical documentation, regulations, or heterogeneous operational data
• Developing or operating distributed, event-driven, and containerized services
• Building AI systems in which traceability, evidence, and human oversight are important

What the job involves

• At Beacon AI, we're building AI-powered systems that help aviation professionals navigate complex, high-stakes operational decisions
• This role sits at the intersection of agentic AI, information retrieval, and production software engineering
• We build systems that reason over diverse sources of information, use tools to investigate problems, and produce reliable, explainable results
• We're looking for engineers who can think critically about how these systems should work, design their underlying information and orchestration architectures, and take them from experimental prototypes to dependable production services
• Design and implement agent architectures involving tool use, planning, orchestration, state management, and memory
• Develop effective context engineering strategies, balancing relevant information, model capabilities, latency, and cost
• Make thoughtful architectural decisions about when to use autonomous agents, explicit workflows, conventional algorithms, or combinations thereof
• Design tool interfaces and structured model interactions that support reliable, inspectable behavior
• Develop an understanding of complex document collections and data sources: their structure, semantics, authority, relationships, and scope
• Translate domain knowledge into effective retrieval systems, structured representations, and agent-accessible tools
• Apply information retrieval, information extraction, and knowledge modeling techniques to heterogeneous structured and unstructured data
• Design strategies for handling temporal relevance, provenance, conflicting evidence, and incomplete information
• Develop rigorous evaluations for AI-powered systems, including representative datasets, human annotations, automated assessments, and targeted failure analysis
• Investigate system-level failures across retrieval, context construction, reasoning, tool execution, and final outputs
• Experiment with alternative architectures, models, and prompting strategies, using empirical results to guide decisions
• Help establish engineering practices for measuring and improving the reliability of agentic systems
• Develop reliable services and asynchronous workflows with attention to concurrency, fault tolerance, observability, and maintainability
• Integrate model inference, retrieval systems, and external data services into production applications
• Build and operate containerized workloads and cloud infrastructure
• Collaborate with engineers and domain experts to translate complex operational requirements into working systems
• Examples of problems you might work on:
• Design an agentic workflow that identifies operational risks from aviation weather, airspace restrictions, aircraft-specific information, and regulatory guidance
• Develop retrieval and tool interfaces that allow agents to navigate multiple information sources while preserving source authority, provenance, and temporal relevance
• Build an evaluation framework that measures the factual completeness, correctness, and operational relevance of AI-generated analyses
• Investigate when an agent should perform additional research, use a specialized tool, or terminate its investigation
• Improve the reliability, latency, and cost of a multi-stage AI workflow through architectural changes informed by evaluation results