Want to know if this job is worth applying to?
United States
Remote only
Not Available
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Engineer, AI Systems based in the United States.
This is a high-ownership engineering role focused on building production-grade AI and LLM systems for enterprise safety applications.
You will design and operate multi-step reasoning pipelines that combine language models, retrieval, tools, and knowledge graphs.
Your work will directly influence how investigators and safety leaders analyze incidents and turn organizational knowledge into actionable insights.
The role emphasizes precision, traceability, evaluation, observability, and human oversight because system outputs can inform consequential decisions.
You will work across agent orchestration, retrieval, evaluation, model selection, production operations, and AI security.
As part of a small, early-stage team, you will collaborate closely with technical and product leadership and take ownership from design through production.
The position offers the opportunity to solve technically challenging problems while seeing your work reach enterprise customers quickly.
Build and operate production LLM pipelines coordinating model calls, tool calls, graph queries, retrieval, quality gates, and specialist-agent handoffs.
Extend agent orchestration systems that move incidents from evidence collection through analysis, review, and organizational learning.
Design grounding and retrieval strategies using graph traversal, vector search, and hybrid retrieval to provide models with relevant evidence and organizational knowledge.
Develop evaluation datasets, scoring systems, regression suites, model comparisons, LLM-as-judge workflows, and human-review loops for extraction and reasoning tasks.
Implement production AI operations capabilities including tracing, tool-call auditing, cost and latency monitoring, failure handling, and quality dashboards.
Identify and address hallucinations, agent loops, silent model drift, regressions, and other production-quality issues before they affect customers.
Evaluate and select models across providers based on quality, latency, cost, context capabilities, and operational risk.
Partner with product and knowledge engineering teams to shape technical direction and the AI roadmap.
Contribute to enterprise AI security practices, including prompt-injection protection, context-leak prevention, tenant isolation, access controls, and policy separation where applicable.
Professional AI/ML engineering experience with a demonstrated track record of shipping production LLM systems used by real users.
Hands-on experience building and debugging multi-step, tool-calling agent workflows using LangGraph, LangChain, or an equivalent framework.
Strong understanding of LLM evaluation, including representative datasets, regression testing, LLM-as-judge approaches, and/or human evaluation loops.
Experience designing retrieval and context-assembly systems, with the ability to make informed decisions about what information to retrieve, how much to provide, and why.
Demonstrated ownership of production systems through deployment, monitoring, troubleshooting, and incident response, including experience diagnosing and resolving failures or regressions.
Strong judgment when working across multiple model providers and evaluating tradeoffs involving quality, latency, cost, context, and operational risk.
Experience with Neo4j and Cypher or a comparable graph database is strongly preferred, along with the ability to quickly learn graph data modeling.
Strong Python development skills and production experience with technologies such as FastAPI, asynchronous services, automated testing, observability, and maintainable software interfaces.
Experience with graph schema evolution, embeddings, and operating live knowledge graphs is a plus.
Familiarity with enterprise AI security, including prompt injection, context isolation, tenant separation, role-based access, and policy-layer controls is advantageous.
Experience with Azure and hybrid search technologies such as Azure AI Search, Pinecone, MongoDB Atlas, pgvector, Elasticsearch, or similar platforms is a plus.
Previous experience in B2B enterprise SaaS environments is strongly preferred.
Ability to work effectively in a small, fast-moving team where ownership, adaptability, and independent technical decision-making are important.
Must be based in the United States; relocation is not considered for this position.
Visa sponsorship is not available for this role.
High-impact opportunity to solve complex AI reasoning problems in a safety-critical enterprise environment.
Evaluation-first engineering culture focused on making AI quality measurable, observable, and continuously improvable.
Direct exposure to customer needs and feedback from safety teams across energy, utilities, infrastructure, construction, and manufacturing.
High level of technical ownership within a small, early-stage team.
Close collaboration with technical leadership, product, and knowledge engineering teams.
Opportunity to make consequential architectural and product decisions and see implementations reach customers quickly.
Exposure to advanced LLM orchestration, knowledge graphs, retrieval systems, AI evaluation, and production AI operations.
Remote role for candidates based in the United States.
