Want to know if this job is worth applying to?
United States
Remote only
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Director of AI Operations and Insurance Domain Quality based in the United States.
The Director of AI Operations and Insurance Domain Quality will build and lead a specialized team responsible for evaluating AI-generated outputs across enterprise commercial insurance workflows.
This role combines deep Property & Casualty (P&C) insurance expertise with operational leadership and AI quality governance.
You will establish the evaluation function from the ground up, initially managing a team of approximately 10 domain experts and scaling operations as demand grows.
Your focus will be on developing rigorous quality standards, optimizing reviewer productivity, and ensuring consistent accuracy across complex insurance use cases.
Working closely with client product leaders and machine learning engineers, you will translate domain insights into actionable feedback that improves AI performance.
This is a remote leadership opportunity for an experienced insurance professional passionate about operational excellence and the practical application of AI in the insurance industry.
As the operational leader of the AI evaluation function, you will oversee team development, quality governance, workflow performance, and continuous improvement. Your responsibilities will include:
Program Stand-Up and Scaling
Design and implement the operational framework for the AI evaluation function.
Launch and manage an initial team of approximately 10 insurance domain evaluators.
Scale team capacity and operational infrastructure in response to client demand.
Monitor launch speed, queue capacity growth, and reviewer time-to-proficiency.
Quality Assurance and Domain Governance
Establish productivity and evaluation standards, quality audit rubrics, and calibration procedures.
Conduct secondary reviews of complex insurance cases and provide targeted coaching to improve evaluation accuracy.
Audit policy edge cases and identify recurring errors or inconsistencies in AI-generated outputs.
Track quality metrics and implement corrective actions to reduce error rates across active evaluation queues.
Operational Queue and SLA Management
Build and manage daily workflow queues, ensuring efficient distribution of evaluation tasks.
Monitor reviewer throughput, productivity, and operational capacity.
Optimize resource allocation to meet strict client turnaround times and service-level agreements.
Identify and eliminate workflow bottlenecks while maintaining consistent service quality.
Cross-Functional Product and Engineering Collaboration
Consolidate audit findings and identify systemic AI failure patterns.
Lead recurring feedback sessions with product and machine learning engineering teams.
Translate insurance domain insights into actionable recommendations for improving prompts, model behavior, and evaluation logic.
Track defect reporting quality, model accuracy improvements, and reductions in recurring AI processing errors.
Team Leadership and Development
Recruit, mentor, manage, and evaluate junior and senior insurance domain evaluators.
Partner with learning and development teams to create continuous training programs.
Ensure evaluators remain up to date on insurance policy standards, domain practices, and AI audit tools.
Monitor team retention, skill development, performance, and calibration consistency.
The ideal candidate brings extensive commercial insurance experience, proven operational leadership, and the ability to apply specialized domain knowledge to emerging AI evaluation workflows. The following qualifications are expected:
At least 8–10 years of progressive leadership experience in commercial Property & Casualty insurance operations, underwriting quality control, brokerage management, or insurtech program operations.
Active Resident or Non-Resident Property & Casualty (P&C) Producer/Agent License.
Professional designations such as CPCU, CIC, or ARM are strongly preferred.
Deep technical knowledge of commercial P&C insurance lines, including General Liability, Commercial Property, Workers' Compensation, Umbrella/Excess, and Cyber.
Strong understanding of standard insurance forms and documentation, including ACORD forms, ISO forms, and manuscript endorsements.
Demonstrated experience building operational teams and establishing workflow processes and quality assurance frameworks from the ground up.
Proven ability to manage team capacity, monitor operational SLAs, and scale teams from initial groups of approximately 10 employees to larger enterprise operations.
Familiarity with AI/ML evaluation workflows, document extraction platforms, or Reinforcement Learning from Human Feedback (RLHF) methodologies.
Experience with insurance or document management platforms such as Applied Epic, Vertafore, Hyperscience, or Glider is advantageous.
Strong analytical and problem-solving skills, particularly in identifying quality issues and translating findings into operational improvements.
Excellent executive communication and stakeholder management skills, with the ability to collaborate directly with client product leaders, machine learning engineers, and executive sponsors.
Proven ability to lead, coach, and develop teams while maintaining high standards of quality and performance.
Annual salary range of $135,000–$145,000 USD, depending on qualifications, relevant experience, technical expertise, and role requirements.
Comprehensive healthcare coverage for employees and their families.
Employee Assistance Program, including counseling and coaching support for employees and family members.
401(k) retirement plan with company matching contributions.
Paid time off.
Paid parental leave.
Volunteer Day Off.
Life insurance coverage.
Business travel accident insurance.
Remote work arrangement within the United States.
.jpg?1761221989)