Sr. Engineer II, ML Ops

Do you want to shape the future of fintech and healthtech? Energized by challenges and inspired by bold goals? Ready to elevate your career alongside driven and talented colleagues? If that sounds like you, explore a career at Alegeus today. Opportunity Happens Here. 

ROLE SUMMARY

We are seeking a highly technical Senior/Expert Engineer to establish enterprise-grade AI operations at Alegeus and, once those practices are embedded and team-owned, lead the transition into advanced AI research and emerging capabilities. This is a two-phase role by design: disciplined operational buildout first, then research and innovation leadership — not because one matters less than the other, but because research at scale requires the operational foundation to land in production.

The successful candidate is a hands-on builder who can lead. You will own the full AI model lifecycle, govern releases in a regulated environment, train engineers on production practices, and build systems designed from the start to run without your constant involvement. When operations are stable and team-owned, the role pivots deliberately toward emerging architectures, agentic systems, and capabilities that differentiate Alegeus.

PHASE 1: ESTABLISH & OPERATIONALIZE

Own the design and implementation of MLOps and LLMOps practices from scratch. Success here is measured by what the team can do without you.

MLOps & LLMOps Pipelines

• Design end-to-end model lifecycle: versioning, CI/CD, artifact registry, environment promotion, and deployment automation using Azure Machine Learning, MLflow, and Azure DevOps.
• Build LLMOps practices: prompt versioning, prompt testing harnesses, regression detection across prompt iterations, system instruction governance, and rollback.
• Implement automated retraining with scheduled, drift-triggered, and manual intervention points. Build self-healing mechanisms that detect degradation and remediate without human intervention.

Evaluation, Monitoring & Observability

• Build evaluation frameworks for traditional ML and LLM systems: golden datasets, automated test suites, and human-in-the-loop review.
• Monitor model performance, hallucination risk, token cost, latency, and production anomalies. Create dashboards, alerts, and incident response playbooks.
• Establish baseline metrics and production readiness criteria. Own incident triage, root cause analysis, and remediation.

Governed AI Operations

• Define and operationalize an AI Definition of Done: what evidence is required before production release — test results, evaluation scores, security review, rollback plan, monitoring readiness.
• Capture operational evidence for all releases: model version, prompt version, approver, deployment environment, evaluation metrics, and production readiness status.
• Implement separation-of-duties controls, human-in-the-loop approval gates, and audit trails for sensitive deployments in healthcare-regulated environments.
• Work with Security, Governance, Risk, Privacy, and Internal Audit to embed compliance into the release lifecycle — not bolt it on.
• Build safe rollout patterns: feature flags, canary releases, A/B testing, and phase-gated promotion based on evidence, not intent.

Engineering Leadership & Knowledge Transfer

• Establish playbooks, runbooks, and decision frameworks so teams own operations, not just execute tasks.
• Mentor AI engineers, data scientists, and platform engineers on production practices, deployment governance, and incident response.
• Lead architecture reviews and production readiness assessments. Codify patterns. Influence Security, Governance, Product, and Platform stakeholders without direct authority.

Phase 1 is complete when: all AI releases follow a shared governed process. Teams operate independently. Audit trails are complete and defensible. You are no longer in the critical path of every deployment.

PHASE 2: RESEARCH & INNOVATION LEADERSHIP

Operations are team-owned. Your focus shifts to emerging capabilities, applied research, and defining where AI takes Alegeus next. You remain grounded in production constraints — research that cannot ship is not research.

Emerging AI Research & Assessment

• Continuously monitor AI research: large language models, multimodal systems, agentic workflows, advanced RAG architectures, and new model paradigms.
• Evaluate build-versus-buy decisions. Run proof-of-concepts. Convert successful experiments into production-ready capabilities that integrate with the established platform.

Agentic AI & Advanced Capabilities

• Design autonomous and semi-autonomous AI workflows using agentic frameworks (AutoGen, CrewAI, LangChain, Semantic Kernel). Build orchestration patterns for multi-agent systems.
• Architect advanced RAG systems integrating Alegeus's Snowflake-native data platform with vector databases and embedding models. Improve retrieval quality, grounding, and hallucination controls.
• Extend the AI platform for new paradigms: fine-tuning, distillation, hybrid retrieval, and domain-specific model architectures. Optimize inference cost, latency, and accuracy.

Applied Research & Product Impact

• Drive research into domain-specific problems — claims accuracy, document processing, workflow automation — and translate findings into product capabilities.
• Shape AI platform strategy. Influence product roadmap through research insights. Partner with engineering, data science, and product leadership on differentiation through AI.
• Mentor next-generation AI practitioners. Share research findings with teams and leadership. Remain hands-on in high-impact proof-of-concepts.

Phase 2 is successful when: 2-3 new AI capabilities have launched from research leadership. The platform supports novel architectures. Product strategy is shaped by your findings. Teams view you as Alegeus's technical North Star for AI.

QUALIFICATIONS

Required

• 7+ years software engineering; 5+ years leading AI/ML initiatives at production scale
• Deep hands-on expertise in 4+ of: MLOps, LLMOps, evaluation frameworks, drift monitoring, CI/CD for ML, model governance
• Strong Python; able to write, review, and debug production systems
• Experience deploying AI in regulated environments: healthcare, fintech, benefits administration, or equivalent
• Proven ability to establish practices teams adopt and operate independently
• Comfortable with ambiguity; prioritizes ruthlessly between operational rigor and research exploration

Preferred

• Azure ML, MLflow, Azure DevOps, Azure OpenAI, or comparable LLM platforms
• Agentic AI frameworks: AutoGen, CrewAI, LangChain, Semantic Kernel
• Graph DB, Vector databases, embedding models, RAG architectures, context engineering, harness engineering
• Healthcare benefits, claims processing, or consumer-directed healthcare
• Docker, Kubernetes/AKS, Terraform, secrets management, RBAC
• Track record converting AI research into production capabilities

SUCCESS PROFILE

Builder-first: 60-70% hands-on in Phase 1. Writes code, designs systems, ships infrastructure.

Pragmatist: Establishes "good enough" governance first, iterates to enterprise-grade. Balances perfection with progress.

Operations-minded: Thinks in runbooks, monitoring, and audit trails. Owns compliance without being paralyzed by it.

Teacher & multiplier: Measures success by what teams can do without them. Codifies patterns, grows others.

Research-curious: Follows AI research. Runs experiments. Comfortable moving between operations and emerging architectures.

Strategist: Sees beyond current products. Articulates a point of view on where AI should take Alegeus.

WHY ALEGEUS

Alegeus is at an inflection point: foundational AI services in flight, a strong Snowflake-native data platform, and growing demand for AI-enabled capabilities. What we need is someone to establish operational rigor and governance first — then lead the research and innovation that follows. This role is not forever operational. The first phase is foundational. The second phase is where you define the future of AI at Alegeus.

You will work in a regulated, mission-critical domain where your decisions affect real systems and real users. You will influence product strategy, platform architecture, and engineering culture. And you will have the rare opportunity to define not just what gets built, but how AI adoption happens at enterprise scale.

BECAUSE WE CARE, WE OFFER: 

We are committed to a policy of Equal Employment Opportunity and will not discriminate against an applicant or employee on the basis of race, color, religion, creed, national origin or ancestry, sex, age, physical or mental disability, veteran or military status, genetic information, sexual orientation, marital status, or any other legally recognized protected basis under federal, state or local laws, regulations or ordinances. The information collected by this application is solely to determine suitability for employment, verify identity and maintain employment statistics on applicants.

At Alegeus, being transparent about our compensation philosophy and approach is more than just a legal requirement. As our organization continues to grow and evolve, we have made a commitment to ensure that our compensation framework is equitable, data-driven, consistent, and unbiased, with allowable pay differences based on factors unique to each candidate (think: skills, experience, qualifications, etc.) in order to attract and retain a highly talented and committed workforce.

We are taking an “inside-out” approach to pay transparency by first educating our valued managers and internal workforce and then moving to publishing compensation ranges externally.  In the interim, if you are a California, Colorado, Connecticut, Maryland, Nevada, New Jersey, New York, Ohio or Washington resident and this role is physically available in your state or classified as remote, you may be eligible to receive additional information about the compensation and benefits for this role, which we will provide upon request. Please send an email identifying the title of the role you are interested in and the state you reside in to talentacquisitionteam@alegeus.com.

Alegeus may use AI technology during candidate interviews. The uses include recording, note-taking, and summarizing candidate interviews. The information generated by the AI technology will be used by Alegeus during the hiring process. If you wish to opt out of having AI technology transcribe your interview, please notify your recruiter in advance of the interview. Otherwise, by agreeing to an interview with us, you consent to the use of AI technology during your interview.