Want to know if this job is worth applying to?
Pune Job Posting Location - India, India
Onsite
Location: Pune, India
Seniority: Senior (7-12 years)
Purpose: Lead FNZ's unified AI evaluations and red teaming capability, ensuring all AI agent solutions meet rigorous safety, performance, and compliance standards before production deployment.
Key Responsibilities:
Define and evolve AI evaluation strategy aligned to FNZ's six-pillar framework (Task Performance, Safety & Compliance, Efficiency, Groundedness & Reasoning, Robustness, Suitability)
Establish evaluation standards, methodologies, and scoring criteria integrated into FNZ's SDLC as mandatory release gates
Represent evaluations function in AI Governance Committee, providing risk assessments and release recommendations
Build, mentor, and manage team of 4 evaluation specialists across generalist and specialist roles
Design and execute complex evaluations for high-risk AI agents; lead red teaming exercises for critical deployments
Communicate evaluation findings to technical and non-technical stakeholders; influence product roadmaps
Skills and Experience
7-12 years in AI/ML engineering, quality assurance, or security testing with 3+ years in evaluation/red teaming
Deep understanding of LLM-based agents, RAG architectures, and agentic AI systems (not model training)
Strong programming background; with real world experience developing evaluation solutions for agents.
Proven ability to design evaluation methodologies for probabilistic AI systems
Experience in regulated industries (financial services, healthcare) with compliance awareness
Leadership experience managing cross-functional teams and influencing stakeholders
About FNZ
FNZ is committed to opening up wealth so that everyone, everywhere can invest in their future on their terms. We know the foundation to do that already exists in the wealth management industry, but complexity holds firms back.
We created wealth’s growth platform to help. We provide a global, end-to-end wealth management platform that integrates modern technology with business and investment operations. All in a regulated financial institution.
We partner with the world’s leading financial institutions, with over US$2.4 trillion in assets on platform (AoP).
Together with our clients, we empower nearly 30 million people across all wealth segments to invest in their future.