AI Enablement & Governance– AI Quality & Evaluation Lead

Our story

At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.”  We are passionate about connecting purpose with impact.  Alight empowers clients to build a healthier and more financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, navigation, and absence management.

Our Benefits

With a comprehensive total rewards package, Alight offers programs and plans that support your mind, body, wallet, and life. Benefits include health, dental and vision coverages starting Day One. Additionally, Alight colleagues enjoy wellbeing programs, retirement plans with contribution matching, generous time off, parental leave, continuing education, and career growth opportunities – all within a thriving global organization.

Flexible Working

So that you can be your best at work and home, we consider flexible working arrangements wherever possible. Alight has been a leader in the flexible workspace and “Top 100 Company for Remote Jobs” 6 years in a row.

Great Place to Work

Thanks to the work of every colleague, Alight has received multiple awards of recognition including “Great Place to Work” for the past 7 years and Fortune’s “Best Companies to Work For.” To learn more about our company culture and awards Click Here.

If you, Champion People, seek to Grow with Purpose, and embody the meaning of Be Alight – We invite you to join our team!  Learn more at careers.alight.com. 

The Role

The AI Quality & Evaluation lead enables responsible and scalable AI adoption by defining technical quality standards, evaluation framework and control requirements across the AI lifecycle. As a function owner of AI quality and robustness standards, the role translates enterprise trust principles and statistical rigor into practical evaluation standards and enforceable technical controls.  The role partners closely with AI Engineering, Data Scientists, QA CoE, and Product teams to define quality and performance requirements that are embedded by design, ensuring AI solutions—including RAG systems and Agents—are accurate, grounded, and aligned with enterprise risk and trust expectations.

Responsibilities

Quality-by-Design Partnership

• Partnering directly with AI Engineers, Application Developers and Data Scientists during the design phase to define technical quality acceptance criteria and fit-for-use requirements.
• Embedding quality considerations into model and system architecture from the onset, specifically for complex patterns like RAG and autonomous Agents.
• Defining golden truth requirements and evaluation dataset standards; partner with Data Science teams to ensure datasets reflect production-level complexity.
• Defining quality and evaluation expectations for third-party AI systems and vendor-supplied models, ensuring consistent governance standards regardless of model origin.

Technical Evaluation & Metric Engineering

• Designing and maintaining structured evaluation framework that assesses AI system against defined quality bars (e.g. Goodness-of-Fit, Calibration, Stability).
• Developing automated metrics for Generative AI performance including Groundedness (Hallucination detection), Faithfulness, Completeness, and other domain-relevant metrics.
• Defining and operationalize fairness and bias evaluation criteria, including demographic parity assessments and disparate impact testing for client-facing AI systems.
• Calibrating evaluation thresholds and monitoring cadence to AI risk tier, ensuring proportionate controls without over-engineering lower-risk use cases. 

Technical Control & Monitoring

• Identifying and document technical AI governance controls that enable automated compliance with performance and risk obligations.
• Establishing drift and ongoing monitoring requirements, defining statistical triggers for feature and concept drift that necessitate model intervention.
• Developing clear control statements that articulate the expected evidence artifacts (e.g. test results, model cards) required go/no-go decisions.

Governance & Evidence Enablement

• Providing objective, data-driven evaluation outputs that support AI governance reviews and risk classification.
• Translating governance expectations into clear, testable quality criteria that engineering teams can apply consistently within their CI/CD pipelines.
• Maintaining authoritative documentation of AI controls to support audit, regulatory review, and internal assurance activities.

Requirements

• Technical Depth: 5–8+ years of experience in Data Science, ML Engineering, or AI Quality, with a focus on evaluation and statistical validation.
• System Design: Practical experience partnering with engineers to design RAG, LLM-based Agents, or traditional ML pipelines.
• Analytical Skills: Expert-level Python (Pandas, Scikit-learn) and experience with evaluation frameworks (e.g., RAGAS, TruLens, or MLflow).
• Governance Mindset: Demonstrated ability to translate abstract trust concepts into mathematical metrics and enforceable technical controls.
• Stakeholder Influence: Demonstrated ability to work without direct authority, driving quality adoption across engineering and product teams through enablement. 
• Communication: Ability to bridge the gap between high-level governance policy and low-level code implementation.
• Bachelor’s degree in a technical field (e.g., Computer Science, Computer Systems Design) or equivalent professional experience

Application and Interview

By applying for a position with Alight, you understand that, should you be made an offer, it will be contingent on your undergoing and successfully completing a background check consistent with Alight’s employment policies. Background checks may include some or all the following based on the nature of the position: SSN/SIN validation, education verification, employment verification, and criminal check, search against global sanctions and government watch lists, credit check, and/or drug test. You will be notified during the hiring process which checks are required by the position.

Alight requires all virtual interviews to be conducted on video. 

Our commitment to Inclusion

We celebrate differences and believe in fostering an environment where everyone feels valued, respected, and supported. We know that diverse teams are stronger, more innovative, and more successful.

At Alight, we welcome and embrace all individuals, regardless of their background, and are dedicated to creating a culture that enables every employee to thrive. Join us in building a brighter, more inclusive future.

As part of this commitment, Alight will ensure that persons with disabilities are provided reasonable accommodations for the hiring process. If reasonable accommodation is needed, please contact alightcareers@alight.com.

Equal Opportunity Policy Statement

Alight is an Equal Employment Opportunity employer and does not discriminate against anyone based on sex, race, color, religion, creed, national origin, ancestry, age, physical or mental disability, medical condition, pregnancy, marital or domestic partner status, citizenship, military or veteran status, sexual orientation, gender, gender identity or expression, genetic information, or any other legally protected characteristics or conduct covered by federal, state, or local law.   In addition, we take affirmative action to employ disabled persons, disabled veterans, and other covered veterans.

Alight provides reasonable accommodations to the known limitations of otherwise qualified employees and applicants for employment with disabilities and sincerely held religious beliefs, practices and observances, unless doing so would result in undue hardship. Applicants for employment may request  reasonable accommodations/modifications by contacting their recruiter.

Authorization to work in the Employing Country

Applicants for employment in the country in which they are applying (Employing Country) must have work authorization that does not, now or in the future, require sponsorship of a visa for employment authorization in the Employing Country and with Alight.

Note, this job description does not restrict management's right to assign or reassign duties and responsibilities of this job to other entities; including but not limited to subsidiaries, partners, or purchasers of Alight business units.

Our story

At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.”  We are passionate about connecting purpose with impact.  Alight empowers clients to build a healthier and more financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, navigation, and absence management.

Our Benefits

With a comprehensive total rewards package, Alight offers programs and plans that support your mind, body, wallet, and life. Benefits include health, dental and vision coverages starting Day One. Additionally, Alight colleagues enjoy wellbeing programs, retirement plans with contribution matching, generous time off, parental leave, continuing education, and career growth opportunities – all within a thriving global organization.

Flexible Working

So that you can be your best at work and home, we consider flexible working arrangements wherever possible. Alight has been a leader in the flexible workspace and “Top 100 Company for Remote Jobs” 6 years in a row.

Great Place to Work

Thanks to the work of every colleague, Alight has received multiple awards of recognition including “Great Place to Work” for the past 7 years and Fortune’s “Best Companies to Work For.” To learn more about our company culture and awards Click Here.

If you, Champion People, seek to Grow with Purpose, and embody the meaning of Be Alight – We invite you to join our team!  Learn more at careers.alight.com. 

The Role

The AI Quality & Evaluation lead enables responsible and scalable AI adoption by defining technical quality standards, evaluation framework and control requirements across the AI lifecycle. As a function owner of AI quality and robustness standards, the role translates enterprise trust principles and statistical rigor into practical evaluation standards and enforceable technical controls.  The role partners closely with AI Engineering, Data Scientists, QA CoE, and Product teams to define quality and performance requirements that are embedded by design, ensuring AI solutions—including RAG systems and Agents—are accurate, grounded, and aligned with enterprise risk and trust expectations.

Responsibilities

Quality-by-Design Partnership

• Partnering directly with AI Engineers, Application Developers and Data Scientists during the design phase to define technical quality acceptance criteria and fit-for-use requirements.
• Embedding quality considerations into model and system architecture from the onset, specifically for complex patterns like RAG and autonomous Agents.
• Defining golden truth requirements and evaluation dataset standards; partner with Data Science teams to ensure datasets reflect production-level complexity.
• Defining quality and evaluation expectations for third-party AI systems and vendor-supplied models, ensuring consistent governance standards regardless of model origin.

Technical Evaluation & Metric Engineering

• Designing and maintaining structured evaluation framework that assesses AI system against defined quality bars (e.g. Goodness-of-Fit, Calibration, Stability).
• Developing automated metrics for Generative AI performance including Groundedness (Hallucination detection), Faithfulness, Completeness, and other domain-relevant metrics.
• Defining and operationalize fairness and bias evaluation criteria, including demographic parity assessments and disparate impact testing for client-facing AI systems.
• Calibrating evaluation thresholds and monitoring cadence to AI risk tier, ensuring proportionate controls without over-engineering lower-risk use cases. 

Technical Control & Monitoring

• Identifying and document technical AI governance controls that enable automated compliance with performance and risk obligations.
• Establishing drift and ongoing monitoring requirements, defining statistical triggers for feature and concept drift that necessitate model intervention.
• Developing clear control statements that articulate the expected evidence artifacts (e.g. test results, model cards) required go/no-go decisions.

Governance & Evidence Enablement

• Providing objective, data-driven evaluation outputs that support AI governance reviews and risk classification.
• Translating governance expectations into clear, testable quality criteria that engineering teams can apply consistently within their CI/CD pipelines.
• Maintaining authoritative documentation of AI controls to support audit, regulatory review, and internal assurance activities.

Requirements

• Technical Depth: 5–8+ years of experience in Data Science, ML Engineering, or AI Quality, with a focus on evaluation and statistical validation.
• System Design: Practical experience partnering with engineers to design RAG, LLM-based Agents, or traditional ML pipelines.
• Analytical Skills: Expert-level Python (Pandas, Scikit-learn) and experience with evaluation frameworks (e.g., RAGAS, TruLens, or MLflow).
• Governance Mindset: Demonstrated ability to translate abstract trust concepts into mathematical metrics and enforceable technical controls.
• Stakeholder Influence: Demonstrated ability to work without direct authority, driving quality adoption across engineering and product teams through enablement. 
• Communication: Ability to bridge the gap between high-level governance policy and low-level code implementation.
• Bachelor’s degree in a technical field (e.g., Computer Science, Computer Systems Design) or equivalent professional experience

Application and Interview

By applying for a position with Alight, you understand that, should you be made an offer, it will be contingent on your undergoing and successfully completing a background check consistent with Alight’s employment policies. Background checks may include some or all the following based on the nature of the position: SSN/SIN validation, education verification, employment verification, and criminal check, search against global sanctions and government watch lists, credit check, and/or drug test. You will be notified during the hiring process which checks are required by the position.

Alight requires all virtual interviews to be conducted on video. 

Our commitment to Inclusion

We celebrate differences and believe in fostering an environment where everyone feels valued, respected, and supported. We know that diverse teams are stronger, more innovative, and more successful.

At Alight, we welcome and embrace all individuals, regardless of their background, and are dedicated to creating a culture that enables every employee to thrive. Join us in building a brighter, more inclusive future.

As part of this commitment, Alight will ensure that persons with disabilities are provided reasonable accommodations for the hiring process. If reasonable accommodation is needed, please contact alightcareers@alight.com.

Equal Opportunity Policy Statement

Alight is an Equal Employment Opportunity employer and does not discriminate against anyone based on sex, race, color, religion, creed, national origin, ancestry, age, physical or mental disability, medical condition, pregnancy, marital or domestic partner status, citizenship, military or veteran status, sexual orientation, gender, gender identity or expression, genetic information, or any other legally protected characteristics or conduct covered by federal, state, or local law.   In addition, we take affirmative action to employ disabled persons, disabled veterans, and other covered veterans.

Alight provides reasonable accommodations to the known limitations of otherwise qualified employees and applicants for employment with disabilities and sincerely held religious beliefs, practices and observances, unless doing so would result in undue hardship. Applicants for employment may request  reasonable accommodations/modifications by contacting their recruiter.

Authorization to work in the Employing Country

Applicants for employment in the country in which they are applying (Employing Country) must have work authorization that does not, now or in the future, require sponsorship of a visa for employment authorization in the Employing Country and with Alight.

Note, this job description does not restrict management's right to assign or reassign duties and responsibilities of this job to other entities; including but not limited to subsidiaries, partners, or purchasers of Alight business units.

Our story

At Alight, we believe a company’s success starts with its people. At our core, we Champion People, help our colleagues Grow with Purpose and true to our name we encourage colleagues to “Be Alight.”  We are passionate about connecting purpose with impact.  Alight empowers clients to build a healthier and more financially secure workforce by unifying the benefits ecosystem across health, wealth, wellbeing, navigation, and absence management.

Our Benefits

With a comprehensive total rewards package, Alight offers programs and plans that support your mind, body, wallet, and life. Benefits include health, dental and vision coverages starting Day One. Additionally, Alight colleagues enjoy wellbeing programs, retirement plans with contribution matching, generous time off, parental leave, continuing education, and career growth opportunities – all within a thriving global organization.

Flexible Working

So that you can be your best at work and home, we consider flexible working arrangements wherever possible. Alight has been a leader in the flexible workspace and “Top 100 Company for Remote Jobs” 6 years in a row.

Great Place to Work

Thanks to the work of every colleague, Alight has received multiple awards of recognition including “Great Place to Work” for the past 7 years and Fortune’s “Best Companies to Work For.” To learn more about our company culture and awards Click Here.

If you, Champion People, seek to Grow with Purpose, and embody the meaning of Be Alight – We invite you to join our team!  Learn more at careers.alight.com. 

The Role

The AI Quality & Evaluation lead enables responsible and scalable AI adoption by defining technical quality standards, evaluation framework and control requirements across the AI lifecycle. As a function owner of AI quality and robustness standards, the role translates enterprise trust principles and statistical rigor into practical evaluation standards and enforceable technical controls.  The role partners closely with AI Engineering, Data Scientists, QA CoE, and Product teams to define quality and performance requirements that are embedded by design, ensuring AI solutions—including RAG systems and Agents—are accurate, grounded, and aligned with enterprise risk and trust expectations.

Responsibilities

Quality-by-Design Partnership

• Partnering directly with AI Engineers, Application Developers and Data Scientists during the design phase to define technical quality acceptance criteria and fit-for-use requirements.
• Embedding quality considerations into model and system architecture from the onset, specifically for complex patterns like RAG and autonomous Agents.
• Defining golden truth requirements and evaluation dataset standards; partner with Data Science teams to ensure datasets reflect production-level complexity.
• Defining quality and evaluation expectations for third-party AI systems and vendor-supplied models, ensuring consistent governance standards regardless of model origin.

Technical Evaluation & Metric Engineering

• Designing and maintaining structured evaluation framework that assesses AI system against defined quality bars (e.g. Goodness-of-Fit, Calibration, Stability).
• Developing automated metrics for Generative AI performance including Groundedness (Hallucination detection), Faithfulness, Completeness, and other domain-relevant metrics.
• Defining and operationalize fairness and bias evaluation criteria, including demographic parity assessments and disparate impact testing for client-facing AI systems.
• Calibrating evaluation thresholds and monitoring cadence to AI risk tier, ensuring proportionate controls without over-engineering lower-risk use cases. 

Technical Control & Monitoring

• Identifying and document technical AI governance controls that enable automated compliance with performance and risk obligations.
• Establishing drift and ongoing monitoring requirements, defining statistical triggers for feature and concept drift that necessitate model intervention.
• Developing clear control statements that articulate the expected evidence artifacts (e.g. test results, model cards) required go/no-go decisions.

Governance & Evidence Enablement

• Providing objective, data-driven evaluation outputs that support AI governance reviews and risk classification.
• Translating governance expectations into clear, testable quality criteria that engineering teams can apply consistently within their CI/CD pipelines.
• Maintaining authoritative documentation of AI controls to support audit, regulatory review, and internal assurance activities.

Requirements

• Technical Depth: 5–8+ years of experience in Data Science, ML Engineering, or AI Quality, with a focus on evaluation and statistical validation.
• System Design: Practical experience partnering with engineers to design RAG, LLM-based Agents, or traditional ML pipelines.
• Analytical Skills: Expert-level Python (Pandas, Scikit-learn) and experience with evaluation frameworks (e.g., RAGAS, TruLens, or MLflow).
• Governance Mindset: Demonstrated ability to translate abstract trust concepts into mathematical metrics and enforceable technical controls.
• Stakeholder Influence: Demonstrated ability to work without direct authority, driving quality adoption across engineering and product teams through enablement. 
• Communication: Ability to bridge the gap between high-level governance policy and low-level code implementation.
• Bachelor’s degree in a technical field (e.g., Computer Science, Computer Systems Design) or equivalent professional experience

Application and Interview

By applying for a position with Alight, you understand that, should you be made an offer, it will be contingent on your undergoing and successfully completing a background check consistent with Alight’s employment policies. Background checks may include some or all the following based on the nature of the position: SSN/SIN validation, education verification, employment verification, and criminal check, search against global sanctions and government watch lists, credit check, and/or drug test. You will be notified during the hiring process which checks are required by the position.

Alight requires all virtual interviews to be conducted on video. 

Our commitment to Inclusion

We celebrate differences and believe in fostering an environment where everyone feels valued, respected, and supported. We know that diverse teams are stronger, more innovative, and more successful.

At Alight, we welcome and embrace all individuals, regardless of their background, and are dedicated to creating a culture that enables every employee to thrive. Join us in building a brighter, more inclusive future.

As part of this commitment, Alight will ensure that persons with disabilities are provided reasonable accommodations for the hiring process. If reasonable accommodation is needed, please contact alightcareers@alight.com.

Equal Opportunity Policy Statement

Alight is an Equal Employment Opportunity employer and does not discriminate against anyone based on sex, race, color, religion, creed, national origin, ancestry, age, physical or mental disability, medical condition, pregnancy, marital or domestic partner status, citizenship, military or veteran status, sexual orientation, gender, gender identity or expression, genetic information, or any other legally protected characteristics or conduct covered by federal, state, or local law.   In addition, we take affirmative action to employ disabled persons, disabled veterans, and other covered veterans.

Alight provides reasonable accommodations to the known limitations of otherwise qualified employees and applicants for employment with disabilities and sincerely held religious beliefs, practices and observances, unless doing so would result in undue hardship. Applicants for employment may request  reasonable accommodations/modifications by contacting their recruiter.

Authorization to work in the Employing Country

Applicants for employment in the country in which they are applying (Employing Country) must have work authorization that does not, now or in the future, require sponsorship of a visa for employment authorization in the Employing Country and with Alight.

Note, this job description does not restrict management's right to assign or reassign duties and responsibilities of this job to other entities; including but not limited to subsidiaries, partners, or purchasers of Alight business units.

We offer you a competitive total rewards package, continuing education & training, and tremendous potential with a growing worldwide organization.

Salary Pay Range

Minimum :

150,000.00 USD

Maximum :

200,000.00 USD

Pay Transparency Statement: Alight considers a variety of factors in determining whether to extend an offer of employment and in setting the appropriate compensation level, including, but not limited to, a candidate’s experience, education, certification/credentials, market data, internal equity, and geography. Alight makes these decisions on an individualized, non-discriminatory basis. Bonus and/or incentive eligibility are determined by role and level. Alight also offers a comprehensive benefits package; for specific details on our benefits package, please visit: Wellbeing and Benefits Selector Page - Alight

DISCLAIMER:

Nothing in this job description restricts management's right to assign or reassign duties and responsibilities of this job to other entities; including but not limited to subsidiaries, partners, or purchasers of Alight business units.

Alight Solutions provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, creed, sex, sexual orientation, gender identity, national origin, age, disability, genetic information, pregnancy, childbirth or related medical condition, veteran, marital, parental, citizenship, or domestic partner status, or any other status protected by applicable national, federal, state or local law. Alight Solutions is committed to a diverse workforce and is an affirmative action employer.