PATH current employees - please log in and apply Here
PATH is a global nonprofit dedicated to achieving health equity. With more than 40 years of experience forging multisector partnerships and with expertise in science, economics, technology, advocacy, and dozens of other specialties, PATH develops and scales up innovative solutions to the world’s most pressing health challenges.
About AI for Health
PATH’s Artificial Intelligence (AI) Initiative is our flagship technical team working across divisions to deliver best-in-class evaluation and implementation of AI in health, spanning model evaluation to product development and service innovation. We aim to use AI to address the world’s largest health inequities.
We’re looking for a clinician with research experience to help deliver PATH’s work evaluating whether different AI models and products can provide safe and effective mental health/wellbeing support for those most in need. You will collaborate with leading AI evaluation labs and with clinical, research, lived experience, and policy experts across sub-Saharan Africa and the UK to develop new ways of contextualising evaluation so that we can better understand what works, for whom, and in which contexts. You’ll bring clinical expertise to the design and execution of our evaluations, as well as coordinate day-to-day delivery across an international consortium. This role is a strong fit for an early-career who wants their work to change how AI is built and used for mental health.
Responsibilities:
Bring Clinical Expertise to Evaluation
- Contribute to evaluation design, including which AI systems we test, who we test them for, how we test them, and how we learn from our partners to refine our approach. Keep this all current as the field evolves.
- Ground our simulated users/patients and automated (LLM) judges in clinical practice: help our partners specify personas and presentations and draft behaviour rubrics, and review transcripts for clinical errors before they go to the panels of clinicians and lived experience experts in each country. Expect to work hands-on alongside our partners.
- Lead PATH’s evaluation of consumer AI products, from general-purpose assistants to dedicated mental health apps, while tailoring evaluations to specific populations, cultures, and languages. Coordinate with developers on evaluation of their models and products.
- Co-author research on new methods for contextualised AI evaluation with consortium partners, bringing clinical expertise to how we build and validate simulated users/patients and LLM-judges.
Coordinate Delivery across the Consortium
- Coordinate day-to-day delivery across our partners in South Africa, Kenya, Ghana, the UK, and the US (each of whom leads their own strand of the work): work with our ops team to track progress, organise working meetings, follow up on actions, and solve problems as they appear.
- Matrix manage members of the AI4Health team supporting the delivery of consumer AI product evaluations, including both fellows doing their PhDs (whom you will contribute to the supervision of), as well as peers leading engineering for the programme.
- Contribute to donor reports and to presentations for funders and external audiences.
Publish Evidence
- Prepare content for a website presenting all results of PATH’s AI evaluations in mental health, including plain-language summaries for people seeking support; contribute to writing seminal reports on the state of AI safety and effectiveness for global mental health; and support presentation of evidence to international commissions and panels.
- Write up our results for high-impact academic venues and established policy fora.
Contribute to the Broader AI programme
- Support research and related activities led by other members of the AI team (where your expert skill set is relevant, and sometimes just as another pair of hands), including the soon-to-be-launched ‘Nature Commission on Consumer AI and Wellbeing’ for which PATH is hosting the secretariat.
- This is a full-time post based in London. You’ll report to PATH’s Deputy Director for AI for Health. This role includes international travel to partner sites and convenings (around 10–15% of time).
Required Skills and Experience
- A professional qualification in a clinical discipline (e.g., clinical psychology, medicine, mental health nursing, or equivalent), with hands-on clinical experience in mental health (including training placements or rotations).
- Research experience, with evidence of scientific writing (e.g., publications, preprints, or a doctoral thesis).
- Expertise/experience in measurement and evaluation methods—reliability, validity, inter-rater agreement.
- Some programming experience (e.g., Python or R), and hands-on use of LLMs (e.g., writing prompts, or writing and testing rubrics).
- Excellent written and verbal communication for scientific, clinical, public, and policy audiences.
- Ability to coordinate work and deadlines across complex, multi-stakeholder projects.
Preferred Qualifications
- Two+ years’ clinical experience in mental health since qualifying.
- Research experience in digital mental health or experience evaluating digital health products (including their clinical safety).
- Experience working with collaborators or patients in low- and middle-income countries, ideally in sub-Saharan Africa.
- Experience coordinating a multi-partner or multi-site research program.
- Research experience in simulating users or patients with LLMs, validating automated judges against human raters, or building LLM-based evaluation pipelines.
- Experience of industry engagement and/or evaluation governance.
- Experience working with lived experience or patient and public involvement panels.
We know great candidates won’t always meet every listed qualification. Research shows that some groups, on average, are more likely to self-select out if they feel they don’t meet all requirements. If you’re excited about the role and think you’d be a good fit, we encourage you to apply.
To be selected, you must have legal authorization to work in the United Kingdom.
.png?1603752378)
