Sydney, New South Wales, Australia
Do Work That Matters
AI & General Model Validation (AIGMV) provides independent technical challenge and review of the Group's most important ML and AI models, across business and support units.
See yourself in our team
As a Principal Data Scientist within the team, you will lead independent review, challenge and validation of ML, statistical and AI models across the Group, including GenAI and agentic systems. You will assess whether they are fit for purpose and operate as intended, and whether the risks they introduce are understood and appropriately controlled. This will involve:
Scope and design validation of complex ML and AI systems, split the work into a structured plan, and lead other data scientists through it
Form independent risk opinions and provide credible challenge to senior business, engineering and governance stakeholders
Set and improve the team's AI validation practices, tooling and standards, and mentor other data scientists
We’re interested in hearing from people who:
Extensive experience in developing, evaluating or validating ML/AI, GenAI or statistical models, or equivalent depth
A courageous mindset and independent judgement. You enjoy self-directed learning, and have the curiosity and discipline to stay ahead of fast-changing AI techniques and risks
Close attention to detail, and the judgement to see which problems matter
Deep working knowledge of GenAI and agentic architecture, enough to validate and challenge orchestrators, sub-agents, RAG pipelines, tool use, guardrails, multi-agent routing, memory and persona handling, and LLM-as-a-Judge
Hands-on experience designing, building and evaluating AI and Generative AI solutions. Fine-tuning transformer models is a plus.
Advanced Python, with strong GitHub, version control and CI/CD experience. SQL, R or TypeScript is a plus.
Ability to read and navigate unfamiliar codebases, reproduce evaluation pipelines, write test harnesses and inspect trace data
Hands-on experience with AI coding agents, including Claude Code and Codex, and the ability to build and maintain skills, sub-agents, hooks and plugins that automate workflows.
Working knowledge of traditional machine learning models, including regression, random forests, XGBoost, and related evaluation methods, and the ability to validate and challenge them.
Demonstrated experience as the technical lead on projects, leading other data scientists through the work rather than as a people manager, including turning a complex system into a structured test plan
Strong communication skills, with the ability to explain complex technical concepts to mixed audiences, including senior stakeholders and governance forums.
Practical experience with PyTorch and AWS SageMaker, and with GenAI tooling such as LangChain, LangGraph, OpenAI Agents SDK, Pydantic and Langfuse/HoneyHive
Degree in Statistics, Data Science, Computer Science, Actuarial Science, or a related field, or equivalent industry experience
Working with us:
At CommBank, we're committed to creating an accessible, inclusive and respectful workplace. If you require support or adjustments, please let us know. We welcome applications from people of all backgrounds and we're particularly committed to making a positive difference for Aboriginal and/or Torres Strait Islander Peoples. For support please contact 1800 989 696.
If this opportunity excites you, we’d love to hear from you. Apply now and help shape smarter ways to protect our customers and communities.
If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We’re keen to support you with the next step in your career.
We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.
Want to know if this job is worth applying to?
Onsite