AI Data Quality Analyst

Data Analysis

• Quality Audits - Perform quality audits on annotated datasets to ensure that they meet

• established guidelines and quality benchmarks.

• Statistical Reporting - Leverage statistical based quality metrics such as F1 score and

• inter-annotator agreement to evaluate data quality.

• Root Cause Analysis - Analyze annotation errors, trends, project processes, and project

• documentation to identify and understand the root cause of errors and propose remediation strategies.

• Edge-Case Management - Resolve and analyze edge-case annotations to ensure quality and identify areas for improvement.

• Tooling - Become proficient in using annotation and quality control tools to perform

• reviews and track quality metrics.

• Guidelines - Become an expert in the project specific guidelines and provide feedback for

• potential clarifications or improvements.

Continuous Improvement

• Automation - Identify opportunities to use automation to help enhance analytics, provide

• deeper insights, and improve efficiency.

• Documentation - Develop and maintain up-to-date documentation on quality standards,

• annotation guidelines, and quality control procedures.

• Feedback - Provide regular feedback that identifies areas for improvement across the

• annotation pipeline.

Collaboration & Communication

• Cross-Functional Teamwork - Work closely with key project stakeholders and clients to understand project requirements and improve annotation pipelines.

• Training - Assist with training annotators, providing guidance, feedback, and support to

• ensure data quality.

• Reporting - Provide regular updates that highlight data quality metrics, key findings, and

• actionable insights for continuous process improvements.

Experiences you’ll bring:

• 1+ years of experience as a data analyst with exposure to data quality and/or data annotation - ideally within an AI/ML context.

• Familiarity with the basic concepts of AI/ML pipelines and data.

Core skills you'll need:

• Strong analytical and problem-solving skills with an exceptional eye for detail.

• Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams.

• Ability to work independently and manage time effectively to meet deadlines.

• A strong problem-solver who thinks critically and drives innovation and continuous optimization.

• A quick learner with the ability to work independently in a fast-paced environment.

• A strong focus on detail, balanced against strategic priorities.

• A positive can-do attitude and the ability to easily adapt to new environments.

• Not afraid to speak up.

Nice to have:

• Familiarity with data annotation tools (e.g. Labelbox, Dataloop, LabelStudio etc.).

• Experience working with multi-modal AI/ML datasets (images, videos, text, audio).

• Prior experience in an agile or fast-paced tech environment with exposure to AI/ML pipelines.

• Knowledge of programming languages (e.g. Python).

• Knowledge of the concepts and principles of data quality for AI/ML models and the impacts it can have on model performance.

• Working understanding of common quality metrics and statistical methods used in AI/ML data quality.

• Knowledge of AI/ML concepts and experience with data for AI/ML models.

• Experience in prompt engineering and leveraging LLMs in your day-to-day work.

Why you’ll love this role:

• Work remote: Work from anywhere you’re most productive!

• ​Every dataset is unique: Audit, analyze, and improve complex AI/ML datasets, no two projects are ever the same.

• Visible impact: Your quality checks, root cause analyses, and automation ideas will directly shape how AI models perform in real-world applications.

• Room to grow: Build deep expertise in AI data quality and scale your influence as our projects and capabilities expand.

• Mission-driven team: Work alongside analysts, engineers, and stakeholders who share a passion for creating accurate, reliable, and trustworthy AI.

The impact you’ll make:

• Guard data integrity through audits: Conduct quality audits on annotated datasets against established guidelines and statistical benchmarks (e.g., F1 score, inter‑annotator agreement) to uphold data reliability. 

• Pinpoint and resolve annotation issues: Analyse error patterns, edge-case annotations, and root causes to surface insights and elevate annotation quality.

• Optimise annotation processes: Become proficient in annotation and QA tools, propose automation, and maintain clear documentation of quality standards, guidelines, and procedures. 

• Collaborate and coach: Work closely with stakeholders and annotators to clarify requirements, provide constructive feedback, and support skill development within the annotation pipeline.

• Track and communicate quality performance: Regularly report on data quality metrics, key findings, and actionable insights to drive continuous improvements across projects.