Want to know if this job is worth applying to?
New York, New York, United States
Onsite
Not Available
Company Overview:
We are building Protege to solve the biggest unmet need in AI — getting access to the right training data. The process today is time intensive, incredibly expensive, and often ends in failure. The Protege platform facilitates the secure, efficient, and privacy-centric exchange of AI training data.
Solving AI’s data problem is a generational opportunity. We’re backed by world-class investors and already powering partnerships with some of the most ambitious teams in AI. The company that succeeds will be one of the largest in AI — and in tech.
We’re a lean, fast-moving, high-trust team of builders who are obsessed with velocity and impact. Our culture is built for people who thrive on ambiguity, own outcomes, and want to shape the future of data and AI.
The Human Data team produces the human judgments that make AI training data trustworthy. That includes annotating data, verifying model output, and the human review that makes healthcare data safe to use. We work with an in-house team of annotators and a network of partners ranging from annotation platforms to practicing physicians. Our work feeds the research teams building AI evaluations and training environments, and the AI labs that license our data.
You'll own human data projects from start to finish. A project might be 800 scanned medical documents reviewed by five annotators over three weeks, or thousands of physician ratings on model output for a research team. You scope it with the team that needs it, design and test the task, staff it, keep it on schedule and on budget, catch quality problems before they spread, and deliver results the customer can trust. This role is built for someone early in their career who wants to own outcomes quickly, and who is as comfortable in a spreadsheet as in a room with researchers.
Own human data projects end to end: scope them with the requesting team, set timelines and budgets, and deliver on them.
Design tasks and instructions with the requesting team, and pilot them on a small batch before anything scales.
Staff projects across our in-house annotators and partner vendors, and manage those vendor relationships day to day.
Run calibration before production. Every annotator passes a known-answer test before their work counts, and anyone who doesn't gets feedback and a retry.
Track throughput, cost and quality by annotator and vendor, and report progress to leadership every week.
Catch problems early (an annotator drifting, a vendor slipping, an instruction that isn't working) and fix them before they reach the customer.
Handle sensitive data, including protected health information, under strict security procedures.
Travel on short notice when a project needs someone on site.
Make every project easier to run than the last one.
30 days: learn the machine. You've shadowed a live project, taken over its roster and returns intake, and met each of our annotation partners.
60 days: own a project. You've taken one project from scoping to delivery on your own, and every open project is tracked in one place.
90 days: own the portfolio. You're running two to three projects at once, you've scoped a new one directly with a requesting team, and you've delivered our first cost-and-quality comparison across vendors.
Must have
One to three years in consulting, finance, a startup, or another operationally intense environment where you owned complex projects with real stakes. Strong new grads with that kind of experience are welcome.
Strong analytical skills, with instincts for statistics
Comfort and experience with agentic coding/work tools, willingness to learn new ones
Comfort working directly with demanding stakeholders, including researchers and outside partners.
The judgment to notice when something is off and the instinct to raise it early rather than guess.
Clear, fast written communication.
Authorization to work in the U.S. as a W2 employee (required for access to protected health information).
Nice to have
Experience running data annotation, labeling or AI training data projects
Healthcare, clinical research or HIPAA-regulated work
Experience managing contractors, vendors or freelancers
Python or SQL
Pass the Loved Ones' Test. We act with integrity and do the right thing — especially when it's hard and no one is watching.
Always Find a Way. We are resourceful, resilient builders who solve hard problems and push through obstacles.
Go Fast and Grow Fast. Velocity matters. We move with urgency, learn quickly, and continuously improve as individuals and as a company.
Practice Kindness and Candor. We communicate directly and respectfully, building trust through honest feedback and genuine care for one another.
Deliver Together. We win as one team. Collaboration, accountability, and shared ownership drive our success.
Own the Outcome. Hone the Craft. We take pride in our work, sweat the details, and continuously raise the bar for excellence.
How this role lives the values: the loved ones' test is literal here, since this person handles patients' health information every day, and the whole job is owning the outcome.