Want to know if this job is worth applying to?
India
Onsite
A Little About Us
We are a global service organization providing human evaluation and diagnostic services including content analysis and insights, testing and training data, and product quality optimization.
A Lot About You
As Content Analysts and Knowledge Engineers, we do human evaluation, content management, data analysis, training data development and evaluation for tuning models, taxonomy and tagging systems, and other information management projects for Yahoo-branded products and applications. Through structured data extraction processes, we transform unstructured data in non-semantic HTML pages into structured data using internal tools and workflows. That transformed structured data helps Yahoo create products that enhance and personalize user experiences.
Our team also plays a critical role in bridging human judgment and AI systems by supporting the development, evaluation, and continuous improvement of Large Language Models (LLMs), Machine Learning (ML) systems, and human-in-the-loop workflows.
Your Day
Establish order from complexity by classifying diverse, unstructured data and synthesizing user analysis to articulate explicit user wants and underlying needs.
Design and refine prompts, evaluation frameworks, annotation strategies, and training data methodologies to improve model quality and performance.
Develop and maintain efficient human-in-the-loop workflows, including assisted labeling, AI-supported evaluation, and QA processes.
Identify opportunities to integrate AI into workflows to improve efficiency, scalability, consistency, and operational quality.
Analyze model behavior, conduct error analysis, and contribute to dataset, guideline, and evaluation improvements.
Execute content management and structured data extraction with precision, balancing data-driven metrics, editorial standards, and AI-assisted tooling.
Balance automation and human judgment to ensure reliable, high-quality outputs and trustworthy user experiences.
Innovate and champion next-generation content management, evaluation methodologies, and quality assurance practices.
Collaborate directly with Engineering, Research, and Product teams on the development, evaluation, and deployment readiness of LLMs, ML algorithms, ranking systems, classification models, and NLP initiatives.
Communicate complex technical and operational concepts clearly across cross-functional stakeholders.
Remain detail-oriented, adaptable, and highly organized while managing multiple priorities in a fast-paced environment.
You Must Have
A BA or BS degree. Preferred fields: Linguistics, Computer Science, Library & Information Science, Information Management, or related disciplines.
At least 2 years of relevant work experience.
Exceptional organizational and communication skills, including the ability to articulate complex technical concepts clearly and effectively.
Demonstrated ability to exercise strong editorial, analytical, and business judgment.
Strong technical aptitude and proven ability to collaborate effectively with Engineering, Product, and Research teams.
Expertise in taxonomy development, metadata strategy, content categorization systems, and data classification schema.
Proven analytical ability to collect, interpret, and translate complex data into actionable content requirements and evaluation frameworks.
Experience creating and managing keyword lists, including synonyms, variants, and common misspellings.
Familiarity with prompt engineering, AI-assisted workflows, model evaluation concepts, or human-in-the-loop systems preferred.
Understanding of how annotation quality, evaluation rigor, and dataset design impact ML and LLM performance preferred.
Important notes for your attention
Applications:
All applicants must apply for Yahoo openings direct with Yahoo. We do not authorize any external agencies in India to handle candidates’ applications. No agency nor individual may charge candidates for any efforts they make on an applicant’s behalf in the hiring process.
Our internal recruiters will reach out to you directly to discuss the next steps if we determine that the role is a good fit for you. Selected candidates will go through formal interviews and assessments arranged by Yahoo direct.
Offer Distributions:
Our electronic offer letter and documents will be issued through our system for e-signatures, not via individual emails.
Yahoo is proud to be an equal opportunity workplace. All qualified applicants will receive consideration for employment without regard to, and will not be discriminated against based on age, race, gender, color, religion, national origin, sexual orientation, gender identity, veteran status, disability or any other protected category. Yahoo will consider for employment qualified applicants with criminal histories in a manner consistent with applicable law. Yahoo is dedicated to providing an accessible environment for all candidates during the application process and for employees during their employment. If you need accessibility assistance and/or a reasonable accommodation due to a disability, please submit a request via the Accommodation Request Form (www.yahooinc.com/careers/contact-us.html) or call +1.866.772.3182. Requests and calls received for non-disability related issues, such as following up on an application, will not receive a response.
Yahoo has a high degree of flexibility around employee location and hybrid working. In fact, our flexible-hybrid approach to work is one of the things our employees rave about. Most roles don’t require specific regular patterns of in-person office attendance. If you join Yahoo, you may be asked to attend (or travel to attend) on-site work sessions, team-building, or other in-person events. When these occur, you’ll be given notice to make arrangements.
If you’re curious about how this factors into this role, please discuss with the recruiter.
Currently work for Yahoo? Please apply on our internal career site.
