This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior AI Scientist based in Portugal.
As a Senior AI Scientist, you’ll play a key role in continuously improving the quality, accuracy, and effectiveness of AI-powered products.
You’ll analyze real-world AI interactions to uncover opportunities for better responses, stronger user experiences, and more reliable outcomes.
The role combines hands-on work across prompt engineering, LLM evaluation, retrieval-augmented generation, and conversational AI.
You’ll design experiments, define quality metrics, and use data-driven insights to validate and prioritize improvements.
Working closely with AI Engineers and other stakeholders, you’ll help turn successful experiments into production-ready enhancements.
You’ll contribute to evaluation frameworks, benchmarking, responsible AI practices, and the ongoing evolution of generative AI capabilities.
This is an opportunity to have a measurable impact in a remote-first environment while working at the forefront of applied AI.
Accountabilities
- Analyze AI conversations, interaction data, and user feedback to identify opportunities to improve response quality, accuracy, relevance, and overall user experience.
- Design, implement, test, and optimize prompt engineering strategies and conversational behaviors.
- Optimize RAG pipelines, retrieval mechanisms, vector-based search, and overall retrieval quality.
- Evaluate different LLMs and recommend the most effective models for specific product use cases, including exploring the development or training of customer-specific models where appropriate.
- Define, track, and continuously improve AI quality metrics such as accuracy, relevance, latency, and user satisfaction.
- Design and execute structured experiments to assess AI improvements and determine their impact.
- Develop and maintain evaluation frameworks, benchmarks, testing methodologies, and supporting documentation.
- Collaborate closely with AI Engineers to validate and deploy successful improvements into production environments.
- Monitor the performance of production AI systems and identify opportunities for optimization at scale.
- Stay informed about advances in generative AI, conversational AI, speech and voice technologies, and recommend practical applications that can improve existing products.
- Help ensure AI solutions are developed and evaluated according to appropriate governance, safety, and responsible AI principles.
Requirements:
- 5+ years of professional experience in AI, machine learning, NLP, or a related technical field.
- Strong practical understanding of LLMs, generative AI, and conversational AI systems.
- Hands-on experience with prompt engineering, prompt evaluation, and LLM optimization.
- Proven experience evaluating conversational AI using both qualitative and quantitative methodologies.
- Strong Python and SQL skills, with the ability to analyze data and build practical evaluation or experimentation workflows.
- Experience with AI evaluation frameworks, benchmarking, and systematic model or system assessment.
- Knowledge of RAG architectures, vector databases, embedding models, and retrieval optimization.
- Experience working with production AI systems at scale.
- Experience with speech and voice AI technologies is highly valuable.
- Understanding of AI governance, safety, and responsible AI practices.
- Strong analytical and problem-solving skills, combined with a structured and experimental approach to complex challenges.
- Excellent communication and stakeholder management skills, with the ability to translate technical findings into clear recommendations.
- Proactive mindset, strong ownership, and accountability for measurable outcomes.
Benefits:
- Competitive salary with individual performance-based bonuses paid quarterly.
- 28 days of paid annual leave.
- Remote-first working environment with flexibility across your local time zone.
- Core working hours from 10:00 AM to 3:00 PM local time, with flexibility outside those hours.
- Referral bonuses and additional flash bonus opportunities.
- High-quality, top-of-the-line work equipment.
- Annual company retreats designed to strengthen collaboration and internal networking.
- Opportunity to work on impactful AI products and contribute to the evolution of generative and conversational AI.
- International, remote-friendly environment with significant autonomy and flexibility.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
