Want to know if this job is worth applying to?
Barcelona, Catalunya, España
Remote only
⭐ Role Overview
We are seeking a Senior AI Engineer specializing in LLMs to lead the design, evaluation, and deployment of production-grade generative AI systems. In this role, you will own end-to-end LLM solutions, from prototyping to scalable production, while establishing best practices in evaluation, reliability, and responsible AI.
Own and evolve the LLM evaluation (evals) strategy, including designing gold-standard datasets and benchmarks, building automated eval pipelines and scoring systems and defining metrics for factuality, grounding, robustness, and user impact.
Diagnose and resolve complex failure modes (hallucinations, retrieval issues, agent breakdowns).
Optimize systems for latency, cost, scalability, and reliability in production.
Mentor junior engineers and guide best practices in LLM development and evaluation.
Collaborate cross-functionally with product, data, and leadership to shape AI strategy.
Set standards for responsible AI, including safety, bias mitigation, and observability.
5+ years of experience in software engineering, machine learning, and applied AI with a track record of driving projects to completion.
Strong software engineering fundamentals (testing, modular design, dependency injection) in Python.
A track record of taking AI and LLM-powered features from initial concept through deployment and long-term production maintenance.
Experience implementing automated testing strategies for non-deterministic systems, and strong debugging and analytical skills for ambiguous model behavior.
A strong understanding of prompt engineering and prompt lifecycle management, RAG architectures and retrieval evaluation, and LLM limitations and failure patterns.
Solid experience in using Data Analytics techniques (SQL, analysis and visualizations) to inform Product decisions and delivery.
A heavy product mindset to deeply understand our product and our customer needs to design the right solutions for them.
Strong tech leadership and mentorship skills, and the ability to independently drive projects to completion.
Clear communication of trade-offs, risks, and system performance to stakeholders.
Proven experience driving ambiguous projects to completion, mentoring teams, and communicating complex technical risks to stakeholders.
The ability to design robust, production-grade evaluation at scale using advanced metrics and statistical validation.
Deep expertise in model fine-tuning, adversarial red-teaming, and safety testing to protect the system from edge-case vulnerabilities.
🏠 Remote Flexibility: The freedom to work from home any day that works for you.
🌴 Time to Recharge: 25 working days of paid vacation and Jornada Intensiva in August..
💊 Alan Health Insurance: Premium health, dental, and mental health support via Alan. Pre-existing conditions are covered.
😋 Meal Perk: €150/month allowance on your Alan card + 50% off Ametller Origen prepared dishes at the office.
💸 Tax-Free Savings: Increase your take-home pay by using Flexible Remuneration for extra meal costs (up to €70/mo) and public transport (up to €136/mo).
🖥️ Home Office Gear: We provide a table, ergonomic chair, and monitor for your home setup.
🇪🇸 Language Learning: Free Spanish classes.
🤑 Referrals: Cash rewards for bringing in new talent.
🌟 Social Life: Daily office breakfast and monthly team events
🎯 Dynamic Hub: A high-energy, inclusive environment designed for collaboration and connection with a team that represents over 60 countries.
*Benefits offered may differ based on the type of contract that is issued
So, what are you waiting for? Apply now!
All applications and CVs must be submitted in English 😉