AI Prompt Engineer

Job Summary:

As an AI Prompt Engineer at Caravel, you'll build the production layer of our AI practice — the prompts, prompt chains, and LLM-driven components that power client solutions and the internal accelerators our consultants rely on. You'll partner with our AI Specialists to turn solution designs into reliable, evaluated, production-ready builds. This is a hands-on build role for someone who cares about getting model behavior right, not just getting it to run.

Key Responsibilities:

Design, build, test, and iterate on prompts and prompt chains for production LLM workflows across client delivery and internal automation.
Implement evaluation harnesses, guardrails, and regression tests so output quality is measurable and stays stable over time.
Integrate LLM components with business systems and data — APIs, retrieval/RAG pipelines, and structured outputs.
Partner with AI Specialists to convert solution designs into production builds, and document patterns for reuse.
Tune for accuracy, cost, latency, and safety against real evaluation data — not vibes.
Grow Caravel's library of reusable AI accelerators.

Preferred Qualifications

2+ years building software or AI/LLM components, including hands-on experience crafting and iterating prompts for real applications.
Working knowledge of LLM fundamentals: context windows, few-shot prompting, function/tool calling, structured outputs, and RAG basics.
Proficiency in Python (or similar) and comfort with APIs, JSON, and version control (Git).
A test-and-measure mindset — you evaluate output quality, you don't eyeball it.
Clear written English and strong documentation habits.
Ability to collaborate across time zones with an onshore/nearshore team.
Preferred Skills:
Experience with evaluation frameworks (e.g., promptfoo, Ragas, or custom eval harnesses).
Familiarity with embeddings, vector databases, and retrieval pipelines.
Exposure to ERP/CRM ecosystems (NetSuite, Salesforce) or business-process automation.
Experience with agentic frameworks and orchestration.