Want to know if this job is worth applying to?
United States
Remote only
We are seeking a collaborative, product-minded backend engineer with strong Agentic AI fundamentals and a passion for building reliable, scalable systems that power AI applications.
In this role, you will:
Design, build, and scale evaluation frameworks for Zillow’s agentic AI experiences.
Build tracing, observability, and quality measurement systems for production AI agents.
Create platform infrastructure that enables domain teams across Zillow to build on top of the Evals platform without bottlenecks.
Partner with applied scientists and machine learning engineers to integrate new AI evaluation capabilities into production systems.
Help evolve how Zillow measures AI quality, reliability, trustworthiness, and product impact.
Stay current with emerging agentic AI paradigms, evaluation techniques, and LLM tooling, and translate them into practical platform innovation.
Support scaling, reliability, performance optimization, incident response, and cost management for the evaluation layer.
Apply first-principles thinking to ambiguous problems and iterate quickly on novel solutions.
You are a roll-up-your-sleeves technical expert who can combine state-of-the-art AI technology with large-scale backend engineering. You thrive in ambiguous environments, enjoy solving challenging problems, and care deeply about creating reliable systems that other teams can trust.
You have:
4+ years of backend engineering experience, with a track record of designing, shipping, and operating scalable production ML services
Experience building platform infrastructure or developer-facing services consumed by multiple teams; you've thought carefully about APIs, user experience, reliability, and what it means to have internal customers
Hands-on experience with ML Evals & Observability frameworks (Databricks MLflow, or evolving LLM frameworks preferred - LangSmith, Braintrust, Promptfoo, or similar)
Experience evaluating third-party AI platforms and making principled build vs. buy decisions for platform infrastructure
Fluency working across applied science, ML, product, design, and engineering; you translate between disciplines without losing precision
Comfort with LLMs, agentic systems, and evaluation frameworks; familiarity with orchestration tools (LangChain, LangGraph, MCP, or similar)
A habit of reaching for AI-assisted development tools in your own workflow; you don't just build for AI, you build with it
Strong ownership and a growth mindset; you're energized by ambiguity, not slowed by it
Preferred qualifications:
Advanced degree in Computer Science, Machine Learning, or a related field, or equivalent practical experience building products using frontier LLMs, multimodal models, or agent-based systems
Experience designing and operating evaluation pipelines: offline batch evals, online scoring, cost/latency/signal tradeoffs, and closing the loop from user signals back to training data.
At Zillow, we’re reimagining how people move—through the real estate market and through their careers. As the most-visited real estate platform in the U.S., we help people navigate buying, selling, financing and renting with greater ease and confidence. Whether you're working in tech, sales, operations, or design, you’ll be part of a company reshaping an industry and helping more people make home a reality.
How we work is a key part of that transformation. Through Cloud HQ, our strategic approach to distributed work, most employees have the flexibility to work from wherever they’re most productive—enabling us to move fast, stay connected and deliver on our people promise.
Zillow is honored to be recognized among the best workplaces in the U.S. Zillow was named one of FORTUNE 100 Best Companies to Work For® in 2026, and included on TIME’s America’s Best Companies 2026 list, reflecting our commitment to creating an innovative, inclusive, and engaging culture where employees are empowered to grow.
No matter where you sit in the organization, your work will help drive innovation, support our customers, and move the industry—and your career—forward, together.
Zillow Group is an equal opportunity employer committed to fostering an inclusive, innovative environment with the best employees. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. If you have a disability or special need that requires accommodation, please contact your recruiter directly.
Qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable state and local law.
Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
