Austin, Texas, United States
Remote only
Job posting may be removed earlier if the position is filled or if a sufficient number of applications are received.
This role can be performed anywhere in the Central and Eastern United States.
Role Summary
Splunk is seeking a senior individual contributor to serve as the technical lead for the AI agents that make our technical sellers more productive, setting direction through thought leadership and influence across field teams. The role spans the full agent lifecycle across multiple platforms: setting best practices for cost, performance, design, and integrations; deciding which agents are built where; completing the last-mile build; maintaining a subset in production; and directing and reviewing a contractor who builds additional agents. You will combine agentic engineering (designing and building agents) with agentic operations (running them reliably, safely, and cost-effectively) as the single technical authority guiding quality from design through production. Your immediate mission is to deliver an initial 3 to 5 agents that streamline technical sellers’ daily workflows, freeing them to spend more time customer-facing and advancing Splunk’s revenue and customer-success goals.
Responsibilities
Agentic Platform Strategy (Architect)
Reference Design & Standards: Advise on the end-to-end design for agentic systems across platforms, contributing inputs to the reference designs, patterns, and standards agents are built against, and informing system-level decisions on how agents reason, integrate, and scale.
Cross-Platform Best Practices: Define, document, and evangelize best practices for cost, performance, design, and integrations across the organization’s agent platforms, so teams build consistently and avoid duplicated effort.
Platform Selection & Build Allocation: Guide which agents are built on which platform, balancing capability, cost, integration fit, security, and maintainability, and set the criteria the organization uses to decide.
Contractor Direction & Review: Set technical direction for a contractor building agents, specifying what to build and where, and reviewing their work for quality, security, and adherence to standards. (Technical direction and review, not people management.)
Technical Thought Leadership & Influence: Serve as the recognized technical authority for agentic systems across field teams, contributing to reference designs and standards, mentoring engineers, and driving alignment with IT, Engineering, Cloud Operations, Security, Product, and field stakeholders through influence.
Strategic Impact: Advance Splunk’s revenue and customer-success goals by increasing technical sellers’ customer-facing time and making the agents behind demonstrations and Field Solutions Apps more capable, reliable, safe, and cost-effective.
Agent Delivery & Engineering (Build)
Deliver Seller-Productivity Agents: Own delivery of an initial 3 to 5 production agents that streamline technical sellers’ daily workflows, prioritizing the use cases that return the most selling time and measurably reduce manual effort.
Last-Mile Build & Integration: Complete the last-mile build of agents, wiring up tools, Model Context Protocol (MCP) servers and clients, data sources, and integrations, and hardening them from prototype to dependable production use.
Agent Design & Orchestration: Translate the reference design into working agentic systems with evaluation and guardrails designed in from the start, and define retrieval (RAG) and tool-integration strategies that keep agents grounded and accurate.
Agentic Operations (Operate)
Observability & Monitoring: Establish comprehensive logging, monitoring, and alerting for agents in production. Because agents make non-deterministic decisions, ensure deep visibility into agent health, behavior, and performance, and lead root-cause analysis when agents fail.
Performance & Cost Evaluation: Track resource consumption across agentic workloads, including API usage and token limits across LLMs. Define and monitor SLOs/SLIs, manage cost-per-task economics, and drive model-routing and efficiency decisions that keep performance high and spend predictable.
Governance & Guardrails: Define the boundaries that keep agents safe, within budget, and compliant with security and regulatory standards, establishing policy, permissions, audit trails, and human-in-the-loop checkpoints so every agent action is bounded, traceable, and auditable.
Continuous Optimization & Maintenance: Maintain a subset of production agents, resolving regressions and keeping them current as platforms and models evolve, while using telemetry and evals to refine prompts, tools, topologies, and guardrails. Validate changes in simulation before release and automate away manual toil.
Qualifications
Technical Skills:
3+ years of software, ML, or platform engineering experience, including a demonstrated track record of leading technical direction through influence (no direct-report or people-management requirement)
Proven ability to act as a technical authority and force multiplier (contributing to reference designs and standards, mentoring engineers, and driving adoption) without formal management authority
Hands-on experience designing and building production agentic systems, including multi-agent orchestration and Model Context Protocol (MCP) integrations
Experience selecting among and building across multiple agent platforms, with the judgment to match use cases to the right platform
Strong tool and API integration skills, including designing retrieval (RAG) strategies that keep agents grounded and accurate
Experience building evaluation harnesses and offline/online evals, and validating agentic systems in simulation before release
Experience implementing guardrails, policy and permission models, and audit trails for autonomous systems
Familiarity with token and cost management across multiple LLM providers, including model routing and fallback strategies
Experience directing and reviewing the work of contractors, vendors, or peer engineers through technical review (without formal management authority)
Hands-on experience operating agents in production, including observability and monitoring with Splunk and Splunk Observability (Splunk o11y)
Experience with Splunk products (Splunk Cloud/Enterprise, Splunk IT Service Intelligence, Splunk Observability, or equivalent) for telemetry, monitoring, and observability
Proficiency in cloud-native architectures, preferably AWS
Experience in programming/scripting languages such as Python and Bash
Understanding of distributed systems, microservices architecture, and API design
Strategic & Business Skills:
Strong strategic thinking and ability to translate business objectives into technical roadmaps
Excellent communication skills with ability to present technical concepts to non-technical stakeholders
Track record of delivering results in fast-paced, dynamic environments
Data-driven decision-making approach with focus on measurable outcomes
Core Values and Mindsets:
Commitment to Excellence: A dedication to delivering high-quality solutions, strengthening technical skills, and taking pride in continuous improvement to consistently raise the bar.
Adaptability / Flexibility: A growth mindset, openness to feedback, and the ability to respond effectively to new challenges and rapid change.
Collaboration: A strong team player attitude, excellent communication skills, and a commitment to empowering colleagues and fostering an open, supportive environment.
Ownership: A proactive approach to improvement, accountability for outcomes, and a drive to deliver on commitments without waiting for direction.
Innovation: A passion for creative problem-solving, technical rigor, and pushing boundaries while maintaining high standards.
Fun: An ability to embrace fun, celebrate success, and contribute to a positive and energetic culture where everyone can thrive.
At Cisco, we’re revolutionizing how data and infrastructure connect and protect organizations in the AI era – and beyond. We’ve been innovating fearlessly for 40 years to create solutions that power how humans and technology work together across the physical and digital worlds. These solutions provide customers with unparalleled security, visibility, and insights across the entire digital footprint.
Fueled by the depth and breadth of our technology, we experiment and create meaningful solutions. Add to that our worldwide network of doers and experts, and you’ll see that the opportunities to grow and build are limitless. We work as a team, collaborating with empathy to make really big things happen on a global scale. Because our solutions are everywhere, our impact is everywhere.
We are Cisco, and our power starts with you.
Individual pay is determined by the candidate's hiring location, market conditions, job-related skillset, experience, qualifications, education, certifications, and/or training. The full salary range for certain locations is listed below. For locations not listed below, the recruiter can share more details about compensation for the role in your location during the hiring process.
U.S. employees are offered benefits, subject to Cisco’s plan eligibility rules, which include medical, dental and vision insurance, a 401(k) plan with a Cisco matching contribution, paid parental leave, short and long-term disability coverage, and basic life insurance. Please see the Cisco careers site to discover more benefits and perks. Employees may be eligible to receive grants of Cisco restricted stock units, which vest following continued employment with Cisco for defined periods of time.
U.S. employees are eligible for paid time away as described below, subject to Cisco’s policies:
10 paid holidays per full calendar year, plus 1 floating holiday for non-exempt employees
1 paid day off for employee’s birthday, paid year-end holiday shutdown, and 4 paid days off for personal wellness determined by Cisco
Non-exempt employees** receive 16 days of paid vacation time per full calendar year, accrued at rate of 4.92 hours per pay period for full-time employees
Exempt employees participate in Cisco’s flexible vacation time off program, which has no defined limit on how much vacation time eligible employees may use (subject to availability and some business limitations)
80 hours of sick time off provided on hire date and each January 1st thereafter, and up to 80 hours of unused sick time carried forward from one calendar year to the next
Additional paid time away may be requested to deal with critical or emergency issues for family members
Optional 10 paid days per full calendar year to volunteer
For non-sales roles, employees are also eligible to earn annual bonuses subject to Cisco’s policies.
Employees on sales plans earn performance-based incentive pay on top of their base salary, which is split between quota and non-quota components, subject to the applicable Cisco plan. For quota-based incentive pay, Cisco typically pays as follows:
.75% of incentive target for each 1% of revenue attainment up to 50% of quota;
1.5% of incentive target for each 1% of attainment between 50% and 75%;
1% of incentive target for each 1% of attainment between 75% and 100%; and
Once performance exceeds 100% attainment, incentive rates are at or above 1% for each 1% of attainment with no cap on incentive compensation.
For non-quota-based sales performance elements such as strategic sales objectives, Cisco may pay 0% up to 125% of target. Cisco sales plans do not have a minimum threshold of performance for sales incentive compensation to be paid.
The applicable full salary ranges for this position, by specific state, are listed below:
New York City Metro Area:
$168,800.00 - $277,400.00Non-Metro New York state & Washington state:
$148,800.00 - $248,200.00* For quota-based sales roles on Cisco’s sales plan, the ranges provided in this posting include base pay and sales target incentive compensation combined.
** Employees in Illinois, whether exempt or non-exempt, will participate in a unique time off program to meet local requirements.
Want to know if this job is worth applying to?
