How Cisco CX Built AI Agents That Actually Work in Production
Most enterprise AI projects fail at scale - either drowning in hallucinations or struggling with adoption. Cisco's Customer Experience team cracked the code by evolving from simple chatbots to full AI teammates handling $26B in renewals. Here's what worked (and what didn't) in their 2-year journey.
The $26B Renewals Challenge
Cisco's Customer Experience (CX) organization oversees $26-27 billion in annual recurring revenue - more than half the company's total business. When renewal rates dipped slightly in early 2025, the team faced intense pressure to scale their operations without proportionally increasing headcount.
Their initial approach followed the industry trend - bolting AI onto existing workflows. "We were pissing off customers faster with AI than we did before," admits Carlos, Cisco CX Fellow and Chief Architect. The breakthrough came when they stopped asking how AI could improve existing processes and started questioning whether those processes were even needed.
Key insight: Traditional workflows with human orchestration DNA often can't be "fixed" by AI augmentation - they need to be completely reimagined as AI-native systems from the ground up.
From Chatbots to AI Teammates
Cisco's journey began with guided chatbot interfaces in early 2025, using genAI only for answers while maintaining rigid question structures. This lasted just three months before renewals teams rebelled against the constraints. "I don't want to be slave to an interface - give me open prompt," became the common feedback.
The team evolved to an agentic foundation combining multiple specialized agents (renewals, sentiment analysis, adoption tracking) with traditional ML models for risk prediction. This hybrid approach achieved 95% accuracy - but accuracy alone proved insufficient.
"We plateaued," Carlos explains. "People would try the system a handful of times then ghost it." The missing piece was positioning AI as a teammate rather than a tool - something that could receive delegated tasks, proactively surface insights, and prove its value through business outcomes.
The Planner-Supervisor Architecture
Cisco's production architecture centers on a dynamic planner that decomposes complex queries into executable subgraphs. When a renewals manager asks "Show me my top 10 customers by ATR for Q1 FY26 with their sentiment scores," the system:
- Disambiguates acronyms (ATR = "Available to Renew")
- Identifies dependent stages (renewals data → sentiment analysis)
- Creates an execution plan with context passing between agents
- Monitors and replans if any stage fails confidence thresholds
Production reality: At Cisco's scale, you can't hardcode all possible query paths. The planner must dynamically generate execution graphs while maintaining strict policy compliance for a $26B business.
Each agent itself became a subgraph with internal planning capabilities. For example, the renewals agent understands Cisco's fiscal calendar (different from calendar year) and can decompose queries accordingly - something no general-purpose LLM would know.
Balancing Creativity With Determinism
Cisco discovered LLMs would gradually "creep" outside defined workflows - following steps precisely on first tries, then getting increasingly creative (and non-compliant) on subsequent executions. Their solution: deterministic task shortcuts modeled after developer tools like Cursor or Codex.
"Morning" commands like /top-renewals --include-outages --check-sentiment execute predefined workflows without LLM interpretation. This ensures critical business processes run exactly as designed, while maintaining flexibility for ad-hoc queries.
Enterprise lesson: Not all workflows benefit from LLM creativity. For high-stakes processes, sometimes you need the AI equivalent of a stored procedure.
Overcoming the Adoption Plateau
Despite 95% accuracy, Cisco hit an adoption wall. "People would use it a few times then revert to spreadsheets," Carlos recalls. The problem? Optionality - the AI was just another tool rather than an embedded workflow component.
The solution came from what Cisco calls "forced curiosity" - making the AI system the path of least resistance for daily tasks. They:
- Consolidated tools to eliminate alternatives
- Personalized agent behavior to individual users
- Shifted from Q&A to proactive task delegation
- Embedded agents directly into existing workflow surfaces
This flipped adoption from a training challenge to a convenience advantage - users engaged because the system made their jobs easier, not because they were told to use it.
Flipping the Human-AI Workflow Paradigm
Cisco's most profound insight was philosophical: "We stopped asking how AI could improve existing workflows and started asking whether those workflows were even needed."
This led to a complete inversion of their automation approach:
Old model: Humans orchestrate workflows, AI assists at specific steps
New model: AI orchestrates workflows, humans intervene only at confidence thresholds
By focusing on context rather than sequential steps, Cisco enabled parallel execution of workflow components. The system automatically handles everything within its confidence bounds, only surfacing exceptions for human judgment.
6 Hard-Won Production Lessons
After two years running AI agents at scale, Cisco distilled these key lessons:
- Start bounded then extend: Let agents help before letting them act autonomously
- Self-correction isn't optional: Production systems must fix their own outputs
- Observability is non-negotiable: You can't manage what you can't measure
- Quality ceilings matter: Invest in knowledge upfront rather than chasing benchmarks
- Routing comes first: Solve classification before building agents
- Personalization drives adoption: Make the system adapt to users, not vice versa
As Carlos summarizes: "Pretending the newest model will solve your problems is a fallacy. The model is a helper - you need to architect the system around business outcomes."
Watch the Full Tutorial
At 12:30 in the video, Carlos walks through a real example of how their planner handles complex renewals queries with multiple dependent stages. This demonstrates the power of their dynamic execution graphs in action.
Key Takeaways
Cisco's journey proves enterprise AI success requires more than accurate models - it demands architectural thinking, workflow reinvention, and psychological adoption strategies. Their $26B renewals operation now runs on AI teammates that blend deterministic workflows with dynamic planning.
In summary: Production AI systems must balance creativity with compliance, focus on outcomes over accuracy metrics, and become indispensable rather than optional. Cisco's planner-supervisor architecture provides a blueprint for scaling agentic systems beyond pilot projects.
Frequently Asked Questions
Common questions about this topic
Cisco started with chatbot interfaces using genAI only for answers, not questions. They quickly realized this wasn't sufficient for enterprise needs and evolved to agentic systems with dynamic planning capabilities.
The initial guided interface lasted just three months before renewals teams demanded open prompting. This early feedback shaped their entire architecture evolution.
- Phase 1: Rigid Q&A with predefined answer structures
- Phase 2: Open prompting with accuracy verification
- Phase 3: Full agentic systems with dynamic planning
Their initial system achieved 95% accuracy by combining LLMs with traditional machine learning models for predictions. However, they found accuracy alone wasn't enough - personalization and workflow integration were critical for adoption.
The hybrid approach used sentiment analysis agents, adoption tracking agents, and traditional ML risk prediction models working together. This combination proved more reliable than any single approach.
- LLMs for interpretation and reasoning
- Traditional ML for numerical predictions
- Rule-based systems for policy compliance
Rather than treating AI as a tool, Cisco positioned it as a teammate that could delegate tasks, receive assignments, and prove value through business outcomes. This required focusing on workflows rather than just individual agent capabilities.
The teammate model changed user expectations - instead of judging the AI on answer quality, teams evaluated it on business results delivered. This shifted the success metrics from accuracy to impact.
- Receives delegated tasks like a human colleague
- Proactively surfaces relevant insights
- Measured on outcomes rather than outputs
They implemented deterministic task-based workflows for critical operations, similar to coding assistant shortcuts. For example, backslash commands would execute predefined tasks without LLM interpretation, while maintaining flexibility elsewhere.
This hybrid approach gave them the best of both worlds - creativity where needed, reliability where essential. Critical renewals processes could run exactly as designed every time.
- Backslash commands for repeatable workflows
- Confidence thresholds for automatic human handoff
- Strict policy enforcement layers
They discovered optionality kills adoption - when AI was just another tool, users would revert to spreadsheets. The breakthrough came when they embedded AI directly into existing workflows and made it a required collaborator.
This "forced curiosity" approach, combined with personalization, transformed the AI system from something people could use to something they had to use as part of their normal workflow.
- Eliminated alternative paths to complete tasks
- Positioned AI as primary interface for workflows
- Personalized behavior to individual user patterns
They flipped from 'how can AI improve existing workflows' to 'is this workflow even needed?' focusing on outcomes rather than steps. This allowed parallel execution based on context rather than sequential human-driven processes.
The new philosophy treats AI as the primary workflow engine, with humans stepping in only at confidence thresholds. This inverted the traditional automation paradigm completely.
- From step-by-step human orchestration
- To context-driven parallel execution
- With humans only intervening at defined thresholds
Key lessons included: start bounded then extend capabilities, make self-correction mandatory, invest in observability, focus on quality ceilings rather than benchmarks, and solve routing classification before building agents.
These technical insights complemented their workflow philosophy changes, creating a comprehensive framework for production AI deployment at enterprise scale.
- Prove value in bounded domains first
- Build self-healing into the architecture
- Instrument everything for observability
GrowwStacks specializes in building enterprise-grade AI agent systems tailored to your business workflows. Our team can design, implement and optimize AI automation solutions that drive measurable results.
We offer free 30-minute consultations to discuss your specific needs and develop a customized roadmap for AI agent deployment. Whether you need a complete system or targeted workflow automation, we can help.
- Custom AI agent architecture design
- Workflow analysis and optimization
- Free consultation to discuss your requirements
Ready to Build Your Production-Ready AI Agents?
Don't waste time reinventing Cisco's hard-won lessons. Let GrowwStacks design an AI agent system tailored to your business workflows - with measurable ROI from day one.