PTP Key Takeaways:
- A divide is growing between those seeing major gains from agentic AI for business and those not.
- Cognitive limitations make agents far more effective at handling clear, straightforward use cases with the aid of effective training.
- Agents are faring far worse at autonomy than augmentation, revealing low success rates at complex tasks but huge boosts in efficiency and cost reduction.
- Data cleaning for AI remains a consistent challenge for companies to overcome first to get the most value from systems.
- Customer segmentation with AI is one means of extending data to drive effective sales gains with current sales automation platforms.
- Successful AI-driven sales strategies take advantage of AI’s ability to scale and handle repeated tasks like scheduling, contacts updating, customer reactivation, and targeted outreach.
Faster and cheaper but significantly worse.
In a nutshell, that’s the current state of most agentic AI for business circa November 2025, compared to humans doing the same work.
While agentic solutions bring significant upside to the right tasks, their value is incredibly uneven, both between tasks and implementations, leaving many companies struggling to get advertised gains from agents while watching others thrive.
And while most businesses are using AI in some form and are committed to adopting agents, many are still moving slowly and cautiously, trying to avoid falling into early adopter pitfalls.
We’ve previously covered the impact of AI coding tools and agentic AI in software testing. In today’s PTP Report we look at the state of AI agents in sales. We cover the promise and shortfalls, profile new research results from sources both academic and professional, and look at use cases where current systems are increasing revenue in sales.
While autonomy may be the agentic promise of the future, today it’s more hype than reality, with augmentation still the name of the game.
Agentic Upside: The Promise and Peril in Autonomous AI Sales Tools
In their study of 1,250 firms worldwide (released in September), the Boston Consulting Group found just 5% are getting measurable AI value at scale.
They found 60% to be getting no value at all (despite significant investment), with the remaining 35% (+13% over 2024) still scaling up and just starting to see returns.
This is one in a sea of studies that have come out this year pointing to the difficulty of undergoing substantive AI transformation.
Late last year, AI agents were all the rage, with 2025 dubbed “the year of the agent.” But it’s November, and the actual returns remain highly varied.
Of companies that have already implemented AI agents, 66% say they are delivering measurable value in the form of greater productivity (per PwC).
This is a far cry from the 5% above, or MIT’s NANDA study released in August that famously found that, despite $30–40 billion in spending, 95% of the enterprise AI pilots they studied failed to deliver meaningful ROI.
Or Gartner’s forecast from June, which said 40% of current agentic projects would be scrapped by 2027.
In each case, there is abundant fine print, such as Gartner’s belief that a significant number of these are actually using “agent washing” (they estimated that just 130 of the thousands of vendors hawking AI agents are offering the real thing).
Or the MIT NANDA research pointing to flawed pilot designs, aiming for big, splashy results, and avoiding back-of-office use cases. They also noted that enterprises fared worse in part because of scale, goals, and very slow movement.
As one of the founding members of OpenAI and former director of AI at Tesla (also creator of the first deep learning course at Stanford), Andrej Karpathy has a knack for saying timely truths about the state of AI systems, like coining the name “vibe coding” for wholly AI-generated code.
In an October interview with podcaster Dwarkesh Patel, Karpathy made waves again by saying AGI is easily a decade away, and that agents need a decade to become truly autonomous. While he uses AI agents every single day, and stressed that they perform many useful functions, Karpathy said they’re nowhere near to functioning like full employees because they lack continual learning and have substantive cognitive issues.
He once again emphasized that in the effort to reach AGI or jump to full autonomy, companies are missing the useful things they can do now.
In areas like coding, AI systems (which mimic us extremely well) are also limited to repeating the ways things have already been done. This makes them good at writing boiler plate code, for example, and they can also lower the bar for languages you don’t already know.
This same truth often applies to other areas of business, too, be it marketing, sales, customer service, recruiting, or finance.
Agentic AI Sales Performance Insights from Carnegie Mellon and Stanford
Maybe not coincidentally, AI agents appear to approach most business problems as if they are coding problems.
Or at least this was found by researchers from Carnegie Mellon and Stanford. In a study that sought to understand and measure how AI agents really do human work, they compared them, working across skillsets like design, engineering, data analysis, computation, and writing.
In matching computer-use activities along variable, structured workflows, they found that:
- AI agents take an “overwhelmingly programmatic approach across all work domains” (even open-ended tasks) in sharp contrast to human methods.
- The AI agents produced work of notably lower quality and also mask it, with apparent progress updates, data fabrication, and misuse of some tools, overall faring 32.5–49.5% worse than humans.
- The AI agents are much faster (88.3%) and cost far less (90.4-96.2%).
- Humans who used AI in their study did so 75% of the time for augmentation, not automation, and they enjoyed a 24.3% acceleration in their work (vs a 17.7% drag on those attempting full automation).
- Once again, teaming up AI and humans to the best of their abilities ensured both quality and increased efficiency, with boosts measured at 68.7%.
You can read the full research yourself here, but it isn’t the only study of its kind measuring agents vs humans and finding similar results.
As discussed in our October AI news roundup, Scale AI with the Center for AI Safety (CAIS) created a new Remote Labor Index benchmark with the goal of measuring how well AI agents can do freelance work tasks.
And so far, the results aren’t good. The best agents managed less than 3% of tasks successfully, earning around $1,810 from a possible $143,991.
The Salesforce SCUBA AI Metric and What It Shows
Barron’s reported at the end of summer about the “decision fatigue” some Salesforce customers have expressed, overwhelmed by the speed of change with AI overall. This has been one barrier for some companies leading the way in AI adoption.
Another has been successfully measuring the change.
While most benchmarks focus on web navigation or software manipulation (or are academic or consumer-oriented), Salesforce AI Research created the SCUBA benchmark with the goal of filling this gap. Measuring 300 task instances from real user interviews (platform admins, sales reps, and service agents) and run in sandbox environments, it, like the study above, aims to understand real agentic AI capacities and not just measure how well they answer questions.
Their research looked at how models used the UI, manipulated data, triggered workflows, and troubleshot issues, in the effort of seeing how well agents can operate inside enterprise software systems.
And like the study profiled above, they found huge gaps in performance, along tasks and also favoring the costlier, closed models over open-source systems. At their best, agents managed just 50% success rates but with time reductions of 13% and cost reductions of 16%.
Their insights included:
- Real world autonomy is still hard, demonstrated by a drop in performance on SCUBA vs OSWorld (desktop apps).
- Demos and tutorials really help the agents. Human demonstrations increased success and lowered both time and token use. But the quality of the demonstration matters a lot, as less effective demos can actually increase token use while not improving results.
- Meaningful improvement requires success in three areas, all at once: effectiveness, time, and cost. Improvements need to keep all three in mind, and measure things like latency, the token cost, number of steps, and error recovery.
- For the future improvements, training data needs to shift from text to UI/action context, UX to become more agent-friendly (structured actions with better logs), and effective demonstration libraries for workflows can be really beneficial.
Practical Uses for AI Agents in Sales
Companies that are thriving now are doing so by notching less flashy wins, but at scale. Rather than looking to create complex, autonomous systems, human co-piloting of AI remains highly effective, with short, effective loops for verification.
More, smaller agents can give bigger benefits over massive, do-all systems, playing to the current strengths of AI.
Tool use remains problematic, with UX being one of the first stumbling blocks companies are working rapidly to beat.
Reliable outputs come from fine-tuning and well-defined use cases, with strong context.
There are numerous areas that are seeing real gains, and one example that’s actionable and scalable was reported in The Information in November.
Data management startup Cribl uses an AI agent to assess contracts, determining where overpayment is occurring. Its agent then contacts vendors to ask for lower prices.
And while their human team takes on all the largest contracts, they just don’t have the bandwidth for smaller deals and trust the agent with these.
This process has saved the company $3 million this year alone. While the humans may not have time to “nickel and dime all these vendors” to adjust contracts, the agent does.
AI Sales Outreach Tools in Action
A common thread is scale, and straightforward use cases that exceed human bandwidth. From healthcare to finance to recruiting, these approaches are seeing some of the biggest wins for businesses making effective use of agents today.
For sales use cases, this plays out in tasks like drafting and summarizing, initial research, first contact handling (inbound or outbound), contacts cleanup and outreach, scheduling automation with reminders and follow-ups, and records updating and improvement.
At PTP, we’ve helped companies successfully implement AI sales automation for many of these uses cases and have seen easy wins that scale and bring efficiency and outreach that’s out of reach today.
This includes returns like:
- 1.5 times improvement in customer acquisition
- 20% reductions in churn
- 40% faster conversions
- 20% expansions through upsell and cross-sell
- Overall increases in portfolio value as high as 35%
We’ve seen AI calling pilots, for example, yield, from just their first 1,000 calls: additional contact details from 35 clients, 25 leads for the sales team, and 12 closed deals, here generating $21k in revenue.
This form of sales outreach serves as an extension of a sales team’s own efforts, or effective augmentation, handling the “nickel and dime” calls they can’t, transferring opportunities to humans, and while maintaining consistent, clean records updated back to the CRM.
Once again, these cases remove manual work that is often outside a human team’s reach, rather than attempting to automate entire processes.
Prep and Data Cleaning for AI Remains Essential
In addition to making wise choices on use, effective agentic AI also requires data that’s clean and ready.
And along with the sprawling variety of sources, structural issues, and rapid speed of growth, some problems begin with how companies are using their CRMs in the first place.
Validity put out a report this summer drawing from 600+ CRM administrators and business users that showed a high level of concern for this quality, which can directly impact how effectively they can be used for AI augmentation and automation cases alike.
76% of those surveyed acknowledged that that less than half of their data is both accurate and complete, with 45% saying their data isn’t ready for AI. An additional 37% said they currently lose revenue as a direct result of these quality issues.
AI tools can also help with these corrections, too, such as resolving completion issues and updates.
Regardless, addressing data is an essential first step for organizations struggling to implement successfully and wondering why rivals may be faring better.
[We’ve written a few times about the challenges of AI data readiness. You can check out this PTP Report for more on ensuring data privacy and security with AI.]
Customer Segmentation with AI
Putting data issues aside, another way businesses can improve their ability to get sales results ASAP from AI is through customer segmentation.
With a capacity to conduct outreach at scale, it’s increasingly valuable for businesses to know why customers have gone dormant or inactive, for example.
As in the example above with contract renegotiation, reactivating even small numbers of idle accounts can translate into significant gains, and an action that AI agents can undertake successfully today.
This begins with asking:
- What differentiates your inactive contacts?
- Can you establish a finite list of reasons contacts become inactive in the first place (root cause to address)?
- Can you segment by value or assigned sellers?
- Have personnel turnovers affected contact activity?
- Can you group by sales/transaction volume and/or order frequency?
Forming contact segments by answering questions like these is one way we’ve helped companies directly apply agentic AI systems to expand their reach in highly targeted, and effective ways.
Conclusion
At PTP, we’re finding the good with agentic AI today is how it can significantly improve efficiencies and lower costs.
The bad stems from, as Karpathy termed it, cognitive deficiencies that can lead from inconsistencies to outright failures with some automation efforts, covered by a tendency to mimic or imitate success (and buried by hype).
But where AI agents are truly, or justifiably, hype-worthy is through augmentation and partnerships that merge human and agentic quality and efficiency together. In sales, we’ve seen this steadily increase through outreach and inbound contact handling, scheduling, and data maintenance and updating.
When will agentic systems get above the 50% mark with any consistency on benchmarks like Salesforce’s SCUBA? While everyone has a guess, it’s safe to say no one really knows for sure.
But if you are in need of assistance with implementation, data preparation or cleaning, customer segmentation, or great talent today—or just want to talk AI—contact PTP. We are AI-first but implement with care and an eye for safety, quality, and compliance.
References
The Widening AI Value Gap: Build for the Future 2025, BCG
Over 40% of agentic AI projects will be scrapped by 2027, Gartner says, Reuters
Andrej Karpathy — AGI is still a decade away, Dwarkesh Podcast
How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupations, arXiv:2510.22780v2 [cs. AI]
Salesforce AI Agent Adoption Hits a Hurdle. Customers Have ‘Decision Fatigue,’ Analyst Says., Barron’s
SCUBA: Salesforce Computer Use Benchmark, arXiv:2509.26506 [cs. AI]
Agents Learn to Bargain Down Software Prices, The Information: Applied AI
The State of CRM Data Management in 2025, Validity
FAQs
Why are there so many reports of AI agents failing alongside reports of how much money they’re saving companies?
This disconnect is real, and it’s more than just hype. Agents today can excel at certain tasks where there are well-defined use cases, and with effective training and oversight. But as research is increasingly showing, they fare far better in augmentation roles working alongside people in tight loops to ensure their quality and effectiveness.
How should companies evaluate vendors claiming “autonomous agents”?
Gartner has reported on the extent of “agent washing” that’s prevalent in the industry by estimating that of the thousands of vendors promising agentic AI, just around 130 actually provide them. To get more detail on the autonomy, ask for detail on the exact tasks agents are performing, their success/efficiency/cost metrics (ala SCUBA), how they read from and write to data sources like CRMs, their human-in-the-loop controls, and how they ensure compliance, governance, and auditability.
What is customer segmentation, and why is it useful for getting value from AI sales agents?
One thing agents can do very well today is get to work that humans can’t, often because it lacks sufficient value in isolation. These “nickel and dime” tasks can really add up in total and agentic AI’s capacity to conduct outreach, for example, that’s both personalized and at scale is one example of a substantial revenue-gainer available from today’s agents.
Customer segmentation is a way companies can improve the context for agents and their capacity for personalization, by grouping contacts that have gone dormant or fallen off by things like transaction volume and frequency, and the reps who worked them.
