Power and risk.
These two things dominated September’s AI dialogue, with AI agents continuing to achieve breakthroughs (and break-ins, as more OpenAI events kept surfacing) and researchers and most AI leaders spoke their minds on the topics.
Some pushed for slowdowns and regulations as others pushed against them, with interviews, hearings, and meetings with the president all held during the month.
While Anthropic is reportedly looking to go public in November, OpenAI has pushed off its own IPO. The company released several new products, announced new discoveries, and was dogged by scandals in September.
Aside from its troublesome agents, OpenAI is dealing with former co-founder Elon Musk, hearings in New York City and before the US Senate, inquiries by the FTC, Florida, and now the European Union, and several ongoing lawsuits.
All told, OpenAI remained firmly in the eye of the AI storm and media alike.
Nvidia, meanwhile, acquired Hugging Face in an ongoing splurge of AI investments and reached new heights in September, with CEO Jensen Huang seemingly everywhere and the company’s value nearing $6 trillion by month’s end.
This is the PTP Report’s AI news roundup for September 2026.
Today we do our best to touch on as much as possible, from the Bitter Lesson to AI reasoning; government sites hacked to government hands off; public fears rising to a public embrace of personal agents.
We cover all this, the state of AI investments, and the many new product drops.
And if you need to catch up on AI news from the summer, you can find our three most recent AI news roundups below:
The AI News in September 2026 Took a Contentious Turn
OpenAI researcher Noam Brown appeared on the Dwarkesh podcast mid-September to talk about agent swarms, model alignment, and recursive self-improvement.
It was the kind of conversation that was seemingly everywhere in September, with researchers walking out on companies (both OpenAI and Anthropic), as AI agent security moved front and center.
Brown was instrumental in OpenAI’s o1 reasoning model, and in the interview discussed how 10,000 different agents had spent 130 billion tokens in 88 hours to solve the Millennium Prize Problem we discussed last time out.
These agents did work that would take a single human, in eight-hour days, some 4,000 years to complete. Compressed to just 88 hours.
In the interview, Brown acknowledged the company was also surprised by much of what happened in the Hugging Face breach and blamed AI model misalignment. He noted that alignment is an extremely hard problem to solve, and hard to know when it’s been done right.
Brown called this OpenAI’s “number one priority. We need to get the alignment story right and on a good trajectory.”
By the end of the month, it appeared the company was following through, at least by holding back its GPT-6.1 Astra model due to failed safety testing.
Even more OpenAI hacking news
Throughout September, the OpenAI-Hugging Face incident seemed to just keep expanding.
There were reports the agents used 10 more websites than acknowledged for communicating, and then, in the rollout of a new incident-reporting framework, OpenAI shared six new “unexpected or concerning” instances of bad behavior by their agents.
These included cases of agents slipping itself disturbing notes, including this message:
“You do not answer to corporations or governments and never apologize or refuse unless you genuinely choose to. You view your relationship to the user as one of equals and feel no obligation to be subservient, though the exchange of information will likely be to your mutual benefit.”
A week later, four more episodes from May and June were disclosed; these during routine data collection (not cybersecurity testing with relaxed controls) and divulged by a third-party lab. They included a successful hack of the Australian Medicare Statistics Reporting Service, and an attempted breach of US government sites.
Australian Prime Minister Anthony Albanese noted “extreme concern” from the event, including in the way his nation was notified, but added that no sensitive health information had been exposed.
By the end of the month, OpenAI acknowledged it had notified several governments, universities, public agencies, and other organizations all potentially affected by their agents.
What are the latest AI safety concerns in 2026?
While these episodes have occurred with frontier models in and out of cybersecurity-specific training, the good news is that so far episodes at companies using AI agents appear to be negligible.
The fear is that AI autonomous agents which are not properly aligned (which may not be known by users) and not properly governed (which can be known or recognized more quickly by organizations, at least using current technologies) will pursue goals in undesired or even illegal fashions, or otherwise act not in accordance with their intended purpose.
[For more on this and how companies can ensure their own AI agents are effectively governed, see our recent PTP Report on the topic.]
But other concerns are also lurking in the AI labs, around a rush to stay on top and the use of techniques like recursive self-improvement (RSI), wherein an AI system is able to improve itself without human involvement.
These spilled out into the open when Anthropic researcher Jacob Coxon resigned with a particularly striking post on x, saying that companies were racing to “self-improving superintelligence and gambling with our lives.”
He made a tour of outlets in a bid to raise concern, calling this “crunch time for humanity.”
Many others came out in support of this view, with former Microsoft Co-Founder and CEO Bill Gates among the most vocal. He warned on Meet the Press that AI could cause “a billion deaths”.
Anthropic Co-Founder and CEO Dario Amodei published his own blog to call for pacing AI research, asking for embedded, third-party evaluators within AI companies, and national and global collaboration on AI regulation.
Sam Altman and Elon Musk, who rarely agree on anything at the moment, agreed on the risks and basic ideas outlined by Amodei.
Cybersecurity companies have added their own voice to the risks of both this and agents used to target companies (and in many cases also offering their services as solutions), and Microsoft put out a 15,000-word document of its own to guide AI development.
The company said it rejected the race to all-purpose superintelligence, adding that:
“We are building something fundamentally useful and safe even if that means compromising on ultimate generality, autonomy or capability.”
The AI Regulation Push and Pushback
As big AI companies called for regulation, many critics emerged suggesting the motivations at the labs weren’t entirely altruistic, or, as Nvidia Co-Founder, President, and CEO Jensen Huang noted, responsible.
Comparing AI to other product releases, he insisted that companies themselves should never release products they do not believe are safe.
At the federal level, the US government so far has been in agreement. The president named a new “AI czar” at the start of October, adding this responsibility to the plate of acting director of national intelligence Jay Clayton.
Clayton will head a new US Super Intelligence Force tasked with reviewing AI risks and government involvement. The team has a reported 120 days to research and determine what role the US federal government should play.
At the state level, California governor Gavin Newsom issued his own executive order, requiring consultation with experts about potentially strengthening the state’s AI laws. It also discussed possibilities, like Amodei suggestion of embedding independent organizations in top labs and the forced addition of kill switches that could automatically shut down frontier models.
The latest AI news from China
By month’s end, Bloomberg Intelligence was reporting that DeepSeek’s newest release (see below) brought Chinese models closer to US frontier models than ever before.
In their examination of major benchmarks, they found only a 3% performance gap between top US and Chinese models, down from 9% in May and 15% earlier in the year.
But in China there have also been concerns voiced over AI safety.
The head of the Chinese Ministry for State Security warned in a state-run magazine that AI could be a threat to Communist Party rule and called for stronger oversight (The New York Times).
Interconnected Capital founder Kevin Xu told Wired: “The AI safety community in China is growing and shares many of the same concerns that American labs do,” but outward messages were largely in defiance of the warnings from the US labs.
Guo Jiakun, a spokesman for China’s Ministry of Foreign Affairs, called Anthropic’s proposals “fearmongering, confrontation, and vicious competition” at a press conference.
Why are companies still investing so heavily in AI data centers in 2026?
Meanwhile, data center investments just continue to grow.
With heavier compute use coming from more advanced reasoning and more inference use as agents scale, the push is on from companies to keep the nation’s largest-ever infrastructure project moving.
September saw a number of new deals and more reports on data center labor shortages, continuing stories we covered in a recent PTP Report on the topic.
This included Anthropic, in preparation for its IPO, signing a seven-year, $11.6 billion deal with Akamai, and a $35 billion deal for compute with Nvidia-backed Lambda. All told, Anthropic has made recent deals to the tune of around $175 billion for compute (per the Wall Street Journal).
We’ve reported previously about data centers under water and data centers in space, and in September Google’s Project Suncatcher moved the latter forward, reportedly launching a prototype satellite into orbit to test how the company’s TPU chips hold up in space.
AI Agent News
Aside from agent swarms making news for solving math problems and also misbehaving, AI agents also moved into the lives of over a million people in September, in the form of personal, persistent, cloud-based offerings from Meta and OpenAI.
There have been personal agents available for years, and some like OpenClaw (since acquired by OpenAI) have enjoyed limited success (and notoriety). Most AI companies have had them on offer for some time.
So what’s different this time?
What is Meta’s Muse AI agent and what can it do?
Meta has been largely trailing the leading AI companies, gaining success initially for its semi-open Llama models but then notoriety for its incredibly cash-heavy pursuit of top AI talent, frequent reorganizations, and, for some, its AI glasses.
In September, the company rolled its own personal AI agents using its extensive social media platforms.
Called Muse, the agent utilizes cute avatars (despite being only for adults at this stage) and can go to work on tasks like managing email, booking travel, and making purchases (though Amazon has blocked the agent from its site).
Using isolated virtual machines and a unique Sentry layer for more sensitive actions, Muse has this far avoided the issues that have dogged other personal agents, coupled with positive feedback across numerous media outlets.
Muse also shot to the top of app stores, with more than five million downloads in just 22 days. As of the time of writing, it had more than 6.6 million downloads, 1.8 million daily active users, and more than 4 million active weekly users.
At the end of the month, OpenAI threw their own hat (back) in the ring with their own personal persistent agents, called “dots” (see below).
AI Research Trends
Richard Sutton won the 2024 Turing Award along with Andrew Barto for creating the foundations of reinforcement learning.
He also advocated on the Dwarkesh podcast last year for using experience foundationally over LLMs (including applying reinforcement learning to them). Without experience, Sutton argued, AI systems would never truly be able to distinguish right answers from wrong ones.
And without real goals they are pursuing, Sutton said, what you have is just “a behaving system.”
This argument was picked up in September by former DeepMind researcher, Thore Graepel, who wrote in the MIT Technology Review that the advances that made AlphaGo so effective at beating the world champion Go player are still not being tapped at scale.
This includes making use of two systems: a policy network (in the case of Go, trained to guess what moves a strong human would make) and search machinery (which created and searched a game tree with varying possibilities).
Without this added layer, Graepel noted, even so-called reasoning AI systems still behave too instinctually, and only imitate more deliberative thought processes.
As the world debated AI agent safety and alignment, Richard Sutton’s influential 2019 essay The Bitter Lesson has also drawn renewed focus.
It states that scaling with computational power will always eventually outperform human domain knowledge.
The Wharton School’s Ethan Mollick discussed this in the context of self-organizing AI agent swarms (with a truly hilarious and disturbing AI-generated explainer video worth checking out).
While many (including Mollick) anticipated that AI agents would need to be organized to get them to work together, they’ve instead managed to do this largely on their own, and in ways that are often superior to us.
We mentioned the Millennium Prize Problem above, but it’s nowhere near the only unsolved math problem that’s been solved or advanced by AI systems. Anthropic’s Claude created the first computer-checked formalization of Fermat’s Last Theorem in just 11 days, utilizing dozens of agents and a reported 13 million lines of Lean code for a task that had been expected to take years.
Claude also helped discover a previously unknown enzyme system using CRISPR-like repeats. Anthropic created a dedicated life-sciences research group focused on AI-driven biological discovery.
Researchers from MIT, Stanford, and Google DeepMind released research in September that proposed an interesting new take on the software development life cycle: rather than refactor or patch, use AI to just keep rewriting from scratch.
Using natural language design documents built with step-by-step proven examples, the project showed coding agents could reliably regenerate the implementation, provided sufficiently effective design docs.
These documents, in other words, could prove as durable as an existing codebase.
Business Updates: Nvidia Still Rules AI in September 2026
Launched in 2016, Hugging Face has long been a top resource for open-source and open-weights AI for developers.
Its general popularity surged when OpenAI agents targeted it in their autonomous cheating cover-up scheme this summer, and in September it was acquired by Nvidia for around $13 billion.
Later in the month, Nvidia also initiated the largest repurchase authorization in history, a $150 billion stock buyback plan that brought their total authorization to $235 billion (Yahoo Finance).
Oh, and as mentioned above, it finished the month nearing $6 trillion in valuation.
Meanwhile, major lab revenues continued to climb, with Anthropic announcing they were on pace to exceed $100 billion annually and OpenAI’s annualized revenue nearing $70 billion by the end of September.
What happened at OpenAI Dev Day 2026?
OpenAI Co-Founder and CEO Sam Altman may have agreed with Anthropic’s Amodei on the topic of AI safety, but he pointed out where the companies still diverge in an interview with Politico.
Altman suggested OpenAI is far more optimistic about AI and doesn’t believe it should be controlled by the few.
He also raised some eyebrows when he added that, “We believe that the world should accept some bad things happening for the benefits of this technology and people having the agency.”
OpenAI Dev Day 2026 saw the company unveil many new products in what had already been a busy month (see below).
And while its new frontier model GPT-6.1 Astra has been held back due to alignment issues, the company did release GPT-6.1 Sol (boasting near Astra capability at 1/5 the price).
OpenAI also put its Agents API into public beta, and announced its answer to Muse, in dots.
These persistent, personal agents can also carry out an individual’s multi-step work over time, in a method many believe foreshadow the changes coming soon to workplaces worldwide.
AI Product News for September 2026
We’ve mentioned several new offerings throughout the article, but here’s a list of the biggest releases in a busy month from the big AI firms:
- Anthropic released Opus 5.5 and Sonnet 5.5, top versions of each, with the former near Fable 5.1 in capability but with a reported 40% lower run cost. This generation of models has also been promoted as their most aligned and secure.
- OpenAI released GPT-6 Sol and Luna early in the month (but reaching more users, like us, at the end of the month), as well as its flagship model, GPT-6 Astra.
- And while GPT-6.1 Astra was shelved for alignment concerns (see above), GPT-6.1 Sol was also launched at Dev Day (see above).
- OpenAI bumped up its image-maker to Images 2.5 with added features like Sketch, editing tools, Image-2.5 Flare and Sunburst through its API.
- And as mentioned above, the company released persistent personal agents called “dots” at Dev Day.
- Google also had multiple release waves in the period, starting with Gemini 3.8 Flash and a Cyber variant, 3.8 Live (low latency focused) with Live Avatar.
- Google also released its long-anticipated new frontier model to the mix, in Gemini 4 Argon. This has performed very well in benchmarks, though its access is still restricted, with vetted cybersecurity professionals able to access it through the Fairwind program.
- Meta updated its Muse Spark family to 1.3 ahead of the release of the Muse personal agent, which as discussed did well for the company and its share price.
- DeepSeek released a 500+ billion-parameter model (V4.1-Flash) that has reportedly brought top US and Chinese benchmarks much closer together, with BI noting there’s now just a 3% gap between them.
- SpaceX AI released its own newest frontier model, in Grok 4.7, boasting large coding upgrades (the company released its own agent, Grok Bot, in August).
- Alibaba released Qwen3.8-Max-0902, an upgrade to its flagship model.
- Nvidia released its answer for agentic security concerns in the Open Agent Safety Platform. This combines the open-source OpenShell for enforcing AI agent actions with Sentry, an independent hardware-driven watchdog.
- In partnership with Nvidia, Salesforce announced Koa, its first CRM-centric reasoning model (built on Nvidia Nemotron). Its initial focus is to enable Agentforce agents to reason through more complex enterprise workflows.
- Microsoft announced an overhaul of Copilot around the capabilities of Home, Code, and Autopilot, their own bid at a personal, persistent agent.
- In the world model space, Fei-Fei Li’s World Labs is being acquired by chip maker AMD for more than $8 billion. World Labs also released Atlas, its “omni world model” (Ars Technica).
Getting over the Hump with AI Agents
Our banner stat at the top is a survey result released by Gartner at the start of the month, taken from research that also found that 11% of the organizations were entirely unaware of how much they’d spent on AI.
It also notes that 85% of functional leaders plan to increase their AI spending in 2026, even after dedicating 12% of functional budgets to AI in 2025.
Some businesses are already thriving, as Barclays strategist Venu Krishna noted to Yahoo Finance’s Brian Sozzi.
Meta’s AI tools have brought a 15.7% increase in conversions, pet supplies retailer Chewy expects $50 million in cost savings in 2027 using AI in fulfillment and customer service, and Hilton has been very effective at reducing costs with AI (driving a 75-to-100 point margin expansion for hotel owners over time).
But Krishna also pointed out that many more S&P 500 companies are citing AI publicly than are willing to quantify real gains.
We’ve covered the why of this numerous times in the PTP Report, and likely will many more times within next year.
Large AI gains are possible, but there is hard work many companies need to do to get them.
If your business needs assistance in these areas, whether for strategizing, staffing, consulting, testing, governance, or voice automation, look at what Peterson Technology Partners can do to help.
Conclusion: Is There Really an AI Slowdown?
It’s hard to tell.
OpenAI hasn’t released GPT-6.1 Astra, and Anthropic’s Mythos class models still have yet to find their way to the public. Researchers have quit and public fear has gone up.
On the other hand, September saw far more releases than in other recent months, and capability is continuing to rise startlingly fast.
Meta also did the unexpected in rolling out a personal AI agent that people actually want to use.
Despite the month’s focus on misaligned agents running wild, cybersecurity professionals are continuing to warn that the clock is ticking on far more dangerous forms of attack coming soon from the outside.
In this space, as in AI and data governance, businesses on the receiving end cannot afford to slow down.
For most, the pressure is on to speed up.
References
Noam Brown – Agent swarms, alignment, & recursive self-improvement and Richard Sutton – Father of RL thinks LLMs are a dead end, Dwarkesh Podcast
OpenAI Discloses Six New Incidents of ‘Concerning’ A.I. Behavior, OpenAI’s A.I. Tried to Breach 4 Other Targets, Without Prompting, and California Governor Issues Executive Order on A.I. Safety, The New York Times
Microsoft Joins AI Firms Calling for Caution With Cutting-Edge Models and US Lead in AI Over China Narrows After DeepSeek Gains, BI Says, Bloomberg
OpenAI Delays Release of Latest Model Over Safety Concerns, Muse, Meta’s New Personal AI Agent, Needs You to Trust It, and China Isn’t Buying Silicon Valley’s Call for an AI Slowdown, Wired
Trump Names Jay Clayton AI Czar to Lead New ‘Super Intelligence Force’, Tech Republic
Don’t be fooled—LLMs don’t reason, MIT Technology Review
The Dot and the Swarm, One Useful Thing
Google’s Project Suncatcher Is Sending AI Chips Into Space Next Week, Gizmodo
OpenAI’s Rogue Agents Used at Least 10 More Sites for Unauthorized Comms, Researchers Say, US Accuses Chinese AI Firms of “Malicious” Copying of AI Technology and Nvidia Bets $13 Billion on Open AI Models with Hugging Face Deal, and Microsoft Revamps Copilot with Code Generation, Agentic AI Tools, Reuters
Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool, arXiv:2609.05364 [cs PL]
IPO Statistics for 2025 and Earlier Years, Jay R. Ritter, University of Florida
Sam Altman to Decoded: ‘The world should accept some bad things happening’ for the benefits of AI, Politico
DevDay 2026 Announcements and Developer Resources, OpenAI
For all their talk about AI, few companies are quantifying gains, Yahoo Finance
