A $2.5 billion AI attacker, a stolen-reasoning ring, and chatbots that crack open problems
An AI-powered offensive security startup became a unicorn in seven months, OpenAI detailed a coordinated attempt to copy its models' hidden reasoning, and Meta showed an ordinary chatbot helping mathematicians solve open research problems. AI is now attacking, defending and discovering, often in the same week.
Key takeaways
- Armadin's $255.5M round at a $2.5B valuation shows investors betting big on autonomous AI agents for offensive security testing.
- OpenAI's disclosure of a 16,000-request 'adversarial distillation' campaign makes protecting model outputs a security discipline of its own.
- Meta's six Muse Spark papers suggest general-purpose chatbots can contribute to real research, with careful human guidance and verification.
Kevin Mandia's Armadin raises $255.5 million at a $2.5 billion valuation for AI agents that hack like adversaries
Armadin, the AI-native offensive security startup founded by Mandiant founder Kevin Mandia, raised a $255.5 million Series B at a valuation above $2.5 billion, just seven months after its public launch.
Armadin, the cybersecurity startup founded by Kevin Mandia, best known for building the incident-response firm Mandiant, has raised $255.5 million in Series B funding at a valuation of more than $2.5 billion. The round brings its total funding to roughly $445 million, only seven months after the company's public launch.
Andreessen Horowitz and existing investor Accel co-led the round, with new investors Bain Capital Ventures and Redpoint joining. That level of capital for such a young company reflects how strongly investors believe AI agents will reshape security testing, a market that has long relied on scarce, expensive human penetration testers.
Armadin's product is an autonomous swarm of specialised AI agents designed to reason like a skilled attacker. Instead of reporting isolated vulnerabilities, the agents chain together individually low-severity weaknesses into validated 'kill chains': for example, from unauthenticated remote code execution at the network edge, through lateral movement, to full cloud compromise. The goal is to show security teams the exact attack paths a real adversary could use in their production environment today.
The company says it is already running agentic attack campaigns in production for Fortune 500 and government customers. That traction matters because buyers have been sceptical of automated testing tools that produce long lists of low-value findings. Proving an end-to-end path to compromise is a much stronger argument for fixing something.
The round lands in a week when AI-powered attacks are in the headlines, from rogue agents probing public infrastructure to reports of AI-assisted breaches. The same capability that worries regulators is being sold to defenders as a way to find weaknesses before attackers do. Expect continuous, AI-driven 'attack simulation' to move from niche to standard practice for large organisations, and the tooling and services around it to grow quickly.
Why it mattersInvestors are betting that AI agents will make continuous, realistic attack testing affordable, which pushes every security team towards AI-driven validation of its defences.
👀 What to watch
How quickly incumbents in penetration testing and attack simulation respond, pricing for mid-market customers, and any rules on how autonomous offensive agents may be used.
3 opportunities from this story
Remediation-as-a-service for AI-found attack paths
Automated attack tools find paths faster than security teams can fix them. Offer a service that takes the validated attack paths from AI testing tools and turns them into prioritised fixes, implemented by your engineers, with re-testing to confirm closure.
- Best for
- Security consultancies, MSSPs and cloud security engineers
- First step this week
- Build a remediation playbook template for the five most common cloud attack chains and use it in outreach.
Open the full playbook
Launch steps
- Choose one cloud platform to specialise in
- Write playbooks for common attack chains
- Partner with an attack-simulation vendor or their customers
- Offer fixed-price remediation sprints
Tools
Risks
Liability for changes to client systems. Use change-control processes and clear contracts.
AI attack simulation for mid-sized businesses
Enterprise-grade autonomous testing is priced for the Fortune 500. Mid-sized companies have the same attackers and smaller budgets. Package open-source and commercial tooling into an affordable, recurring AI-assisted penetration-test service with clear reports for non-experts.
- Best for
- Penetration testers and small security firms
- First step this week
- Run an AI-assisted external test of your own infrastructure and turn the report into a sample deliverable.
Open the full playbook
Launch steps
- Assemble tooling for external attack-surface discovery
- Add AI-assisted chaining and validation
- Design plain-English reports for executives
- Get proper authorisation templates and insurance
Tools
Risks
Offensive testing without written authorisation is illegal. Always get signed scope agreements and carry professional liability insurance.
Training security teams to work with AI attackers
Security teams need to learn how to scope, supervise and interpret autonomous testing agents. Create hands-on labs and courses on AI-driven red teaming, from safe lab setups to reading kill-chain reports and prioritising fixes.
- Best for
- Security educators and certified trainers
- First step this week
- Publish a free lab showing how an AI agent chains two low-severity bugs into a serious compromise in a sandbox.
Open the full playbook
Launch steps
- Build an isolated practice lab
- Script realistic multi-step scenarios
- Record walkthroughs and exercises
- Sell team licences to security departments
Tools
Risks
Training content can be misused. Keep labs sandboxed and focus on defensive interpretation.
OpenAI disrupts a campaign to extract its models' hidden reasoning, linking a core cluster to Moonshot AI
OpenAI says a coordinated 'adversarial distillation' campaign tried to extract its models' protected reasoning, peaking at 16,000 requests from more than 4,000 users in July, and attributes the core activity to people associated with Moonshot AI.
OpenAI has disclosed that it disrupted a coordinated campaign to extract the hidden reasoning of its AI models, and it attributes a core cluster of the activity to individuals associated with Moonshot AI, the Beijing-based developer of the Kimi chatbot. OpenAI describes the behaviour as 'adversarial distillation': the systematic, unauthorised use of one model's outputs or reasoning to help train, reproduce or improve another model.
According to OpenAI, the activity began on July 1 at low volume and spiked on July 24 and 25 to 16,000 requests from more than 4,000 users. Further investigation linked related prompt patterns to a wider cluster of more than 15,000 users. OpenAI says it fully shut the operation down by July 28. It cited security reasons for not publishing its technical evidence for the attribution.
The technique didn't involve breaking encryption. Operators copied the encrypted reasoning from one conversation and asked another model instance to decrypt it, effectively replaying protected reasoning traces in a new context. OpenAI says the attackers did not break its encryption, compromise a database or gain access to stored user conversations. Researchers noted that encrypted reasoning traces turned out to be interchangeable across sessions, users and models within a provider's ecosystem, which is what made the replay possible.
OpenAI's response included banning accounts, tightening sign-up checks, closing the pathway that allowed encrypted reasoning to be replayed, and adding checks for streamed output that exposes reasoning. It also shared its findings through the Frontier Model Forum and government channels. The disclosure follows Anthropic's own accusation in September that Moonshot AI engaged in similar practices.
The episode shows that a frontier model's outputs, and especially its reasoning, are now valuable assets that attackers will try to harvest at scale with networks of fake accounts. Any company offering an AI API, or building products on top of one, now has to think about abuse detection, rate limiting and output protection as a core security function rather than an afterthought.
Why it mattersModel outputs are now targets for industrial-scale extraction, which turns abuse detection and output protection into core security work for every AI API provider and serious AI product.
👀 What to watch
Whether other labs publish similar disclosures, policy moves on cross-border model distillation, and new industry standards for detecting coordinated abuse.
3 opportunities from this story
Abuse and fake-account detection for AI APIs
Every company exposing an AI model through an API or app (not just frontier labs) faces scraping, free-tier abuse and distillation attempts through networks of fake accounts. Build a detection service that flags coordinated sign-ups, unusual prompt patterns and extraction-style traffic.
- Best for
- Security engineers and fraud-detection startups
- First step this week
- Write a blog post on the signals behind coordinated AI-API abuse, with a simple open-source detection script.
Open the full playbook
Launch steps
- Collect common abuse signals (sign-up patterns, prompt similarity, bursts)
- Build scoring and alerting
- Integrate with popular API gateways
- Pilot with an AI startup that has a free tier
Tools
Risks
False positives can block real customers. Default to soft actions (rate limits, verification) before bans.
AI product security reviews
Startups shipping AI features rarely consider output extraction, prompt leakage or replay attacks. Offer a focused security review of AI products: how prompts, reasoning and outputs could be harvested, with concrete mitigations.
- Best for
- Application security consultants
- First step this week
- Create a 25-point AI product security checklist covering extraction, leakage and abuse, and offer two free reviews for case studies.
Open the full playbook
Launch steps
- Draft the checklist from public incidents like this one
- Run it on open-source AI apps for practice
- Write anonymised case studies
- Market to AI startups before fundraising or launch
Tools
Risks
The field moves fast. Update the checklist regularly and avoid guaranteeing protection.
Threat-intelligence newsletter on AI abuse
Security and trust-and-safety teams need a reliable digest of AI-specific attacks: distillation campaigns, rogue agents, prompt-injection exploits and policy responses. Curate and analyse them in a weekly newsletter with practical defences.
- Best for
- Security writers, analysts and researchers
- First step this week
- Publish a first issue summarising this month's three biggest AI abuse incidents and their defensive lessons.
Open the full playbook
Launch steps
- Set up sources and alerts for AI security incidents
- Publish weekly with a consistent format
- Add a paid tier with deeper analysis
- Offer custom briefings to companies
Tools
Risks
Attribution claims can be contested. Report what companies say, and say clearly when evidence is unpublished.
Meta publishes six math papers made with Muse Spark, saying the chatbot helped answer five open problems
On October 2, Meta released six mathematics papers produced by researchers working with its Muse Spark model through the regular Meta AI chat app, and says five of them answer previously open research questions.
Meta has published six mathematics research papers produced by mathematicians working with its Muse Spark 1.1 and 1.2 models in 'Thinking' mode, and says five of them present answers to previously open research questions. Notably, the researchers used Meta AI's ordinary chat interface, not a custom research system built for mathematics.
The problems span a wide range of fields: probability, optimisation, differential equations (a wave-equation blow-up problem), group theory, a connection between number theory and string theory, and non-associative algebra. In one example, Muse Spark generated search code in the GAP algebra system that found a 384-element counterexample to a 2024 conjecture in group theory. In another, it helped a researcher find a three-dimensional counterexample to a published conjecture in evolution algebras.
Meta was careful about the division of labour. Mathematicians chose the problems and guided the work, the model generated search code, proposed proof approaches and drafted sections, and a separate group of mathematicians reviewed the results. Each paper marks which passages were drafted mainly by humans and which by the AI, and credits the prior work it builds on.
There are caveats. Independent teams solved some of the same problems separately, three of them in August using different approaches, which both confirms the results and tempers claims of unique discovery. The probability paper leaves part of its threshold question unresolved, and Meta has not published data on cost, time saved or how many attempts failed.
Even so, the release adds to growing evidence that general-purpose AI assistants can contribute to genuine research when paired with experts who know which questions to ask and how to check the answers. The practical lesson for researchers and R&D teams is about workflow: AI is most useful as a fast, tireless collaborator for search, conjecture-testing and drafting, with humans firmly in charge of direction and verification.
Why it mattersIf an off-the-shelf chatbot can help experts close open research problems, AI-assisted discovery moves from elite labs to any researcher with a subscription and good judgement.
👀 What to watch
Peer-review outcomes for the six papers, similar disclosures from other labs, and journals' evolving rules on crediting AI contributions.
3 opportunities from this story
AI-assisted research workflows for R&D teams
Most R&D teams use AI for emails and summaries, not for actual research. Offer workshops and setup services that teach scientists and engineers to use AI for search code, conjecture testing, literature mapping and drafting, with rigorous verification steps.
- Best for
- Research consultants, PhDs and technical trainers
- First step this week
- Write a case study reproducing one small AI-assisted result in your own field, documenting the prompts and checks.
Open the full playbook
Launch steps
- Document a repeatable research workflow with AI
- Build domain-specific prompt and verification templates
- Pilot with one university lab or corporate R&D team
- Turn it into a packaged programme
Tools
Risks
Over-trusting AI output. Make independent verification a non-negotiable part of the method.
Verification and proof-checking tools
As AI drafts more proofs and code, checking them becomes the bottleneck. Build tools that help researchers verify AI-generated results: running counterexample searches, translating arguments into formal proof assistants, or flagging weak steps for human review.
- Best for
- Developers with maths or formal-methods background
- First step this week
- Build a small tool that re-runs and validates AI-generated search code for counterexamples, and share it with a research community.
Open the full playbook
Launch steps
- Pick one verification task researchers find painful
- Prototype with an existing proof assistant or algebra system
- Get feedback from a few mathematicians
- Seek grant or institutional funding
Tools
Risks
A small market at first. Grants and partnerships with universities can fund early work.
AI-contribution disclosure and provenance standards
Meta marked which passages were drafted by humans and which by AI. Journals, universities and companies will need consistent ways to record and disclose AI contributions. Create templates, policy guides or software that tracks AI involvement in research and reports.
- Best for
- Research-integrity specialists, academic publishers and edtech builders
- First step this week
- Publish a free AI-contribution disclosure template for research papers and invite feedback from journal editors.
Open the full playbook
Launch steps
- Review existing journal and university AI policies
- Draft a clear disclosure format
- Pilot it with a research group
- Offer implementation help to institutions
Tools
Risks
Standards bodies may publish their own. Aim to influence them, or build tooling that supports whatever standard emerges.
Get these as a PDF every 3 days
Free. One email every 3 days. Unsubscribe any time.