Pentesters' willingness to rely on fully autonomous AI testing fell from 29% to 9% in a year. The reasons they give are specific, and a security product that wants their trust has to answer each one.
The figures come from Cobalt's June 2026 report, as covered by Dark Reading. In 2025, nearly three in ten security professionals thought fully autonomous AI could meet their testing needs. In 2026 the figure is 9%. The same report found that 78% of organizations have had automated systems miss significant vulnerabilities. Most buyers now want a hybrid, human-in-the-loop model, or automation limited to non-critical work.
The five objections
- Missed findings. Automated tools have blind spots, including high- and critical-severity issues. A tool that misses a critical path and reports a clean result is worse than no tool, because it produces false comfort.
- False positives. Every unverified finding becomes triage work for a human. If the tool cannot prove a finding, the cost moves to the reader.
- Too much output. Derek Rush of Bishop Fox, quoted in the coverage, says these systems produce more data than humans can handle without experienced experts. Analysts from FIRST, quoted in the same article, expect human verification to become the bottleneck.
- Judgment. Deciding which lead matters, and building a validated attack chain, is still human work.
- Unpredictable cost. AI service fees are hard to forecast, and budgets can run out mid-engagement.
The skeptics are mostly right about today's tools. They are not arguing against automation. They are arguing against automation that cannot show its work, cannot say what it missed, and removes the person who is accountable.
What the objections require of a product
Each complaint maps to a design rule.
- Prove it or don't report it. A finding should arrive with the evidence that it is exploitable, so a reviewer can reproduce it instead of re-deriving it.
- State the blind spots. The report should say what the tool could not see, so a clean result is never mistaken for a complete one.
- Rank, don't flood. Show what is connected to what, and which combinations of low-risk items add up to a high-risk path, so a person starts with the few things that matter.
- Keep a person accountable. Testing that touches live systems should be a separate, optional step with the customer's written sign-off, and a certified human should be available to review.
- Make it checkable. Publish evidence people can verify without buying anything.
How Redthread approaches this
Redthread is built around those rules. Discovery is agentless and read-only: the customer creates a role they can delete, and nothing is installed. The result is a graph of the AI agents, tools, models, code, cloud identities and data the platform can see, with a statement of what it could not see. Findings are ranked, and toxic combinations are flagged. Findings come with a working exploit and a fix, so a reviewer can reproduce what we claim. Testing is a separate step that requires written sign-off. Certified human review is available.
For teams that want the testing step itself, Redthread Pentest applies the same rules to AI applications. It probes for prompt injection, broken authorization, data leaks and chained exploits. Every reported finding includes a working proof of exploit and reproduction steps, and leads it could not prove are labeled as unproven instead of being passed off as findings. Results map to the OWASP LLM Top 10 and MITRE ATLAS. Scope is limited to what the customer specifies, active testing needs written authorization for each target, and a named expert can sign off on a report. The first scan is free.
Why we scan the MCP registry, and how
Why. AI agents now install tools from the official Model Context Protocol registry much as developers pull packages from npm. The registry checks one thing: that the publisher owns the namespace it publishes under. It does not check code security, whether tool descriptions are honest, whether the requested permissions are right, or whether a server still does what it did when someone approved it. Whoever approves an agent's tools is making a trust decision with almost no data. A public dataset gives the market that data, and it lets anyone check our claims instead of taking our word. It is also a standing test of our own rules: publishers and researchers can see exactly why a server scored as it did and tell us when a check reads wrong.
How. The scan is static. It reads only what each server declares in its registry manifest. Nothing is fetched and nothing is executed. Each server is checked against the rules that apply to it, and a rule that does not apply is dropped instead of counted as a failure. For example, a server that ships no package skips the pinning check. A server counts as passing when it passes every applicable check. The checks cover:
- whether the server authenticates with a long-lived static secret, which cannot be scoped per action or rotated by the installer
- whether it publishes a source repository at all
- whether its container image is pinned to an immutable digest, since a tag can be repointed at different bytes after approval, and whether packages are pinned to an exact version
- whether its repository is also claimed by unrelated servers, and whether the repository belongs to the same account as the namespace
- whether remote endpoints use TLS, and whether it ships secrets with default values
What gets published. Every scored server gets a permanent detail page showing each check and what failing it means. The result is also issued as a signed statement (in-toto format, Ed25519 signature) bound to the hash of the manifest that was scanned, so a claim cannot be moved to a different version. Publishers can add a badge, either pinned in their repository or live, and anyone can verify a statement offline with the atpp verify tool. A badge goes grey after seven days without a fresh scan, so it cannot keep claiming something that stopped being true. Publishers did not ask to be scored, which is why the rules are public and why we correct a rule or a score when someone shows it is wrong.
What it found. The 7 October run covered 40,653 servers. 51% passed every applicable check. 24% authenticated with a long-lived static secret, and 28% published no source repository. Only 11 of 1,058 container-packaged servers pinned an immutable digest, the worst result of any check. One generic library was named as the source repository by 2,356 unrelated servers, because nothing verifies that a publisher owns the repository it names. The checks that held up were TLS on remote endpoints at 100%, no default-value secrets at 99.8%, and 91% of packages pinned to an exact version. The live figures are on the scan page.
What it does not show. A clean score is a claim about a manifest at one point in time. It does not prove the code is safe, that tool descriptions are honest, or that runtime behavior matches what was declared. It is the layer almost nobody checks before installing, and it is where this kind of evidence starts.
What we do not claim
That is the standard the 9% figure sets. Trust in AI pentesting will come back through verification by the people whose names are on the reports.
To see how this works in practice, try Redthread Pentest or talk to us.