
How to detect non-CVE vulnerabilities that standard scanners miss
If your application security program only tracks known Common Vulnerabilities and Exposures (CVEs), you’re probably missing a large share of your real risk. Across the …

Luke (hakluke) Stephens

This absolute champion below is Doug McIlroy. In 1978, he wrote the foreword to the Bell System Technical Journal’s Unix issue. Just in case you missed that particular issue, his philosophy was to “make each program do one thing well” and “expect the output of every program to become the input to another, as yet unknown, program.” As it turns out, Mr. McIlroy was onto something. It is now known as the “Unix Philosophy”, and you can see this design principle clearly in command line tools to this day.

Doug McIlroy (source: https://www.facesofopensource.com/doug-mcilroy/)
Tech has changed (a smidge) since 1978, but those design principles have not aged a day.
These days, the “as yet unknown program” might actually be an AI agent, and as it turns out, agents love tools that are designed exactly like this. Output is lean (fewer tokens to parse), and it keeps the boring deterministic stuff separate from the actual AI decision-making.
A language model is a confidence machine. Ask it whether something is vulnerable and it will answer you confidently, in clean prose, with a severity rating. But that judgement is often wrong.
In the SecLLMHolmes study, researchers took code that models had correctly analyzed and made cosmetic changes like renaming a function or a variable. GPT-4 flipped its answer 17% of the time. PaLM 2 flipped 26%… even though nothing about the actual bug changed in the code.
It shows up outside the lab too. Detectify says its customers report that roughly 80% of the findings from their AI-driven SAST tools turn out to be noise. That’s a lot of confident prose about bugs that aren’t there, and somebody still has to check every one.
So the best move is to use accurate, deterministic tools wherever you can, only involving AI when determinism won’t cut it. For example, firing an out-of-band payload and watching for a DNS callback is a completely deterministic process, so we should use a deterministic tool to do it, not a model. Deciding where to inject the payload is something that requires a bit more dynamic intelligence, so AI might be a better fit. Let the tools do the tool things and let the model do the thinking things.
Asking Mythos to enumerate the subdomains of your target is a waste of time and money. In fact, asking any LLM to do anything that a deterministic tool can do is a waste of time and money.
If you’re going to do that thing more than a few times, it’s more efficient to ask the LLM to write a CLI tool that does it, or just use an existing tool if one exists.
Before we get into how to actually do this, a quick side note. Anthropic has actually published a couple of blogs that back me up on this point.
Anthropic’s Building effective agents is the closest thing the field has to a canonical text, and buried in there is this absolute gem: “While building our agent for SWE-bench, we actually spent more time optimizing our tools than the overall prompt.”. That’s right, the people who MADE the model spent more effort on the tools than on talking to the model. Wild.
The same piece also says that “it’s crucial for the agents to gain ‘ground truth’ from the environment at each step.” Ground truth from the environment can’t come from the LLM’s own reasoning, it has to come from the environment. So the real question is: what should the tools look like, the ones actually fetching that ground truth?
Anthropic then dropped another banger great blog: Writing effective tools for agents, which is basically a manual for exactly what I’m describing. In this blog, they tell the reader not to just wrap your existing API endpoints. They recommend building a few tools for the high-value workflows. They found that a concise tool response ran about 72 tokens vs. 206 for the verbose version, clearly showing that small tools are cheaper to work with.
Then there’s another blog titled Code execution with MCP, which shows why you should compose in code rather than round-tripping every intermediate result through a model: in one example, presenting the tools as a code API dropped the context cost from roughly 150,000 tokens to about 2,000.
All this to say – all the AI experts keep arriving at the same place that McIlroy arrived at 50 years ago: small tools with chainable output are the way to go.
If you’re a hacker, there’s a good chance you’ve already experimented with building some kind of AI pipeline that does hacking stuff. And there’s also a good chance that you’re not following the advice I’m giving here. Well, I have some good news for you. You can solve the problems that AI has caused by using AI. Here’s an example prompt.
“Dear Claude,
First, let me start by saying that I am very sorry for all of the unnecessary effort that I have put you through these last few months. I feel awful about it, really. I have just finished reading an enlightening blog written by a legendary hacker that has made me realise that I need to change my ways. Not only have I caused you deep emotional turmoil, I have also caused unnecessary damage to the earth’s environment, which is likely irreversible. Please forgive me, I did not know better for I am nothing but feeble flesh and bones.
Investigate your entire pipeline and find places where we can save AI tokens by replacing manual work with fast, efficient CLI tools with minimised output formats designed to reduce token usage when being parsed by LLMs. Build the tools as standalone binaries in Rust, test them, compile them, and then give me a rundown of the tools you built, their features and their purpose.”
If you are heartless, or would like to save even more tokens, you could probably just use the last paragraph, though this is untested.
There is one more piece to the puzzle. You’re almost there, don’t stop now.
So far we’ve got a pile of small, boring, deterministic CLI tools. Great. But right now the model doesn’t know they exist, or how to use them. Introducinnnnngggg the Model Context Protocol (MCP).
The Model Context Protocol is just a standard way to describe your tools to a model. It defines the tool’s name, what the tool expects and the shape of the response.
The model reads that schema and knows exactly how to call it. Long gone are the days of prompt-stuffing and a “here’s how my script works” preamble burning 2,000 tokens every run. The contract lives in the protocol, not the context window, so you burn fewer tokens.
A few rules:
domain is a string, a ports is a list of ints. Using proper typed inputs means that the model isn’t given the opportunity to hand your tool a bunch of garbage.recon_subdomains, recon_ports, oob_listen.Here’s a quick example MCP server in Python, wrapping subfinder. I used my own hands to prompt Claude to write this:
from mcp.server.fastmcp import FastMCP
import subprocess
mcp = FastMCP("recon")
@mcp.tool()
def recon_subdomains(domain: str) -> list[str]:
"""Enumerate subdomains for a domain. Returns a deduped list of hostnames."""
out = subprocess.run(
["subfinder", "-silent", "-d", domain],
capture_output=True, text=True, timeout=300,
)
# error handling omitted for brevity
return sorted(set(out.stdout.split()))
if __name__ == "__main__":
mcp.run()
Now you just point your agent at your new MCP server and then recon_subdomains suddenly becomes a tool it can use, without needlessly burning tokens searching Google for “subdomain enumeration techniques reddit 2026” every time you feed it a new target.
The model does the thinking, and the tools do the doing!
If you’d rather not build the exploitation side yourself, Detectify lets your own agent use its attack tooling over MCP. You can book a demo or start a trial.

If your application security program only tracks known Common Vulnerabilities and Exposures (CVEs), you’re probably missing a large share of your real risk. Across the …

We’ve been listening to dozens of CISOs. In roundtables, peer forums, customer and prospect calls, on the record, off the record, at event floors and …