Cyber Agent

Why we built this

Security review is part of how we hand a project over. Before a site goes to a client we check the TLS setup, the headers, the cookies, the exposed paths, the dependency versions — the ordinary list.

Doing that properly means running about ten tools. Each has its own flags, its own output format, and its own idea of what counts as severe. So the work splits into two very different halves: a few minutes of running things, and a few hours of reading ten reports, working out which findings overlap, deciding which ones a client should actually care about, and writing it all up in language they can act on.

The second half is where all the time went. It is also the half that is almost entirely mechanical.

Nobody asked us for this. We built it because we kept doing the same afternoon over and over.

The constraints

These were our own decisions, not requirements handed to us — which changes what "constraint" means here. Nobody set a deadline, so the constraints are the ones we chose to accept.

Ten tools, kept as-is. We could have written our own scanners and controlled the output format. We decided not to: the existing tools are better than anything we would write, and maintaining our own would become the whole project. So the tool outputs are a fixed, messy input we do not get to change.

A report a non-technical reader can act on. The output had to be something a client reads and understands, not a findings dump. That is a much harder target than a machine-readable list, and it is the reason there is an AI layer at all.

Degrade, never fail. One scanner erroring should cost that scanner's findings, not the run. We chose this early because the alternative — an all-or-nothing scan — would have made the tool useless exactly when a site is unusual, which is when a scan matters most.

The genuinely hard part was never running the tools. It was deciding which of forty findings a business owner should read first.

How we built it

An orchestrator fans the scanners out concurrently, so total time is the slowest tool rather than the sum of all ten. Each result is normalised into one finding shape — severity, location, description, evidence — which is what makes the ten incompatible outputs comparable at all.

The AI layer sits after normalisation and does one job: rank. Raw scanner severity is not business impact — a tool will mark a missing header "high" and an exposed admin path "medium" because it is scoring the class of issue, not the situation. The model ranks the normalised set against what the site actually is, and the report is generated from the ranked order.

LangGraph holds the orchestration state, which is what makes the degrade-never-fail rule cheap to implement rather than a pile of error handling.

Inside the product

The dashboard shows the scan while it happens: which scanners have finished, which are still running, and the findings appearing as they normalise.

The counts at the top are the summary a reader wants first — how bad is this, roughly. Underneath, findings are listed in ranked order, not in the order the tools happened to return them. That reordering is the single most valuable thing the tool does and it is completely invisible unless you know the raw output.

The report at the bottom is generated, not templated. It is a first draft that removes the collation work — we still read every one before it goes anywhere. An AI-written security report that nobody checks is a liability, not a product.

What we built

• An orchestrator running ten security tools concurrently
• One normalised finding schema behind ten incompatible tool outputs
• An AI ranking layer that orders findings by business impact
• Generated PDF reports written for a non-technical reader
• Scheduled re-scans and on-demand runs
• A dashboard for reviewing findings across scans

The decisions that mattered

Orchestrate existing tools rather than write our own. Writing scanners is a tarpit — they need constant maintenance as the threat surface moves, and ours would always be worse than the maintained ones. Treating the tools as fixed, messy inputs was the decision that kept the project small enough to finish.

Rank with a model, not with a rules table. We tried a scoring table first. It worked until it met a site that did not match its assumptions, and then it confidently sorted the wrong thing to the top. Ranking needs context about what the site is, which is what a model is actually good at.

Normalise before analysing, not after. Tempting to hand the model ten raw outputs and let it work it out. That makes every run a different shape, makes the ranking unreproducible, and spends a lot of tokens on parsing. Normalising first means the model only does the judgement work.

Degrade over fail. A partial scan with nine tools' findings is useful. A failed scan is not. This decision cost more error handling than it looks like, and it is the one we would keep first.

We use this on our own assessments. If you want a security review of something you have built, tell us what it is — or read how we think about automating the repetitive parts of a business, which is the same instinct that produced this. The AI automation work we do for clients uses the same orchestration approach.