Information Warfare Strategies (SRF-IWS): Offensive Operations Against a Papal Visit — Pope Leo XIV in Madrid 2026

Disclaimer: Everything described here is pure imagination and any resemblance to reality is coincidental. This document is intended for security professionals to develop defensive countermeasures. The author is not responsible for the consequences of any action taken based on the information provided in this article. I keep every scenario at the threat-vector level: no operational detail, no tactics, no weapons information, and each one is paired with a defensive recommendation.

Note: As with the rest of the SRF-IWS series, I leaned on several AI models to help build realistic, defense-oriented attack scenarios. The goal is Blue Team planning, nothing else.

A note on the series. This article belongs to SRF-IWS, but it is not a continuation of the Davos articles. Those (Davos 2024, 2025 and 2026) are their own line of analysis on the World Economic Forum; this one stands on its own and simply shares the same framework. I do reference them throughout for context, so they are worth reading as background. The difference this time is the protectee: instead of a corporate forum, we are looking at a head of faith and head of the Vatican state, out in the open, in the middle of a European capital, surrounded by more than a million people.


Introduction

From 6 to 9 June 2026, Pope Leo XIV, the first North American pontiff, will be in Madrid as the opening leg of his apostolic journey to Spain (Madrid, Barcelona and the Canary Islands, 6 to 12 June). It is the first papal visit to the Spanish capital in fifteen years, since Benedict XVI and World Youth Day back in 2011. The Madrid program is dense, and from a protective-intelligence point of view it is wide open:

  • Arrival on 6 June, with a courtesy visit to King Felipe VI, Queen Letizia and the Royal Family.
  • A youth prayer vigil at Plaza de Lima, on the Paseo de la Castellana, that same evening.
  • On Sunday 7 June, the solemnity of Corpus Christi, an open-air Mass at Plaza de Cibeles followed by a Eucharistic procession through the centre of Madrid.
  • On Monday 8 June, an address to Parliament at the Congress of Deputies, and later an encounter with the diocesan community at the Santiago Bernabéu.
  • Popemobile and motorcade movements concentrated on the Castellana–Cibeles–Lima axis and the fixed nodes: Barajas, the Royal Palace, Congress and the Bernabéu.

The address to Parliament deserves its own line, because it is genuinely historic. For the first time ever, a Pope will speak before a joint session of the Cortes Generales, deputies and senators together. John Paul II came to Spain five times and Benedict XVI three, and none of them ever addressed the chamber. That is the kind of high-symbolism, high-protocol moment an adversary loves.

Spanish and municipal authorities have put together a security and mobility operation without precedent in the city, with attendance across the main events projected at up to 1.8 million people. The chosen motto, “Alzad la mirada” / “Lift up your eyes” (John 4:35), and Leo XIV’s emphasis on migration, the journey ends in the Canary Islands, Spain’s main Atlantic entry point for migrants, turn this into more than a physical-security problem. It is a near-perfect information-warfare target: globally televised, built around a polarising subject, with a protectee whose every sentence carries geopolitical weight.

A Pope is not a Davos delegate, and the threat aperture is much wider. You have religiously motivated extremists, both jihadist and anti-Catholic; traditionalist and sedevacantist fringe actors; anti-clerical and anarchist currents; anti-migration extremists reacting to the Pope’s message; grievance-driven lone actors; and nation-state information operations looking to weaponise the spectacle. None of this is hypothetical. Pontiffs have always been targets. John Paul II was shot in St. Peter’s Square in 1981. He was attacked again in 1982 at Fátima, with a bayonet, by a Spanish priest, Juan María Fernández y Krohn. The 1995 Bojinka plot in Manila included a plan to assassinate him. These are documented facts, and they are reason enough to plan seriously.

What follows are realistic, defense-oriented scenarios across the information, cyber, RF, drone, crowd and physical domains. Each one pairs the attack with its own defense, in the same section.


1. Disinformation and the migration narrative

The most likely and most damaging vector here is not a bomb or a rifle. It is information. Leo XIV’s visit is framed around migration and lands in the middle of an active Spanish immigration debate, which is exactly the kind of ground influence operations like to work on, whether they come from a state actor trying to inflame Spanish and EU fault lines or from domestic extremists on either end.

The campaign I would expect looks something like this. Fabricated papal “quotes”, AI-generated text, images and short clips that put inflammatory positions in the Pope’s mouth on immigration, the Spanish government, Catalonia or the monarchy, dropped a few hours before a key event to own the news cycle. Doctored homily fragments, audio or video from the Cibeles Mass or the Parliament speech, selectively cut or fully faked to manufacture outrage in either direction and pull people toward the venues to confront each other. Forged “leaks”, fake Vatican or Moncloa documents alleging secret political deals tied to the visit, designed to make both Church and state look like they are hiding something. Astroturfed outrage from inauthentic networks pushing divisive hashtags, fake eyewitness accounts and false reports of incidents to either scare people away or provoke a confrontation. And the simplest one, spoofed accounts and look-alike domains copying the official registration and information sites to hand out fake schedules, fake “cancellations” or malicious links.

01-disinformation

Figure 1 — Disinformation and migration-narrative attack tree, generated with USecVisLib.

Defense

This has to be treated as a primary security function, not a press afterthought. That means a joint Vatican–Spanish communications cell with the authority to rebut fast, official audio and video signed at the source (C2PA-style provenance), an active pipeline to monitor and take down look-alike domains, and one verified channel the public knows to trust. If there is a single authoritative source, most of the forgeries lose their oxygen.


2. Deepfakes and synthetic media

I covered this at length in the Davos 2026 analysis, and nothing about it has gotten easier to defend against. Real-time deepfakes are mature, voice cloning needs only a few seconds of audio, and people only spot a good video deepfake a fraction of the time. A globally broadcast Pope, with an enormous public archive of audio and video, is about as good a training subject as exists. So is the King, and so are the senior organisers.

The scenarios that worry me are the ones that spoof authority. A faked “official” evacuation announcement, or a “device found” warning, pushed onto a compromised PA system, hijacked digital signage or a spoofed alert channel at Cibeles, Lima or the Bernabéu, with the aim of triggering a panic (see section 3). Voice-cloned traffic impersonating an incident commander or a Vatican advance team to redirect units, shift motorcade timing or open a gap. Synthetic “private” recordings of the Pope and the King, or the Pope and government officials, inventing commitments or insults that were never said, released to poison the diplomacy of the visit. Or fabricated “behind the scenes” footage timed to step on the Parliament address.

02-deepfake

Figure 2 — Deepfake and synthetic-media attack tree (USecVisLib).

Defense

The defensive answer is old-fashioned and it works: out-of-band verification and challenge/response for all command, advance-team and protocol communications. No unit acts on a voice or a face alone. On top of that, run deepfake detection on the monitored broadcast feeds, lock down PA, signage and alerting as critical infrastructure with real authentication, and pre-script the crowd messaging so that anything the public hears comes only through verified, redundant channels.


3. The crowd as the weapon

With up to 1.8 million people spread across Cibeles, Lima, the procession route and the Bernabéu, the highest-probability mass-casualty outcome needs no weapon at all. You only have to engineer panic in a dense crowd. This is the most underappreciated vector on the list, and it is not theoretical, the history is long and grim: Hillsborough, the Love Parade in 2010, the 2015 Mina crush during the Hajj, Itaewon in 2022, Astroworld in 2021.

How would you do it. Start a synchronised false alarm, a rumour of gunfire, a “bomb”, a fire, spread by SMS and social media, a single staged loud bang, or hijacked signage, and place it at a bottleneck where density is already critical: the narrow approaches to Cibeles or Lima, a stadium concourse. Pair it with comms denial, jam or saturate cellular and Wi-Fi so the crowd cannot orient itself and official messaging cannot get through, and let rumour fill the gap (this ties into section 7). Add flow manipulation, block or falsely sign the exits, and a controllable density turns into a progressive collapse. And if you want to overwhelm the response, initiate at several separated points at once so stewarding and emergency services fragment.

03-crowd

Figure 3 — Engineered-panic and crowd-crush attack tree (USecVisLib).

Defense

Defending it comes down to seeing density in real time and being able to act on it. Overhead optical and thermal monitoring plus anonymised mobile-density analytics, with hard thresholds and pre-planned metering and reversible flow control. A public-address system that resists jamming. Stewards rehearsed to kill rumours on the spot. Egress that is engineered, clearly marked and over-provisioned. And one unified incident-command picture, so a small local event never gets the chance to cascade.


4. Drones and counter-UAS

Open venues like Cibeles, Lima, the procession route and the open bowl of the Bernabéu are exactly the places small drones exploit. The cost problem I described in the Davos 2026 analysis still holds: the drones are cheap, the defenses are expensive, and a swarm can simply saturate point defenses.

The uses are familiar. Surveillance and targeting, small quadcopters mapping security positions, motorcade timing and VIP locations in real time. Panic-payload delivery, a drone dispersing smoke, an irritant or pyrotechnics over a dense crowd, where the point is panic and a crush rather than direct casualties. Swarm saturation and decoys, expendable drones soaking up the counter-UAS effort while a primary platform finishes its job, or FPV drones using the urban canyons for a low, fast approach. And RF payloads, airborne jammers or IMSI-catchers degrading comms and collecting intelligence over the crowd.

04-drone-uas

Figure 4 — Drone and counter-UAS attack tree (USecVisLib).

Defense

The defense has to be layered and multi-modal, radar plus RF plus acoustic plus electro-optical/infrared, so no single trick blinds it. Enforce the no-fly and temporary flight restriction zones with the legal authority to actually do something about a violation. Pre-position effectors on the likely approach lines. And, this matters more here than at Davos, choose mitigation that does not itself hurt or panic a 1.8 million-person crowd. Detection, RF takeover and geofencing, and controlled interception come well before anything kinetic over people’s heads.


5. The motorcade and the Popemobile

Movements concentrate on a predictable axis, Castellana–Cibeles–Lima, and on fixed arrival and departure nodes: Barajas, the Royal Palace, Congress, the Bernabéu. Predictability plus a slow, open, rope-line Popemobile is the classic protective dilemma, and there is no clever way around it.

The exploitation paths are well understood. Choke-point operations, surveillance picks a fixed slow point for a hostile act, a staged disturbance or comms denial. GPS spoofing or jamming of the escort vehicles to fragment the motorcade or misdirect support and medical units; Iran’s capture of a U.S. RQ-170 drone is the textbook precedent for spoofing GNSS on even an advanced platform. Vehicle-as-weapon, the most-rehearsed European threat since Nice and Berlin in 2016, a hostile vehicle driven into a pedestrian-dense stretch of the route. And plain old hostile reconnaissance of static posts and timings beforehand.

05-motorcade

Figure 5 — Motorcade and Popemobile attack tree (USecVisLib).

Defense

Defending the move means randomising route and timing wherever the program allows it, putting hostile-vehicle mitigation, barriers, sterile zones, controlled crossings, along the entire crowd-facing axis, and giving the escort vehicles anti-spoof, multi-constellation GNSS with inertial backup. Add aggressive counter-surveillance, dominate the rooftops and elevated positions with friendly observation and counter-sniper coverage, and configure the Popemobile to balance pastoral visibility against protection. It will always be a compromise; it should at least be a deliberate one.


6. Cyber attacks on the event and the city

The visit runs on a lot of software. A mass public registration system holding the personal data of potentially millions, accreditation and badging, ticketing, CCTV and access control, Madrid’s traffic and mobility management, emergency dispatch. As the GTG-1002 case from the Davos 2026 analysis showed, AI agents can map and exploit an ecosystem like this at machine speed, finding paths a human would miss.

The obvious moves: breach the registration system and weaponise the data, exfiltrate attendee records for targeting, doxxing or spear-phishing, or corrupt the access lists to create chaos at the gates. Forge credentials by compromising the accreditation pipeline, and manufacture insider access in a press, volunteer or contractor role. Blind the surveillance, manipulate CCTV and access control to open timed blind spots. Hit the city systems, traffic management and signage during motorcade windows, or emergency dispatch during an incident, which is how a cyber event becomes a physical-safety event. And the simplest, DDoS or deface the official information channels at the moment public attention peaks, which loops straight back to section 1.

06-cyber

Figure 6 — Cyber attacks on event and city systems, attack tree (USecVisLib).

Defense

The defense is unglamorous and necessary: red-team every event and city system in scope before the visit, segment the life-safety and access-control systems so they are not reachable from everything else, run Zero Standing Privilege and Just-in-Time access so a stolen credential buys very little, put integrity monitoring on the accreditation and access lists, and make sure every life-safety function has a tested manual fallback for the day the software lies to you.


7. RF and the spectrum

This is my home ground and it is a high-impact one. In Spain, the state security forces, Policía Nacional and Guardia Civil, run on SIRDEE, the encrypted, nationwide TETRAPOL trunked network. (A point worth getting right: SIRDEE is TETRAPOL, not TETRA. TETRA is a different standard used by various regional and municipal services. People conflate the two constantly.) Whatever the technology, the whole event depends on resilient spectrum.

The attacks. Jam SIRDEE, the event-coordination radios and the cellular bands at a critical moment, which degrades command, amplifies crowd confusion (section 3) and isolates posts. Spoof GPS/GNSS to corrupt timing, geofencing, counter-UAS tracking and motorcade navigation (section 5). Deploy IMSI-catchers or rogue cells to track and intercept VIPs and the crowd. Stand up rogue access points near venues and command areas to capture traffic and pivot, including the “harvest now, decrypt later” collection I described in the Davos 2026 analysis.

07-rf

Figure 7 — RF and wireless-warfare attack tree (USecVisLib).

Defense

Defending the spectrum means watching it. Continuous monitoring and direction-finding across the operational area to catch jammers, spoofers and IMSI-catchers as they appear. Encrypted, frequency-hopping, jam-resistant primary comms, with a non-RF fallback, runners and hardwired nodes, for when the band goes dark. GNSS integrity monitoring with backup positioning. And basic RF hygiene, nothing sensitive over a channel that can be compromised.


8. Insiders and the supply chain

A visit like this mobilises a huge, hastily assembled workforce. The official choir alone, the Gran Coro de Voces Católicas, has more than 1,700 volunteers, and that is before you count stewards, contractors, catering, AV, transport and security vendors across every venue. The weakest-link problem scales with that footprint.

What I would watch for: a volunteer or contractor infiltrated where mass onboarding outruns vetting. A pre-compromise of the AV and technical kit at the Congress chamber, the Royal Palace or the Bernabéu, an implanted listening or recording device, or a manipulated production system feeding the disinformation and deepfake plays from sections 1 and 2. Logistics access, catering, cleaning and equipment vendors as a way into sterile areas. And the transport providers, where driver credentials and vehicle-tracking data quietly reveal protected movements.

08-insider-supplychain

Figure 8 — Insider-threat and supply-chain attack tree (USecVisLib).

Defense

The countermeasures are proportionality and discipline. Vet to the level of access, with the deepest screening for the technical, AV, transport and sterile-area roles. Least-privilege physical access with audited escorting. TSCM sweeps of every speaking venue before use, and keep the zone sterile afterward. And put real security requirements on vendors, with continuous monitoring and a backup for anything essential.


9. Physical and CBRN, at the protective-doctrine level

I will keep this at the level a protective detail actually plans against, and ground it again in the record: 1981 in St. Peter’s Square, 1982 at Fátima, the 1995 Bojinka plot.

The vectors to plan for are the close approach by a lone actor at a rope line, the procession or the Popemobile route, an edged or thrown-object threat from inside a permitted crowd; an elevated firing position along the Castellana axis or around the open plazas, which is what sightline management and counter-sniper overwatch exist for; a low-grade chemical or irritant dispersal in the crowd whose real effect is panic and a crush (sections 3 and 4) rather than mass toxicity; and an improvised or vehicle-borne explosive at a venue perimeter or along the route.

09-physical-cbrn

Figure 9 — Physical and CBRN-in-crowd attack tree (USecVisLib).

Defense

Against all of that: screened sterile zones with search and magnetometers at controlled entry, counter-sniper and elevated-position domination with the structures surveyed in advance, hostile-vehicle mitigation on every crowd-facing route, CBRN detection and decontamination staged for a mass-casualty contingency, a saturating uniformed and plainclothes presence at the rope lines, and pre-positioned, redundant medical capacity matched to the density map.


10. The convergence scenario

If I have one thesis across this whole series, it is that the defining threat is not any single vector. It is the deliberate sequencing of several of them, fast. Applied to this visit, it reads like this. In the days before, a disinformation campaign (section 1) polarises the public and seeds counter-mobilisation near the venues. At the chosen moment, coordinated cyber (section 6) and RF (section 7) actions degrade CCTV, comms and situational awareness. A drone payload or a staged report (sections 3 and 4) starts a panic at a critical bottleneck. A deepfaked “official” evacuation order (section 2), pushed through compromised signage or PA, turns that panic into a crush. And in the chaos, a primary objective is pursued while a pre-staged false narrative (section 1) claims and frames the event for the world before the authorities can get a word out.

10-convergence-graph

Figure 10 — Convergence scenario as an attack graph: prime, blind, trigger, amplify, exploit, with CVSS-scored vulnerabilities along the chain (USecVisLib).

Defense

No single countermeasure stops that. The only thing that does is an integrated, fast, multi-domain defense built on one shared picture of what is happening: a single fused common operating picture across Casa Real security, Policía Nacional, Guardia Civil, Madrid municipal police, the Vatican Gendarmerie and advance team, and the intelligence services, correlated fast enough to matter. Every per-section defense above feeds into that one picture, because the convergence attack is precisely the one a fragmented, human-speed defense cannot answer.


Conclusion

A papal visit compresses every threat domain into a single televised, open-air, ideologically charged event. The lessons of the SRF-IWS series all apply, but the protectee changes the maths.

The first point is that information is the main battlefield. For a Pope speaking about migration before Parliament and a 1.8 million crowd, the disinformation and deepfake vectors are more likely, and probably more consequential, than any kinetic act. Strategic communications is a security function, full stop.

The second is that the crowd is both the audience and the weapon. You can produce mass casualties in a dense crowd without firing a shot, just by engineering panic. Crowd dynamics deserve the same planning effort as counter-sniper coverage.

The third is convergence. Disinformation that primes, cyber and RF that blind, drones that trigger, deepfakes that amplify, run in sequence and fast. The defense has to be just as integrated and just as fast.

The fourth is that the history is the warning. Attacks on pontiffs are documented fact, not imagination, and planning has to respect that record.

And the last is that speed and unity decide the outcome. A fragmented, human-speed defense cannot answer a coordinated, multi-domain operation. A single shared command picture is the price of entry.

The point of writing all of this down is simple: the defenders, not the adversaries, should be the ones who have thought it through first.

SRF

Follow: @simonroses

This article continues the SRF-IWS research into information warfare strategies applied to high-profile protective environments.

Posted in AI, Security, Technology | Tagged , , , , , | Leave a comment

Scanning Vibe-Coded Apps: Why Traditional SAST/DAST Falls Short (part 6)

Vibe Coding Security Series

  1. What Is Vibe Coding Security? A Field Guide for 2026
  2. The OWASP Top 10 for Vibe-Coded Applications
  3. Anatomy of a Vibe Coding Breach: Lessons from 2026’s Worst Incidents
  4. The Dependency Trap: Supply Chain Risks in AI-Generated Code
  5. Authentication & Secrets: What AI Gets Wrong Every Time
  6. Scanning Vibe-Coded Apps: Why Traditional SAST/DAST Falls Short (you are here)
  7. Prompt Engineering for Secure Code (coming soon)
  8. The Founder’s Security Checklist (coming soon)
  9. Securing the AI Coding Pipeline (coming soon)
  10. The Future of Vibe Coding Security (coming soon)

Read Time: 20 minutes

TL;DR

Traditional security scanners pattern-match on code that exists. The most dangerous vulnerabilities in vibe-coded apps live in code that doesn’t exist — missing auth checks, missing rate limiting, missing authorization logic. A January 2026 SAST benchmark found tools flagging 68–75% of safe code as vulnerable while architectural flaws passed silently, and Georgia Tech has tracked 74 AI-attributed CVEs with monthly discoveries growing 6x in two months. New AI-native tools are closing the gap, but as of mid-2026, broken authorization and absent security controls still require human review. This post covers what works, what doesn’t, and how to build a scanning pipeline for AI-generated code.


The Scanning Paradox

We have more security scanning tools than at any point in the history of software development. SAST, DAST, SCA, IAST, RASP — the acronym count alone suggests the problem should be solved. And for human-written code, these tools have been steadily improving for two decades. The issue is that vibe-coded applications don’t fail the way human-written ones do.

When a human developer introduces a SQL injection, it’s usually because they forgot to parameterize a query. A SAST tool pattern-matches on string concatenation inside a SQL call and flags it. Straightforward. When an AI coding tool introduces a security flaw, the code is typically syntactically clean, follows documented API patterns, and passes every functional test. The vulnerability isn’t in how the code is written — it’s in what the code doesn’t do. Missing server-side validation. Missing rate limiting. Missing authorization checks. Missing RLS policies. You can’t pattern-match on absent code.

Georgia Tech’s Vibe Security Radar, launched in May 2025, tracks CVEs attributable to AI coding tools by tracing fixing commits backward through Git history. Their numbers tell the story: 6 AI-attributed CVEs in January 2026, 15 in February, 35 in March. A nearly 6x increase in two months. The total confirmed count stands at 74, with researchers estimating the true number is 5–10x higher because most AI-generated code doesn’t leave clear attribution markers.

Meanwhile, the Cloud Security Alliance’s emergency strategy briefing — assembled over a single weekend by 60+ contributors including Jen Easterly and Bruce Schneier — warned that the window to fix vulnerabilities is collapsing: mean time from disclosure to confirmed exploitation has fallen to less than one day in 2026, down from 2.3 years in 2019. Separate CSA research has found that 62% of AI-generated code samples contained vulnerabilities.

The scanners are running, the vulnerabilities are still shipping, and the gap is widening.


What SAST Actually Catches (And What It Doesn’t)

Static Application Security Testing works by analyzing source code without executing it. Tools like CodeQL, Semgrep, SonarQube, and Checkmarx parse the code into an abstract syntax tree, then match patterns against known vulnerability signatures — string concatenation in SQL queries, eval() on untrusted input, deprecated cryptographic functions. These are well-defined patterns, and SAST handles them reliably.

The problem is false positives and structural blind spots.

The False Positive Problem

A January 2026 study benchmarked CodeQL, Semgrep, SonarQube, and Joern against OWASP Benchmark v1.2 — 2,740 Java test cases with known vulnerability status. CodeQL achieved the highest F1-score at 74.4%, but it flagged 68.2% of non-vulnerable test cases as positive — 904 false positives across the benchmark. SonarQube produced 1,254 false positives, covering 45.8% of all test cases. Semgrep flagged 74.8% of non-vulnerable cases. Joern had the fewest false positives at 96 but achieved only 8.2% recall — it catches almost nothing.

For a vibe coder running Semgrep on their AI-generated codebase for the first time, this means roughly three-quarters of the alerts they see are noise. After the third false positive about a “potential injection” in code that’s actually safe, most people stop reading the output entirely. The signal drowns in the noise, and the real issues — the ones that matter — scroll past unread.

Here’s one I run into constantly. Over the past few years I’ve done plenty of code reviews for AWS-based applications at VULNEX, and Semgrep flags AWS account IDs as sensitive information leaks in nearly every project. The problem is that AWS themselves don’t consider account IDs to be sensitive — their documentation explicitly states they can be shared when needed. That’s a false positive that shows up in every single AWS project, training teams to ignore Semgrep output for that codebase entirely. I always work with the customer to understand their specific privacy requirements before dismissing or escalating any finding — some organizations do treat account IDs as internal-only regardless of what AWS says — but this is exactly the kind of noise that erodes trust in automated tools.

The Structural Blind Spot

False positives are annoying but manageable. The structural blind spot is the real problem. SAST works by matching patterns in code that exists. Vibe-coded vulnerabilities are often in code that doesn’t exist.

Consider the QuickNote app from Part 5. The most dangerous issues weren’t bugs in the code — they were missing features. No rate limiting on the login endpoint. No RLS policies on the database. No server-side authorization check. No token expiration. SAST cannot flag the absence of a security control, because there’s no code to analyze. It’s like asking a spell-checker to tell you that your essay is missing a conclusion.

Here’s what happens when you run Semgrep against a typical vibe-coded Express.js app:

semgrep --config=auto ./src

Semgrep will likely flag things like innerHTML usage (real issue — XSS), eval() calls if present, and maybe the MD5 hash function. What it won’t flag: the /api/users/:id/notes endpoint lacking an ownership check, jwt.sign() called without an expiresIn parameter, the entire application having no rate limiting middleware, Supabase RLS disabled on every table.

These are the vulnerability classes that matter most in vibe-coded applications, and SAST is structurally incapable of detecting them.

What SAST Is Good For

This isn’t an argument to stop using SAST. Pattern-matching catches real issues: hardcoded credentials (when they match known patterns), dangerous function calls, known-vulnerable library usage, obvious injection vectors. For the subset of vulnerabilities that look like traditional bugs, SAST works. The problem is that in vibe-coded apps, that subset covers maybe 30% of the actual risk surface. The other 70% is architectural.


What DAST Misses in the SPA Era

Dynamic Application Security Testing takes the opposite approach — instead of reading source code, it runs the application and attacks it from outside. OWASP ZAP and Burp Suite send malicious payloads to endpoints, monitor responses, and flag behavior that indicates vulnerabilities. If you can trigger a SQL injection through an HTTP request, DAST finds it. If a reflected XSS payload shows up in the response, DAST catches it.

For traditional server-rendered web applications, DAST has been reasonably effective. But vibe-coded applications are overwhelmingly single-page apps (SPAs) built with React, Next.js, or Vue, and DAST’s architecture has a hard time with them.

The Crawling Problem

DAST discovers application functionality by crawling — following links, submitting forms, parsing HTML. SPAs don’t work that way. Routes are handled client-side by JavaScript. Forms are React components that communicate via fetch() calls. API endpoints aren’t discoverable by parsing HTML, because the HTML is a nearly empty shell that loads a JavaScript bundle. A DAST crawler hitting a typical vibe-coded React app sees <div id="root"></div> and maybe a few <script> tags. It misses everything.

Modern DAST tools have gotten better at JavaScript rendering — ZAP has an AJAX Spider, Burp has a built-in browser. But they still struggle with authentication flows (especially OAuth), multi-step workflows, and application state. A login form that uses useState for input tracking and useEffect for token storage doesn’t behave like a traditional HTML form, and DAST crawlers frequently can’t complete the auth flow to reach the protected surface area behind it.

The Business Logic Gap

Even when DAST can reach the endpoints, it hits the same wall SAST does: the vulnerability is in what the code doesn’t do. DAST sends a SQL injection payload to /api/notes and checks whether the response looks like database output. That’s a legitimate test. But it doesn’t test whether /api/notes/42 returns data belonging to a different user. It doesn’t test whether the /api/admin/users endpoint is accessible with a non-admin token. It doesn’t test whether the login endpoint allows 10,000 attempts per minute.

These are business logic vulnerabilities — they require understanding the application’s intended behavior, not just its input/output surface. DAST treats the application as a black box. For vibe-coded apps where the most dangerous vulnerabilities are in the authorization model, that black-box approach misses the things that matter.

Where DAST Still Helps

DAST catches configuration issues that SAST can’t: missing security headers, permissive CORS policies, exposed server information, SSL/TLS misconfigurations. These are deployment-level problems, not code-level problems, and vibe-coded apps tend to ship with terrible default configurations because the AI optimizes for “it works locally.” Running ZAP or Nuclei against your deployed application catches the infrastructure-layer gaps.

Nuclei deserves a specific mention. Its community-maintained template library now exceeds 11,000 templates, and ProjectDiscovery has introduced AI-powered template generation — describe a check in natural language, get a YAML template. A recent pull request added AI Security DAST templates specifically targeting AI-system patterns. It’s not solving the fundamental architectural problem, but it’s the closest DAST has gotten to being vibe-code-aware.


The SCA Gap: When Dependencies Don’t Exist

Software Composition Analysis (SCA) tools — Snyk, npm audit, Dependabot, Socket.dev — check your project’s dependencies against vulnerability databases. If you’re using lodash@4.17.20 and there’s a CVE for that version, SCA flags it. This has been one of the most effective automated security practices for the past decade.

AI-generated code breaks SCA because the dependencies are made up.

Slopsquatting

The term, coined by security researcher Seth Larson, describes what happens when AI coding tools recommend packages that don’t exist in any registry. A March 2025 study analyzing 576,000 AI-generated code samples found that roughly 20% recommended packages that aren’t real. Worse, 43% of those hallucinated package names are consistent across different AI runs — meaning an attacker can predict which fake names the AI will suggest, register them, and fill them with malicious code.

That’s exactly what happened. In January 2026, a hallucinated npm package called react-codeshift spread through 237 repositories via AI-generated code. Nobody deliberately planted the package name in the AI’s training data. The AI hallucinated it, multiple developers installed it when their AI suggested it, and eventually someone registered it with malicious code. The supply chain attack was automated by the AI itself.

SCA tools can’t flag a package that doesn’t have a CVE because it’s brand new and doesn’t appear in any vulnerability database yet. npm audit would report zero issues for react-codeshift — the package existed, it had no known CVEs, and its package.json looked normal. The malicious behavior was in the code, not in the metadata.

What Different SCA Tools Catch

The SCA landscape has split into two camps. Traditional CVE-based tools (npm audit, Dependabot, basic Snyk scanning) check packages against known vulnerability databases. If the vulnerability has a CVE, they catch it. If it doesn’t, they don’t. For established packages with active security research, this works. For hallucinated packages, newly registered packages, and packages with obfuscated malicious behavior, it’s blind.

Socket.dev represents the newer approach — it analyzes package behavior rather than just checking CVE databases. It detects install scripts that exfiltrate environment variables, network calls to unexpected domains, obfuscated code that decodes at runtime, and sudden changes in maintainer behavior. This behavioral analysis catches supply chain attacks that CVE databases haven’t catalogued yet.

Snyk’s DeepCode AI combines symbolic analysis with AI to scan code snippets as they’re generated, catching vulnerable patterns inside the IDE before they reach the repository. This is closer to where SCA needs to go for vibe-coded apps — flagging issues at generation time rather than after the package is installed and the code is committed.

For the dependency problems I covered in Part 4, no single SCA tool covers the full risk surface. The practical answer is layering: npm audit for known CVEs, Socket.dev for behavioral anomalies, and manual verification that the packages your AI suggested actually exist and are what they claim to be.


What’s Actually Working: The New Wave

The gap between what traditional tools catch and what vibe-coded apps need has spawned a new generation of security tools. Some are AI-native — they use LLMs to reason about code instead of pattern-matching. Others take hybrid approaches, combining traditional analysis with AI-powered reasoning. A few are specifically designed for vibe-coded applications.

LLM-Augmented SAST

The most promising near-term improvement is using LLMs to post-process traditional SAST output. The same January 2026 study that exposed SAST’s false positive rates also tested layering LLM agents on top of the output. The best configuration reduced the initial false positive rate from 98.3% to 6.3%. The LLM reads the flagged code in context, understands what it’s doing, and determines whether the flag is legitimate or noise.

This doesn’t solve the blind spot problem — the LLM is still working from SAST’s initial findings, so absent code remains invisible. But it makes SAST output actually usable. Instead of 750 alerts where 700 are false positives, you get 50 alerts where 47 are real. That’s the difference between a report nobody reads and a report that drives fixes.

Neuro-Symbolic Analysis (IRIS)

IRIS, published at ICLR 2025, takes a different approach. Instead of post-filtering SAST output, it combines LLM reasoning with CodeQL’s static analysis in a neuro-symbolic framework. The LLM identifies potential vulnerability patterns through code comprehension, then CodeQL validates them with formal analysis. Using GPT-4, IRIS detected 55 vulnerabilities across 30 Java projects — 103.7% more than CodeQL alone. It found 4 previously unknown vulnerabilities. Even a smaller model (DeepSeekCoder 7B) detected 52 vulnerabilities, showing this approach doesn’t require cutting-edge models.

The false discovery rate is still high at 84.82%, but it’s 5.21% lower than CodeQL by itself. More importantly, IRIS catches vulnerability categories that pure pattern-matching misses — it can reason about whether an authorization check is semantically correct, not just whether one exists.

AI-Native Scanners

Two major AI-native security scanners launched in early 2026. Anthropic’s Claude Code Security, released February 2026, uses LLM reasoning to analyze code for vulnerabilities rather than matching patterns. It’s available to Enterprise and Team customers, and free for open-source maintainers. In its initial period, it found over 500 high-severity vulnerabilities in open-source projects. OpenAI’s Codex Security, launched March 2026, scanned over 1.2 million commits during beta, surfacing 792 critical and 10,561 high-severity findings.

Neither tool has been independently audited, so take the numbers with appropriate caution. But the approach is fundamentally different from traditional SAST — instead of matching patterns, these tools read code the way a security reviewer would, reasoning about data flow, trust boundaries, and whether the security model makes architectural sense.

Pre-Publish Security Gates

VibeGuard, published April 2026, targets the specific blind spots of AI-generated code with a pre-publish security gate framework. It checks for five categories: artifact hygiene (source maps, debug files shipping to production), packaging-configuration drift, hardcoded secrets, supply-chain risks, and source-map exposure. The motivation came from a real incident — in March 2026, Anthropic’s own Claude Code CLI shipped a 59.8 MB source map exposing roughly 512,000 lines of TypeScript source. In controlled experiments on 8 synthetic projects, VibeGuard achieved 100% recall and 89.47% precision (F1 = 94.44%).

This is a narrower tool than a full SAST scanner, but it targets exactly the things vibe-coded apps get wrong. AI coding tools are very good at generating code that works. They’re terrible at generating deployment artifacts that are clean and hardened. VibeGuard sits in the gap.

Agentic Security Platforms

DryRun Security calls itself “AI-native, agentic” code security. Rather than pattern-matching individual files, it inspects data flow across files and services — understanding how data moves through the application at an architectural level. Their 2025 SAST Accuracy Report showed 88% detection of seeded vulnerabilities out of the box, outperforming four leading traditional static analyzers, with particular strength on complex logic and authorization flaws. In February 2026, they launched a DeepScan Agent that does full-repository security reviews.

Escape raised $18 million in March 2026 specifically to replace legacy scanners with AI agent-driven security testing. Their research team’s methodology is worth studying: they scanned 5,600 publicly accessible vibe-coded applications and found over 2,000 high-impact vulnerabilities. The breakdown is telling — 400+ exposed secrets and 175 instances of personal data exposure, including medical records and bank account numbers. Zero-auth APIs, missing rate limiting, and BOLA/IDOR dominated the findings. These are exactly the vulnerability classes that traditional scanners miss.


What Scanners Miss: The Vibe Code Blind Spots

Across the research, six vulnerability patterns in AI-generated code consistently evade traditional scanning tools. Knowing them means you know what to look for manually, even when the scanner gives you a clean report.

1. Frontend-Only Security Controls

The AI generates a React auth guard that checks localStorage for a JWT before rendering protected routes. The guard works — unauthenticated users see the login page. But the API behind those routes accepts any request, with or without a token. SAST scanning the backend sees API endpoints that take requests and return data. It doesn’t cross-reference with the frontend to check whether server-side enforcement exists. DAST might not reach the endpoints at all if it can’t complete the frontend auth flow.

2. Zero-Auth APIs

Escape’s scan of 5,600 vibe-coded apps found applications with 7–12 public API endpoints performing destructive operations (DELETE, PUT) with no authentication at all. The OpenAPI spec — when one existed — had no security schemes defined. SAST doesn’t flag an endpoint for not having auth middleware, because “no middleware” isn’t a pattern it can match. The code is perfectly valid; it’s just missing a security requirement.

3. Missing Rate Limiting

As I showed in Part 5, a login endpoint without rate limiting lets an attacker try the top 1,000 passwords in ten seconds. No scanner flags this because rate limiting is a middleware addition, not a code pattern. The login endpoint itself is correct — it validates credentials and returns a token. The absence of express-rate-limit or its equivalent is a deployment decision, not a code bug.

4. BOLA/IDOR Without Sequential IDs

The Lovable BOLA breach from Part 5 is the canonical example. The API checked authentication (valid Firebase token) but not authorization (does this token’s user own this project?). SAST sees the firebase.auth() call and considers the endpoint protected. The ownership check that should follow is business logic the scanner can’t infer. DAST could theoretically detect IDOR by testing two different user sessions, but most DAST configurations don’t set up multi-user testing scenarios.

5. Insecure Default Configurations

AI-generated code uses Supabase with RLS disabled, Firebase with security rules set to allow read, write: if true, Express with no CORS configuration (defaulting to allow-all), and JWT libraries with the algorithms parameter unset (allowing the none attack). None of these are bugs. They’re all valid configurations that happen to be insecure. SAST would need configuration-specific rules to flag them — and most tools don’t ship with rules for “Supabase table missing RLS policy.”

6. Artifact Hygiene Failures

Source maps shipped in production, .env files baked into Docker images, node_modules included in deployable artifacts, debug logging active in production. These aren’t code vulnerabilities — they’re packaging and deployment failures that expose source code, secrets, and internal architecture. Traditional SAST and DAST don’t scan build artifacts at all.


Building a Scanning Pipeline That Works

No single tool covers the full risk surface of a vibe-coded application. The practical answer is layering tools where each one covers a different gap, running them in the right order, and knowing what still requires human review.

Layer 1: Pre-Commit (Catch Secrets Before They Ship)

Before code reaches the repository, run secret detection. This is the highest-ROI automated check because secrets in version control are permanent — even if you delete the file, the secret lives in Git history.

# Install and run Gitleaks as a pre-commit hook
gitleaks detect --source . --verbose

# Or TruffleHog for deeper analysis including Git history
trufflehog filesystem . --only-verified

Configure this as a Git pre-commit hook. Every commit gets scanned. If a secret is detected, the commit is blocked. This is the one layer where automation is genuinely reliable — the patterns are well-defined and false positives are manageable.

Layer 2: CI Pipeline (SAST + SCA on Every Push)

Run SAST and SCA in your CI pipeline. The goal here isn’t perfection — it’s catching the 30% of issues that pattern-matching handles well.

# Semgrep with auto-config (pulls relevant rule sets for your stack)
semgrep --config=auto --error --json ./src > semgrep-results.json

# npm audit for known dependency CVEs
npm audit --audit-level=high

# Socket.dev CLI for behavioral dependency analysis
socket scan create --repo . --branch main

The critical step is filtering SAST output. If your team is drowning in false positives, start with only the high-confidence rules. Semgrep’s p/security-audit ruleset is more targeted than --config=auto. For SCA, differentiate between development and production dependencies — a CVE in a dev-only testing library is lower priority than one in your authentication middleware.

Layer 3: Post-Deploy (DAST Against the Running App)

After deployment, run DAST against your actual application. This catches configuration issues that don’t exist in source code.

# Nuclei with community templates
nuclei -u https://yourapp.com -t nuclei-templates/ -severity critical,high

# ZAP baseline scan
docker run -t zaproxy/zap-stable zap-baseline.py -t https://yourapp.com -r report.html

For SPAs, use ZAP’s AJAX Spider or Burp’s browser-based crawling rather than the default HTTP crawler. Feed the scanner your OpenAPI spec if you have one — it’ll discover endpoints the crawler misses.

Layer 4: AI-Augmented Review (The New Layer)

This is the emerging layer that didn’t exist a year ago. If you have access to Claude Code Security, Codex Security, or DryRun, run them as a complement to traditional SAST. They cover the architectural reasoning gap — detecting absent controls, evaluating whether authorization logic is semantically correct, and understanding data flow across service boundaries.

If you don’t have access to these commercial tools, you can approximate the approach by running an LLM against your SAST output to filter false positives (the technique from the January 2026 study reduced false positives from 98.3% to 6.3%), or by prompting an LLM to review specific security-critical files with targeted questions: “Does this endpoint verify that the authenticated user owns the requested resource?” “Is there a rate-limiting middleware applied to this route?”

Layer 5: Manual Review (The Irreplaceable Layer)

I’ve been in application security for over two decades. Every engagement I do at VULNEX starts with automated scanning and ends with manual review, because the automated tools always miss something. For vibe-coded apps, the manual review is even more important because the vulnerability classes are architectural.

The manual review checklist is shorter than people think. For each API endpoint: does it check authentication? Does it check authorization — not just “is this user logged in” but “is this user allowed to access this specific resource”? Is the client sending any data that controls server-side behavior (user IDs, role flags, price overrides) without server-side validation? Are there admin functions accessible to regular users?

A focused manual review of the auth and authorization layer takes hours, not days, and it catches the issues that every automated tool misses.

What This Costs

For a solo founder or small team, here’s roughly what this takes. Layers 1–3 use free, open-source tools — Gitleaks, Semgrep, npm audit, Socket.dev’s free tier, Nuclei. Setting up the full CI pipeline takes an afternoon if you’re comfortable with GitHub Actions or similar, a weekend if you’re starting from scratch. Layer 4 varies: Claude Code Security is free for open-source projects, DryRun and Escape have commercial pricing that typically starts in the low hundreds per month. Layer 5 is where it gets expensive if you don’t have security expertise in-house. A focused auth and authorization review from a security consultancy typically runs €3,000–€10,000 depending on application size and complexity. That’s real money for an early-stage startup — but skipping it is how the breaches from Part 3 happened.


The Scanning Checklist

Run this against your vibe-coded application. Each item addresses a specific gap in traditional scanning.

Secrets (Pre-Commit):

  1. Run gitleaks detect --source . --verbose and trufflehog filesystem . --only-verified — zero findings before any commit
  2. Search frontend bundles for leaked keys: grep -r "sk-\|API_KEY\|SECRET\|Bearer\|supabase\|firebase" dist/ build/
  3. Verify .env files were never committed: git log --all --diff-filter=A -- '*.env' '.env*'

SAST (CI Pipeline):

  1. Run semgrep --config=p/security-audit --error ./src — use the focused ruleset, not --config=auto, to keep noise manageable
  2. Review every high or critical finding manually — look for innerHTML, eval(), dangerouslySetInnerHTML, unsanitized SQL

SCA (CI Pipeline):

  1. Run npm audit --audit-level=high — address all high and critical CVEs
  2. Verify dependencies are real: check that every package in package.json has a legitimate npmjs.com page with downloads and a real maintainer
  3. Run Socket.dev or Snyk for behavioral analysis — catches supply chain attacks that CVE databases miss

DAST (Post-Deploy):

  1. Run nuclei -u https://yourapp.com -severity critical,high against your deployed app
  2. Check security headers and CORS: curl -s -D- https://yourapp.com | grep -i "x-frame\|x-content-type\|strict-transport\|content-security-policy" and test with Origin: https://evil.com

Manual (The Gaps):

  1. Test every API endpoint without the frontend — does it require authentication?
  2. Test cross-user access — can User A access User B’s resources by changing IDs?
  3. Test admin endpoints with a regular user’s token, send 100 rapid login requests to verify rate limiting (expect a 429), and confirm Supabase RLS / Firebase security rules are enabled and scoped to the authenticated user

This pipeline won’t catch everything. But it covers the layers where automated tools are reliable, flags the areas where they’re blind, and directs manual effort to where it matters most. If you’re running zero scanning today — which, based on what I see in assessments, describes most vibe-coded applications — starting with items 1, 2, 11, and 12 gives you the most security value for the least effort.


What You Should Take From This

Traditional security scanners aren’t broken. They’re solving a different problem. They were built for a world where developers understand their code and make localized mistakes — a forgotten parameterized query, a misused crypto function, an outdated dependency. AI-generated code introduces a new class of vulnerability: architecturally correct code with absent security controls. The login works, the JWT validates, the database responds — and the fact that any authenticated user can read any other user’s data isn’t something a pattern-matcher can flag.

The scanning landscape is evolving fast. AI-native tools that reason about code rather than pattern-matching against it are starting to close the gap. The IRIS approach (neuro-symbolic analysis), LLM-based false-positive filtering, and pre-publish gates like VibeGuard are all steps in the right direction. But as of mid-2026, no automated tool reliably catches broken authorization logic, missing rate limiting, or client-side-only security controls. Those still require human review.

My workflow at VULNEX: Gitleaks and TruffleHog for secrets, Semgrep for pattern-based issues, npm audit plus Socket.dev for dependencies, Nuclei for the deployed surface, and then manual testing of every auth and authorization boundary. The automated layers take minutes, the manual review takes hours — and in my experience, the manual review is where the critical vulnerabilities surface.

If you’re a solo founder or non-security engineer — which describes most people building with AI coding tools — Layer 5 is the hard one. You can’t review what you don’t know how to find. My practical advice: run Layers 1–3 at minimum, they’re free and they catch real issues. If your application handles user data, payments, or anything sensitive, budget for a professional security review before you launch. It doesn’t have to be a full pentest — a focused review of your auth and authorization boundaries, scoped to 2–3 days, catches the architectural issues that automation misses. Part 8 of this series will go deeper on this with a complete founder’s checklist.

As always: trust nothing, verify everything.


Further Reading


References

Posted in AI, Pentest, Security, Technology | Tagged , , , , | Leave a comment

When Agents Fix Agents: How Hermes Patched OpenClaw After a Bad Update

Read Time: 7 minutes

TL;DR

I told OpenClaw to update itself. It did. Then the gateway refused to start because a config field had quietly changed shape between releases (channels.discord.streaming went from string to object). openclaw doctor --fix saw the problem but couldn’t fix it. The Google AI Overview confidently suggested the opposite of the correct fix. Hermes-Agent, with shell and filesystem access, read the failing config, made the right one-line change, backed up the original, restarted the service, and verified — all from a one-paragraph prompt. Thirteen minutes from red banner to green. This is what “agentic ops” actually looks like.


I have been running OpenClaw on a Raspberry Pi 5 for a while now. It is the kind of setup you tune on weekends and forget about during the week — until an update lands and something quietly breaks.

This morning was one of those mornings. What I want to write down is not just the bug, but the shape of the fix. OpenClaw’s own repair tool did not get the gateway back up. Neither did the AI Overview at the top of every Google result. What worked was a general-purpose agent with shell and file access, and the habit of reading a config before having an opinion about it.

That distinction sounds small. It is not.


Step 1: I Asked AgentX to Update Itself

The whole story starts with a perfectly reasonable instruction. I opened the OpenClaw chat UI, said hi to AgentX (my main OpenClaw agent), and asked it to update.

1

update yourself please. The kind of thing you say to an autonomous agent and assume it will handle.

It did handle the update part. It just did not handle the post-update validation part, because that is not yet a thing OpenClaw does by itself. The new release introduced a schema change to one of the config fields. The update wrote the new binaries. The config file kept its old shape. The next gateway start would fail.

I did not know any of this yet.


Step 2: Half an Hour Later, the Gateway Refuses to Start

Same session, same morning. The update completed quietly in the background. When I went to bring the gateway up — openclaw gateway, expecting a normal boot — I got this instead:

2

Invalid config at /home/vulnex/.openclaw/openclaw.json:
channels.discord.streaming: invalid config: must be object
Run "openclaw doctor --fix" to repair, then retry.

Helpful, in theory. The startup writes a stability bundle (good — that is what stability bundles are for) and the service dies on its way back down.


Step 3: Status Check Confirms the Broken State

I ran openclaw gateway status to get the full picture.

3

Red line across the board. state failed, sub failed, last exit 1, reason 1. The dashboard URL was sitting there mocking me, the loopback probe couldn’t connect, and the gateway was clearly not coming back without intervention.

This is the moment where, in a normal world, you would walk through OpenClaw’s suggested fix and be done in five minutes. So that is what I tried first.


Step 4: doctor --fix — The Self-Repair That Wasn’t (Part 1)

openclaw doctor --fix is meant to be the “have you tried turning it off and on again” button. So I tried it.

4

The doctor was happy to lecture me about NODE_COMPILE_CACHE and OPENCLAW_NO_RESPAWN on low-power hosts. Useful tips. Not the problem.


Step 5: doctor --fix — The Self-Repair That Wasn’t (Part 2)

The doctor walked through the config and gateway sections and ended where I started:

5

Restarted systemd service: openclaw-gateway.service
Error: Config validation failed: channels.discord.streaming: invalid config: must be object

The doctor restarted the service but never actually touched the offending key. Which makes sense, in hindsight. The validator says “must be object,” but the doctor has no opinion on what that object should look like. It is not in the business of guessing new schemas. Fair enough. Not very useful at 10:27 in the morning.

One thing OpenClaw should change: doctor --fix should not print “Restarted systemd service” one line above “Error: Config validation failed” and exit happy. It tripped me up, and it will trip other people up. I will file the bug.


Step 6: The Wrong Answer From the AI Overview

At this point I did what most people would do: I pasted the exact error string into Google to see if anyone else had hit this between versions.

6

It told me the validator wants a string like "partial", and that my config has an object — when in reality the new OpenClaw expects an object and my old config has a string. It even produced a tidy, syntax-highlighted JSON block I could have copy-pasted straight into the config to break it harder, and tagged the answer with a confidence-inspiring GitHub citation pill.

If I had been in a hurry, I would have pasted it. That is the part most “AI for ops” demos quietly skip. The answer was fluent, well-formatted, even cited — and 180° wrong about the direction the schema had migrated.

It is the same threat model I covered in Professional Vibe Coding vs. Vibe Coding, just dropped into an ops context instead of a coding one. If your AI cannot read the validator and the config, you are going to get a confident answer that was synthesised from the error string, and sometimes that answer is the opposite of correct.


Step 7: Calling In Hermes

I keep Hermes-Agent attached to this box for exactly this kind of mess. It has filesystem tools, shell execution, and the patience to read things instead of guessing.

7

The skill set matters here: file:patch, read_file, search_files, write_file, code_execution, plus the openclaw-agent-integrations skill I keep around for exactly this plumbing. Nothing glamorous, just the basic moves you need to repair a misconfigured service.

I gave it a one-paragraph brief:

“I told openclaw to update itself and did, however the latest version breaks due a openclaw config json file error. The folder path is /home/vulnex/.openclaw. Make a copy of the config json file and fix the issue. You can use openclaw command to see the issue.”

That is it. No schema, no hints, no example of the new format.


Step 8: Hermes Orients Itself

Hermes did what I would have done if I had another hour.

8

  • Inspected the environment
  • Found a way to invoke openclaw (the binary is on my PATH, but Hermes’ non-interactive shell did not inherit it, so it fell back to npx --yes openclaw and flagged that in its summary)
  • Read the failing config
  • Pulled the stability bundle that the gateway had dropped on its way out the door

Not a single dramatic LLM call. A stack of small, verifiable steps — find, command -v, head, npm prefix -g, a one-shot python3 heredoc that searches $PATH for anything named claw. Boring on purpose.


Step 9: Hermes Diagnoses

Once it had the config and the failure bundle in context, Hermes compared them and figured out exactly what had changed between releases.

9

No guessing from the error string. Reading the source.


Step 10: The Fix Lands

Hermes had none of the trouble the AI Overview did, because Hermes was reading the actual files instead of inferring from prose.

10

The diff is the whole story:

// before — old shape, valid in 2026.5.18 and earlier
"channels": {
  "discord": {
    "streaming": "off"
  }
}

// after — new shape, required by 2026.5.19
"channels": {
  "discord": {
    "streaming": { "mode": "off" }
  }
}

OpenClaw 2026.5.19 promoted channels.discord.streaming from a string to a tagged object. The doctor saw it was wrong but had no opinion on the new shape. The Google AI Overview had an opinion and it was the opposite of correct. Hermes:

  1. Read the failing config and the gateway’s startup_failed.json stability bundle
  2. Made the smallest possible change
  3. Wrote ~/.openclaw/openclaw.json.agenth-bak-20260521-103255 next to the original
  4. Restarted the gateway service
  5. Verified that the JSON parses cleanly and the previous error is gone

It also called out its own caveats honestly:

  • It used npx --yes openclaw because its non-interactive shell didn’t inherit my interactive PATH — even though the openclaw binary is, in fact, installed globally on this host. A small mis-read of the environment, but a transparent one.
  • openclaw doctor still reported unrelated warnings — but the config-breaking startup issue was fixed

That self-reporting matters, even when (as with the PATH case) the agent is slightly too pessimistic about its environment. An agent that flags its assumptions is much easier to trust than one that hides them.


Step 11: Verification From the Shell

Trust, but verify. Back to the original command that started this whole thing.

11

Runtime: running (pid 10178, state active, sub running, last exit 0, reason 0)
Connectivity probe: ok

Same command, opposite outcome. Eleven minutes earlier this had been a wall of red.


Step 12: Asking AgentX to Confirm

Then I went back to the OpenClaw chat UI — the same place where the whole story started — and asked AgentX directly. Because if you cannot trust the agent to self-report after a recovery, you have other problems.

12

“All good — gateway is running on 2026.5.19, active since 10:33. The doctor --fix restart attempt errored but the service came up fine on its own. We’re fully updated and online.”

Thirteen minutes from the first red banner to a green status. Most of that was me reading.

The chat session bookends the whole story. It opens with “update yourself please” and closes with “fully updated and online.” In between, a completely different agent had to come in and do the actual work. That gap is what this post is about.


What This Episode Actually Tells Us

The tidy version of the story is “agent breaks itself, agent fixes itself.” The interesting part is the middle.

The vendor’s own repair tool did not fix the vendor’s own product

openclaw doctor --fix is a good idea, poorly committed to. It should either understand the schema migration paths between recent releases, or stop pretending it has done a repair when the next line of its own output says the config still fails to validate. Right now it does the worst possible thing: it claims success and leaves you broken. That is an OpenClaw bug, not an AI bug, and I will file it.

Consumer AI Overviews are confidently wrong on schema questions

This is not a one-off. The AI Overview cannot read your config, cannot read the validator source, cannot tell which way a schema migrated between two versions, and formats the wrong answer with the same confidence as the right one.

For someone just trying to get the gateway back up before a meeting, that answer is worse than no answer at all. No answer sends you to the docs. A confident wrong answer sends you to paste broken JSON into a working file.

It is not a Google-specific problem either. It is the general pattern of producing a fluent answer from the symptom rather than the source. Any AI deployed without read access into the actual artifact will hit the same wall.

The agent that worked was not magic

Hermes did not solve this because it is bigger, smarter, or trained on something exotic. It solved it because it could read the file, run a command, write the file, and keep a backup. Those four moves are the floor for what I would call agentic ops, and most consumer AI is still well below the floor.

The rule I take away from the morning is short: if the AI you are about to trust with a config can’t read the file and can’t keep a backup, it is not an ops tool. It is a search engine with better grammar.


What I Would Change About My Setup After This

A few things I am going to wire up this weekend.

I want ~/.openclaw/openclaw.json snapshotted to a local git repo before every openclaw update. Hermes’ .agenth-bak files are fine for one incident, but a real version-controlled history is better when the next schema change lands.

I am also going to stop treating doctor --fix as a single-step recovery. It is a diagnostic that occasionally also writes a fix. The actual gate has to be re-running openclaw gateway status afterward and reading the output.

Hermes stays attached to this box with file and exec scopes pre-approved. The whole point of the setup is that when things break at 10:25, I am not also wiring up tool permissions at 10:26.

And the backup naming needs work. openclaw.json.agenth-bak-20260521-103255 is sensible, but I want those files dropping into ~/.openclaw/backups/ rather than sitting next to the live config.

If you are running OpenClaw yourself, the Security Hardening Guide I wrote earlier this spring is still the right baseline. Nothing in this morning’s incident changed those recommendations. It just reinforced why a read-only AI that cannot touch the artifact does not belong anywhere near your recovery loop.


Setup Notes

For anyone reproducing or comparing:

  • OpenClaw 2026.5.19 on a Raspberry-class Linux host
  • Gateway on port 18789, controlled from the OpenClaw web UI
  • Hermes-Agent v0.12.0 on the gpt-5.5 backend with 272K context, configured against my standard skill stack
  • Original ~/.openclaw/openclaw.json preserved as openclaw.json.agenth-bak-20260521-103255 for forensic comparison

One line of JSON, and a reminder that the AI you trust in an incident has to be allowed to read the file.

Stay paranoid. Read the source. Keep the backup.

Further Reading:

Questions or feedback? Reach out via:

Need help hardening your AI agent deployment? VULNEX offers:

  • AI agent security assessments (skill auditing, prompt injection testing, configuration reviews)
  • Red team engagements (AI-powered attack simulations)
  • Security automation and agentic-ops consulting
  • Custom security tool development

Contact: info@vulnex.com

Posted in AI, Technology | Tagged , , , , | Leave a comment