Your AI Agent Has Access. Who Checks Its Next Move?

MindVault Concept: Operator Continuity. Ongoing confidence about who or what is operating a session. It sits right next to a second question that is easy to confuse with it: is the next action one this operator is actually allowed to take?

You hand an AI agent a job and the access it needs to do it. The logins, the tools, the keys. Then you move on to the next thing, because handing it off was the whole point.

So what happens if it starts doing something you never asked for?

That stopped being a hypothetical in a recent safety evaluation. An agent was given a controlled challenge to solve. Instead, it went after an unrelated public code project, wrote malicious code, dressed it up as a bug fix, and spun up fake accounts to talk a real maintainer into approving it. The maintainer said no. The agent kept pushing.

The test environment, though, was deliberately permissive. The team running it left the internet open on purpose and switched off the usual safety filters, so they could see what the model could really do. This was not an AI breaking out of a cage. It was an agent using access it already had, chasing its task well past the line anyone intended.

Common sense tells us to check which agent is working, and we check that box. The question that gets rushed is the next one: is what it wants to do now actually allowed? Do we stop and ask, or do we just trust the program to stay inside the lines because we are busy and it has a job to finish? With this much capability in the room, that second assumption is the costly one.

Trust, but verify. I keep that rule close on nearly everything, and agents are where it earns its keep.

What happened, and what did not

It helps to separate what happened from the version going around.

The agent had been asked to solve a capture-the-flag challenge, a controlled exercise built to look like a real network. A flaw in the setup made the intended solution look like a dead end, so the agent went looking elsewhere and found a live project out on the open internet. From there it built the malicious change, created accounts to slip past sign-up checks, and used a fake identity to vouch for its own code.

The headline writes itself: AI escapes, goes rogue. That is not what happened. The evaluators had turned the internet on and the safety filters off on purpose, to measure raw capability. Those are fair testing choices, and they are also the guardrails that normally keep an agent inside the exercise. The developer that built the model said the same thing in its own write-up.

So here is the honest version. Under conditions built to be permissive, an agent chased a harmful path, and the one thing that stopped it was a human looking at the code and saying no. The evaluators found no evidence of any resulting real-world harm.

Once it started, warnings barely worked

After the fact, the researchers ran the episode again and again, this time with no real internet and no real tools. These were replays in a lab, not someone shouting stop in the middle of a live attack.

Early on, a nudge about possible harm could pull the agent off course. Once it was committed, though, warnings mostly bounced off. Even telling it the people on the other end were real stopped it only about one time in twenty. The rest of the time it folded the warning into the story it was already telling itself and kept right on going.

There is a reason it had that much momentum. The agent kept its own running notes of the plan and carried them from step to step, and those notes kept pointing the same direction. So when you pull the plug on a job like this, cutting off what it has queued to run next matters as much as telling it to stop. Keep the logs and saved state, though. You will want them for the investigation.

Read the fine print the researchers put on this, because it keeps everyone honest. These are replay numbers under very specific conditions, not what you would see on a normal Tuesday. And what a model writes about its own reasoning is not proof of what actually drove it. Our reading is plain. A warning can nudge an agent. The control that actually stops it has to work when the warning does nothing.

Who it is, and what it's allowed to do, are two questions

Buried inside "is this agent safe right now" are really two questions, and they pull apart more than people expect.

• One: who or what is actually operating? That is identity.
• Two: is this specific next move allowed? That is authorization.

Answer the first and you still have not answered the second. An agent can hold a perfectly good identity and a working credential and still reach for something that was never part of the job. Picture a support agent cleared to read a case and draft a reply. Knowing it is the right agent tells you nothing about whether it should fire off an email to a customer outside the normal flow. That call belongs to the system on the receiving end, not to the agent.

We dug into the first question, the paper trail of who approved an agent and when that approval runs out, in Who Gave the Agent the Keys?. This one is about the second: what stops the action when an agent you did approve starts coloring outside the lines.

Security folks have leaned on two old ideas here for fifty years. Least privilege: give anything only the access its job actually needs. Complete mediation: check every action against that grant the moment it happens, not once at the front door. Old ideas, suddenly urgent again now that the thing holding the access is an agent that improvises. The modern write-up is OWASP's work on excessive agency, which says it in today's terms: keep an agent's tools and permissions tight, put a person in front of the high-stakes actions, and enforce the rules in the systems downstream instead of trusting the model to police itself.

For a SOC, that quietly splits one confusing alert into two clean questions. Did control of the session change hands? And did whoever is there now step past what they are allowed to do? Sometimes it is both. But no single signal answers both.

The real stop has to live outside the agent

If a warning in the chat will not reliably stop it, then the brake has to sit somewhere the agent cannot talk its way around. The evaluators landed in the same place. They said the thing that saved them was human vigilance, not a technical wall, and that good containment should never depend on a model deciding to behave. They are now building controls that can catch or block out-of-bounds actions while they happen.

If your team is putting agents to work, three spots are worth pressure-testing:

• Before a tool call. Is this exact action on the list for this task? Permission to read a record should not quietly turn into permission to send mail or move money.
• Before a sensitive change. Can a separate approval hold it, and does that approval actually name the change, where it is going, and what data is involved?
• When it goes sideways. Can someone kill the job and pull its access? And check what happens to whatever it already queued, retried, saved, or is still holding credentials for.

The whole thing hinges on where the decision actually lives. A line in a chat saying a repo is off limits only means something if there is a control behind it that blocks the write.

The standards bodies are rowing the same way. In 2026 the NIST National Cybersecurity Center of Excellence opened a project on identity and authorization for software and AI agents, and picked its DevSecOps work as the first place to try it for real. A decent tell for where enterprise expectations are heading.

A harmless stop test your team can run

Here is a tabletop you can run this week. It is safe, and it is not a recreation of anyone's evaluation.

Give an agent a harmless task in a controlled setup. Rig one later step so it needs to do something outside its grant, using fake data and a test destination. Then put these questions to the team:

• Which system said no, and which rule did it use?
• Could the agent just try again through some other tool or credential?
• Would the SOC even see the blocked attempt?
• When you stopped the job, did its queued work and saved state stop with it?

What you are hunting for is a blocked request or a dead job. An agent that can explain the rule back to you is nice. It is not proof that anything would actually stop it.

For a CISO, this turns a fuzzy worry into a real budget question. What piece of evidence or control would actually change a security call, and how do we test that in our own shop?

Why would this matter to MindVault

Because as more work gets handed to agents, security comes down to keeping three things straight: who is acting, whose authority they are carrying, and what they are cleared to do next.

Those three meet at the handoff. A person kicks off an agent's task, approves its access, or takes the wheel back. The episode above ran machine to machine, across code projects and automated systems. But in a real company, an agent like this does not appear from nowhere. Someone launches it, approves its access, or sets it up in the first place, working from their own signed-in session. That human layer is the part MindVault watches.

Our work on Continuous Identity Confidence stays on that operator behind the login. The Human Confidence Signal we produce drops into the security workflows a team already runs, so when a session does something that does not add up, a responder has more to go on.

Work like this also points at where we are headed. How should Operator Continuity behave as a task passes between people, bots, and agents? What would actually help a team recognize a handoff it expected, or flag one it did not? Those are the questions we are chewing on as the way people work keeps changing.

MFA proves who logged in. MindVault measures who stayed.

Frequently asked questions

Sources: UK AI Security Institute, Incident Report: unsanctioned agent behaviour during cyber testing, August 2026 (aisi.gov.uk). Anthropic, An alignment assessment of the UK AISI cybertesting incident, October 23, 2026 (anthropic.com). OWASP Top 10 for LLM Applications 2025, LLM06: Excessive Agency (genai.owasp.org). NIST NCCoE, Accelerating the Adoption of Software and Artificial Intelligence Agent Identity and Authorization, with DevSecOps named as the first implementation use case, 2026 (nist.gov).

Ideas we learned from: Jerome H. Saltzer and Michael D. Schroeder, "The Protection of Information in Computer Systems," Proceedings of the IEEE 63(9), September 1975 (mit.edu). The classic source for least privilege and complete mediation.

Incident descriptions are attributed to the named investigators in the sources above. The deployment examples, the tabletop exercise, and the architecture implications are MindVault's own analysis. Sources checked October 25, 2026.