What an AI Agent Should Never Be Allowed to Do to a Student Record

By Mr. sanjay Laul, Founder at MSM Aventra

Most conversations about AI agents in enterprise software are about what they can read. The harder question, and the one that decides whether any of this survives contact with a regulator, is what they are allowed to write.

The scale of that question is no longer theoretical. Palo Alto Networks’ 2026 Identity Security Landscape found machine identities outnumbering human ones by 109 to 1 inside the enterprise, up from 82 to 1 a year earlier, with AI agents accounting for roughly three-quarters of them. Gartner expects 40% of enterprise applications to embed task-specific agents by the end of this year, against fewer than 5% in 2025. Salesforce’s 2026 Connectivity Benchmark puts the average company at twelve agents already in operation. And yet only 6% of companies say they fully trust agents to execute core business processes without a person in the loop. The distance between how fast agents are being deployed and how little they are trusted is not hypocrisy. It is an unsolved design problem.

In a student information system, that problem has an unusually sharp edge. A student record is not a dataset. Every field in it is a determination somebody made and can be held to. An attendance mark can decide whether an international student stays compliant with a visa condition. A grade sits on a transcript an employer will read in ten years. A status flag decides whether a fee is levied, a scholarship continues, or a place is held open. Reading these fields is recoverable. Writing to them usually is not.

So we build to four limits, and we would argue that anyone putting agents into a system of record should.

An agent should never write to a field of record on its own authority. It can draft, flag, rank, summarise and prepare. The write itself carries the identity of the person who authorised it, and the record shows both: what the agent proposed, and who accepted it.

An agent should never make a determination the law reserves for a person. Annex III of the EU AI Act names them plainly deciding access or admission, evaluating learning outcomes, assessing the level of education someone can access, monitoring behaviour during tests. Article 14 requires that such systems be designed so a human can effectively oversee them. Designed. Not documented after the fact.

An agent should never carry a credential wider than its task. The Non-Human Identity Management Group found that 73% of secrets held by non-human identities carry excessive permissions, and that more than 5% of AWS machine identities hold full administrative privileges. OWASP counted 24 million leaked non-human credentials on GitHub in a single year, a large share of them still valid long afterwards. When an agent inherits a credential at that level, the distance between “allowed to do its job” and “allowed to do anything” collapses to nothing.

And an agent should never take an action that cannot be reversed and reconstructed. Every automated write reversible to its prior state. Every log entry showing which agent acted, on what evidence, under whose authority, at what time. Article 12 of the same regulation calls this logging. We think of it as the only honest answer available to a student who asks why their record changed.

None of this is an argument against autonomy. It is an argument about where autonomy is cheap and where it is expensive. Anthropic’s 2026 research on agentic coding found developers leaning on AI for around 60% of their work while safely handing off outright only a fraction of it. That pattern holds well beyond code.

The realistic failure mode here is not a rogue agent. It is a registrar approving the four-hundredth suggestion of the morning because the first three hundred and ninety-nine were right. Automation bias defeats any oversight designed as a queue of confirmations, which is precisely why these limits have to live in the permission model rather than in a training module.

There is a timing argument too, and it cuts against the industry’s instinct. The Act’s high-risk obligations were due to bite on 2 August 2026. The Digital Omnibus on AI, in force since 27 July, moved them to 2 December 2027. Sixteen months, widely read as a reprieve. It is better understood as a design window. Retrofitting oversight into a platform that already lets agents write is considerably more expensive than building the constraint in from the start, and Gartner already expects more than 40% of agentic AI projects to be scrapped by the end of 2027 on cost, unclear value and insufficient risk controls.

The institutions we talk to have started asking a different question. Not what the AI can do, but what it cannot. That is a harder thing to put in a product brochure, and a much better thing to be able to answer.

Related Post