Your AI Stack Has No Governance Layer
LangChain handles orchestration. OpenAI handles inference. Nobody handles what happens between them. That gap has a name now.
If you've built an AI agent recently, the stack probably looks familiar: a prompt, an LLM API call, some tool definitions, a loop around all of it. Maybe LangChain or a hand-rolled harness. Maybe streaming. Maybe a database standing in for memory.
What it almost certainly doesn't have is a governance layer - not because anyone forgot to add one, but because there genuinely wasn't one to reach for.
What orchestration gives you
LangChain, LlamaIndex, the Vercel AI SDK - these frameworks are good at chaining LLM calls, managing tool definitions, routing between agents, handling retries and streaming.
What they don't do, and aren't really designed to do, is decide whether an action should happen at all. The model says "call this tool with these arguments" and the framework calls it. That's the right scope for an orchestration layer. It's not a complete production system on its own.
What observability gives you
The LLM observability platforms are genuinely good at telling you what happened after it happened. Latency, token counts, cost, traces. None of it stops anything. It watches.
A beautiful trace of exactly how your agent deleted the wrong data doesn't undelete the data.
The gap in between
Between "execute what the model decided" and "record what happened," there's a missing step: pre-call enforcement. The ability to intercept a call before it runs, check it against a policy, and decide - allow it, modify it, or block it outright.
Concretely: PII caught in a prompt before it ever reaches the model. A tool call checked against an allowlist before the tool runs. A call paused for human review when it crosses a risk threshold you've defined. Every one of those decisions signed and chained so it can't quietly be edited later.
Why agents raise the stakes
For a single chat completion, the stakes are fairly low - the user sends a message, the model replies, worst case is a bad answer.
Agents with tool access are a different animal. One agentic loop can read files, write to a database, send email, call paid APIs - all from a single user prompt, with the model making a string of individually consequential decisions along the way. Running that without a governance layer is a bit like handing a new hire root access on day one because they interviewed well.
What the layer needs to do
A governance layer for an LLM stack needs to get three things right. It has to intercept at the right level - not just the tool call, but the LLM call itself, since the prompt is where the intent actually lives. Stopping only at the tool level means the model has already decided to do something bad before you get a chance to catch it.
It has to enforce deterministically. PII patterns, tool allowlists, spend caps - these are things you can and should check in microseconds, without another LLM sitting in the decision path adding latency and a dependency you didn't need.
And it has to leave proof behind. Every allow, block, and redact decision signed and chained, so the record can't be quietly rewritten after the fact.
That's roughly the shape of what obsvr does. Wrap your LLM client, and the governance layer is in the execution path from that point on, generating its own audit trail as it goes.
Why this layer keeps getting skipped
The gap is not a secret, and teams that hit it are not careless. It gets skipped for three structural reasons worth naming, because each one predicts when it will bite.
Nothing fails without it. Skip authentication and requests break immediately. Skip governance and everything works fine, right up until an incident or an audit. Absent controls produce no error, so they lose every prioritization argument to features that do.
It looks like it is already covered. Teams have logging, they have a tracing tool, they have provider-side safety filters. Each of those is real and none of them answer the actual question. The overlap is close enough that it takes a specific prompt, usually from a customer security questionnaire, to notice the difference between recording what happened and being able to prove it.
The bill arrives later. The cost of not having it is paid in an incident review or an audit window, months after the decision not to build it. By then the missing artifact cannot be created retroactively, which is the property that makes this different from most technical debt.
Build versus adopt, honestly
Plenty of teams build this internally, and for a narrow case that is a reasonable call. It is worth knowing which parts are cheap and which are not.
Cheap: a pre-call hook, a regex pass for obvious identifiers, an allowlist check, writing rows to a table. A capable engineer has a working version in a week.
Expensive, and usually underestimated: making the record actually tamper-evident rather than merely stored; keeping enforcement identical across two languages so a Python service and a Node service do not quietly diverge; normalizing text so obfuscated identifiers do not walk past the scanner; making the whole thing fail without taking the host application down with it; and versioning policy so a decision from four months ago can be reconstructed under the rules that were live then.
That second list is where internal versions stall. Not because it is unachievable, but because none of it is the product anyone was hired to build, so it stays at ninety percent indefinitely. Ninety percent is fine for a debugging tool and not fine for an evidence artifact.
The test for whether you need it yet
One question, and it is not about scale.
If someone asked you to prove what your AI did last Tuesday, and to prove the record had not been altered, what would you produce and how long would it take?
If the answer is a query you could run and hand over, the layer is doing its job wherever it currently lives. If the answer involves reconstructing intent from application logs, or if it depends on someone believing your team did not edit anything, then the gap is real. Whether it matters yet depends on your regulatory exposure, your customers' security requirements, and how much your agents can actually reach.