It's Just RAG. Except That Isn't the Interesting Part.

I've had a few people look at the work I've been doing with the Executive Positioning System (EPS) and say some version of, "Isn't this really just RAG?" There is some truth to that, but there is also a lot missing from it.

For anyone who hasn't followed this work, EPS started as an experiment in applying Governed AI to a real and sufficiently complex problem: developing executive application packages from a large body of career evidence. It wasn't because the world needed another AI resume writer. There are plenty of those. I wanted a problem complicated enough to test a different architectural approach to using AI.

EPS maintains a governed body of evidence about a candidate — roles, achievements, scope, outcomes, metrics and other career information. It analyzes an employment opportunity, determines what matters, selects the relevant evidence, develops the positioning, writes and optimizes the application material, evaluates the result and ultimately produces the application package.

There are Large Language Models (LLMs) throughout that process, but the LLM isn't the system. That distinction has been fundamental to the architecture from the beginning.

EPS separates probabilistic reasoning from deterministic governance. Models can reason, recommend, write and evaluate, but they don't decide what the authoritative evidence is. They can't quietly change a fact because different wording happens to sound better. Generated content doesn't become authoritative simply because a model produced it. Evidence has provenance, artifacts have state, changes are validated, and movement through the workflow is controlled.

The principle behind it is fairly simple: use probabilistic AI for the things it is good at, and deterministic controls for the things it should not be trusted to govern itself.

As I've put that architecture through real cases, however, it has led to another question that I hadn't really anticipated.

What Happens When the AI Should Stop?

One of the things EPS does is iterative optimization. A section is generated, evaluated against a defined framework, revised and evaluated again. In testing, I started noticing a fairly consistent pattern.

A first round might score 82. The next optimization takes it to 92 — a significant improvement. Another round might move it to 93. Run it again and it might fall back to 91.

The interesting part was that the evaluator could still find things to improve. Of course it could. Ask an LLM to find another way to improve a piece of writing and it will almost always suggest something. Feed those recommendations back into another model and it will happily rewrite it again.

We could keep doing this for quite a while, consuming more time and more tokens without necessarily producing a better product.

That made me realize we were asking the wrong question. It wasn't simply, "What should we change in the next round?" It was, "Should there be a next round at all?"

That is a governance question.

The Trajectory Tells Us More Than the Latest Score

Consider a simple optimization sequence: 82 → 92 → 93 → 91.

Looking only at the latest result tells us that we have a 91-point artifact and perhaps several remaining recommendations. Looking at the trajectory tells us something quite different. The first intervention produced a substantial improvement, the second produced very little, and the third actually made the product worse.

A governed system shouldn't assume that the newest artifact is the best artifact simply because it came last. It should know that a previous round remains the best validated state and preserve it.

But there is no reason we need to wait for a regression to discover this. Before generating another round, EPS already knows quite a bit. It knows the current quality, how much the previous round improved it, what deficiencies remain, whether those deficiencies are actually addressable with the governed evidence available, and what another optimization cycle is likely to cost in time and tokens.

That raises a more useful question: what is the expected improvement from another AI operation?

Perhaps there is significant quality headroom and two material, addressable issues remain. Another round probably makes sense. Or perhaps the current object is already strong, the last round added one point, the remaining recommendations are marginal, and another rewrite carries a meaningful risk of regression. In that case, the correct decision may simply be to stop.

There is an even cheaper model call than one that uses fewer tokens: the one you determine isn't necessary.

Turning That Into an Architectural Control

The practical response in EPS is to introduce a shared optimization control layer.

The evaluation framework still has an important job. It determines how good an artifact is, where the weaknesses are and what could be improved. But a separate self-optimizer can look across the history of those evaluations and answer a different question: is another round actually likely to be worthwhile?

That optimizer can preserve the best valid round, recognize plateaus and regressions, identify diminishing returns, determine whether the remaining issues are actually addressable, and estimate the expected improvement before authorizing more generation.

The distinction is important:

The evaluator asks whether something could be better. The optimizer asks whether trying to make it better is worth doing.

Those aren't the same question.

It also means optimization doesn't need to be based on an arbitrary rule such as "run three rounds" or "continue until the score reaches 90." The system can make the decision from the trajectory, the remaining material issues, the available evidence, the expected gain and eventually what it has learned from previous optimization attempts.

From Governance to Bounded Self-Governance

This is where I think the architecture starts to become particularly interesting.

I use the term self-governance carefully. I am not suggesting that an AI system should be free to establish its own rules, determine its own authority or decide what constitutes truth. In the EPS architecture, those boundaries remain governed. Evidence authority, artifact state, validation and promotion remain deterministic controls.

What the system can begin to govern is its own use of probabilistic reasoning within those boundaries.

Should another reasoning cycle occur? What should it target? How much improvement should we reasonably expect? What is the risk of regression? Is the remaining problem actually solvable with the evidence available? Which previous state should remain authoritative if the new attempt performs worse?

Those are decisions about the use of AI rather than decisions made by AI about the business process itself. I think that distinction matters.

More importantly, the system can learn whether those decisions were right.

Did the System Get Its Prediction Right?

Suppose EPS determines that another optimization round has a 70% probability of producing a material improvement and predicts an expected gain of four points. The round is authorized and the actual improvement is one point.

Now we have something useful: a prediction and an observed outcome.

Keep both.

As those prediction/outcome pairs accumulate, the system can start measuring its own forecasting error. Perhaps it discovers that second-round optimization is generally productive but third-round optimization above a particular quality level rarely creates meaningful value. Certain types of recommendations may consistently result in improvement, while others mostly produce different wording. Some optimization patterns may have a surprisingly high probability of regression.

After enough observations, the decisions don't need to depend entirely on thresholds I designed. EPS might discover, for example, that when an employment section reaches 91–93 after a large second-round improvement and only writing-density issues remain, another round produces material improvement 12% of the time but regression 24% of the time.

At that point, stopping isn't simply a prompt instruction or my opinion. It is a decision informed by the system's measured history.

The important point is that EPS can compare what it expected to happen with what actually happened and use the difference to calibrate future decisions.

The loop becomes: predict → act → measure → compare → calibrate → predict again.

The system isn't simply learning how to produce better content. It is learning when using the model is likely to be valuable.

To me, that is a much more interesting form of self-governance. The deterministic architecture continues to establish the boundaries, while the system becomes progressively better at deciding how much probabilistic reasoning is justified inside them.

Why Solve the Same Problem Twice?

The same principle applies across cases.

If EPS has already processed five or ten opportunities for the same person, why should the next opportunity start from scratch? We may already have employment summaries that have been selected, optimized and validated. We may know which achievements have been strongest for particular types of mandates. We may have positioning approaches that have worked well before, along with the optimization history showing which changes improved them and which did not.

That is governed knowledge. It should be reused.

When a new opportunity resembles a problem EPS has already solved, the starting point should increasingly be the proven object rather than the raw evidence. Determine what is different about the new opportunity, reason about the difference, and validate the adapted result against the new mandate.

This changes the economics and behaviour of the system. Early in its use, EPS may need considerable probabilistic reasoning because it has little accumulated knowledge. As its governed knowledge base grows, it should increasingly retrieve validated objects, adapt only what is different and avoid rediscovering things it already knows.

In other words, the system should become more capable while requiring less unnecessary AI.

This Isn't Really About Resumes

EPS happens to be the environment where I'm testing these ideas, but there is nothing particularly resume-specific about the control pattern.

A contract-drafting system could ask whether another review is likely to materially reduce risk. A policy system could determine whether another revision is likely to improve compliance or clarity. A software system could decide whether another autonomous repair attempt has sufficient expected value or whether the last known good state should be preserved.

The artifact changes, but the governance problem is remarkably similar: establish authority, control what probabilistic reasoning is permitted to change, measure the intervention, preserve the best valid state, predict whether another intervention is worthwhile and learn from whether that prediction was correct.

That is where this starts to become more interesting to me as an architectural pattern rather than simply an EPS feature.

So, Is It RAG?

Yes, parts of EPS are Retrieval-Augmented Generation (RAG). If the system retrieves relevant evidence before asking a model to reason about an opportunity, that's RAG. If it later retrieves previously validated objects that are relevant to a new problem, retrieval is involved there as well.

RAG is useful. I have no interest in inventing a new name for something simply because an established technique is already doing the job.

But describing the entire architecture as RAG misses what I think is becoming the more interesting question.

RAG is principally concerned with what knowledge we should provide to the model. The governed architecture is also concerned with whether the model should reason at all, what it is permitted to reason about, whether its last intervention actually improved the outcome, whether that improvement matched what the system predicted, which artifact should remain authoritative, and what should be learned from the result.

Over time, it should also be able to recognize when something has been learned well enough that probabilistic reasoning is no longer necessary for much of the problem.

That's not a semantic layer around RAG. It is a control architecture around probabilistic reasoning.

RAG made models more useful by giving them better information. Agents are making models increasingly capable of taking action. I think there is another architectural problem sitting between those ideas and dependable enterprise AI: a control system that can govern probabilistic reasoning, measure whether it created value, learn from the result and eventually recognize when that reasoning is no longer necessary.

The models remain extremely important. They do the things they're very good at. But they don't govern the system.

What I am now interested in is whether a governed system can progressively learn to govern how much it needs them.

There is something slightly counterintuitive about where that leads. The measure of a mature AI system may not be how much AI it can use.

It may be how much unnecessary AI it has learned not to use.