13 AI governance decisions to make before the technology makes them for you

AI governance can become abstract very quickly. Principles, frameworks and policies all matter, but eventually somebody has to decide what an AI system should actually do when it is working with company information and the real world does not behave as expected. That is where governance stops being a policy exercise and becomes an operational one.

I have been running an AI system across my fractional CTO practice as more than a chatbot. It supports recurring intelligence work, correspondence triage, meeting preparation, document drafting, security analysis, institutional memory and scheduled work. The objective is not to replace judgement. It is to improve the information, retrieval, monitoring and drafting layer underneath it.

My CTO experience shaped the initial boundaries around the system. Operating it against real work then exposed a series of situations that demonstrated why those boundaries matter. There are 13 I think leadership teams should consider.

They are not 13 bugs to fix. They are 13 decisions somebody needs to own.

1. Capability does not equal authority

An AI may technically be capable of taking an action. That does not mean it should be authorised to take it. Our system can prepare an external email or message. Sending it remains behind a human boundary.

The principle is to understand the blast radius before granting authority. A low-consequence internal action and a client-facing commitment should not be governed in the same way.

Leadership question: What is the worst plausible consequence each AI-enabled action could create, and where is human approval actually enforced?

2. Know which part of the system has failed

An apparent AI failure may have nothing to do with the model. We encountered working areas that appeared equivalent from the outside but had different write permissions. The resulting behaviour could easily have been interpreted as poor AI reasoning when the underlying problem was infrastructure.

That distinction matters because otherwise the team fixes the wrong thing. Model behaviour, application logic, permissions, integrations, data and business rules need to be observable separately.

Leadership question: When something fails, can your team tell you which layer actually caused it?

3. An integration being connected does not mean it can do the job

A system can show an integration as available while lacking the permissions required for the intended workflow. We found dependencies that appeared live but were effectively read-only or unavailable for the operation being attempted. That becomes particularly important when the work is unattended. A process can reach a dependency everyone assumed was available and simply stop.

Leadership question: Are you verifying what each dependency can actually do, or merely that it appears to be connected?

4. Decide what your AI is allowed to remember

Persistent memory makes an assistant far more useful, but not everything it encounters deserves to become organisational knowledge. Our own system has finite persistent memory, which forced us to curate it. Stale entries need removing, related information needs consolidating and temporary working context should not quietly become permanent fact.

The capacity constraint is specific to our implementation. The governance principle is not.

Leadership question: Who decides what gets remembered, corrected and eventually retired?

5. Retrieval is not the same as retrieving the truth

We have encountered situations where the system returned a perfectly plausible answer from information that had become stale. The authoritative information had moved elsewhere. Nothing in the quality of the answer revealed that.

A knowledge base is only as useful as its understanding of which source should be trusted when information conflicts.

Leadership question: For important business information, does the system know which source is authoritative and what happens when sources disagree?

6. Confidence and correctness need separate signals

A weak information match can produce prose as polished as a strong one. Tone therefore tells you very little about reliability. Where the evidence is thin, the system should be able to expose that uncertainty or decline to reach a conclusion rather than turning weak evidence into confident prose.

Leadership question: Can users see the difference between a strongly supported answer and a plausible guess?

7. If a rule matters, formalise it

A governance rule that exists because somebody remembers to repeat an instruction is fragile. We found that conversational corrections were not sufficient for recurring behaviour. Our approach became straightforward: the first correction can be conversational; if the behaviour matters enough to keep correcting, it needs to become an explicit rule or control.

Leadership question: Which important AI behaviours currently depend on a particular person remembering what to tell the system?

8. Do not use AI judgement where software can know the answer

AI is convenient enough that there is a temptation to use it for decisions that do not require interpretation. A date comparison, numerical threshold or permission check should not need probabilistic reasoning if deterministic software can give the right answer consistently. The occasional AI error in a deterministic task is particularly unhelpful because it may be silent and inconsistent.

Leadership question: Which of your AI decisions genuinely require interpretation, and which should simply be application logic?

9. Technical completion is not the same as business success

We encountered a simple example of this. A file could be generated correctly. The system could accurately report that the operation had completed. The person who needed the file still could not see it.

Technically successful. Operationally useless. That distinction applies to far more than files.

Leadership question: When a workflow reports success, are you measuring completion of a technical step or whether the intended person actually received the outcome?

10. Unattended processes should not silently inherit changing defaults

A live AI conversation normally has someone watching it. A scheduled process does not. That changes the level of control required.

If an unattended process inherits whichever model or software default happens to be current, an upstream change can affect behaviour without anybody being present to notice. For recurring unsupervised work, we make important dependencies explicit rather than relying on silent defaults.

Leadership question: If a supplier changes a model or dependency tomorrow, which of your unattended processes could change with it?

11. Separate contexts need to remain separate

As AI expands across customers, projects, teams or business units, overlapping language becomes inevitable. That creates a difficult failure mode: correct information from one context can influence an answer in another. Because the resulting output may still sound plausible, proofreading is not a reliable control.

Our boundary is strict: information from independent contexts should not be combined simply because it appears related. Ambiguity should cause the system to establish context rather than guess.

Leadership question: What actually prevents information from one customer, project or department leaking into the reasoning for another?

12. Preserve the difference between a fact and a hypothesis

A plausible idea gets written down. Another process retrieves it. It becomes established context. A later output repeats it with greater confidence.

Soon the organisation appears certain about something nobody actually validated. Human organisations have always done this. AI simply allows the propagation to happen faster.

We therefore distinguish explicitly between validated information and hypotheses.

Leadership question: Does your information architecture preserve whether something is fact, assumption, opinion, interpretation or hypothesis?

13. Retrying a failed action is not resilience

When an automated process fails, trying again can be appropriate. But repeating the same action when nothing has changed can also create duplicate work, noise or repeated external actions. Our policy is deliberately simple: identify the blocker, explain why the action cannot continue, then stop until something materially changes.

Leadership question: When should your automated process retry, and when should it stop and require a person to make a decision?

What these 13 decisions have in common

My conclusion from operating the system is not that AI is too risky to use. Quite the opposite. It is useful enough that removing it would mean recreating a significant amount of work manually.

But its usefulness depends on deciding where it sits within the operating model. What can it do? What can it trust?

What information can it retain? Where does it need approval? What happens when it is uncertain?

What happens when something breaks? Those decisions need to exist before the system becomes trusted with increasingly consequential work. And they are technology leadership decisions, not model-selection decisions.

This is also why leadership visibility matters

The assistant gives somebody the ability to interrogate information, explore context and understand a decision. As AI-supported workflows increase, the dashboard we are developing needs to solve a different problem: showing leadership where attention is required. What changed?

What is blocked? Where is risk increasing? Which information is not yet validated?

Where does somebody need to make a decision? A founder should not need to understand the underlying AI architecture to have proper visibility and control over how technology is affecting the business. That is the direction we are taking with Scryla and CTO in Your Pocket.

Not an AI pretending to replace an experienced CTO. A stronger intelligence and visibility layer that helps experienced judgement go further, while giving founders who do not have a CTO better access to the questions, controls and oversight that role would normally provide. You do not need to copy these 13 decisions exactly. Different organisations will have different data, customers, systems and risk.

But somebody in the leadership team should be able to explain where AI has authority, what it is allowed to trust, how uncertainty is handled, how contexts are separated and what happens when it gets something wrong. If nobody can yet answer those questions, the governance has probably not caught up with the technology.