AI can do far more than answer questions. The hard part is knowing what it should do

Most businesses are still approaching AI by asking what the technology can do. Can it read our documents? Connect to our systems? Automate a process? Prepare reports? Build an assistant?

Those are sensible questions, but they are becoming the easier part. The harder question starts when AI moves beyond being something you occasionally ask for help and becomes part of how the business operates. That is the problem I have been working through in my own fractional CTO practice.

I already know AI can produce a decent answer. What interests me far more is whether it can take on some of the operational work that sits underneath a CTO role, while keeping the judgement, accountability and control where they belong.

This is a technology leadership question before it is an AI question

A large part of being a CTO is deciding where technology should and should not be trusted. What should we build? What should we buy? What can safely be automated? Where do we still need somebody to make a judgement? Which information can we rely on? What happens if a system gets something wrong?

AI does not remove those questions. It makes them more important because the technology can now interpret information and act on it rather than simply follow a fixed set of instructions. So our starting point was never to make an assistant capable of doing as much as possible.

It was to work out where AI genuinely improves the way I operate, and where experienced judgement still needs to remain firmly in control.

Where it has become genuinely useful

The system now supports a meaningful part of the work underneath my CTO role. It can run recurring intelligence work and prepare briefings without needing me to remember to ask for them each time. It can work across correspondence and calendars, identify things requiring attention and prepare responses for review.

Before meetings, it can bring together previous decisions, outstanding actions and relevant background. It supports structured document and commercial-risk drafting, and it can analyse a codebase for security and vulnerability issues based on the evidence it finds. One of the more valuable capabilities is something much less glamorous: institutional memory.

A question such as:

What did we agree about this? can otherwise mean going back through months of emails, meeting notes and documents. Being able to retrieve that history and bring the relevant information together materially changes the amount of manual work needed before I can make a decision. Some of this can also run on a schedule rather than waiting for me to prompt it.

That is very different from having a chatbot open in another browser tab. It starts to become an operational intelligence layer.

But I am not trying to build an AI CTO

That distinction matters. The value is not in pretending that more information automatically produces better leadership. Senior technology decisions involve context, commercial priorities, competing risks, people, timing and consequences. Two businesses facing what looks like the same technical problem can quite reasonably make different decisions.

Experience still matters. What AI can do extremely well is improve the layer underneath that judgement. It can retrieve information, monitor changes, bring together context, prepare analysis and identify things that deserve attention.

That gives the person making the decision a much better starting point. And that is also why I think the assistant on its own is not enough.

Leadership should not need to know the right question before discovering a problem

As the assistant became more useful, a separate problem became obvious. If a founder or leadership team only discovers something important when they happen to ask the right question, visibility is still poor.

A business should not need to ask:

Is anything going wrong? before it finds out that something needs attention. That is why we are developing a smart dashboard alongside the assistant. The two have different roles.

The assistant is there when somebody wants to interrogate information, understand what happened or explore a decision. The dashboard should surface what deserves attention before somebody knows to ask. For a founder, that does not mean another 50 engineering charts.

It means being able to see things such as:

What has changed? What is blocked? Where is risk increasing?

What currently needs a decision? What assumptions are we making? Is delivery starting to move away from what the business actually needs?

What should I be asking my technical team? The underlying technical information can be as detailed as necessary. Leadership visibility should not require somebody to become a technologist to understand it.

That is where I think AI becomes considerably more interesting for founders and SMEs.

Greater capability also creates harder decisions

There is another side to this. Once AI has access to real operational information, you have to decide where its authority stops. What can it do independently?

What can it prepare but not execute? What always requires human approval? Which information should it retain?

How does it know which source is authoritative? What happens when two sources disagree? How do you stop information from one client or project influencing another?

What happens when it does not have enough information to answer reliably? These are not theoretical issues.

Operating the system has exposed situations where a convincing answer could be produced from stale information; where an assumption could begin to look like an established fact; where correct information could be applied to the wrong context; and where a technical process could report success even though the person who needed the result had not received it.

We have also seen situations where AI was being asked to infer something that deterministic software could simply calculate. On the surface, these can look like AI problems. Most are actually problems of architecture, information integrity, permissions, accountability and governance.

In other words, normal technology leadership applied to a new kind of system.

The advantage will not come from giving AI the most authority

I expect businesses to get enormous value from AI. But I do not think the strongest businesses will necessarily be those that automate the most or give AI the greatest freedom. They will be the businesses that make better decisions about where AI belongs.

That is the thinking behind what we are developing with Scryla and CTO in Your Pocket: experienced CTO judgement supported by a much stronger intelligence, retrieval and visibility layer. Not AI replacing technology leadership. AI making good technology leadership more accessible and giving founders better visibility over what is happening underneath their business.

The next question is the one that becomes unavoidable once you start trusting AI with real work:

Who is accountable when it is confidently wrong?