We have been seeing the same pattern for some time in organisations that have invested in AI for years: the models are in place, the data is in place, and the investment has been made. And yet, when something goes wrong or the regulator asks questions, the same uncomfortable silence emerges.
There is a conversation that still arrives too late in many executive committees. It is not about whether to use AI. In most organisations, that decision has already been made. It is about something more uncomfortable: which decisions are being shaped by models that very few people in the room could clearly explain if asked.
According to Deloitte, nearly 60% of employees already have access to approved AI tools. Yet 84% of companies have not redesigned a single process or role to integrate them.
AI has been layered on top of the workflow. It has not been embedded into the decision-making process.
That gap has consequences.
The model is already in the room, even if it does not appear on the agenda.
In banking and insurance, scoring, fraud detection and pricing models already generate outputs in real time. The decision is shaped before the committee meets. In retail, recommendation and inventory optimisation algorithms influence what is purchased, at what price and for whom. In telecommunications and energy, network operations and predictive maintenance increasingly follow the logic of the model rather than that of the engineer.
This is not a technology issue. It is a governance issue.
When a model fails, or when its recommendations contradict an executive’s experience, a question arises that many organisations have not answered before the system went into production: who is accountable for this decision?
Shaping is not the same as deciding. But confusing the two comes at a cost.
Gartner estimates that by 2028, at least 15% of day-to-day decisions will be autonomous through agentic AI.
For now, fully autonomous deployment remains at an early stage. What already exists across almost every sector is something different: models that prepare the ground, narrow the options and assign probabilities. Models that shape decisions within frameworks that are not always explicitly defined.
The difference between shaping and deciding may appear technical. It is not.
When a model shapes a decision that an executive assumes as their own without understanding the mechanism, accountability becomes opaque. When the model is wrong, it is difficult to determine where the error begins. When the regulator asks questions, the explanation is not always available.
The EU AI Act, DORA and the Basel frameworks were not introduced to make life more difficult for technical teams. They were introduced because this pattern has already led to documented failures.
All three converge on the same principle: effective human oversight is not optional. It is mandatory. And it must rest with individuals who have genuine authority to intervene.
What separates organisations that govern from those that improvise
It is not the model. It is the question asked before deployment.
Which specific decisions will this system shape? Who has the authority to review its recommendations? What happens when the model and an executive’s judgement conflict? Who signs off on the final decision?
These are not technical questions. They are governance questions. In organisations where they still have no clear answers, it is not due to a lack of intent—it is because these issues were not brought to the table before the system entered production. We see the same pattern in organisations that are making progress: the moment these questions are asked clearly is precisely when governance stops being an aspiration and becomes a practice.
IBM measures this in terms of maturity: the difference between leading organisations and laggards is not having better models. It is having a decision architecture that integrates AI, business rules and human accountability in a coherent way.
Without that architecture, the model works. But the organisation does not govern.
Three questions for senior leadership
Before approving a new data or AI initiative, it is worth answering the following questions clearly:
- Which decisions in your organisation are already being shaped by models, even if this is not documented anywhere?
- Who currently has the real authority to challenge, correct or stop one of those models if its recommendation is wrong?
- Could someone on your executive committee explain, in non-technical language, how the most critical model that shaped a recent decision actually works?
If any of these questions does not have a clear answer, the problem does not lie in the model. It lies in the framework used to govern it.
Frequently Asked Questions about AI and Decision-Making
Do AI models make decisions on their own?
In many cases, they do not make autonomous decisions, but they do shape the options, recommendations and probabilities on which people base their decisions.
Who is accountable for a decision shaped by a model?
Accountability should rest with individuals who have real authority to review, challenge and, if necessary, stop the model’s recommendations.
What is effective human oversight in artificial intelligence?
It is the ability to understand how a model influences a decision, intervene when necessary and assume ultimate responsibility for the outcome.
How do AI models influence business decision-making?
They narrow options, assign probabilities and prioritise recommendations that shape decisions in areas such as scoring, pricing, fraud detection and predictive maintenance.
