AI is exposing a human problem

"95% of AI projects fail."

The statistic has become a fixture of AI conferences, board presentations, and consultancy reports. Depending on who is speaking, the figure changes slightly. Sometimes it is 80 per cent. Sometimes 85. Recently, 95 per cent has become the preferred statistic. Yet when Graeme McDermott heard that figure presented to a room full of technology leaders, his response was not to challenge the number itself. It was to ask a different question.

"Define failure."

That question exposes a deeper problem than the statistic ever could. Organisations are spending millions on AI. They are building strategies around it. Boards are discussing it. Investors are asking about it. Yet many organisations still struggle to explain what success would actually look like. They can describe the tools. They can discuss the vendors. They can point to pilot programmes and implementation roadmaps. What often remains unclear is the outcome they are trying to achieve.

This was one of the central themes of Episode 6 of The AI Values Podcast, where Edosa Odaro and Lindley Gooden spoke with Graeme McDermott, Chief Data Officer at Tempcover, about AI adoption, organisational value, and the future of work. The discussion revealed an uncomfortable observation that the biggest challenges emerging around AI are not necessarily technical, but human.

AI has become ordinary. Human judgement has not

One of the most striking moments in the conversation came when McDermott reflected on how quickly AI had become part of his own working life.

"Eighteen months ago, I was using some form of AI once a month."

A year later, it was a weekly tool.

Today, it is used every day across multiple systems and workflows.

His experience mirrors what is happening across many organisations. AI has moved from novelty to infrastructure in remarkably little time. It sits inside productivity tools, development environments, customer systems, and workplace platforms. Increasingly, people encounter AI not because they seek it out but because it has become embedded in the tools they already use.

What has not evolved at the same pace is the way people evaluate the information that those systems generate. The problem is not that AI occasionally gets things wrong. Every technology has limitations. The problem is that many people increasingly treat AI outputs as authoritative simply because they appear polished and confident.

McDermott described a growing tendency to accept AI-generated responses without applying the same scrutiny that would be applied to information from any other source.

"You're just going to take the answer it gave you and say, well, AI told me."

That observation carries significant implications for governance. Much of the current discussion around responsible AI focuses on controlling models, monitoring outputs, and implementing safeguards. Those issues matter. But they can distract from a more immediate challenge: ensuring that the people using these systems retain the ability to question them.

The organisations most vulnerable to AI-related failures may not be those with the weakest technology. They may be those whose people have become least willing to challenge what the technology tells them.

The organisations that ask better questions will learn faster

The conversation repeatedly returned to a capability that receives surprisingly little attention in discussions about AI readiness: the ability to ask good questions.

During an internal training exercise, McDermott watched employees tackle the same challenge using the same AI system. The results varied dramatically. Some participants entered a short prompt with almost no context. Others took time to explain the objective, define the audience, and describe the outcome they were seeking. The quality of the responses reflected the quality of the thinking that went into the request.

"I actually need to think about the question and the context and how I use it."

That lesson extends far beyond prompting techniques. Much of the current market treats AI literacy as a technical competency. Organisations invest in training programmes designed to help employees use tools more effectively. Yet the people who consistently gain the greatest value from AI are often not the most technically sophisticated users. They are the people who understand their domain well enough to recognise weak reasoning, incomplete answers, and flawed assumptions.

In other words, expertise still matters, perhaps more than ever. The popular narrative around AI often suggests that knowledge is becoming less valuable because information is now instantly available. The opposite argument may be closer to the truth. As information becomes easier to generate, the ability to evaluate it becomes more important.

The future advantage may belong less to organisations with access to the most powerful AI systems and more to organisations whose people have developed the judgement to use them well.

Most AI failures begin before the technology arrives

The debate around AI project failure usually begins after implementation. Did the model work? Did the deployment succeed? Did the technology perform as expected?

These are important questions. They are rarely the first questions leaders should ask.

When McDermott was challenged on the now-familiar claim that most AI projects fail, he immediately focused on the assumptions behind the statistic.

"The first question was define failure."

That response highlights a common weakness in how organisations approach AI investments. Failure is often discussed as though it were self-evident. In reality, different stakeholders define failure differently. A project may work technically but fail commercially. It may deliver cost savings while damaging customer trust. It may improve operational efficiency while creating governance concerns that outweigh the benefits.

The problem is often present before any technology is deployed. Many organisations begin with a fascination for AI itself rather than a clear understanding of the problem they are trying to solve. The conversation quickly becomes focused on models, tools, and implementation pathways rather than outcomes. As a result, value becomes difficult to measure because it was never clearly defined. 

McDermott's point is ultimately a simple one. Success criteria should exist before the project begins. Organisations should know what outcomes matter, how they will be measured, and which trade-offs are acceptable. Without that discipline, even successful projects can appear to fail, while failed projects may be celebrated as progress.

For boards and executive teams, this is not merely a project management issue. It is a governance issue. Accountability becomes impossible when success itself remains undefined.

Why AI is moving into the boardroom

For many years, emerging technologies were delegated to specialist functions. Digital transformation belonged to the CIO. Data initiatives belonged to data leaders. Technology strategy sat comfortably within IT.

AI is beginning to challenge that arrangement. According to McDermott, responsibility for AI is steadily moving towards CEOs and executive leadership teams because the implications of AI now extend far beyond technology itself.

"I think this is bigger than that."

His comparison was striking. AI, he suggested, may be closer to electricity than to any previous wave of enterprise technology. The comparison is not about capability. It is about reach. Electricity did not create a new department. It changed every department. It altered how organisations operated, how work was performed, and how value was created. AI appears to be following a similar trajectory.

If that assessment is correct, governance structures must evolve accordingly. Boards cannot treat AI as a technical issue delegated to specialists. The decisions being made about AI increasingly affect workforce planning, customer experience, operational processes, risk management, and organisational strategy. These are leadership responsibilities.

The workforce challenge leaders are not ready for

One of the most revealing parts of the conversation focused on employment. Not the distant future of work that dominates so many AI discussions, but the present.

McDermott described conversations with employers, colleagues, and parents whose graduates were struggling to secure roles despite strong academic credentials. One example involved a graduate holding a degree in AI who remained unable to find work months after completing university. The issue was not a lack of technical knowledge, but a lack of practical experience.

Across many industries, organisations are seeking people who can apply knowledge in commercial environments. At the same time, AI is beginning to automate some of the entry-level tasks that historically provided the route into those professions.

That creates a dilemma. If organisations remove the work that allows people to gain experience, where will future expertise come from? The answer cannot simply be to hire experienced people forever. Every experienced professional was once inexperienced.

This is why the discussion around apprenticeships, workplace learning, and practical development pathways is becoming increasingly important. The organisations that succeed in an AI-enabled economy may be those that rethink how expertise is developed rather than assuming the market will continue supplying it.

The workforce challenge is therefore larger than employment. It is about capability, and capability is a strategic asset. This means that the organisations that fail to invest in developing future expertise may eventually discover that efficiency gains today have created skills shortages tomorrow.

AI will not replace the need for human judgement

A recurring theme ran through the entire episode. The greatest risks associated with AI are often human rather than technical. They are characterised as poor questions, weak oversight, undefined objectives, blind trust, or a lack of accountability. These are not failures of technology but failures of governance.

The pace of AI development continues to accelerate. Organisations are becoming more capable. Models are becoming more sophisticated. New tools appear almost weekly. Yet organisational maturity is not advancing at the same speed.

This is why responsible AI cannot be reduced to compliance frameworks, technical controls, or regulatory obligations. Those mechanisms matter, but they address only part of the challenge. The deeper question is whether organisations are strengthening the human capabilities needed to use AI responsibly.

The organisations that thrive will not necessarily be those with access to the most powerful tools. They will be those that maintain strong judgement while using them. As AI becomes woven into everyday decision-making, the ability to ask better questions may become one of the most valuable capabilities any organisation can develop.

Listen to Episode 6

‍

Going deeper: why judgement is becoming a governance capability

One of the most important insights from Episode 6 is that AI governance is increasingly becoming a question of organisational behaviour rather than technology management.

Many governance frameworks focus on outputs. They monitor model performance, bias, transparency, and compliance. These are necessary safeguards. Yet they often overlook the conditions that produce poor decisions long before an AI system generates a response.

The conversation with McDermott repeatedly points towards human judgement as the missing variable. Organisations frequently ask whether AI can be trusted without asking whether the people using it have the skills required to challenge it. They focus on technological capability while paying less attention to the quality of decision-making surrounding its use.

That distinction matters because governance failures rarely begin with a model. They begin with assumptions.

They begin when leaders fail to define success, when accountability is unclear, and when outputs are accepted without scrutiny because they appear authoritative. Responsible AI, therefore, requires more than policies and controls. It requires organisations that can think critically about evidence, incentives, and consequences. The future of AI governance may depend less on what machines can do and more on whether institutions can preserve the human judgement needed to use them wisely.