Human judgement still matters

The most consequential question facing organisations is no longer simply what AI can do. It is what people should allow it to do.

That distinction becomes particularly clear in healthcare. AI can already analyse medical images, identify patterns across vast quantities of information and accelerate tasks that previously demanded significant human effort. Yet when a decision affects someone's health, wellbeing or life, technical accuracy is only part of what matters. Context, experience and judgement remain essential.

Ming Tang has spent much of her career working where those questions have immediate consequences. A senior digital, data and technology leader, she led digital technology and data for NHS England and helped drive work towards a digital-first NHS. In Episode 14 of The AI Values Podcast, she joins Edosa Odaro and Lindley Gooden to examine what responsible AI looks like when decisions affect real people.

Her perspective reaches well beyond healthcare. AI is changing work, increasing cyber risk and forcing organisations to reconsider which human capabilities become more valuable as machines become more capable. For leaders, the challenge is not simply adopting AI. It is deciding what should remain distinctly human.

Curiosity is becoming a core professional skill

The debate about AI and employment often concentrates on which technical skills people will need. Tang points towards something more fundamental: curiosity.

Her own career began in data and analytics, but she describes an early interest in understanding why people behaved as they did and why organisations worked in particular ways. That curiosity became part of how she approached problems throughout her career.

The same capability may become more valuable as AI makes information and technical assistance easier to access.

"The curiosity used to be a soft skill, now it's an essential hard skill."

The distinction matters. AI can increasingly provide an answer, generate an analysis or suggest a course of action. It cannot remove the need for someone to determine whether the right question has been asked in the first place.

Curiosity enables people to challenge assumptions, investigate unexpected results and connect information across different contexts. These capabilities become particularly valuable when AI makes standardised knowledge easier to obtain.

For organisations, this changes the workforce discussion. Preparing people for AI cannot be limited to teaching employees how to use particular tools. Leaders also need to preserve the human capabilities that enable people to question those tools, interpret their outputs and recognise when something does not make sense.

The goal is not to automate everything

The assumption that more automation is automatically better creates its own problems.

Tang captures this problem with a deliberately everyday example:

"I want AI to do my washing on my Monday. I don't want it to do my creative writing and design, right?"

Behind the humour is a serious distinction about value. People may welcome AI when it removes repetitive work while resisting it when it begins replacing activities connected to creativity, identity, expertise or personal meaning.

This makes the future of work more complicated than a simple calculation of which tasks can be automated. Organisations also need to ask which activities should be automated and what may be lost when they are.

The distinction matters for workforce design. An organisation could theoretically automate a task and still decide that maintaining human involvement produces a better outcome. The reason might be creativity, trust, professional development, accountability or the need to retain expertise.

Responsible AI therefore requires leaders to distinguish between technical possibility and organisational desirability. The fact that a system can perform an activity does not establish that handing the activity to the system creates greater value.

Healthcare shows why human judgement cannot be an afterthought

Healthcare provides one of the clearest tests of where AI should sit in relation to human decision-making.

Tang is optimistic about applications where AI can analyse information at a scale or speed that improves diagnosis. Image interpretation is one example. AI can assist with identifying cancers, examining images of the eye and detecting patterns that may help clinicians make better decisions.

But diagnosis and treatment are not identical activities.

"The judgment of how to treat the person is still the human."

That boundary matters because healthcare decisions depend on more than recognising a pattern. Treatment involves a person's circumstances, preferences, medical history and wider needs. The technically optimal answer may not always be the appropriate answer for the individual sitting in front of a clinician.

Mental healthcare makes the problem even clearer. A system optimised to solve a user's stated problem may produce an answer without understanding the human context in which the problem exists. Tang warns that this can become dangerous when a machine is designed to achieve an outcome without the years of experience through which healthcare professionals learn how to respond when situations are uncertain or sensitive.

This is why caution in healthcare should not automatically be interpreted as resistance to innovation. In high-consequence environments, caution can be part of responsible deployment.

The governance lesson extends beyond medicine. Organisations need to identify where AI can provide analysis, support or recommendations while ensuring that consequential judgement remains with people who can understand context and carry responsibility for the outcome.

Human in the loop has to mean something

Keeping a human involved in an AI process sounds reassuring, but the phrase can become meaningless if the person's role is not clearly defined.

Human oversight is valuable only when the person has the expertise, authority and opportunity to challenge what the system produces. A human who simply approves an automated recommendation without meaningful scrutiny provides little additional protection.

Healthcare illustrates why the contrast matters. Clinicians develop judgement through years of training and experience, including learning from situations where expectations and outcomes diverge. That accumulated knowledge cannot be reduced to placing a person at the final stage of an automated workflow.

The organisational question should therefore be more precise than whether there is a human in the loop. Leaders need to understand what decision the human retains, what evidence they receive, when they can override the system, and who remains accountable for the outcome.

This turns human oversight from a governance slogan into an operating principle.

AI is raising the stakes in cybersecurity

AI risk does not begin and end with model outputs. As organisations become more dependent on digital infrastructure, AI is also changing the scale and economics of cyber threats.

Tang describes cybersecurity as one of the issues that can keep senior technology leaders awake at night. Her concern is not based on the outdated image of an isolated hacker attempting to penetrate a system.

"It's big. It's big industries, big business and deploying millions of agents to find your weak points in any organization and just blanketing it with these attacks."

AI changes the economics of attack because malicious actors can automate more of the work required to search for vulnerabilities. Once those capabilities can operate repeatedly and at scale, organisations face a threat environment in which attackers may need only one successful entry point.

The response cannot be limited to minimum compliance.

Tang argues for regular penetration testing, clear contingency plans, identification and containment of risks, and collaboration with organisations that possess specialist expertise. She also stresses the importance of assuming that an attack will eventually happen rather than designing security around the hope that it will not.

This is an important principle for AI governance more broadly. Resilience depends on preparing for failure as well as designing for success.

For boards, cyber risk should therefore form part of the AI adoption discussion from the beginning. Increasing an organisation's use of AI without considering how the same technologies change the external threat environment creates an incomplete view of risk.

Leadership should start with purpose and failure

Towards the end of the discussion, Odaro asks Tang what senior leaders should be thinking about now if they want to move AI in the right direction.

Her response begins with purpose.

"What is your source of value for your organization, for your purpose, your reason for being, what you're trying to achieve."

This gives leaders an anchor for AI decision-making. Instead of beginning with the technology, organisations can evaluate AI against what they exist to achieve and the value they are trying to create.

Tang then proposes a second question that is less comfortable but equally important.

"What don't you want AI to do to your organization?"

That question changes the nature of governance. Rather than considering risk only after a system has been designed, leaders can identify unacceptable outcomes before deployment and use them to shape decisions from the beginning.

Tang recommends treating this as a form of pre-mortem. Leaders should imagine what could go wrong and work backwards to determine which safeguards, boundaries and decisions would reduce the likelihood of that outcome.

For senior executives accustomed to discussions centred on growth and opportunity, this can feel overly negative. Yet responsible leadership requires organisations to consider both sides of technological change.

Purpose establishes what AI should help the organisation achieve. A pre-mortem establishes what the organisation is unwilling to sacrifice in pursuit of it.

Together, they provide a more practical basis for responsible AI governance than enthusiasm for capability alone.

The future of AI depends on remaining proudly human

Tang's perspective is not pessimistic about AI. Quite the opposite.

She points to opportunities in medicine, genomics, environmental conservation, scientific discovery and other areas where AI could accelerate advances that create substantial human benefit.

"AI can help in accelerating that discovery and also the deployment of those things."

But optimism about those possibilities does not require organisations to treat human involvement as an obstacle that technology must eventually remove.

Tang argues that organisations should retain a clear appreciation of what people contribute.

"We have to stand back and make sure that we're proud to be human and that we appreciate those human factors."

This may become increasingly important as AI improves.

When machines become better at routine analysis, standardised knowledge and repeatable tasks, the value of human contribution shifts towards judgement, curiosity, imagination, empathy, accountability and the ability to understand circumstances that do not fit neatly within established patterns.

Those qualities should not sit outside an organisation's AI strategy as optional soft skills. They need to become part of how AI systems, jobs and governance structures are designed.

The objective is not to preserve human involvement for sentimental reasons. It is to retain the capabilities that remain necessary for good decisions when technology reaches the limits of what it can understand.

Listen to Episode 14

Going deeper: Responsible AI requires institutional judgement

The central leadership issue in Episode 14 is not whether humans or AI are better.

It is how organisations determine which form of capability should be responsible for which decisions.

That requires institutional judgement.

A healthcare organisation may decide that AI should identify patterns while clinicians retain responsibility for treatment. Another organisation may automate routine administrative work while protecting activities that develop junior expertise. Cybersecurity teams may use AI defensively while simultaneously preparing for attackers using the same capabilities at greater scale.

There is no universal boundary that can be applied to every organisation.

This is why responsible AI governance needs to begin with purpose, consequences and accountability. Leaders must understand what value they are trying to create, which human capabilities remain essential to that value and what outcomes they would regard as unacceptable.

Tang's recommendation to conduct a pre-mortem is particularly useful because it makes governance practical. Instead of discussing responsible AI only through principles, leaders can identify plausible failures before they happen and design safeguards accordingly.

As AI becomes more capable, the organisations best positioned to use it responsibly will not necessarily be those that automate the most. They will be those that know where automation creates value, where human judgement remains essential and where the consequences demand greater caution.