What AI doesn’t know, and why it matters

Artificial intelligence has taken the wired world by storm, but the backlash came almost as fast. Progressives complain of job losses, environmentalists question the ecological impacts of huge data centers, and local activists are clamoring for assurances that household utility bills won’t skyrocket because of the centers’ voracious electricity requirements. Others simply worry that the technology will overwhelm humans’ ability to control it.

At least in part, these reactions stem from the overselling of AI.

AI is super cool, but it’s not superhuman nor is it super intelligent. AI is simply very fast processing of vast amounts of data.

Intelligence, knowledge, understanding and wisdom are all different concepts; the distinction between them elucidates the scope and limits of both human and electronic “intelligence.”

Intelligence is the ability to process information into an internally coherent framework that’s useful and adds or detracts from knowledge to the extent it is more or less accurate. Knowledge is the accumulation of information organized into coherent frames or models that help us understand. Understanding is awareness of the significance, purpose, or meaning of accumulated knowledge.

And wisdom is judgment seasoned by experience and the awareness that intelligence, knowledge, and understanding are limited, inherently flawed, and useful only to the extent they advance a worthwhile purpose.

Nearly 2,500 years ago, the Oracle of Delphi reportedly declared that no man was wiser than Socrates. Socrates claimed to be stunned by this because he was keenly aware of how much he didn’t know. But after talking to others widely acclaimed to be knowledgeable, such as the leading politicians, poets, philosophers, and artisans of his day, he discerned this Delphic wisdom: Those claiming knowledge were ignorant of their own ignorance, whereas Socrates knew he knew nothing.

For this insight, Socrates was put to death for impiety and corrupting the youth of Athens, thereby proving for all time both the foolishness of his accusers’ certainty and the wisdom of Socratic questioning.

This bears repeating today, as we enter the Age of Artificial Intelligence: it’s wise to question the “intelligence” of machines, the “knowledge” they propagate, and our understanding of the significance and limits of the technology.

AI models are amazing and useful despite being incomprehensible to most of us, but AI is not infallible. AI will expand human knowledge and understanding of the world only if and to the extent that human users are encouraged to question AI results, processes, and functions.

People make mistakes, as do the people making and training the machines. Still, people tend to trust machines more than people, especially with respect to processing information that’s harder to process. For example, tennis players have more faith in electronic line calls over human line calls, although that faith in the new technology has been shaken by errors, such as when ball marks are inconsistent with the electronic line calls.

As AI use spreads, people will increasingly rely on AI and trust its results for routine tasks (like Google searches), while most people remain more skeptical of AI results for more complex tasks and do not trust AI to act to handle certain tasks for its users without human intervention.

It’s wise to question AI’s results; errors are common even in routine searches.

Examples of AI errors, hallucinations and political bias are rife. A Northwestern University business school professor of my acquaintance recently asked ChatGPT for advice evaluating investment alternatives. ChatGPT recommended he invest in a particular fund and described in detail that fund’s returns, risks, and assets. When the professor went to invest in ChatGPT’s recommended fund, he discovered the fund did not actually exist; ChatGPT made it all up (a phenomenon commonly referred to as “AI hallucination”).

Indeed, AI can screw up even mundane tasks: In my research for this piece, a Google AI summary ascribed quotes to Socrates that are not supported by any historical record.

Artificial intelligence – like human intelligence – is prone to error and is not always reliable, but that’s to be expected, especially in a fledgling technology. AI is artificial intelligence, not artificial knowledge, understanding, or wisdom.  AI is a processor, a very fast processor, that organizes and distills information – and organized information is easier to evaluate and use by humans than vast amounts of unorganized information.

Properly understood, AI supplements and does not replace human intelligence, knowledge, or understanding; plus, the limitations and faults within these amazing models remind us that human intelligence is limited, too. Human intelligence imperfectly organizes the imperfect data to which a human has access and frames data in a subjective, not an objective, manner.

Many of us expect the machines that humans make to have “better” intelligence than the intelligence of its human creators – more objective, more comprehensive, more insightful. This is a naïve hope. In one sense, it is “better.” AI organizes more information faster than humans can. But who do they think programmed the thing? Every AI model is regurgitating imperfect information collected, created, and input by imperfect, subjective human beings.

What to make of all this?

First, perhaps the math nerds creating AI are mistakenly training machines to handle information processing on human topics as if human topics are math problems with a specific answer.  Perhaps instead, machines should be trained to suggest questions to consider instead of answers to accept with respect to human inquiries relating to politics, economics, psychology, child rearing, crop science – the full range of arts, humanities, and social sciences.

Second, people training these machines should be explicit about the biases and perspectives being built into how the AI organizes, sorts, and frames information. (My own bias on this topic is that I believe American AI companies should be building AI with quintessentially American framing.)

Third, AI creators should consider the political, regulatory, and legal risks of “overselling” what AI is and what it can do. For example, should AI creators anticipate a duty to warn users of shortcomings with AI’s results and/or disclaimers of warranties?

Fourth, AI creators need to consider improving the quality of data upon which the systems are being trained, recognizing that many online data sources intentionally mislead to advance political agendas. Perfectly “unbiased” information is impossible to obtain, but some information is more accurate and less biased than other information; trainers should exercise better judgement about data.

The creation of AI large language models is an incredible feat of engineering. It’s quite useful, and will soon be essential, but it is still a product of human invention. As such, we need to recognize that AI is ultimately just the latest, greatest – but still imperfect – implement invented and used by homo sapiens to make life better for homo sapiens.