Albert Einstein would intuitively appreciate the limits of AI

getty

There’s a missing link in artificial intelligence. It is not capable of making that leap from number-crunching to actual original or novel thought, spurred by real-world circumstances.

That’s the view of Tom Zahavy, discovery team co-lead at Google DeepMind, in a new paper that points out that AI is incapable of making original discoveries on its own. While the paper is meant to discuss the shortfalls of AI for scientific discoveries, it has implications for AI’s creative powers in business settings as well.

Zahavy dissected the process that went into Einstein’s formulation of his General Relativity Theory, noting that the scientist’s thinking evolved through three phases – deduction, induction, and finally, abduction.

AI has mastered induction, based on statistical pattern matching, and is rapidly conquering deduction, which is formal proof, Zahary illustrated. However, it doesn’t clear a third of three hurdles, which is abduction, which is “the generation of novel explanatory hypotheses.”

If scientific discovery were merely the sum of deduction and induction, "modern large language models (LLMs) should theoretically be capable of inventing theories like General Relativity given sufficient compute.”

Einstein’s discovery process was outlined in his letter to Maurice Solovine, where he “conceptualized discovery as a cyclical process involving an intuitive ’jump’ from sensory experience to axioms, followed by logical deduction,” Zahavy explained.

Generative AI is capable of absorbing information, and then making logical deductions about the implications of that information. But it can’t tell you why the information it has assembled for you is new or exciting.

The three stages of discovery include the following:

1. Deduction (rule + case = result): “The analytic application of a rule to a case to predict a result. It is the only mode that guarantees truth, e.g., executing code to verify output.”
2. Induction (case + result = rule): “The synthetic derivation of a rule from the accumulation of cases and results. It validates hypotheses through statistical frequency, e.g., generating a function to satisfy unit tests.”\
3. Abduction (rule + result = case): "The inference of a case or a new rule to explain a surprising result."

The bottom line is LLMs are great tools for assembling information, but it still takes humans to add new dimensions or perspectives to what the output is telling us.

Again, there is a connection to any original thinking that we may hope for from AI in non-scientific settings. A separate study out of Duke University questions the creative output of LLMs in a different sense – the conclusions tend to be homogenous. “The creative outputs of commercial LLMs are more similar to each other than users might hope. When challenged with three standard tasks assessing creativity, answers from commercial LLMs are much more alike than their human counterparts.”

“People might wonder if different LLMs will take them in different directions with the same prompts for creative projects,” said Emily Wenger, assistant professor of electrical and computer engineering at Duke. “This paper basically says no. LLMs are less creative as a population than humans.”

Zahary emphasized that LLMs, in his experience, have been valuable in assisting humans in developing new theories or finding answers to problems. But LLMs will not do this on their own. “Unlike deduction, which guarantees truth, or induction, which finds pattern that generalize in data, abduction is a creative leap that invents a cause for a singular phenomenon,” Zahary explains. LLMs are not capable of making this leap.

“Einstein did not discover General Relativity by searching over symbols; he discovered it by simulating the sensual experience of a falling observer,” Zahary explained. “To build an AI capable of true invention, we must move beyond systems that merely read scientific literature to systems that can perceive the physical world. The emergence of physically consistent World Models offers a pathway to a synthetic laboratory.”