Article

2025-08-13 Science & Technology

AI LLMs' ‘’simulated reasoning’’ abilities are a ‘’brittle mirage,’’ researchers find

[ArsTechnica] In recent months, the AI industry has started moving toward so-called simulated reasoning models that use a "chain of thought" process to work through tricky problems in multiple logical steps. At the same time, recent research has cast doubt on whether those models have even a basic understanding of general logical concepts or an accurate grasp of their own "thought process."

Similar research shows that these "reasoning" models can often produce incoherent, logically unsound answers when questions include irrelevant clauses or deviate even slightly from common templates found in their training data.

In a recent pre-print paper, researchers from the University of Arizona summarize this existing work as "suggest[ing] that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text." To pull on that thread, the researchers created a carefully controlled LLM environment in an attempt to measure just how well chain-of-thought reasoning works when presented with "out of domain" logical problems that don't match the specific logical patterns found in their training data.

The results suggest that the seemingly large performance leaps made by chain-of-thought models are "largely a brittle mirage" that "become[s] fragile and prone to failure even under moderate distribution shifts," the researchers write. "Rather than demonstrating a true understanding of text, CoT reasoning under task transformations appears to reflect a replication of patterns learned during training."

Posted by Elmerert Hupens2660 2025-08-13 00:00|| || Front Page|| [249 views ] Top

#1 I'm beginning to think that AI is just bull.

Posted by Grom the Affective 2025-08-13 02:53|| 2025-08-13 02:53|| Front Page Top

#2 AI brings a gargantuan productivity increase in fields related to esthetics. Commercial art, illustrations for ad campaigns, commercial jingles etc is AI's strength.

But AI is unreliable when it comes to facts.

If thoroughly checking AI output for accuracy and suitability is easy then AI is an appropriate tool with a disruptive impact.

If thoroughly checking AI output is hard, perhaps harder than creating on original work using other tools AI will decrease productivity.

The standard SF trope that an AI would resemble an unemotional, fact oriented, rational human mind has turned out not to be true.

AI is more like a simulation of a excentric gifted artist's mind during a fever dream, a product of complex computational activity without consciousness.

Posted by Elmerert Hupens2660 2025-08-13 04:11|| 2025-08-13 04:11|| Front Page Top

#3 "the same guy who shot the iconic Sydney Sweeney x American Eagle campaign just made a full Jack Daniels commercial using Higgsfield Seedance Pro

it used to cost 1 million, now $20 with AI"

Posted by Elmerert Hupens2660 2025-08-13 06:01|| 2025-08-13 06:01|| Front Page Top

#4 LLMs are not what researchers call Artificial General Intelligence. Efforts toward that incorporate multiple agents with different computational inputs and algorithms - think, a self-directing robot with many sensors of different kinds, a shared internal netowrk of inferences from those different modules etc. In other words, how human brains / minds work.

LLMs and the generative approach to making new sentences etc. from patterns learned via LLMs, are not AGI. The AI label stuck because language is key for humans - key, but not the whole story by any means.

Posted by Robin Burk 2025-08-13 07:27|| 2025-08-13 07:27|| Front Page Top

#5 For code:
Here’s how I[sic] use LLMs to help me write code

Posted by Skidmark 2025-08-13 08:17|| 2025-08-13 08:17|| Front Page Top

#6 The one thing that no AI is capable of is answering the "Why" question.

There has to be self awareness to motivate an independent activity to accomplish a goal. Anyone know of any such thing?

That's what all the algorithms and programs do. The hardest thing in programming is deciding what you (the customer) wants to accomplish and why. Lots of discussion about those.

Posted by alanc 2025-08-13 09:10|| 2025-08-13 09:10|| Front Page Top

#7 I'm beginning to think that AI is just bull.

Anybody who knows any differently can feel free to correct me but, to the best of my knowledge, AI is merely the latest buzzword for software and it's only as good as the people who program it.

Software is a tool that can do a lot of wonderful things for us. Software can solve a lot of problems when all of the variables are known or can be calculated with the variables that are known. But the world, the universe, is a vast and complex organism and we simply don't know all the variables yet. At some time in the future we will hopefully know more than we do now. But for now, software (call it AI if you must) should be taking baby steps that are subject to rigorous testing, sticking to situations where the variables are known.

I remain extremely skeptical of anybody's claim that their software can do my thinking for me.

Posted by Abu Uluque 2025-08-13 14:08|| 2025-08-13 14:08|| Front Page Top

#8 I remain extremely skeptical of anybody's claim that their software can do my thinking for me.

There are a lot of people in the world who avoid thinking as far as possible.

Posted by Grom the Affective 2025-08-13 14:53|| 2025-08-13 14:53|| Front Page Top

#9 GIGO. Consider those who chose the inputs, or those who wrote the inputs.

As for it being intelligent, you can't give what you do not have.

Posted by Melancholic 2025-08-13 16:38|| 2025-08-13 16:38|| Front Page Top

#10 AI is merely the latest buzzword for software and it's only as good as the people who program it.

Permit me to retort. The basis for modern AI is something called a convolutional neural network, a data structure loosely modeled on a brain. A traditional program is a list of explicit instructions for solving a particular problem - do this; do that; if something, do a thing, else do a different thing. It works right out of the box. A neural network, on the other hand, must be trained in order to be useful. The neural network is still software, but it *is* a new New Thing, just like the steam engine was in its day You can think of a neural network as a black box with inputs and outputs that is capable of learning any function or pattern.

Traditional software is only as good as the people who wrote it. A neural network is only as good as its training.

It is disturbing how people are taken in by LLMs, but then we fall for slick-talking politicians all the time.

Posted by SteveS 2025-08-13 22:14|| 2025-08-13 22:14|| Front Page Top

17:25 Elmerert Hupens2660
17:09 Bobby
16:37 Procopius2k
16:35 Procopius2k
16:28 Grom the Affective
16:06 Lord Garth
15:33 Grom the Affective
15:31 Mercutio
14:51 Regular joe
14:19 Grom the Affective
14:18 Grom the Affective
14:01 swksvolFF
13:49 Lord Garth
13:27 Grom the Affective
13:24 Grom the Affective
13:02 DarthVader
13:01 Grom the Affective
12:43 swksvolFF
12:42 Besoeker
12:37 Silentbrick
12:36 Silentbrick
12:17 Besoeker
12:13 Besoeker
12:06 swksvolFF