Research paper: AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking

The Watercooler
43

I highly recommend the pre-print article. One need not spend a lot of time (or any time) on the methodology used in the paper; just review the outcomes and conclusions.

Of particular interest to me is

  • Table #1 for a common set of definitions
  • the conclusion that AI-by-Engineering (aka AI-by-Learning) is theoretically incapable of ever achieving even a lackluster approximation of human intelligence (so-called AGI)
  • the inevitability of some makeists marketing AI-by-Learning ($$) as if it could be AGI
  • the need to reclaim AI-as-theoretical-psychology for the purpose of modeling cognitive functions:

The time is apt to reclaim AI-as-theoretical-psychology as a rightful part of cognitive science. As we have argued, this involves embracing all the valuable tools that computationalism provides, but without (explicitly or implicitly) falling into the trap of thinking that we can or should try to engineer (human-like or -human-level) cognition in practice.

To me, bullet #4 = augment how we understand the brain functions pathologically. Then, it is about solving disease or brain injury. But to seize on this statement requires a quick read of the article.

Reflecting on the paper’s conclusions, I think that the importance of labeling things as AI is going to be the critical piece. Hate to go here but, from a capitalist perspective, that might = charging more for content that is guaranteed NOT generated from AI.

So, in the radiology example above – a viable one IMO – someone could pay more for a radiologist to do a non-AI assessment.

But practically, I don’t see how creepy AI is wholly extracted from medical practice, in this scenario. My healthcare system has gone to AI for transcribing patient meetings. Presumably, the doctors are writing up hybrid AI notes in the records. (I can see the notes in my portal.)

1 Like
44

Since we are talking about critical thinking here, I recommend that everyone at least tries to read the article for themselves, and of course try to look for critiques of this article before taking this “proof” for granted.

The author makes so many fundamental assumptions about humans, human cognition, AI, its purpose, and its method of learning that form the basis of the proof, and many of these assumptions are actually hilariously absurd in their detachment from factual reality.

1 Like
45

Yes, I wholly agree!!

46

Very interesting. It has been abundantly clear to me from the beginning the AGI thing is absolutely ridiculous in so many ways. But, I will be following this research closely for sure.

47

I certainly agree that labeling AI generated or assisted “stuff” generally will be needed, but your statement begs the question: why would anybody pay more for a non-AI assessment, if the AI assessment is actually a better option? For example, whether we call it “AI” or not, if computational systems are better at pattern recognition over huge datasets, and it is demonstrated via “the science” that AI assistance can help diagnose cancer with a higher success rate, so ‘Doctor + AI’ scenario, not just AI, why would anyone pay more for a less accurate service, especially if we are talking about capitalism. Maybe I am misunderstanding the context of your statement. I will admit, I have only read the abstract thus far of the study.

1 Like
48

From just glancing at it it seems to give a purported proof that if AI-by-learning is polynomial time computable, then NP ⊆ BPP. From page 22 hidden in an appendix:

Theorem 2 (Ingenia Theorem). If there is a learning
mechanism that solves AI-by-Learning in polynomial
time, then NP ⊆ BPP.

Then they draw upon the widely held assumption that NP ⊈ BPP so we must find that AI-by-learning is not in polynomial time. The give away seems to be in footnote 12 of page 7?

Formally, NP-hard under randomised polynomial-time one-query reductions. See (Hirahara, 2022) for more details. The re-duction proves that the problem is not tractable (i.e., computable in polynomial time) unless NP ⊆ BPP. It is widely conjectured that NP ⊈ BPP (see, e.g., Arora & Barak, 2009, Chapter 7). See the
Appendix for more details.

I haven’t checked how they prove that AI-by-learning in polynomial time is computationally equivalent to NP ⊆ BPP, but I have to say the whole “proof” depends upon a conjecture (which they acknowledge only in a footnote) that NP ⊈ BPP. Which, again, hasn’t been proven.

To be very clear, if this were a real proof I don’t think you would find this in a psychology paper, but rather I’d expect it would be a widely celebrated result in mathematical and computational complexity circles. It isn’t AFAIK.

@BethL & @sujato is that what you understand about the purported proof as well? That even in the best circumstance it is relying upon NP ⊈ BPP that has not been proven?

It is interesting that they are trying to use computational complexity classes to clarify topics in AI vs AGI though! Would have to dig into how they equate AI-by-learning with known complexity class as that seems the real purported innovation.

I have to say this doesn’t give a good first impression:

AI-by-Learning (informal)
Given: A way of sampling from a distribution D.
Task: Find an algorithm A (i.e., ‘an AI’) that,
when run for different possible situations as in-
put, outputs behaviours that are human-like (i.e.,
approximately like D for some meaning of ‘ap-
proximate’).

That has nothing to do with “AI” per se (Charles might be amused by the broadening of the term here :slight_smile: … it rather seems to have to do with a much more general claim that human cognition cannot be modeled with an algorithm at all even approximately. NOTE: in the formal version of the definition the “human-like” part is completely omitted.

I’m actually quite amenable to the claim, but I do think it is a claim and would require a very high bar to state it as a verified formal proof. Quite simply, it would rock the fields of computer science, mathematics, physics, etc and would be in spirit similar to what Penrose had in mind in his famous books describing non-computational human cognition.

__

UPDATE: Thinking about it a little further, it would seem that one consequence if this purported proof were true is that humans would be capable of solving NP-hard problems in polynomial time or at least solving them in a non-negligible way? Hmm, that doesn’t seem to be my experience.

Venerable @sujato, @BethL do you have any examples of humans solving NP-hard problems in polynomial time in a non-negligible way? This purported proof seems to claim that this is what separates us from mechanical algorithms. Do you have examples?

:pray:

1 Like
49

It is a wee bit challenging to discuss my context without reference to the article :thinking:. At least, Table #1.

I don’t doubt that throwing compute capacity at large data sets will produce valuable information which we wouldn’t have otherwise. Sure, some types of radiology may be a good candidate for that.

Still, if I have the means, I’ll pay extra for a human radiologist making the final assessment and discussing it with the specialist. There’s too much nuance involved.

Besides, a major point of the paper is to argue that it is theoretically impossible to ever reach scale to simulate human cognition. @Sasha_A (and @yeshe.tenley?) do not agree with their modeling. Fair enough. (You really have to look at Table #1 to understand what the range of definitions is.)

50

The author is active on Bluesky, so you might want to ask her about these details.

1 Like
51

Agree. I am not convinced the authors of this paper understand what an LLM is or the state of current research into them. All LLMs are fundamentally based on the same algorithm (a relatively simple one, and it completes in polynomial time - in fact, every time a token is generated). A number of people, including myself, believe that we are hitting limits in terms of what can be achieved and a breakthrough is needed, perhaps by finding a way to get LLMs to do real symbolic processing.

What the authors argue is not new - I remember we were discussing all this when I studied theoretical computer science in the 1980s and this paper will not resolve any of the existing open issues regarding the topic.

I am going to stop here, because anything I say will completely irrelevant to this forum.

1 Like
52

I understand where you are coming from. But … if you support this study, or “studies” in general, then you have to consider this …

“Studies have shown that Artificial Intelligence (AI) can significantly improve cancer detection by analyzing medical imaging data like mammograms, CT scans, and biopsies, allowing for earlier identification of cancerous tissue with high accuracy, sometimes even detecting abnormalities that might be missed by human doctors, particularly in cancers like breast, lung, prostate, and skin …”

See: AI and Cancer - NCI

53

This paper is fun, but it doesn’t take our discourse here any further than the content of my previous post above (#7 in this thread).

Nor does either paper address the fact that the so-called cognitive revolution failed to usurp a technology and science of behavior. It has woredk ok as an adjunct, but not a rival.

Did anyone check out the Stuart Russell BBC Reith Lectures I mentioned? BBC Radio 4 - The Reith Lectures, Stuart Russell - Living With Artificial Intelligence - Available now.

54

Unfortunately most of those studies funded by the US National Cancer Institute (NCI) – your cited source – will slow down or disappear altogether because President “AI” Musk just decided unilaterally to cut the US biomedical grants budget by $4B.

So, perhaps someone who’s more philanthropic can keep funding those.

2 Likes
55

Here’s a new paper: The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.

If you’re worried about bias, this one is by Microsoft, major owner of OpenAI. It comes to the same, blindingly obvious, conclusion: AI impairs cognition, critical thinking, and self-confidence. They conclude:

Our work suggests that GenAI tools need to be designed to support worker’s critical thinking by addressing their awareness, motivation, and ability barriers.

So … the AI will fix the problems AI causes? Why ask questions when the answer is always, “more AI”?

Paper:

Excerpts and discussion:

2 Likes
56

Uber drivers and food delivery cyclists are already used to having the jobs they are handled and if they earn minimum wage or not being governed by these quirks and if they ask for support usually there’s a long conversation with an AI

1 Like
57

Thanks, this is a better quality paper. The questionnaire is better designed, and the conclusions are more nuanced.

I agree that the burden of critical thinking will shift from information gathering to information verification with the use of LLMs.

Using AI effectively will become a fundamental skill, just like the 3 Rs, or the ability to code.

In the course I am teaching next trimester, we are altering the assignment to permit students to use AI, in fact it is a requirement. Instead of asking the students to write an essay to answer questions, we will ask the students to get AI to answer the questions, but the students have to critique AI’s answers against their own understanding and reference sources.

Last trimester, I used AI in class to answer the assignment, then I proceeded to explain where AI got it wrong, what the limitations were, and then explained how an educated human will try to answer the question. The students told me afterwards that was revelatory and eye-opening. Now we are asking the students to do the same analysis.

1 Like
58

The problem identified in the article is certain personal qualities and habits of some categories of people using AI. Accordingly, the proposed solution to the problem is the need to take such qualities and habits of people into account when designing AI and training courses, in order to modify the interaction with AI in a way that encourages positive qualities and habits in users and discourages negative ones.

It is the same old story about adaptation of a newly emerged tool to the actual situation of human use, and humans learning how to use this new tool more skilfully and safely.

59

@christie said:

In the course I am teaching next trimester, we are altering the assignment to permit students to use AI, in fact it is a requirement. Instead of asking the students to write an essay to answer questions, we will ask the students to get AI to answer the questions, but the students have to critique AI’s answers against their own understanding and reference sources.

Last trimester, I used AI in class to answer the assignment, then I proceeded to explain where AI got it wrong, what the limitations were, and then explained how an educated human will try to answer the question. The students told me afterwards that was revelatory and eye-opening. Now we are asking the students to do the same analysis.

That’s brilliant, @christie. Just brilliant! Thank you for the work you do as an educator. That’s basically the process I went through. I was on the fence about chat AIs, interacted with a few, and had my eyes opened.

1 Like
60

Thank you Bhante @sujato. I hope to find time to read it soon.

61

@Sasha_A said:

[T]he solution…is…to take such qualities and habits of people into account when designing AI and training courses, in order to modify the interaction with AI in a way that encourages positive qualities and habits in users and discourages negative ones… It is the same old story about adaptation of a newly emerged tool to the actual situation of human use, and humans learning how to use this new tool more skilfully and safely.

Well said @Sasha_A. As long as “skillfully” includes mindfulness of the fact that we are not like machines, but machines are like us, safety is likely to follow. From the erroneous ideas (negative qualities) Descartes’ drew from his observations of the automata in the royal gardens of Saint-Germain-en-Laye, to the modern and constantly reinforced erroneous ideas that cognitions are “processed” (negative habit), reversing the direction of the analogy (positive habit) requires the persistence and patience (positive quality) of those who know not only that materiality and mentality uphold each other as a unit, but also that that unit makes intentional choices inexplicable to mechanistic “laws” of viewed through the models of modern materialism.

62

New study just out from MIT:

They had people write philosophy essays either with or without the aid of ChatGPT while strapped to an EEG monitor.

People who used ChatGPT showed less brain connectivity and couldn’t, in the post-interview, recall anything “they” had written.

7 Likes