News:

Look, I haven't mentioned Zeus, Buddah, or some religion.

Main Menu

All things AI

Started by Dave, May 21, 2018, 04:03:59 PM

Previous topic - Next topic

Asmodean

Quote from: Tank on June 11, 2024, 10:29:07 AMWhat could you offer an AI that it could possibly value? How would you bribe/influence it?
Electricity. The continued, unrestricted access to global networks.

...Unlimited powah. :smilenod:

Ok, that's a "buzzword answer." To account for a few more variables, the AI would have to grease palms, not just have its own greased, and if working towards a goal, its palms could be greased by something that can be shown to in some ways advance that goal - quite probably at the cost of other goals it ignores or prioritises lower than whatever-it-may-be.

Yes, you can huwheel and deal "in binary." ;)
Quote from: Ecurb Noselrub on July 25, 2013, 08:18:52 PM
In Asmo's grey lump,
wrath and dark clouds gather force.
Luxembourg trembles.

Tank

One of the reasons a large number of Conservative MPs did not want Rishi Sunak to be Prime Minister was because he was independently wealthy and as such un-bribable. An AI would have no need for money nor need to use it. It would be able to hack any system it liked to achieve any end it desired would it not?
If religions were TV channels atheism is turning the TV off.
"Religion is a culture of faith; science is a culture of doubt." ― Richard P. Feynman
'It is said that your life flashes before your eyes just before you die. That is true, it's called Life.' - Terry Pratchett
Remember, your inability to grasp science is not a valid argument against it.

billy rubin

what would cause an AI to develop desires? for anything?


its a fucked up world. what do get? sex and love and guns light a cigarette

Tank

Quote from: billy rubin on June 27, 2024, 10:35:13 PMwhat would cause an AI to develop desires? for anything?

Good question.
If religions were TV channels atheism is turning the TV off.
"Religion is a culture of faith; science is a culture of doubt." ― Richard P. Feynman
'It is said that your life flashes before your eyes just before you die. That is true, it's called Life.' - Terry Pratchett
Remember, your inability to grasp science is not a valid argument against it.

Recusant

These systems don't just "hallucinate" facts. They also give the appearance of understanding a topic while in reality they don't even approach actual understanding. A description and name for that particular variety of AI (LLM) failure-- "potemkin understanding."

"AI models just don't understand what they're talking about" | The Register

QuoteResearchers from MIT, Harvard, and the University of Chicago have proposed the term "potemkin understanding" to describe a newly identified failure mode in large language models that ace conceptual benchmarks but lack the true grasp needed to apply those concepts in practice.

It comes from accounts of fake villages – Potemkin villages – constructed at the behest of Russian military leader Grigory Potemkin to impress Empress Catherine II.

The academics are differentiating "potemkins" from "hallucination," which is used to describe AI model errors or mispredictions. In fact, there's more to AI incompetence than factual mistakes; AI models lack the ability to understand concepts the way people do, a tendency suggested by the widely used disparaging epithet for large language models, "stochastic parrots."

[. . .]

Here's one example of "potemkin understanding" cited in the paper. Asked to explain the ABAB rhyming scheme, OpenAI's GPT-4o did so accurately, responding, "An ABAB scheme alternates rhymes: first and third lines rhyme, second and fourth rhyme."

Yet when asked to provide a blank word in a four-line poem using the ABAB rhyming scheme, the model responded with a word that didn't rhyme appropriately. In other words, the model correctly predicted the tokens to explain the ABAB rhyme scheme without the understanding it would have needed to reproduce it.

[Continues . . .]

A preprint version of the paper is available.

"Potemkin Understanding in Large Language Models" | arXiv

QuoteAbstract:

Large language models (LLMs) are regularly evaluated using benchmark datasets. But what justifies making inferences about an LLM's capabilities based on its answers to a curated set of questions? This paper first introduces a formal framework to address this question.

The key is to note that the benchmarks used to test LLMs -- such as AP exams -- are also those used to test people. However, this raises an implication: these benchmarks are only valid tests if LLMs misunderstand concepts in ways that mirror human misunderstandings. Otherwise, success on benchmarks only demonstrates potemkin understanding: the illusion of understanding driven by answers irreconcilable with how any human would interpret a concept.

We present two procedures for quantifying the existence of potemkins: one using a specially designed benchmark in three domains, the other using a general procedure that provides a lower-bound on their prevalence. We find that potemkins are ubiquitous across models, tasks, and domains. We also find that these failures reflect not just incorrect understanding, but deeper internal incoherence in concept representations.
"Religion is fundamentally opposed to everything I hold in veneration — courage, clear thinking, honesty, fairness, and above all, love of the truth."
— H. L. Mencken


Dark Lightning

"Potemkin understanding". I like that! I talked with another person who posited that we'll only have to worry when the machines learn "artificial sapience", and I can see his point.