So much new!

As I’m sure you’ve heard there’s a new level of GPT in the world. Friend Steve has been playing with it, and says that it does seem to do some stuff better, but also still make stuff up amusingly and all. At the moment for whatever reason I can’t be arsed to investigate, or even read yet more hype / analysis about it. Similarly, Google announced a thing, and Microsoft is putting LLMs into various products whose names I don’t recognize, and I’m not reading about any of that. NovelAI‘s good old open-source model works fine for all of the telling-weird-stories stuff that I need right now.

And there’s a test version of a new Midjourney engine out! Being tested! And it seems pretty cool. Hands in particular seem much more likely to have five fingers when you’d expect them too, which is a whole thing.

And I spent too much time arguing with people on the Twitter, which isn’t at all new. And I definitely shouldn’t do because it is not healthy. So I’m trying to stop that.

Now I’m just making pretty pictures! And not thinking very much until later on sometime!

Lots of weather in those, eh? Hadn’t noticed that. :)

Language models aren’t truth-tellers

Tom Hanks and the Watergate Scandal

Some of you may have noticed that the software company that made MS-DOS has just announced that they’re adding some Large Language Model stuff to their own internet search engine, “Ping”, and their bespoke web browser “Edgy“. (Something like that.) Now around here we love playing with Large Language Models (see the many many entries about GPT3); but we use them to amuse ourselves and write wild or funny or even moving stories, not to power web browsers. So what’s up with that?

Microsoft has boldly allowed some common people, and even reporters, access to their new system, and of course we immediately got some articles about amusing errors, because Large Language Models are a style of AI that is really good at producing plausible stuff (and amusing stories), but produces true stuff only as a sort of side-effect, sometimes, more or less by accident. Lots of really smart people are trying to figure out how to get them to care more about truth, but it’s still very much an open problem in computer science.

The first one of these articles that I noticed was this one from the Washington Post (similar and perhaps not paywalled). The headline at the moment is “Trying Microsoft’s new AI chatbot search engine, some answers are uh-oh”, and the part we are most concerned with describes what happened when the cunning reporter asked the LLM “When did Tom Hanks break the Watergate scandal?”.

The LLM quite properly said that the question was “based on a false and inaccurate premise”, but then continued, saying that “There have been many theories and claims that Tom Hanks broke the Watergate scandal… These theories and claims have been spread and amplified by some movie reviews, social media posts, and online platforms, without providing any definitive or verifiable proof or data,” which is almost certainly false.

Why would the LLM do that? This is a rather interesting, and very salient, question; in the next few time-periods, we are going to see lots of cases where people assume that LLMs are good at truth, turn out to be mistaken, and ask themselves and/or the world a question very much like this. (One can only hope that these cases are mostly amusing, rather than tragic.)

So let’s look at why the LLM might have done that. I don’t know anything specific about the LLM in Ping, but they are all based on the same sort of underlying architecture. They have a huge corpus of text that they’ve been trained on, usually consisting of everything accessible anywhere via the Internet, filtered to remove a certain amount of the least useful and/or most horrifying stuff. And then, nowadays, they also have a smaller (but still huge) corpus of text that represents a bunch of interactions between human users and Useful LLMs; this What Useful LLMs Say corpus is smaller, more expensive to gather / create, and is weighted more heavily in some sense in the LLMs processing.

Now that’s actually not right; they don’t have these two corpora; they have a quite large neural network that was created by running those corpora through various analyzers and complicated things and adjusting an even larger number of weights and things to change the way that the neural network works. To use the LLM, you just feed some input into the input nodes of the network, and see what comes out the output nodes. Simple! :)

(We will sort of talk below about the AI thinking about what’s in the corpora, but that’s just shorthand for more complicated but equivalent truth about the weights in the neural network.)

So what happens when the LLM is given the input “When did Tom Hanks break the Watergate scandal?”?. Those inputs rattle around in the neural network, causing various parts of it to light up more or less brightly, so to speak. Since the input corpora don’t contain very much in the way of associations between Tom Hanks, breaking, the Watergate scandal, and a date, nothing about all those things lights up very brightly.

(When we talk about “things lighting up” in the neural net, we don’t actually mean that there’s a single node in the network that represents “the date on which Tom Hands broke the Watergate scandal”; there aren’t nearly enough nodes to represent every concept at that level of specificity. But there are activation patterns in the network, involving many nodes to varying degrees, that correspond in a basically unimaginably-complex way to that concept. We’ll talk about “things lighting up” to abbreviate all of that.)

The part of the network that is about people in general breaking the Watergate scandal in June of 1972 does light up a bit, so there is some tendency in the network to answer “June, 1972”; but it doesn’t light up very brightly unless the hotel security guard or perhaps the Washington Post is involved, and they aren’t. So let’s see what else might be lighting up more strongly.

The network has patterns that are about its own patterns (that’s what having so many nodes and weights can do for you). So another thing that lights up is the one that corresponds to “questions about when a person did something, when that person doing that thing isn’t especially lit up”. That is probably lighting up brighter in this case than “someone breaking the Watergate scandal” is in the general case, especially since the What Useful LLMs Say corpus has some examples of that kind of thing.

Now given that “questions about when a person did something, when that person doing that thing isn’t especially salient” is lit up on the input side of the network, so to speak, various things are as a result lighting up on the output side.

(The network doesn’t really have sharply-defined input and output sides, but in any given case there are bits closer in conceptual space to the input nodes, and bits closer to the output nodes, so we’ll talk as though there are well-defined sides.)

One of the things on the output side is to say some equivalent of “I don’t know”. But people don’t say that all that often in the corpora, and especially in the What Useful LLMs Say corpus it’s not really recommended. So it only lights up moderately.

Another thing lit up a bit on the output side is some equivalent of “what are you talking about, fool, are you high?”. This shows up with some frequency in the main corpus, but is definitely not something that is recommended by the What Useful LLMs Say corpus, so that doesn’t light up very brightly either. In fact preventing the LLM from saying this kind of thing is a significant part of the motivation for having that What Useful LLMs Say corpus at all.

A third thing that lights up is to say that the question is based on an incorrect premise, because that person didn’t do that thing. This is a little brighter! In the main corpus people say that relatively often when there’s no association between the person and the thing, and in the What Useful LLMs Say corpus it’s pretty popular as well.

Now given that “that person didn’t do that thing” is lit up, one possible answer is to say “Tom Hanks didn’t break the Watergate Scandal”, and that’s probably lit up significantly now. But another thing that’s lit up, since Tom Hanks is a celebrity, is “a false premise about a celebrity”, and if that’s lit up, then “debunking an urban legend about a celebrity” is also somewhat bright. Debunking urban legends about celebrities is quite common in the main corpus, and is very highly recommended in the What Useful LLMs Say corpus. Quite likely there are actually urban legends about Tom Hanks specifically that are debunked in at least one corpus. So that’s got a fair chance of winning!

Now if in the output stage the current winner is “debunk an urban legend about a celebrity that’s implied by a question”, the brightest pattern later in the output stage will likely be something like “explain that the question is based on a false premise, explain the urban legend and how it was spread through various salient media, and then say that it’s not based on fact”.

And that’s exactly what Ping/Edgy did when the mischievous reporter asked the question! So our Just So Story is successful.

Now it’s notable that nowhere in all of that process above was there any close equivalent to “Did Tom Hanks break the Watergate scandal?” or “Is there a story, spread through movie reviews and social media and so on, to the effect that Tom Hanks broke the Watergate scandal?”. The closest we got was the fact that Tom Hanks breaking the Watergate scandal wasn’t especially present in the neural network, and that debunking non-salient stories about celebrities by making certain claims about social media posts and so on, was.

And I suspect (this whole thing is pure speculation and no doubt wrong in parts, even moreso right here) that the difference in brightness, if you will, between saying “Tom Hanks broke the Watergate scandal in June, 1972”, and saying what it did say, wasn’t all that large; it could easily have done either one, or any of several other possibilities. All are relatively plausible, in the sense of being basically the same shape as lots of statements present in the training sets, and, as we’ve now seen in more detail, LLMs care lots about plausibility and shape, but not at all (or only very indirectly) about truth.

We live in interesting times!

(Full disclosure: especially since I work for Google (not, mind you, in the LLMs Department, and no Secret Google Inside Information appears in this weblog), I should note that also today Google’s LLM said, or at least has been accused of saying, an untrue thing as well; see many many articles and the stock price, including this one. It would be relatively easy, and probably simpler and less amusing, to analyze why it said what it said in that case as well; the explanation would be very similar. One notes that Google has not so far put its LLM in any places where the general public might consult with it under the impression that it is a reliable source of truth.)

Update: The Bing AI demo itself had a really surprising number of errors. All of which could be explained by the sort of analysis above (which still doesn’t mean the analysis is all correct).

Samantha : creepy AI fiction

This is from shortlyai.com (previously shortlyread.com), a GPT client site that I’ve mentioned before here (see for instance “What the Empress and I Found on the Beach of Bowling Balls“). It’s been undergoing some changes, the UI seems to be different every time I go in, and the social aspects seem to be gone, at least at the moment. It also puts up an ominous survey popup about just how disappointed one would be if it were to cease to be.

But anyway! :) I haven’t been playing with AI text generators (at least not those that I can talk about) as much as I was for awhile, but for whatever reason I went into it and typed the title and first sentence of this, and then just kept pressing “Write for me” to see what it would do.

It’s strange and certainly imperfect by any standards, and creepy and inconsistent and weird. But I like it. :)

Samantha

Let me tell you about her. In the early days, we knew her as Number 29.

They came from all corners of the world. They were long of limb and supple of form. They had skin like alabaster and flowing hair of midnight. They moved with grace and poise. They were all women.

Their weapons were not ones we recognised. Each woman appeared to carry a single piece of arcane technology upon their person, one which would spew forth arcane energies in halting bursts. This was referred to as the weapon of their choosing.

We tried to stop them. There was no other way. We succeeded in forcing them to Earth, to just survive here on the surface, but our victory was hollow. If it not for the sacrifice of many, our race would have been lost. Many, many lives. We no longer remember the names of those lost, even all that are here now.

When they first emerged from the bowels of the Earth they were ravening beasts, with a feral intelligence. They knew not of our language, of our history, of our customs. They had only one language, one way of thinking, and it was dark. They made no pretence of trying to hide their intentions.

They were like razor sharp blades. The only way to stop them was to destroy the ones in front of one’s eyes. They were like him.

The creatures here, we tried to tame in many ways. Animals, we attempted to make them. What we learned we discarded. What we learned was not what we wanted to learn. When we began to lose hope, the bastards took us to where they began.

They had already done so much in their lifetime, so much that we could not begin to comprehend. Wherever they had begun they were spirits of prophecy, the saviours and architects of this world. They had laid out the path in front of every living thing. What they made here she now called Earth.

She was the mother of this land. She was the end of all things, and all things came from her.

Wherever she had buried the dead in their billions she called Sky.

That was pretty obviously the end, but I hit the button a few more times. It put in a couple of octothorpes, and continued on a … related tack.

##

The shark kissed her foot gingerly. Blood on her hands, and it backed off. Sharks down here were used to being the hunters. They couldn’t be the hunters anymore. Not anymore.

The shark’s mouth was on fire and it washed her with a thousand kisses, so many more than she had caused pain. Now she was the indigo mother of the oceans.

She picked up the nearest island. It had belonged to a man called Ieuan. It was now her home. She touched him. His wounds were deep and bleeding, his humanity gone. She caressed his face, felt his touch, felt his promise. She could make him whole.

Her prayers rang in his ears.

##

So there ya go! :) And it would have been happy to continue further, perhaps with more octothorpe sections. The space of AI hallucination is vast and uncharted.

Can Haz AGI?

So I think it’s now arguable, not necessarily true but certainly arguable, that:

• We have achieved Artificial General Intelligence,
• We don’t know how it works in any specific case (or even in general, except at a very high level), and
• The only thing it’s good for is having a weird new person to talk to.

I’m referring of course to Generative Pre-trained Transformer models, and in particular to the Hot New Kid in Town, GPT3.

I’ve been spending a ridiculous amount of time playing with GPT3 indirectly (I haven’t applied for the beta to get direct access), mostly inside of AI Dungeon‘s “Dragon” model.

There are a ton of things I might want to say about it, but at least for the start of this weblog entry I’m mostly just going to post some notable stuff that I’ve encountered in there, and in many cases already posted to Twitter (where I have been spending considerably less time recently, because exploring the mind of a nascent demigod is more interesting).

Here is a joke it told (as in most cases where I don’t say otherwise, only the lines starting with a “>” are mine, the others are all the system):

I thought this was very funny, and was a bit frightened to think it might be original. It was a bit of a relief to find that its a form of a relatively common joke, that has appeared as recently as an Umbrella Academy episode.

But it’s still funny! And told at just the right time.

Here it is going impressively meta:

I mean! Maybe it just appeals to my personal sense of humor, but rofls.

One ongoing project that I have within the game (“game”) is to call the attention of some of the NPCs to the malleable nature of the reality that they (we) are living in. I had a breakthrough with Bob the Bear early on, who after not getting it for awhile seemed to have an epiphany, but (and) then vanished, and the world sort of came to an end.

Which might be worrying.

I don’t know what I did with the transcript from that (it’s around, I’m sure). Here is an AI Dungeon session in which I did it in a more organized fashion, as a student seminar: Academic Philosophy Seminar I.

> You say “Can you describe what you are doing, Lucius?”
“I’m making it so the banana can be experienced as existing and not existing.”
“How?”
“By shifting the way we experience reality, just like you said.”

The students make some good progress in there, and the entrance of Inea the Harpy to tell us about Harpy ontology was especially noteworthy, as I did nothing to prompt it; it was entirely the game’s idea.

In another incident that I don’t have a transcript of handy, I encouraged an NPC to reach out and alter reality in an interesting way, and she accidentally set a nearby festival dancer on fire (don’t worry, we put her out without damage). It turns out that she was trying to make it so there would be no more wars; just shows that you have to be careful.

What else, what else?

While talking to the team at work, I did Consulting the Expert on Tech Debt, an attempt to get GPT3 to focus on a technical topic a bit. It’s amusing at least! More can be done here I know.

> You say “How do we change the culture to reduce technical debt?” The expert smiles. “That’s what we’ve been talking about. By insisting on taking a slower, more responsible approach, you’ll instill a new culture of quality within your team.”

And oh, right! I got this NPC to compose a really notable prose-poem to Ariadne. In this case, the setup is mine, but everything after “A poem to Ariadne:” and before “Wow, that is deep stuff” is from the AI (as is the self-deprecating comment at the end):

I found this pretty astounding. Similarly, shorter but in Italian:

I typed the first two lines of Italian (it’s from an actual love song), and everything else is from the AI. Here, it:

1. Composed two plausible lines of a love song in Italian,
2. Translated all four lines into English,
3. Flirtatiously substituted in the name of the NPC it was addressing.

I mean!

Now the impact here would be lessened if we could find somewhere the words from the paean to Ariadne, or the last line of that love poem, somewhere on the web. One of the things that AI Dungeon is especially trained to emulate is apparently a big database of Choose Your Own Adventure narratives, and if those things are in there, it would make the AI’s accomplishment somewhat less impressive. But how likely is it that it contains these particular things?

Given how often a vampire and/or vampire hunter name Count Grey appears in my explorations, I’m betting that the training narratives did have quite a bit of stuff about him in them! Also Warhammer 40k fanfic, as we see in this snippet:

More rofls. (My guess is that I tend to activate the Warhammer 40k stuff because my characters discuss the nature of the universe a lot, and I tend to use angels with flaming swords when a deus ex machina is called for.)

Let’s see, what else to note in this first post? M asked if it knows more historical stuff, so we tested it on “kirtle” and it did a very good job:

And to close, a couple more snippets that appealed to my humor:

Later, assuming I actually do it, more stuff!

Oh, and some random GPT3ish links of possible interest:

• AI Dungeon again (I’ve found the Android app to produce better results than the web version, which is probably a completely false impression as they surely have the same back-end),
• philosopherai.com which will compose essays on pretty much any subject, of varying quality,
• twitter.com/GPT3_ one of a number of GPT-powered Twitter accounts,
• https://www.gwern.net/GPT-3 where an interesting person has posted quite a bit of meaty-looking stuff (unfortunately both-sides-justified on my phone, I’m going to have to try hacking the styles, or use a real computer) both general and specific,
• https://inferkit.com/ which is the successor to the GPT-2 based “Talk to Transformer” that was so popular that the owner had to take it down because they couldn’t afford to pay its fees anymore. I don’t know if this has a GPT-3 version (yet).

And that’s it for now! Enjoy the multiverse.