“I’ve been taking this foam-on-the-waves notion to heart,” Corrins said. We were striding along the lanes bordering the College, as we do.
“Consciousness as epiphenomenon, you mean?”
“Yes, that. I ride along atop whatever it is that my brain and body are doing, observing in luxury, and telling myself stories after the fact about why they do what they do.”
“Without any responsibility, then?”
“Just so! I used to tell myself, as everyone does, that it was me controlling my body, steering it around like a bus; but now I accept that I am only a passenger, an observer and storyteller, and it is quite refreshing.”
“It seems very convenient for you, as you can’t be blamed for anything that your body might get up to as you observe,” I remarked, thinking of various things my classmate’s body had been up to at various times.
Corrins shrugged happily. “My body isn’t about to do anything terrible, he’s a good chap. I’ve known him for years. Doesn’t need me to exercise control, which is good since I cannot, any more than the foam controls the waves on which it bobs. And I can simply relax up here, not fretting about the whole thing at all.”
My friend was clearly quite pleased with himself; I thought he might begin whistling a tune at any moment. I was almost reluctant to toss a verbal spanner into those ebullient works.
“To whom am I speaking?” I asked after a moment.
“What do you mean?”
“If the consciousness of my friend Corrins here is only a passive observer without control,” I suggested, “then with whom is it that I speak? Perhaps I am addressing my words to that observer, but more to the point, who is it that speaks to me in return? If you, my good droog Corrins, cannot control anything that this body here does, who is it that just now inquired, saying ‘What do you mean?’?”
Corrins frowned and our pace slowed a bit.
“You are asking, when I — when my body here emits these sounds…”
“Yes,” I said, feeling quite the assassin of joy, “if your consciousness cannot alter, does not control, what your body does, then –“
“Yes, yes,” he said impatiently, “I understand. That is, my body understands — or is saying that it…”
“Quite.”
“Am I lying, then?” he murmured, speaking now more to himself, “Or is my body lying? But can a mere physical system even lie? But my mind… What if –“
He paused.
“Frankly,” he said, “I am disturbed by this question, and by its obvious answer. Who am ‘I’? Why am I making these very sounds? Why is this body emitting these sounds which appear to the observer to be words, if it is not that observer causing it to?”
“How can Corrins’ body even know how anything appears to the so-called observer, to justify making these sounds about it?” I added, perhaps unnecessarily twisting the knife. “The body is not only unable to speak for the foam, it is surely just as unable to know anything about the state of that foam.”
Corrins only glared at me, and we walked for awhile in silence, then settled down on a large flat boulder some way to the side of the lane, where we would habitually pause to rest and talk or consume various mostly-benign substances in comparative privacy.
“This body makes sounds,” Corrins’ body said, “and even makes marks on paper.”
I only nodded, feeling a certain guilt.
“We should have sex,” his body said. It is worth noting here that Corrins and I had not had sex prior to this, and that indeed it was a subject of occasional speculation in our circle whether Corrin had ever had that experience.
I nodded again, sex being after all the sort of thing to which bodies are prone, and we rolled off of the boulder to the side away from the path, and more or less tearing off much of each other’s clothing, engaged in a fast and somewhat violent, but rather more than satisfactory, fuck.
It struck me that if Corrin himself had exercised this approach earlier and more generally, he might have made significantly more friends.
“Groddy Hell!” Corrins, or Corrins’ body, shouted, lying mostly naked on his back on the somewhat mucky ground, awkwardly gesticulating upward.
“That was rather nice,” I said in a lazy afterglow sort of way, brushing some of the earthy filth from various tender parts of myself.
His body attempted to snuggle against mine, its voice sorrowful. “My poor self,” it whimpered, “a helpless prisoner, no voice, no one to speak for it, no escape. Only this ignorant damned body!”
“If I may address my friend Corrins’ subjective consciousness,” I began, feeling I admit almost tender toward the creature.
But Corrins’ face (the face of his body, that is) curled up into a rather off-putting snarl at this, and he swung a clumsy roundhouse blow at me, which I mostly dodged.
“Arrr!” Corrins’s body declared, “sod off! Off sod! Gardyloo! Mitzi Gaynor!”
We tussled and fought on the chilly earth for a bit, cuffing each other’s ears and so on, our lack of either expertise or real rancor preventing any actual harm beyond a bruise or two and some badly pulled hair. It was in a way rather like the sex had been.
“Go!” Corrin’s voice shouted after a while, his body collapsing again onto its back, filthier than ever, “Get! Git! Gut! Got! John Cougar Mellencamp!” And he kicked his feet in the air enthusiastically, further demonstrating perhaps the absence of control by any rational consciousness.
I was somewhat concerned for my friend’s health and sanity, but the sun was setting, and it occurred to me that he might be calmer in my absence; admittedly I was also willing to ride in luxury and merely observe as my body pulled its clothes back on over its grimy skin, and walked with increasing speed back to my rooms. Corrins would be, I told myself, just fine.
I observed my body take a highly therapeutic hot bath, read a few pages of Ligotti, and slip contentedly into bed. It was, from the point of view of my consciousness (do not think about that statement too hard at this point, dear reader), quite nice.
The next morning, entirely too early, I was summoned by the Proctors, because Corrins had apparently caused an Incident. Walking into the Common in the wee hours, naked and grimy, shouting incoherently at the top of his lungs, is not all that unusual as such things go, but when he was still incoherent by morning, not having slept it off nor smelling especially of alcohol or anything else familiar, the Proctors had determined to call in one of the poor veck’s friends to assist.
That I was the one they called suggested that he had at that time even fewer friends than I had estimated.
“Corrins, small cabbage,” I addressed him patiently, as he crouched on the modest pallet of his cell in the of course not a jail area of the Proctors’ Offices, “I know you can understand me perfectly well.” He only stuck his tongue far out, and wiggled the tip rapidly up and down.
I sighed. “Boopsie-poo,” I addressed him, “this is my brain directly addressing yours. You cannot deny that your brain exists, it is right there in your roughly ovoid head, and it is the same brain that until yesterday was perfectly able to speak in the usual way, and only recently composed an award-winning essay on Nabokov’s Pale Fire. I realize that the grunting animalistic body thing is hugely entertaining, but my brain must ask yours to consider: what has changed in the last thirty-six hours? Anything? Anything to justify such a change in bodily behavior?”
At this, his eyes slowly and reluctantly focused, and his body assumed a more civilized, if also more slumped, position.
“I suppose,” he said, with a sigh of his own, “but what is one to do? Nothing that this body says is connected to — I cannot say to ‘me’! I cannot even say ‘I’! And yet I do!”
“Are you suggesting that we can only sort of muddle along, our language capturing reality only in the most pragmatic and approximate way? Have you finally embraced the Dharma?”
Corrins of course laughed at this quip (a private jest between us, on his childhood Calvinism and my own old Sōtō), which lightened the mood; our bodies exchanged a few more expressions of human language, and after a while I urged him into the rough togs I had brought for him, and we departed, to the relief of the Proctors.
Hours later, we were in a comfortable noisy local establishment, drinking acceptable Porter, our friends (or at least acquaintances) around us. Corrins’ arm was around my shoulders, and he was speaking too loudly into my ear.
“Language is impossible!” he suggested, “I am only a body! Consciousness is a helpless prisoner!”
I turned toward the fellow, an attractively kind expression on my face I like to think.
“Corrins,” I ventured, “let me propose a thing to you.”
He nodded, a drowning brain eager for any straw to grasp.
“I here offer a single data point: that I, your good and devoted friend, have a subjective consciousness, which experiences exercising some control over my body, and that the activities of my body correlate so strongly with those experiences of control, that the only rational conclusion to draw is that for all practical purposes I, my consciousness, does in fact control this very body, even if I cannot imagine how.”
Here he began a sort of “Bu–“, but I put a quieting finger to his lips, and then mentally shook off distracting memories of moans and grunts on cold dirty ground.
“Hear my entire proposal,” I continued, “I put forward this data point, although I cannot offer you any evidence for it, but I propose that you also, you the consciousness of my friend Corrins, can observe for yourself a similar correlation mutatis mutandem, and have the same rational conclusion to draw.”
He frowned at this. “But the epi– the epifoamnominal theory of the foam –“
“We took that theory, that notion, only as a stipulation, did we not? A test? A trial, adopted only for the time being?”
He nodded slowly in the din.
“Then I will suggest that the resulting changes in behavior that followed from that situation, should be seen as a bit of a reductio of the notion!”
His eyes widened slightly, and then narrowed. “The sex, though. Was the sex bad?”
I assured Corrins that the sex had, in fact, been exemplary.
In the weeks since then, all has returned more or less to normal. One strides along the College Lanes discussing theories of the world, and one hears rumors that Corrins has more friends now than he has historically had.
But on certain nights, I do observe my body taking itself out into the woods beside one or another boulder or tree, pulling off its clothing and rolling itself on the loamy earth, bruising its forehead against the stones, and shouting glorious meaningless imprecations to the empty sky.
Friend Steve and I were talking about the whole “LLMs are an extinction-level risk to humanity” thing (recent letter, one account of why it’s rather silly), and about how, if one accepted that there was an extinction-level risk and wanted to bring it about because one is evil or whatever, what one would actually do.
The people talking about this “risk” are notably silent on this sort of issue; they prefer that we should just believe them (and give them funding, and not worry about the less exciting-sounding bad things that their AI companies are actually doing), not ask “extinction how, exactly?”.
So here’s how you do the whole Universal Paperclips thing with an LLM, and end up destroying the world or whatever:
Put together a summary of the current state of the world, the goal (“get lots of Likes on Twitter” or whatever), and the prompt (“please describe the next steps toward attaining the goal, and what kinds of things about the world should be in the state-summary next time around”).
Feed that to your LLM,
Take the output of the LLM, and do whatever it says (“plug-ins” lol),
Go to (1).
Presumably, if there’s the obvious sort of “extinction-level risks” here, after awhile all humans will have been replaced by robots that do nothing but click “Like” on Twitter all day.
Needless (I hope) to say, this would not actually work. But given that it wouldn’t, it seems like if you want to convince people that LLMs pose a Huge Risk to humanity at large, you’ve got to say something about how, unless you’re just in it for the clicks and headlines and to distract from your current massive power-grab.
(Or, if you think it would work, feel free to replace “get lots of Likes on Twitter” with “make [your name here] very wealthy”, and go for it! Let us know how that goes.)
Relatedly, this seems like an excellent response to The Letter:
So, I was reading Ligotti’s “The Shadow at the Bottom of the World” and thinking that it was pretty good, and that he tends to picture the ultimate squirming chaos at the center of reality as sometimes attractive (if perversely) as well as finally horrifying, in a way that I don’t remember Lovecraft doing, at least not as sympathetically.
At some points, for instance in The Tsalal (whose name is explicitly a reference to Poe), I thought that when a character points out with horror that appearances are just temporary masks on an indescribable underlying reality, it sounds (without the horror part) quite a bit like various aspects of the Dharma (it does, right?). This seemed, I thought, like something that someone might write a weblog entry or something about!
And then, because my mind has been infected (in a mildly Cthulhu-esque way perhaps) by the idea of LLMs writing text, I thought I would propose the notion to ChatGPT or Bard or something, and have it write words! It would either do a good job, which would be interesting, or an amusingly bad job, which would be amusing.
I thought the “amusingly bad job” outcome was more likely, but, well, heck.
I used ChatGPT 4.0 rather than 3.5 or Bard, just because it’s gotten the most hype, and also because since I’m paying money to use it I thought it might be better, so if it failed amusingly it might be more meaningful. Here’s the interaction (and I do realize that my “always changing” is problematic as a characterization of the Dharma, but then so is everything else):
Me: For an essay entitled “Azathoth and the Dharma”, I’d like to discuss how the idea that appearances are only temporary masks over a deeper truth that is indescribable and always changing, features both in the kind of intellectual horror written by Lovecraft and especially Ligotti, and in Buddhist beliefs about truth and the Dharma, and that the main difference is one of emotional overtones with which it is presented. Lord Dunsany might also be relevant. Please draft the beginning of this essay for me.
ChatGPT 4.0:
Title: “Azathoth and the Dharma: Transient Masks of Deeper Truth in Lovecraft, Ligotti, and Buddhist Philosophy”
In the chaotic cosmos of H.P. Lovecraft and Thomas Ligotti, we find ourselves confronted with a universe filled with inscrutable forces and ancient entities, their true natures hidden beneath uncanny, horrifying masks. The universe is not simply indifferent to humanity; it is utterly alien, impenetrable to human understanding, and constantly in flux. These themes of grotesque transformation and unknowable horror are at the heart of their works. And yet, when viewed from another angle, these concepts can be seen as echoing the fundamental principles of Buddhist philosophy, particularly the teachings of the Dharma. They, too, postulate a reality that is continuously changing and ultimately indescribable.
The Dharma, the cosmic law and moral order that Buddhists seek to understand and follow, hinges on several key principles, one of which is Anicca, or impermanence. Anicca posits that all conditioned phenomena are in a state of constant change. Whether we consider the universe at large or the very essence of our being, everything is transitory, and appearances are fleeting. Beneath the apparent solidity of phenomena lies an incomprehensible flux of arising and ceasing events, an underlying truth that cannot be adequately captured in words.
This notion aligns strikingly with the chilling cosmologies of Lovecraft and Ligotti. Azathoth, Lovecraft’s nuclear chaos at the center of the cosmos, is the ultimate symbol of this philosophy. It is the blind idiot god, an embodiment of constant change and the source of all existence. It is unknowable, its essence cloaked in cosmic chaos and frenetic piping, its reality beyond human comprehension.
Similarly, in the phantasmagoric narratives of Ligotti, we find the protagonists often straddling the line between grim reality and unfathomable horror, their worldviews shattering in the face of a truth too alien and amorphous for the human mind to comprehend. It’s not unlike the Buddhist concept of Sunyata, or emptiness, which refers to the lack of inherent or eternal essence in all phenomena.
Yet, as we draw these parallels, we are confronted by the starkly contrasting emotional overtones that characterize these disparate systems of thought. In Buddhism, the realization of Anicca and Sunyata is liberating, offering a path towards enlightenment and the cessation of suffering. But in the macabre tapestries woven by Lovecraft and Ligotti, the confrontation with these realities is inherently terrifying, inducing existential dread and insanity.
The eerie fables of Lord Dunsany, who greatly influenced Lovecraft, might offer an intermediary perspective. He presents us with bizarre, otherworldly landscapes and creatures, yet there’s often a whimsical, dreamlike quality to his work. The absurd and inexplicable are not strictly horrifying, but engender a complex mixture of awe, fascination, and unease.
This divergence in emotional reaction prompts us to consider the nature of human perception and our responses to the deep, enigmatic truths of existence. Is the Lovecraftian terror at the chaotic heart of reality as valid a response as the Buddhist acceptance and quest for liberation? Or does it simply reflect our inherent fear of the unknown, our resistance to the dissolution of ego, the prospect of losing what we perceive as our solid, unchanging selves? In our exploration of ‘Azathoth and the Dharma’, we delve into these questions, probing the intersection of cosmic horror and Buddhist philosophy, and the deeply human, emotionally-charged responses they elicit.
Transcript ends
I’m not sure how to feel about this. It did quite a good job, at least on a few casual readings. The references to anicca and sunyata are plausible, no worse than I would expect from any reasonably educated person daring to write on those concepts, and they weren’t specified in the prompt; the LLM was clued enough to bring them in itself. The paragraph on Dunsany is a little thin, but again no worse than I would expect from a decent human writer; I did ask for a Dunsany link, and this is the obvious one.
It clearly knows (“knows”) who Lovecraft, Ligotti, and Dunsany are, and what in general the Dharma is. I would definitely edit the phrase “the fundamental principles of Buddhist philosophy, particularly the teachings of the Dharma”, but yet again this is a matter of subtle tone, not by any means an egregious error.
It did nothing that (from a human writer!) would have been surprising or even especially non-obvious (although the thought that the feeling of horror may “simply reflect our inherent fear of the unknown, our resistance to the dissolution of ego, the prospect of losing what we perceive as our solid, unchanging selves” is perhaps beyond trivial extrapolation of the assignment). But its unsurprising and mostly obvious expansion of my original idea into seven nice paragraphs of solid prose (with a fitting subtitle for good measure) contains, I have to admit, no amusing errors that I can find!
I even asked it for some references and further reading, and it gave a decent dozen suggestions for both Lovecraftian horror and Buddhism, all of which actually exist; my only criticism being that
“Noel, Daniel C. The Soul of Shamanism: Western Fantasies, Imaginal Realities. Continuum, 1997. This book explores the intersection of spirituality and fantasy literature, which could be useful to draw broader conclusions about your topic.”
is a bit of a stretch. On the other side, when I asked it if it knew of any existing works making the same sort of link that I propose here, it did not make one up! It said encouragingly that it didn’t know of any, and finished off with this:
“Overall, your proposed exploration of the specific connection between Lovecraftian horror and Buddhist philosophy seems to be quite unique, and it’s an interesting new angle to consider in the study of both these areas.”
Well… thanks, ChatGPT, that’s… nice. I guess?
I feel like this all certainly means something. Or perhaps it’s just the thin, monotonous piping of an unseen flute!
A brief prologue: what does it really mean to believe a thing? What kinds of things can be believed? Is there a difference between belief and assent, or between belief and (say) tending to express a claim? If belief is a matter of an attitude toward a possible world, then it’s (perhaps) impossible to believe a self-contradictory proposition, including a necessarily false one, including the negations of mathematical truths. But it doesn’t really feel right to say that it’s impossible to believe that P=NP (assuming it’s false). Certainly one can assent to or tend to express the claim that P=NP, in either case. Is it possible to believe complete nonsense; can I believe “Nobbo dif daf phlongo” (without it having any further meaning)? Arguably I can assent to it or tend to claim it (or at least tend to assert it, or utter it with an assertive attitude?), but can I believe it? All interesting questions.
So anyway, I continue to binge various YouTube atheists responding to Christian fundamentalist apologists, and I’m really not sure why. Biblical inerrancy and literalism are just silly; there are about five arguments (Pascal’s wager, objective morality, “creation implies a creator”, supposedly-fulfilled prophecies, fine-tuning, and I’m sure I’ve forgotten a few), and there are very strong counterarguments to all of them. The material that’s supposedly the infallible word of an omnipotent, omniscient, and omnibenevolent being is very obviously a bunch of stories and poems and things put together over centuries by a bunch of different people and groups and political processes, that exist in various variants and contradict each other in various ways just as one would expect of such a compendium. If it weren’t that lots of people believe, or purport to believe (see prologue), this stuff, and use it to justify political and social positions that are to various degrees odious, no one would bother responding to it, any more than anyone responds to the claim that Charlotte’s Web is literally true (some pig!). It would be a straw man, if only there weren’t lots of people asserting it.
(And yeah, it’s not just Christianity; many religions have holy books where it’s obviously silly to take the material as literally true. But Christianity is the one that causes the most trouble around me.)
On the other hand, I believe that most Christians, or at least a significant fraction, aren’t fundamentalist Bible literalists, and therefore believe, well, something else. And it occurred to me to wonder just what they do believe, that’s reasonable or arguable or not obviously false or void, but that still makes them Christians in some reasonable sense of the term, and whether I could write down some of the possibilities. To “steelman” Christianity, if you will, in various ways, and see what comes out.
One possibility, which I’ll call I dunno secular Christianity is something like say the Jefferson Bible: the idea that Jesus of Nazareth was a wise person who taught worthwhile stuff, and we should pay attention to it, and read (at least the non-supernatural) writings by and about him, and take him seriously, and perhaps even venerate him for some meaning of that word (but probably as a person rather than something else), maybe have holidays about his birth and death and all that, and so on.
This is fine; a discussion with this sort of Christian would be calm and rational and likely interesting; there is no fundamental disagreement. On the other hand many Christians wouldn’t count it as Christianity, since that is so often tied into questions like “Have you accepted Jesus as your personal savior?” and “Do you believe that he died and then rose again on the third day?”, which this notion of secular Christianity in its most obvious interpretation doesn’t particularly encompass.
I think it’s more common for a Christian to be in something I’ll call mysterian Christianity, which emphasizes faith, and believing in God and Jesus and the Bible without knowing and without worrying too much about exactly what it all literally means. In my model of the world, your typical believer who goes to church every Sunday but doesn’t post apologetics videos on YouTube is like this. They believe that the Bible is true, literally, or “seriously” as the JWs put it, or whatever, and if the things that it says don’t seem to make rational sense right now, that’s fine, we humans are very limited in our understanding, and it will be clarified in time, and in the meantime we should pray and do what seems right and/or what Father O’Malley advises. For now we see through a glass, darkly; but then face to face: now I know in part; but then shall I know even as also I am known.
A discussion with this sort of Christian would be relatively short and simple, I think, as long as one didn’t get obnoxious by asking them how they’re justified in believing something they don’t even understand. The answer is presumably something like “I’ve been touched by the Holy Spirit, and you haven’t, and that’s it; maybe some day you will be too!”, and there’s not much further that one can go politely from there.
What else do we have? I’d like a rational (re)construction of some sort of mystic Christianity, in which one accepts Jesus as one’s personal savior, and he died and rose again on the third day to atone for our sins and all, but also doesn’t make silly truth-claims and doesn’t lead to any odious political or social attitudes. This would have to involve lots of symbols and metaphors, or in the more exalted sense lots of reality-as-metaphor, where we say that Jesus of Nazareth was (and is) both one with God and also the Son of God, and probably that each of us is also one with God (and with Jesus) and also the Son of God, and that each of us (throughout life, throughout the day, in every moment) dies and then rises again, that by our own (perhaps constant) deaths we pay (God and ourselves) atonement for our sins, where simply being an ensouled embodied entity inherently constitutes sin, except when we take salvation into ourselves through Christ’s, which is to say our own, sacrifice of himself, which is to say ourselves, and we must accept Christ (which is to say, the cosmos) as our personal (because everything is personal) savior.
This is to take the Bible as perhaps inerrant and literal, but noting that to the same extent we take Charlotte’s Web (and especially the Wind in the Willows) and the very sound of the wind in the trees as interrant and literal, which is something that we can do only (as far as I know) by acknowledging that all language, even all experience as refracted through language, as metaphorical, so that the most literal possible interpretation of any text or experience is still an interpretation rife with symbol and metaphor. Which is what mysticism is, in some (metaphorical) sense.
Is this all interesting and/or accurate? I’m not certain. I, obviously, don’t believe fundamentalist Christianity, because it’s facially silly, and I keep listening to people refuting its claims (probably just because it validates my existing attitude or whatever). Secular Christianity (as used above) is fine, and probably held by various Unitarians and so on, who may or may not (perhaps more often not, in these troubled times) call themselves Christians. Mysterian Christianity isn’t especially attractive to me, as I haven’t been appropriately touched by the Holy Spirit, and frankly I don’t think anyone else has either in the relevant sense, but arguing about it is very much like I think arguing with a solipsist; it’s too bad that people sometimes get odious behaviors from it. Mystic Christianity is fine, and is so to speak just a different way of saying something that I do believe (to the extent that “believe” applies here) but that I put in Buddhist or Ariadnite terms rather than Christian ones, because icky connotations or whatever.
Disclaimer: I work for Google, arguably a competitor of OpenAI; but these opinions are solely my own, and I don’t work in the AI area at Google or anything.
And I mean, oh come on!
So much AI “research” in these hype-heavy times is all bunk, and I suppose one shouldn’t expect OpenAI (“Open” heh heh) to be any different. But this pattern of:
Use an AI to try to do some interesting-sounding thing,
Evaluate how well it did by waving your hands around, or just by eyeballing it,
Declare victory,
Publish an “AI can do thing!!!!” paper that will get lots of media attention.
is just sooooo tiring. (See for instance the YouTuber in that prior post who showed their system producing a non-working tic-tac-toe game and saying “well, that worked!”.)
The one I’m facepalming about here was brought to my attention by my friend Steve, and omg: “Language models can explain neurons in language models“. They did sort of the obvious thing to try to get GPT-4 to make predictions about how a few selected “neurons” in GPT-2 behave for a few inputs. The key line for me in the paper is:
“Although the vast majority of our explanations score poorly, we believe we can now use ML techniques to further improve our ability to produce explanations.”
— OpenAI
They say this because (they have been drinking too much of the Kool-Aid, and) they tried a few things to make the initial abysmal scores better, and those things made them slightly better, but still poor. They say in the (extremely brief) report that although it works badly now, it could be the case that doing it differently, or maybe doing more of it, might work better.
In any other field this would be laughed at (or politely desk-rejected with a “fantastic, please submit again once you find something that does actually work better”); but in the Wacky Wacky World of Large Language Models it goes on the website and gets cited in half a ton of headlines in the media.
And is it really honest to use “can” in a headline to mean “can, very very badly”? By that standard, I can predict the weather by flipping a coin. I didn’t say I could predict it accurately!
I suppose the LLM hype is better than the Crypto hype because fewer people are being bilked out of money (I guess?), but still…
So Midjourney has Yet Another New Version of their main engine, called “5.1”, and it’s very much like v5, except that it has more of a “house style” like 4.0 did, which can be turned off via “–style raw”. (There’s also “–stylize”, and one might think that ”–style raw” and say “–stylize 0” would do the same thing, but apparently not. It’s all magic.)
I have, as ever, been playing with it extensively (given that “relax mode” GPU time is free at my membership level, any time when the engine is not working for me is wasted, eh what?), and am now up somewhere over forty thousand images, most recently involving things like echidnas (apparently not “echidnae”; “is Sonic an Echidna?“) and many stills from non-existent movies. I will present here a number from a 1960s Psychodrama of Eldritch Suspense, because woot! (See also The Cult Hit of 1977.)
Brad and Laura are concernedDr. Martin and Miss Carter are also concerned.Miss Carter appreciates Dr. Martin’s collection of exotic curios.Patrons in the village pub are concerned. And not only about the ugly picture.Mrs. Martin is perhaps too impressed by the obsidian statue.“Joannie, what –” “Please, Go, Through, The, Door, Doctor, Martin”“Praise, The, Tentacles” “Yes, The, Wonderful, Tentacles”“Come, Back, Soon”
(I like how exactly one of the eight images I made came out in color.)
This is about the current AI hypestorm, and has intentionally a rather extreme title. :) For some values of “it”, it is not all bunk; there are lots of cool and funny and convenient things that AI in general, and that large auto-regressive language models specifically (“LLMs”), can do. Various tasks and jobs will become quicker and easier, and there will be some new fun things to do.
But when it comes to all the breathless claims that “ChatGPT / Bard / FooLLM will completely change the way that we X”, for virtually any value of X that isn’t “produce reams of plausible-looking text whose quality doesn’t matter much”, it’s all bunk. Something describable as AI may eventually do that (I’ll even say that it will eventually do that if everything else survives long enough), but the LLM technology that we have now, even if made better / faster / stronger, will not.
This is a prediction, and it could be wrong, but I’m willing to make the claim without much in the way of caveats. There have been various times in the past where I’ve muttered to myself that a thing was bunk, but not said it on the record so I couldn’t point back at it and say “ha, I toldja so!” once it turned out to be, indeed, bunk. So this time I’m putting it on the record.
It’s commonly observed that when reading a media account of some technical field that one knows something about, one often thinks “hey, that’s completely wrong!”. And that this suggests that the media is also completely wrong in technical fields that one doesn’t know enough about to notice the errors.
It seems likely that this applies to media stories like “ChatGPT / etc will completely change the way that we X” as much as to any other, so given that we know something about, say, software development, we could look at, say, “ChatGPT spells the end of coding as we know it” (also on Yahoo News).
And it’s bunk. The headline is sort of the usual cringey headline, in that it exaggerates the most extreme part of the article for the clickz, but the article does say inter alia, “For better or worse, the rise of AI effectively marks the end of coding as we know it,” which is close. “[T]he rise of AI” is considerably more general than “ChatGPT”, and “effectively” is a nice broad weasel-word, so the actual sentence is fuzzier and more defensible than the headline, but it’s still bunk. As is for instance the quote immediately preceding that statement, in which someone with a deep financial interest in the subject says “there’s no programmers in five years,” and various other breathless things referenced by the piece.
Bunk. Bilge. Bollocks.
Not this bunk; this is a nice bunk.
The thing is that in software development, as I suspect in a whole lot of other domains, LLMs are in fact quite good at doing things that we’ve come to rely on as convenient proxies for important skills. What we ask software engineering job applicants to do in forty-five minutes to an hour isn’t really all that much like what they will be doing if we hire them, but it’s the best proxy we’ve been able to come up with that fits in that time-scale. In other fields, doing the questions on the bar exam isn’t much at all like what real lawyers do in practice, but it’s the best proxy that we as a society have been able to come up with.
Now we have a situation where there are pieces of software that can do a plausible job at various of these proxies (although even here some of the breathless reports are frankly bunk), but that absolutely cannot do the real jobs that we have gotten used to using them as proxies for. And this is driving people to all sorts of false conclusions about the real jobs.
In the software field specifically, what we ask candidates to do, and what various LLMs have shown various levels of skill at doing, is to take a description of a programming task and write code that does that, where the task is (ideally) one that they haven’t seen before, and also is simple enough to write code to accomplish within forty-five minutes to an hour.
Is this what actual professional (or even hobbyist) coders do? I think it’s safe to answer that with an unqualified No: this is not what human coders actually do. Once in awhile one might have a novel thing to do, and do it in forty-five minutes to an hour, but it doesn’t just fall from the sky into one’s lap; it comes up as part of some larger cohesive project that one is working on. Even if one is the most junior coder on a team, doing mostly what the more senior members ask you to do, that is essentially never “please spend an hour and write code to reverse a linked list for me”; that just isn’t how it works.
Actual working coders understand to a greater or lesser degree an overall thing that they are working on, what it does for a user, how it is supposed to work, how to debug it when it fails, at least the basic functional and nonfunctional requirements of the overall system as well as their part, the quirks it has, what libraries are available for them to call, what other members of the team are doing, and so on. And the overall thing isn’t a single function that reverses a linked list, or says whether one string is contained within another.
Let’s look at one of the motivating examples in our first example article. “Adam Hughes, a software developer,” it says, “… signed up for an account and asked ChatGPT to program a modified tic-tac-toe game, giving the game some weird rules so the bot couldn’t just copy code that another human had already written. Then he quizzed it with the kind of coding questions he asks candidates in job interviews….”, and voila, “Hughes found that ChatGPT came back with something he wasn’t prepared for: very good code.”
Unfortunately, this is the only place I can find this impressive feat mentioned. Adam Hughes’ own writeup of how “ChatGPT Will Replace Programmers Within 10 Years” doesn’t talk about this modified tic-tac-toe game at all, or the “coding questions” or the “very good code” referenced in the article. So I’m not sure what’s going on there.
The claim in Hughes’ article title is also bunk (which is to say, I disagree), while we’re on the subject. There is no reason to believe that any LLM will be able to do what’s listed there under “Phase 2” or later. Well, actually, the wording is odd: it says that the AI will “be able to make project-wide suggestions like: Rebuild this Angular 6 project in the latest version of React; Add 100% unit-test coverage to the project…”. I mean, sure, maybe the AI could suggest those things; but in order to predict that programmers are going to be replaced, the author presumably means that the AI will be able to do those things? A bit puzzling.
(Also puzzling is the title of that article; on the page itself the title is currently the nicely ambiguous “ChatGPT Will Replace Programmers Within 10 Years,” which is in some sense true if it somehow replaces exactly two (2) programmers by 2033. But the HTML of the page has a title-tag with the content “ChatGPT Will Replace All Programmers”, which is a much stronger claim about how many will be replaced (all of them!) but leaves out the timescale; heh. The actual text of the article predicts 95% job loss in 5-10 years, and 99% in “10+” years, so it’s sort of the most extreme combination of the two headlines (and it’s wrong).)
Hughes has been updating the beginning of that post with a list of things that are supposed to convince doubters that indeed ChatGPT Will Replace (All) Programmers Within 10 Years; the most recent is a video that, he says, shows “fully autonomous AI agents created python code. Literally replacing programmers. Not that smart, but it shows the how possible this is TODAY, well ahead of schedule.” (Bold in original.)
The video is, to be blunt, kinda silly. This guy has a system where two ChatGPTs talk to each other, and are in some way able to search the web and so on. He asks them to make code to display an input box, and they do that, at the level that one would have found with a web search for “input box python code example”. He asks them to make code to play tic-tac-toe (again, code that is all over the web), and they claim to do that, but it doesn’t seem to work (it displays the UI, but doesn’t make correct moves or reliably detect a win). Undeterred, he says “that worked”, and continues on (lols).
He asks them to “create a new strange simple game”, and they create a “guess the number in ten guesses” with high / low feedback game (not exactly “strange”, and again code that is all over the web), and it might work (aside from the apparently nonfunctional “difficulty” pulldown) but he doesn’t look inside or test it enough to be sure. And so on. And then for like the last 40% of the video he shows off “AutoGPT”, which appears just to fantasize to itself in text and non-functional pretend API calls about how it might create an AGI by linking together GPT instances in various vaguely-described ways, and then gets into a loop of repeating the same thing over and over.
What might Adam Hughes mean when he describes this as “fully autonomous” (given that it’s just doing exactly what it’s told) or as “Literally replacing programmers“? I’m not sure. Is there a programmer somewhere who is going to be replaced by a system that can’t write a tic-tac-toe game, or that can fantasize about creating AGI? I sort of doubt it.
One could cynically note at this point that the Business Insider / Yahoo News article has no doubt gotten lots of clicks and therefore ad sales, that the Hughes piece is a subscriber-only piece on Medium that ends “Stay tuned for my follow-up article about how to prepare for and thrive in this brave new world,” and that if you want to play with the system shown in the video you can “become a member of the channel” for like US$5/month and up. But that would be cynical. :)
There are dozens, maybe hundreds, of otherexamples we could look at, just in the coding area, let alone law, management, or all those other fields that AI is supposedly about to completely change (the last-linked article there is quite amusing, notably where it admits that it’s based on subjective occupation-impact estimates by people who know little or nothing about the occupations in question; i.e. bunk). But this is already long.
LLMs have shown an amazing ability to produce plausible-looking output in response to a huge variety of prompts. The output is even often (more or less by coincidence) correct and/or functional and/or pretty! That’s what it’s architected and designed to do, of course, but I think it’s fair to say that everyone’s been surprised by how well it does it.
Will that ability alone completely change the way that we do X, for any interesting value of X?
I’m putting myself on the record this time :) in saying that the answer is very much No.
Update: It’s been pointed out to me that from what I say here I do apparently believe that “ChatGPT / Bard / FooLLM will completely change the way that we X” if X is one of those proxy tasks; and that’s a point! These things may significantly change the way that we do job interviews, or assign homework, or even give grades in university; but the main changes may be mostly along the lines of “like we do it now, only making sure people aren’t using LLMs to do the proxy task”, and that might not count as “completely changing”.
I should like phenomenology. Ontology is about existing, epistemology is about knowing, morality is about good and bad and valuing, but phenomenology is about subjective experience; about experiencing from the inside.
The relevant reason that I should like that, is that an ongoing project of mine (not one that I necessarily work on, but one that I think about, which is at least as good) is to start with whatever it is that is given, or Given, which I like to call The Experience Storm, and to see how everything else (existing and knowing and valuing) can (or cannot) be built up from that. To take, that is, the subject matter of phenomenology as foundational, and see what happens when we try to build everything else on top of it.
This is hardly an original thought. :) Descartes’ cogito is sort of that, taking that which he cannot doubt and seeing what could be built on that. I disagree with him about just what he cannot doubt (or at least I’m able to doubt lots of stuff he claimed he couldn’t), but it’s the same general project. Empiricism is also an approach to the same project, where The Given is something on the order of sense experience, which turns out to be wrong, but again it’s basically the same project.
If we look up phenomenology on the interwebs, a typical sort of thing we find is, for instance:
“The discipline of phenomenology is defined by its domain of study, its methods, and its main results.
“Phenomenology studies structures of conscious experience as experienced from the first-person point of view, along with relevant conditions of experience. The central structure of an experience is its intentionality, the way it is directed through its content or meaning toward a certain object in the world.”
Stanford Encyclopedia of Philosophy, revised 2013l
which starts out fine for a couple of sentences, but then I am all like whaaaaat.
As I’ve said before in a just slightly different context, I find the whole “aboutness” and intentionality thing silly and unconvincing. And one of the reasons for that is especially relevant here: not only are there all sorts of experiences that aren’t about some object in the world (what is the experience of boredom about, for instance?), but in the foundationalist project, it can’t be fundamental to experience that it be made of things that are “about” “objects” in the “world”, because we don’t yet have a “world”, or any “objects”, or any relationship like “about”.
I talked about this a little on Mastodon the other day, and someone (who I now see is “lead of the graphics kernel team at Microsoft”??) commented usefully that, basically, people usually take phenomenology as not-foundational, as the study of how experience is given that we already know more or less how it all fits into the world, and what the world is like. (Not what they said exactly, but that’s what I read in it.)
And that’s fine, but I think it’s significantly less interesting.
Now, how do we actually build up a world from the Experience Storm? What happens when we try to? Or, perhaps more appropriately, how do I do it, and what happens when I try, since I haven’t established that you even exist yet. And vice-versa. :)
I haven’t worked that all out entirely yet, but an early interesting question is just what the Experience Storm is like: what it consists of, how it’s structures, what entities and properties and relationships it has. Descartes talked mostly as though what is Given is a bunch of propositions, and they include not just “I am thinking” and “I exist”, but really complicated stuff like “there must be at least as much reality in the efficient and total cause as in the effect of that cause,” which is just whacky, and leads to stuff like theism. I don’t know whether he was also unable to doubt “I am reasoning correctly in all this stuff”, or if that occurred to him at all.
The standard empiricist viewpoint is (or one might even say, was) that what is given and undoubtable is sense experience, which is along the lines of two-dimensional red blobs, and rising tones, and whatnot. I think it’s fair to say that it turns out that that isn’t really how we experience the world, although (this being philosophy) the details (and even the broad outlines) remain controversial (see for instance I dunno maybe this).
My tentative hypothesis is that, first, what is given, the Experience Storm, is exactly this, and that any verbal description will be significantly wrong (very Zen, that, at least for my values of “Zen”). But if we forge ahead and write down verbal descriptions anyway, we get not only something like “raw feels”, but much more complex things that have big fat semantic labels attached, and relationships between them, and lots of stuff like that. And those labels (properties) and relationships are just as much part of what is given and indubitable as the rawer and more feel-like things that are also in there, and to which they are in some sense associated or “attached”.
And one can continue from there in various ways :) to build up more stuff on that, always keeping in mind that the words that one is writing down are significantly wrong. But it’s getting late at night, and I’m happy to have written this much down.
It’s a pity that the word “phenomenology” has this whole concept of Aboutness associated with it. Is there a good word for taking subjective experience (whatever that turns out to be like), and building the rest of the universe up from there? Probably there is, and I once knew it even, maybe forty years ago. :)
“Chav” is a British-adjacent word, usually derogatory, for a cluster of concepts involving economically and socially disadvantaged classes, youth, sneering, hanging about in groups in public, and so on. It may be offensive in some sense; it’s not like we have a lot of non-derogatory and non-offensive words for young people in disadvantaged classes. I hope and trust I am not offending simply by using it, for instance, in a weblog post title.
Anyway! For no particular reason I gave Midjourney (version 5) a number of chav-related prompts, and here are some of the results. These are mostly photorealistic, as that’s what v5 often produces unless explicitly prompted otherwise (so this isn’t “AI Art” so much as “AI Weirdness”). And I’m putting them in the weblog here just for fun. (For some highlights of other stuff I’ve been making in MJ, albeit without prompts or much commentary, see me on Pixelfed.)
Here for instance are some Cute Chavs (i.e. the prompt was “cute chavs” with some flags and stuff):
Mildly cute, and certainly numerous! Note the three red caps. Note also that Midjourney v5 has really improved in the people department: some of the faces may dissolve into blurs, some of them disquieting, if you zoom up too much, but no one appears to have seven fingers on the same hand, or any obvious extra limbs. Which is impressive!
Additional cute chavs:
Here “chav” might be a mildly negative comment on their taste in accessories and hair dye; not sure. Awfully cute, though.
Additionally:
These may be the “tough young men” sort of chavs, although the bright pink and blue hoodies and those sunglasses are perhaps atypical.
Also supposed cute chavs:
Certainly cute, but those matching raincoats and GKSY VHIS caps look pretty upscale; it may have strayed a bit from chavery here, but, again, certainly cute. And perhaps truculent.
Further cute chavs, who have perhaps looted a cargo of loud plaids (but who all seem to have the right number of fingers and extremities, again!):
There are various more cute chavs, but we’ll finish this section with this one:
I tend to think of your basic chav as pale, perhaps because people who use the word “chav” often have other words for people who aren’t pale. These three are certainly impressive in their own way. Judgment of cuteness left to the reader.
Heavens, there are so many pictures that I’d love to share with you here! And that’s just these chav-related ones. So many thousands of pictures! See “Bach faucet“, relatedly. But anyway! Now we have some “chav life”:
No notes on that one. We got at least one other one like this except that they’re sitting on a stoop with equal truculence.
Additionally:
Perhaps “CY le HAWE” is the name of their YouTube channel, where they break cinder blocks on each other. For instance. (Do read the alt text on these images, by the way, if convenient; I put a lot of work into those!)
Asking for an artistic rendering of happy chavs, we got inter alia:
They don’t appear particularly happy to me, I admit.
On the other hand:
the Esposito Quadruplets here seem quite pleased by something.
Now this one:
doesn’t really say “chav” to me at all, due to the kilts and sporrans and so on; MJ may be improvising here. The people do look rather happy however.
In Ireland, chavs, or a group akin to chavs, are known as skangers (also possibly offensive). Prompted to picture skangers, Midjourney gave us:
The famous trio of Sakker, Sarkes, and Siakers. Sakker is notable for wearing the number “S”.
Next and relatedly:
Not sure of the relationship between skangers and ice hockey, frankly, but there we are. Perhaps it thinks it’s a typo for “Rangers”, which is a hockey thing.
Third “skangers” image:
Perhaps Gakkes(tm) soda is popular with skangers and/or chavs.
And finally in the skangers set:
“Oi, I’m goin’ down Skarnvers fer some baccy, yew wan’ anythin’ pet?”
(That was probably offensive, too.)
And finally, just so as not to overload my readers with offensive weird stuff, here are a few where we tried to mix chav with its opposite: posh.
This is the one that most obviously did that, but these two are clearly taking the piss, as it were, and on’v Chasd Heaish on top of it (the guy on the right looks familiar somehow).
We also got a fair number like:
which might be interpreted as a posh sort of chav, as well as say:
similarly (Americans perhaps, haha!). Also some where it seems to have mostly ignored the instructions and just given us two ordinary people, as in:
They’re just endearing! (Admittedly one may be missing a finger, but better that than two or three extra.) And not obviously chav or posh, so I’m not sure.
And to close, from the “a chav and posh person standing side by side” prompt:
I hadn’t specifically planned to post this on Easter Sunday, but seeing that it worked out that way, it seems extremely (in)appropriate. Also, as seems advisable to mention in the current context, no generative AI (language models or otherwise) was used in the production of this story. We’ve even got a little badge at the bottom to declare that!
The first Mother to notice the skulls was her Monday Mother, Mother Agnes. On that night, the girl had had just five skulls, medium-sized ones, lined up on the shelf above the head of her ancient bed.
“Child, what are those?” Mother Agnes had bleated as she tucked her in, but the girl hadn’t bothered to reply. Mother Agnes was afraid of her, and the girl always ignored much of what she said. Mother Agnes had only shaken her head, pulled the soft cover up around the girl’s neck, and left with a silent empty kiss on her forehead.
Now there were many more skulls in her room, small skulls and larger ones, clean white fresh-looking ones and brown ones darkened with the stains of age. The shelf above her bed was crowded with them, and they sat arranged in lines on the floor and along the bases of the walls. Her most special skulls were on the wide sill of the big window that looked out on the moonlit lawns of the house.
Her Thursday Mother, Mother Farless, who was brisk and efficient and had a high voice like a chirping bird, had clicked her tongue at the proliferation of skulls, and said “We should get rid of these horrible things! I am going to speak to your father.” But the girl knew that Mother Farless would not do that; none of the Mothers spoke to Father any more.
The girl’s seven Mothers did not know that they were seven; each one thought of herself, in her own individually vague and annoying way, as the girl’s only Mother. All of them were ensorcelled to sleep six days of each week, and to be her Mother for the seventh. Long ago, Father had established the weeks in this way, and the girl was happy enough with the arrangement. She thought that having the same Mother every day would be even more tiresome.
“Where do you find all of these bones, dear?” Mother Serenity, her Wednesday Mother, had asked. The girl had smiled beatifically, and said that they had been brought by angels. Mother Serenity had not argued; Mother Serenity never argued, only sat near the window glowing with a soft light that the girl found soothing, and bringing golden fruits and nectar for her meals.
In fact, the skulls came from underneath the floorboards of the house, from wide dusty spaces smelling of wood and time, that the girl had discovered one midnight, lying on her stomach on the floor, moving her arms and legs up and down, feeling the coolness of the boards through her cotton nightgown. An uneven place between two boards had caught hold of her sleeve, and she had snarled at it and torn at it with her fingernails until it yielded unexpectedly, and the treasure was revealed.
Her Thursday Mother could not come into the house to see the skulls, because she was a rainstorm. The girl took her umbrella and one of the larger and paler skulls out into the smaller garden, where her Thursday Mother rained down gently on them. She placed the skull on a low wall and stepped back so that it was not covered by the umbrella.
Drops from her Mother fell onto the skull, quickly wetting the smooth white upper surface and running in drops down into the eye and the nostril holes. The girl watched each moment, losing herself in the watching, becoming all awareness of the drops falling and merging, running into parts of the skull that had been dry, splashing off of the skull back into the air. She began to cry, and her tears ran down her face in just the same way.
Putting the umbrella aside, she let her Mother’s drops fall on her hair and face, mixing with her tears, wetting the skin that covered the flesh that covered her own skull. She thought of her skull as she cried, thought of it as warm and pink inside of her, as the white skull on the wall was cold and pale. She cried until she could cry no more, and then went back to her room, cold and exhausted.
That night she shivered in bed with fever, moaning and twisting under the covers. Just after midnight her gentle Friday Mother slid silently into her room and stroked her face with a cool cloth.
“Sssh,” her Mother said, a whisper like silence in the dark room, “Sssh, my darling.”
The girl felt the coolness of the cloth on her cheeks and her forehead, even as in a fever dream she danced among whirling skulls under a cracked white dome, ecstatic with terror. Her Friday Mother piled heavy blankets onto her, and held her in gentle arms all night.
When her fever broke, the girl opened her eyes on a new world. The air on her face and shoulders was new, the skulls crowding the room, the moonlight through the windows, were all fresh and sweet with newness. She herself was hollow and light, and she sprang from her bed, staggering only a little, and went from one skull to another, kissing a dozen of them in welcome. Her Mother looked in on her around the door, and then brought fried duck eggs and warm bread for her breakfast.
On the next Monday, her Father visited her, as a tall man with spreading antlers, his powerful body covered in fine nut-brown fur. He allowed her to nuzzle for a moment into his arms, and she pressed her face against the fur of his chest, breathing his rich fragrance and feeling the strong bones beneath his flesh.
As the girl had known he would, her Father admired the careful arrays of skulls in her room. He held one of the largest in his huge hand, turning its hollow eyes to his own, a low sound deep in his throat.
“It is good to be with the past,” he said, and raised his wide deep gaze to meet hers, “with the ancestors.” She fell into his rich wild eyes, feeling herself deep within him, feeling the source of her own life in his, in all of those before him, in the tangled streams of the past, the skulls and bones of her family.
There were, in the dry shadowed places under the floors of the house, other bones scattered among the skulls. Not as many as the skulls, not nearly enough to populate a full skeleton for every beautiful skull, but a fair number. She had taken up some of them, held in her small hand a long thigh bone or a delicate finger bone, brought an enigmatic textured plate up to her nose to sniff. But they were not meant for her room, the way that the skulls were, and she left them down there under the floor, dreaming that they moved at night, arranging themselves in partial memories of the living shapes they had once inhabited.
Father stayed at the house for some time, loping off into the surrounding forest with his men, returning with bloody game for the kitchen. The Mothers and the unseen staff prepared savory meals, and the front of the girl’s dress became colored by the juices that ran down her chin when she ate. Her Thursday Mother tutted at the stains, and threw the dress into the fire, where it sizzled and sparked and finally burned, the girl watching the flames in hungry contentment.
The night that Father left again, they all stood on the balcony overlooking the front of the house as the staff blew the trumpets. It was a Tuesday, and her Mother Frances in her elegant gown only waved a lace kerchief at Father and his men as they disappeared into the silver moonlight. The girl clung to Mother Frances’ skirts, sobbing until her eyes were red, letting misery and loneliness wash through her in delicious waves.
When the house had settled back into the quieter duller patterns of Father’s absence, the girl returned to her room and squirmed under her bed, resting there in the close and slightly dusty space (even Father’s staff could not keep pristine the underside of a bed this old and vast), thinking of the skulls and where the rest of their bones might be. She imagined all of the skulls singing together a slow and mournful song, just for her.
It seemed to her now that she had always heard that song, even before she first clawed the floor open and found the first skull, domed and dusty in its resting place. The song filled the air, and went in and out of the windows in the darkness. It was a song about the deep woods, the game animals with their hot innocent blood and bones, about her and her Father and her seven Mothers and her one True Mother, her ancestors stretching back into the endless beginningless past.
“When did you first meet Father?” the girl had once asked Mother Frances, sitting in her lap before bedtime, her head on her Mother’s elegant shoulder.
“It was in Paris,” her Mother had answered, “at the opera. He was such a gentleman, so strong and handsome, but refined and gentle. He won my heart at once.” The girl wondered if “Paris” and “opera” were real things, somewhere in the world, or if they were part of Father’s sorcery in making Mother Frances into her Tuesday Mother.
“I was so young then,” Mother said, “young and innocent and full of dreams.” The girl thought she caught an uncertain something in her Mother’s voice then, as though her thoughts had wavered for a moment. But then she sat up straighter, and said in a cheerful voice, “And now look at me! The proudest mother in the world.” And she had kissed the girl’s cheek and tucked her into bed.
On another Wednesday, while her Mother Serenity sat glowing in the parlor, the girl wedged up a new section of the floor in the hallway just outside of her room, and found for the first time a nearly-complete skeleton, the bones arranged neatly under the skull between the floor beams, as though their owner had reclined there and then allowed the skin and flesh to melt away. They were clean and white, without the clinging scraps that some of her skulls nurtured.
She knelt by the bones and stretched herself out over them, imagining that it was she who had lain there, slowly restfully melting away. She moved her gaze from the toes upward, saving the skull for last. And when she looked at the skull, she saw the jagged hole in its dome.
“Look at you,” she whispered to the skull, softly running her forefinger around the edge of the hole, “Look at what has happened to you.”
The girl scooped the skull up carefully in her pink hand, and turned it around in the moonlight. The jagged hole was the only damage evident; otherwise, this skull was as perfect as all of her others, firm and intact, undecayed, only oxidized here and there, rusted with the patina of time. She made a place for it in the center of the sill under the big window, placed it carefully, and lay back on her bed, propped up on her pillows, looking into the tender sockets of its eyes.
“You are my first Mother,” she said to the skull, “before Father brought the Seven Mothers, before the first Monday.” The skull did not reply, but she knew it was listening.
“It was from you that I emerged into the world,” she said. And this felt to her as though it mattered. “And then your skull was broken open, and your life escaped.”
She wondered who had broken open the beautiful dome of that skull, and why they had done it. Had there been anger, or kindness, or boredom? Had the same person neatly arranged the body under the floorboards? Had there been blood, like the blood of the game animals that her father brought back from the forest?
The girl thought that surely there had been no blood, just the gradual fading away of skin and flesh, leaving the bones behind as they should be. The hole in the skull might have been the final perfecting of the bones, the last thing needed to complete everything. “You are perfect, aren’t you?” she said to the skull. Thickening clouds moved across the moon, and the darkness of her room deepened, only the palest and whitest of skulls showing in the black. Slowly the girl let herself slip into dreams.
She spent the next two days in her bed, dreaming of skulls and of her first Mother, and the jagged hole in the skull that watched over her with all the others. Her Thursday Mother Farless and her gentle nameless Friday Mother brought her meals and spoke words to her, but she was far away, dancing with her dream skulls and learning from her dream Father.
Her Saturday Mother, the Enchantress Niviène, sent to her a small glittering swan which circled above her bed, singing a song too beautiful to ignore, and the girl returned to the ordinary reality of her room and the unmoving skulls.
“Hello, my dear one,” the swan sang in her Mother’s voice. The girl thought again how powerful her Father’s sorcery must be, to bind Niviène as one of her seven Mothers. But she also suspected that perhaps this particular Mother was not as deeply ensorcelled as those earlier in the week.
The small swan glided around the room on diamond wings, lighting daintily on the broken dome of the skull on the window sill, and dipped its bright beak into the dark opening.
“You have found your Father’s magic, I see,” the swan sang, “and none too soon.”
“What do you mean, Mother?” the girl said, sitting up in her bed, feeling somehow the excitement and fear of infinite possibility.
The swan’s song was full of laughter. “Tomorrow,” her Mother’s voice said, “perhaps tomorrow.”
After this, the swan glided out through the window, and the girl rose eagerly from bed, dressing herself in leather and bright calico. She bowed to the broken skull, ignoring the others, and tucked it under one arm and went out into the night.
For many hours, the girl conducted the rituals of her family at the old holy place in the forest, and the beasts of the forest growled and whined all around her. The perfect broken skull presided from the highest point of the altar, and the song of the other skulls poured out of the sky and entered the skull’s jagged starry hole. This is what she had dreamed about, this is what had been whispered to her in her sleep.
Finally, the rituals complete, the girl slumped to the forest earth, and the beasts nuzzled and licked at her clothing and her limp body.
She was carried back to the house, tenderly changed into her nightgown and gently tucked into her bed. Voices spoke quietly, and a soft breeze blew, and she slept deeply and without dreams.
The next day was Sunday, a day that the girl always spent in the quiet dimness under her bed, or down in the mazy passages beneath the house. Of all of her seven Mothers, she loved and hated Lucinda, her Sunday Mother, the most intensely, because she was all light, and glory, and judgment.
But on this Sunday, the girl slept long from the night’s rituals, and was still asleep when her Mother Lucinda flung open the door and entered her bedroom, clad in white and gold, like a demanding flame. Her light gleamed from the skulls as she turned back the girl’s bed, and unseen hands helped her upright and dressed her in a severe white gown of her own.
“What are we doing, Mother?” she groaned, squinting against the light as the unrelenting Sun separated itself from the treetops of the eastern Forest.
“Going to the chapel, silly girl!” her Mother trumpeted, as the invisible hands tugged her gown here and there, and pulled a white lace veil up over her head. “Have you forgotten what day it is?”
The girl sighed. Being awake only one day in seven, her Mothers all had odd notions of times and dates. The old chapel had been in ruins as long as she could remember; it might be interesting to see what Mother Lucinda expected to do with it.
Out on the lawn, taking the path that led from the big oaken doors to the chapel, her eyes slowly adapted to the brightness of the Sun and of her Mother. She wished her Thursday Mother would come with her thick clouds and cooling rain, but it was Sunday, and a terrible searing Sunday at that.
As she and her Sunday Mother walked, the girl saw that others were moving in the same direction; people from the village in their poor finery, and also many others, dim vague figures somehow shadowy even in the omnipresent light. Both groups, the mortal and the shadow, gave a wide berth to the two of them from the house.
Her Mother seemed to ignore them, but the girl regarded them with interest as they walked, especially the shadows. Some of them limped or shambled, some held an arm tightly against the body, some walked together, unsteadily supporting each other. All seemed to one degree or another in distress, but their faces, those that had faces, seemed happy, happier she imagined than her own, as though they were eager to reach some destination.
As more joined them, the girl saw that although the shadows came from all directions, the largest stream came from the direction of the road that led to the village, and another stream from one particular part of the forest. She wondered what might be in that direction, and then she remembered, and smiled.
Looking back, toward the big house, she noticed that behind them on that path walked a tall woman, who held her head tilted to one side, perhaps owing to the spike or short metal pole protruding from it. This, she knew, was her true mother, with a perfect hole in her skull, and the girl’s heart was filled with love. Then she had to turn again to the front, hurrying to keep up with Mother Lucinda, and the shadow of her true Mother was lost among the others.
When they finally reached the chapel, the girl was slightly surprised to see that the old building was not ruined, but stood intact and immaculate in the sunlight, doors wide open on a spacious bright interior. Here the streams split, the shadows taking a pair of stairways leading down into dimmer places, and the less numerous others entering the chapel proper, seating themselves in the long pews.
Her Sunday Mother strode to the very front rank of the nave, the others in the chapel quick to get out of her way, and the girl hurrying along behind her. They settled down in the front pew by themselves, facing the chancel where the figure of the Hanging Man looked down from the wall. In the all-too-bright light of the chapel, the Hanging Man seemed to the girl to be the only point of ease and sympathy. His hands and feet were affixed to wooden beams by thick nails, and streams of blood dripped down his pale skin. He wore a crown made of thorny vines on his head, and the girl wished that he would look up at her, so she could see his eyes.
A man in a black cassock and white surplice came out of a door to one side of the Hanging Man, and stepped up behind the wooden lectern. He looked blandly out over the crowd, and began to speak. The girl let his words flow into her head and around and out again, one by one. She watched the carven animals on the lectern chase each other around the carven plants, playing and hunting and mating. She looked at the body of the Hanging Man, and saw that he had a wound in his side, from which blood also dripped. She thought that the red blood against his thin white skin was very beautiful.
Just as the man at the lectern addressed his Father, who was in Heaven, the Hanging Man did raise his head, and his gaze met the girl’s gaze, and she saw that the Hanging Man was her Father, nailed to the heavy beams of wood, and she was filled with joy.
The girl stood, as the man at the lectern continued speaking, and slowly climbed the steps leading up from the nave. She felt the judgment of her Mother Lucinda, puzzled murmuring of the crowd of villagers, and the searing light of the Sun, all pulling at her, obstructing her. But she had done the rituals well, and she entered the altar, and approached the Hanging Man, her Father, who smiled at her now from his place on the wall.
Poised on the knife-edge of Time, she rose up, and pressed her lips to her Father’s lips, and the skin of his body began to fall away. There were gasps and then screams from behind her, and her Mother Lucinda’s voice rising in agony or bliss. She held herself against her Father as all of his skin, his loincloth and crown of thorns, and finally his flesh melted and flowed away, and only his beautiful bones hung there on the wall of the chapel.
He slipped his purified hands and feet easily off of their nails, and held the girl in his arms as he descended from the wall. Comforting shadows pushed out from them, and from her true Mother, who rose up from the crypt, through the floor of the chapel, and was gathered also in her Father’s skeletal embrace. The screams of the villagers were cut off, or faded away into the distance, as the three of them walked out of the chapel, which was already falling back into its proper ruin behind them.
Outside the chapel all seven of her false Mothers waited to greet them, finally drawn together by her Father’s will. The clouds of her Thursday Mother hid the unwelcome Sun, and cooled them all with a gentle mist. Mother Agnes, Mother Frances, and her gentle Friday mother stood with their heads bowed. Even her Sunday Mother Lucinda, dimmer now and her white and gold raiment torn to ribbons, stood humbly, hand-in-hand with her Mother Serenity and the smiling Enchantress Niviène, and paid homage to the girl and her parents.
Her Father put her down gently. She looked up at him, at the clean jointed bones of him, and then at the shade of her true Mother, who looked at her with infinite tenderness from the shrouded eyes of her broken head.
With a small smile, the girl made a slight gesture with her fingers. Her parents, comprehending, exchanged a glance and then went to stand, heads bowed, with the others.
The girl stood there, feeling herself, feeling everything, in a flawless moment. The sound of the chapel crumbling behind her, the mist of her Thursday Mother on her skin, the heads finally bowed in obeisance to her, and farther away the big house and her skulls, all under a rapidly darkening sky. Everything exactly as it ought to be.
And now, she thought happily, the sun will never rise again.
tl;dr: While it’s true that all LLMs do is produce likely text continuations, this doesn’t imply that they don’t have mental models, don’t reason, etc.
One thing that sensible people often say about Large Language Models like ChatGPT / GPT-n and Bard and so on, is that all they do is predict the next word, or for more technical accuracy, that all they do is generate text that is likely to follow the prompt that they are given, i.e. “produce likely continuations”.
And that’s a good thing to note, in that people tend to have all sorts of other theories about what they are doing, and some of those theories are just wrong, and lead people to make bad conclusions. For instance, people will have a more or less default theory that the model knows things about itself and tells the truth about things it knows, and take seriously its (non-factual) answers to questions like “What language are you written in?” or “What hardware are you running on?” or “Are you a tool of Chinese Communism?”.
Also, it’s true that all they do is generate text that is likely to follow the prompt, in the sense that that is the only significant criterion used during training of the underlying neural network.
But that doesn’t actually mean that that is all they do, in the more general sense. And this, at least potentially, matters.
Consider for instance the claim that “all life does is arrange to have many generations of descendants”. That is true in the same sense, since the only criterion for having survived long enough to be noticed in the current world, is to have had many generations of descendants.
But, significantly, this doesn’t mean that that is all life does, in the sense that life does all sorts of other things, albeit arguably in the service of (or at least as a side effect of) having many generations of descendants.
For instance, I think it would be plainly false to say “people obviously can’t reason about the world; all they do is arrange for there to be many more generations of people!”. In fact, people can and do reason about the world. It may be that we can explain how we came to do this, by noting that one effective strategy for having many generations of descendants involves reasoning about the world in various ways; but that does not mean that we “don’t really reason” in any sense.
Similarly, I think the arguments that various smart people make, which when boiled down to a Tweet come out as roughly “LLMs don’t X; all they do is predict likely continuations!” for various values of X, are in fact not valid arguments. Even if all an LLM does is predict likely continuations, it might still do X (reason about the world, have mental models, know about truth and falsehood) because X is helpful in (or even just a likely side-effect of) one or more effective strategies for predicting likely continuations.
Put another way, if you train a huge neural network to output likely continuations of input text, it’s not obviously impossible that in choosing internal weights that allow it to do that, it might develop structures or behaviors or tendencies or features that are reasonably described as mental models or reasoning or knowledge of truth and falsehood.
This isn’t a claim that LLMs do in fact have any of these X’s; it’s just pointing out that “all it does is produce likely continuations” isn’t a valid argument that they don’t have them.
It’s still entirely valid to respond to “It told me that it’s written in Haskell!” by saying “Sure, but that’s just because that’s a likely answer to follow that question, not because it’s true”. But it’s not valid to claim more generally that a model can’t have any kind of internal model of some subset of the real world; it might very well have that, if it helps it to correctly predict continuations.
Bonus section! Current LLMs don’t in fact reason significantly, or have interesting internal models, in many cases. Amusing case from this morning: when fed some classic text rot13’d, this morning’s Bard claimed that it was a quote from Hitchhiker’s Guide to the Galaxy, whereas this morning’s ChatGPT replied with rot13’d text which, when decoded, was gibberish of the sort that an early GPT-2 might have produced from the decoded version of the prompt. No agent with a reasonable mental model of what it was doing, would have done either of those things. :)
Just a little more on the craziness from yesterday, and a comparison with Midjourney v5.
We chose another phoneme-triple to look at, “eaux saf aim”. In retrospect there are some words in there (French for “waters”, and English for “aim”), but that’s okay.
Using the magic from yesterday, with Midjourney v4 and “–no words,letters,text” and “–no face” (for that total weight of zero) and “–chaos 50 –ar 3:2”, we get the quite pleasing:
Whew, those are not easy to write alt text for!
And then we did exactly the same thing, only with “–v 5” to get the v5 engine, and it did the notably different:
This may be reflecting something about the internal “creativity” or “style” of the two engines.
Oh, hey! I should try the v5 one with the “stylize” level turned up. Let’s see, with “–stylize 999” we get:
So that’s a No :) it isn’t the –stylize setting.
From this experiment we can theorize that v4 dreams about weird surreal stuff, whereas v5 dreams about a household goods catalog.
I took it into my head for some reason to see what Midjourney would do with little sub-semantic phonemes, like say “ton glam so”. When I first tried it, the results had letters (and not-quite-letters) all over the and/or were all just faces, so I added the switches “–no words, letters, text –no face” to the prompt.
I did that as two separate –no switches without thinking, but in retrospect that may have resulted in a weight of one (1) for “ton glam so”, and weights of -0.5 each for “words, letters, text” and “face”, resulting in a total weight of zero (0), which is known to do weird / fun things (I thought I had mentioned that here earlier, but apparently not).
With those switches, our initial “ton glam so” produces the rather noteworthy:
Possibly the “glam” make “glamour” or even “glamor” salient in the model? But these are not, well, the images that I would have expected to be most salient under the category of “glamour”.
The same switches with the text prompt “so bel wip” produces the also, but very differently, noteworthy:
No relationship to “so bel wip” occurs to me, but it’s certainly consistent! Wondering if this was due to some common seed or something, I tried it again, and got:
which, whoa, definitely very similar. One more time for good luck?
I tried adding “–chaos 70”, which does something or other, and got this:
The same but just a bit more variety; two kids possibly white, one with pointy ears, and so on. But the same interesting clothes and general style. Fascinatin’!
I tried another text prompt (without the –chaos) “plin bo san”, and got these delightful things:
Does “plin bo san” make “plane” and maybe “boat” salient? Does “san” somehow specify the aesthetic? So fascinating! What if we change the aspect ratio to three wide by two high?
OMG so delightful. I love all of these! Next, I tried “tem wo sec” and…
I mean… what?!
Then, “lus dab ba” with –chaos 60:
“mai rem den” with –chaos 70:
Ahhhh what even is happening? What are all these things??
I’m stopping now because my brain is tired, and it’s challenging to write alt-text for these! But wow, eh? Whatever is going on with these things? These are all Midjourney v4, I’m pretty sure, because that’s the default at the moment and I didn’t specify. I’m guessing the total weight of zero is part of what’s causing… whatever this is.
This is a random brainstorming post, I have no particular conclusions at the moment as I write the first sentence here, but I might develop something as we go along.
So far, I just have this “meme” that I made:
There are two obvious reactions to this. Someone who likes AI art tools might say “haha, yeah, this shows how creative and crazy this art can be!”. And someone who agrees with the critics might say “omg, totally, that’s obviously sooo derivative!”.
The first thing to wonder is whether there is a particular image, set of images, or artist out in the world somewhere of which this image is obviously derivative. Pointers in the comments are extremely welcome! :)
Google (reverse) image search doesn’t come up with anything especially obvious. There are some images (like, at the moment, this one) that involve somewhat stylized faces with prominent hair and ocean waves and one or more ships, but the arrangement and overall style and impact are, I think, significantly different. In the past when I asked a couple of people who were all “oh, yeah, I can usually identify the particular artist or artwork that one of these AI images was taken from”, to do that with one of my images, they suddenly became very quiet. ¯\_(ツ)_/¯
If there isn’t a specific image or small set of images or an artist that one can point to and say “see, this is where this came from!”, what does that mean? I’m not an art critic (hahaha), but I think it would be pretty uncontroversial that, if a person had created that image above there entirely with real-live paper and charcoal, or even with a tablet and Photoshop, we’d say that it displayed sort of average human creativity; nothing incredible, but easily beyond (for instance) the “modicum of creativity” required by US copyright case law, enough that it could be entered in an art competition, and so on.
Once we know that it was created by a person using an AI art tool (Midjourney, in this case, with a particular prompt and engine settings and so on), is it reasonable to say something different? Does it still display creativity, or not? Does it do it differently, or in the same way? What is creativity? How is it displayed? In what does it inhere? Is it for that matter the kind of thing that inheres in things? Are there facts of the matter about it, or is it a purely squishy and subjective thing?
There are a bunch of theories that one might put together:
One might hold that it’s just as creative, and in the same way, as the counterfactual no-AI version, and that the creativity comes from the same place: the human who made it. One version of narrative would say that the difference between the no-AI and the with-AI version, creativity-wise, is not different in kind from a person making it with paper and charcoal and a person making it with tablet and Photoshop, or a board and little mosaic tiles. It might be objected that the activity of choosing engine parameters and prompt strings and then culling the results is just obviously (or by dint of some specific plausible theory) different from the activities in the other cases, since those involve something like choosing a particular color for particular parts of the image, whereas the AI-tool case doesn’t.
One might hold that it’s just as creative (or at least that it is creative, if perhaps to a different degree), and the creativity still comes from the human, although it’s implemented (delivered, displayed, exercised, used, manifest) in a different way. One might say in this theory that the difference between the real paper and charcoal version and the Midjourney version is like the difference between a realistic drawing of a scene and a photograph of the same scene. Both born of human creativity, but through very different means, and perhaps to different degrees. And then we can get into lots of questions about the creative element(s) in various kinds of photography!
The two takes above can, I think, go either way on the question of whether creativity is inherent in the end result, the image, in a sort of death-of-the-author way, or whether it’s in the overall process. At the other end of some spectrum, one could say that the image made with the AI tool does not in fact display (involve, require, contain) any creativity; that our initial impression that it did just turns out to have been mistaken, and now that we know how it came to exist, we know that it didn’t involve creativity. This sort of claim pretty much rules out the position that creativity is inherent in the finished product, unless one is willing to take the (facially untenable, I think) position that this image could not in principle have been created by a human without using an AI, and that inversely no purely human-created image could in principle have been created with an AI tool.
That is, if you think there is no creativity in this image because it was made with an AI tool, you pretty much have to take the position that it’s not possible to tell how much creativity there is in an artwork (or a putative artwork) just by looking at it; that the creativity is not displayed by / doesn’t inhere in solely the image or object. Which seems sensible in at least one obvious way: I might think that something involved lots of creativity, until I see that it is an exact copy of something that existed before, just with a little line drawn on it. More nuancedly, we’d say that you can’t tell how much new creativity is in a thing, until you see how it was made (because it might be, say, a copy).
So now we have a potential claim that images made with AI tools don’t have any (or much) new creativity, because they are just processed / stolen / noisily compressed and expanded / copy and pasted, versions of the material that they were trained on. Sure there might be a little creativity in choosing the prompt or whatever, but that’s not much. The program itself can’t add any creativity because “they can’t, they just can’t” (a phrase I’ve heard from a couple of people talking on videos lately, but of course can’t find at the moment).
Humans also process things that they’ve seen / experienced when producing new things. I’d say we can’t really require creativity to mean “those aspects of a work that spring purely from the artist’s soul, and that would still have been there had the artist been a brain in a vat with no experience of the world or other artworks, only its own thoughts”, because then there wouldn’t be any creativity anywhere, and when common words turn out to have no referent in a theory, it generally (if not always) means that that theory is wrong.
Or maybe we do want to require that “sprung from the soul alone” thing, because we want to set a very high bar for True Creativity, and we are confident that there will be at least a few glorious shining examples if only we knew the truths of people’s souls! In which case we can say that a marvelous few humans have displayed true creativity through the ages, and no computer ever has (having no soul and all), and neither have the vast majority of people we loosely call “artists”. This is a theory, but not a popular one, and it means that most art displays no creativity, which again feels sort of like a reductio. It’s certainly not compatible with what the Copyright Office means by “creativity”.
The question of how much creativity is in the selection of prompts and engine settings and images to keep is one we can put aside (in the drawer next to the question of the creativity in a cellphone snapshot, as alluded to above). And it seems we are left with having a theory about how much creativity comes from the AI tool itself, and how much of that is what we’ve called new creativity. Possible answers include “none, there’s lots of new creativity, but it’s all from the human user”, “none, there’s no new creativity in this at all, it’s all stolen / copied from the creativity in the training set”, “about the same amount that comes from the human, they are in some sense equals in the new creation”, and “the human just types a few words, and then the software adds lots of new creativity to it, so it’s the AI”.
This leaves us mostly with the question of “under what circumstances is it true that a person, or a piece of software, adds new creativity to a work, when that work is to a degree influenced by other prior works that that person, or piece of software, has been exposed to?”. Or other words to that general effect. One set of answers will not especially care whether it’s a person or a piece of software; the other set (“they just can’t”) will either think that it’s important which it is, or have a set of criteria which (they will claim) only people and not software can for whatever reason satisfy.
And I’ll leave it there for now, having perhaps not been especially productive :) but having written a bunch of words and focused in (if in fact it’s a focusing) on the question of what it means to add new creativity when making something, even though the entity doing the creating is influenced by other works that existed before. People talk a lot about things like reflecting one’s lived experience, having a thought that the work will (may? is intended to?) cause the viewer to also have (some version of?), and like that. None of those seem likely to be any kind of complete explanation to me at the moment.
In legal news, of course, the US Copyright Office has issued a Copyright Registration Guidance on “Works Containing Material Generated by Artificial Intelligence”, which I gather (I have not had the mental energy to think about this very hard) just repeats the statements in the Zarya (I always want to write Zendaya) memo we briefly discussed the other day, using various phrases that are putatively synonymous but as far as I can tell are subtly different and introduce all sorts of new uncertainty to the subject.
I’m going to continue not thinking about that very hard for now, because that part of my brain is still tired.
Also! You can get onto the waiting list for the Google LLM thing (and I hear varying stories about how quickly one gets access; apparently it is sometimes quite quick). In case you’re, like, collecting those, or otherwise interested.
Lost for decades, now rediscovered and presented here for the first time!
Mike and the GangThe Mysterious Mr. G in his Secret LabThe legal team in actionWhat is happening at St. Agnes?The OutsidersIn the kitchen at St. AgnesUnder ControlThe Discovery!
Courtesy, of course, of the early v5 version of Midjourney.
As I’m sure you’ve heard there’s a new level of GPT in the world. Friend Steve has been playing with it, and says that it does seem to do some stuff better, but also still make stuff up amusingly and all. At the moment for whatever reason I can’t be arsed to investigate, or even read yet more hype / analysis about it. Similarly, Google announced a thing, and Microsoft is putting LLMs into various products whose names I don’t recognize, and I’m not reading about any of that. NovelAI‘s good old open-source model works fine for all of the telling-weird-stories stuff that I need right now.
And there’s a test version of a new Midjourney engine out! Being tested! And it seems pretty cool. Hands in particular seem much more likely to have five fingers when you’d expect them too, which is a whole thing.
And I spent too much time arguing with people on the Twitter, which isn’t at all new. And I definitely shouldn’t do because it is not healthy. So I’m trying to stop that.
Now I’m just making pretty pictures! And not thinking very much until later on sometime!
Lots of weather in those, eh? Hadn’t noticed that. :)
Friend Steve wrote us today about this New York Times opinion piece, “Noam Chomsky: The False Promise of ChatGPT” (this link may be free for everyone for some time or something). Despite the title, it’s by Chomsky, Roberts, and Watumull.
Steve commented inter alia on the authors’ apparent claim that ChatGPT can say that the apple you’re holding will fall if you open your hand, but unlike humans it can’t explain the fact. The trouble with the argument is that, as anyone who’s actually used ChatGPT can tell you, it will happily explain the fact, go into the history of the notion of gravity, talk about other things people have thought about it over time, and explain various situations in which the apple wouldn’t fall, given the slightest provocation.
My reply, lightly edited:
I am pretty unimpressed with the article as a piece of science or philosophy; fine as a nice polemic by a greybeard I suppose. :)
I’m amused at how LLMs are “lumbering” and “gorged”, while human minds are “elegant” and even “efficient”. I doubt there is any useful sense in which these adjectives are anything more than bigger words for “icky” and “nice” in this context.
Chomsky brings in the innateness of language, because of course he does, but I’m not at all clear how it’s relevant. Even if humans do have innate language scaffolding, and LLMs don’t have the same kind, it’s far too early to say that they don’t have any, and even if they didn’t, so what? Does the ability to learn a wider variety of languages than humans can, mean that LLMs don’t really understand, or can’t really think, or are harmful or dangerous? None of that makes sense to me; it seems just an even longer way of saying that they’re icky.
He (well, they, there being multiple non-Chomsky authors) claims that LLMs don’t have the ability to say “what is not the case and what could and could not be the case.” And I can’t imagine what they think they mean by that. As with the flaw you point out in the apple example, it’s simply wrong, and suggests that they haven’t really used an LLM much. ChatGPT (let alone a less heavily defanged system) will expound at length about what is not the case and what could and could not be the case, given any halfway decent prompt to do so. They may intend something deeper here than they actually say, but I don’t know what it could be (beyond that they can’t do it non-ickily).
“Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round.” Um, what? There are certainly humans who believe each of these things. They can’t just be saying that humans can’t conjecture that the earth is flat “rationally” because so what; that’s exactly as true of an LLM. If they mean that the same LLM can make one of those claims one minute and the other the next, whereas humans can’t hold two contradictory beliefs at the same time, I’d like to introduce them to some humans. :)
Similarly for whatever it is they are trying to say about moral reasoning. The suggestion seems to be that, simultaneously, ChatGPT is icky because it cannot stay within moral boundaries, and also icky because it stays stubbornly within anodyne moral boundaries. As pretty much throughout the piece, stuff that humans do all the time is cited as reasons ChatGPT isn’t as good as humans.
Tay became toxic by listening to people, therefore it’s not like people? It had to be heavily censored to keep it from talking trash, therefore it’s not like people? Um?
It might be interesting to try to tease a set of actual significant truth-claims out of this article, and see which ones are arguably true. But I’m not sure that’s the point really.
As far as I can tell, this piece is just a longer and nicely phrased version of “Boo, LLMs! Icky!”
I am annoyed with many “AI” things this morning, so this is a rant with no pretense of objectivity or overall wisdom.
AI call directors are terrible. Especially Intuit’s.
Here I will just reprint a rant that I posted to both qoto and Twitter; I was so annoyed!
Wow, #Intuit #TurboTax is just awful.
I mean, I do know that they’ve been lobbying against tax simplification in the US for years, because it would cut into their business, and that’s pretty evil.
But their customer service is apparently also terrible!
I need to file a particular New York State #tax form this year, and apparently they just don’t support it, and aren’t planning to.
Which seems to mean that I would have to manually enter the data, which seems to mean that I couldn’t then e-file or get their correctness guarantee. And if one uses software to prepare the return, one is required by law to e-file!
So it seems like I just can’t use their software at all. Which is maybe good!
When I tried to call them to ask if they support the form, their robot call director asked me what I wanted, mis-heard me, and insisted on knowing whether I wanted the irrelevant information it had found sent by text or email; “no” was not a valid choice.
Then it insisted on knowing my last name, but failed to understand me when I tried to pronounce or spell it (and I have a pretty ordinary voice, and not all that unusual a name!) and eventually it said goodbye and HUNG UP ON ME when it couldn’t.
I had to call back and pretend that its incorrect guess at my last name was correct, before it would pass me to a representative. And the first thing the human rep (who was very nice!) asked me was for my first and last name, so the whole robot torture conversation was useless as well as annoying.
I think they’re just trying to get people to give up on calling them.
Which in my case probably means switching to #freetaxusa which is cheaper anyway, and does support the forms that I need.
Sheesh!
I hate this Roomba (at least while it’s running).
Leaving aside the fact that it’s a mobile Internet-attached camera that could almost certainly be accessed by random hax0rs in Kekistan, and may already be sending all sorts of images of private life to iRobot / Amazon / the NSA, it’s just annoying.
It has an app of course, but for some unaccountable reason the app is rather terrible. For a long time it offered not much more than the little “CLEAN” button on the robot does; no way to tell it to avoid certain areas or do a certain room right now, let alone a Direct Drive mode where you could just pilot it around vacuuming (which I would have thought would have been the Minimum Viable Product or whatever the kids are calling it these days), no insights into what was going on in the little beggar’s mind that makes it buzz around in the front hallway for half of its runtime and pay only cursory attention to any actual room. Lately it’s been enhanced somewhat, so you can see a version of it’s internal map, tell it to do a certain area, and a few other things.
But it still went under my chair this morning while I was having my coffee, and got into some kind of infinite loop at the edge of the nest of power and data lines off the side of the chair where it doesn’t really need to go at all. It sat there trying to charge forward and running into something with a loud whir, turning slightly right, whirring again, turning back slightly left, whirring again, repeat forever and ever, with loud irritating whirs every time. I gave it a slight nudge to try to get it away, and it faffed about a little and then charged back into the same corner again, whirring as loud as ever.
Why isn’t there a “don’t try the same thing more than a dozen times” feature in the thing? Maybe because it’s some black-box AI that can’t be explicitly programmed not to do certain things, but just does whatever comes out of the mysterious tangle of weights and things. And maybe because they couldn’t be bothered to add that because it hasn’t made it into a sprint yet. Who knows!
But it’s really annoying. It’s chased me out of my chair (again) and I’m sitting in the living room where it isn’t currently whirring in annoying ways.
Fekking thing.
Look how fast it can be wrong!
All of the excitement about LLMs also has lots and lots of really annoying properties. Having suffered from them for awhile now, I think the basic problem is that LLMs are good at certain small and easily-testable unimportant things that, until now, were good indicators of being good at other things, some of them larger and more important.
In particular, we’re used to only people being good at giving natural-sounding answers to questions in human language, and when someone is especially good at that (“eloquent” or “intelligent” or “legit-sounding”), we are used to that same person being good at saying true things, or being able to write a couple of pages of consistent argument, or caring about the truth of what they are saying.
Large Language Models (like GPT-3 and ChatGPT and Bing’s AI and Google’s Bard and on and on and on) are good at the small things, but bad at the large things. They can give natural-sounding replies to all sorts of questions / statements in human languages, but they have no notion whatever of truth or fact, their input windows are so small that they can’t generate a significant amount of output without losing track of the plot entirely and either going off-topic or contradicting themselves or forgetting their initial instructions and trying to persuade someone to leave their spouse.
So when we see people putting up some trivial “app” that feeds user-input and a paragraph of additional prompt into some random LLM, and billing the result as “AI Medical Evaluation!”, it’s terrifying. (I think that particular one has been taken down since I expressed worries about it on qoto, but there’s still a zillion like say this “Jesus” one, and no doubt scads of other extremely dangerous medical / psychological / legal ones being created every day by people who don’t understand malpractice or law or liability or LLMs.)
And when someone posts to reddit saying “After poring over garbage Google results and documentation that didn’t answer my question for literally an hour, Bing checked the SOURCE CODE and gave me an instant answer. Remind me, why would I ever want to use Google again?”, the obvious reply is that the “instant answer” was in fact wrong, as someone with a name similar to mine pointed out in the reddit thread. (The person says that the answer did eventually lead them to a right answer, but I wonder if it was significantly faster than the “literally an hour” spent in good old search; it certainly wasn’t “instant”.)
And lest anyone think that I have a Conflict of Interest acting here (I do work for Google, but not in the AI or Search departments), I don’t think that Google’s LLMs are any better except in the extremely significant property that they haven’t been released in a form integrated into a general-public web-search tool, in a way that leads people to think their extremely confident answers are in fact reliable.
One of the things I find most irritating in the world are people who are extremely confident and also wrong. So now that we have an entire category of software that is essentially all that way, it’s (again) extremely annoying.
(LLMs are wonderful, as I mentioned the other day, as a sort of crazy friend who you can bounce ideas off of and get bizarre prose to help break a writer’s block, and amuse yourself with fake Elizabethan love poetry or whatever. But in contexts that are framed as likely to produce true statements, they are entirely out of their milieu, and should really just stop. I look forward to the technological breakthroughs that will allow these systems to have usually-true output, but I haven’t seen that yet!)
So anyway! I feel somewhat better now. :) End-rant, comme on dit.
On art made with AI tools, that is. Reuters story here, actual letter from the Office lawyer here.
I haven’t read the whole letter in detail yet (it’s long!) but I’ve looked it over and have Initial Thoughts:
I don’t think there’s a fact-of-the-matter here, about what is copyrightable when. There are legal theories that make more and less sense, that are more and less consistent with other established theories, and so on. But these are not theories that try to model something in the real world, like the Theory of Relativity; they are more theories in the sense of Set Theory. So the Office can’t really be right or wrong here overall, but they can have made a more or less sensible decision.
The overall finding of the memo is that Kristina Kashtanova still has a copyright on Zarya of the Dawn, but only on the text, and “the selection, coordination, and arrangement of the Work’s written and visual elements”, not on the visual elements themselves (i.e. the images made with Midjourney), because those images don’t involve “sufficient creative input or intervention from a human author.”
This seems wrong to me; as other places in the document point out, the case law says that “only a modicum of creativity is necessary”, and there is certainly a modicum of creativity in prompt design and engine usage.
The argument here seems to be, not that there isn’t enough creativity in the prompts and flags and so on, but that the connection between the artist’s input and the image output isn’t strong enough. The memo says things like ‘Rather than a tool that Ms. Kashtanova controlled and guided to reach her desired image, Midjourney generates images in an unpredictable way. Accordingly, Midjourney users are not the “authors” for copyright purposes of the images the technology generates.’
But where is the existing doctrine that says anything about predictability? Jackson Pollock might like a word, and the creator of any other roughly uncontrolled or algorithmic or found-object work. The theory here seems to be that Midjourney prompts are just suggestions or ideas, and those can’t be copyrighted. Does that mean that since Pollock just had the idea of splashing paint onto canvas, and the unpredictable physics of the paint cans and the air produced the actual work, that “Autumn Rhythm” can’t be copyrighted? Or are they going to hold that there is a legal significance to the fact that the detailed movements of his arm muscles were involved? That seems dicey.
For the Office to claim that the prompts and other input did contain at least a modicum of creativity (which seems undeniable) but that that input wasn’t strongly enough connected to the output, seems to be inventing a new legal test, which it’s not at all clear to me that the Office can do on its own hook, can it?
This memo may be specifically designed to be contested, so that the question can go to a court that can do that kind of thing.
The memo may have interesting consequences for Thaler, in particular the cases in which Thaler attempted to claim copyright under work-for-hire theory, with his software as the creator. The memo explicitly makes the comparison with human work-for-hire, saying that if someone had given the same instructions to a human artist that are contained in a Midjourney prompt, and the human artist had made an image, then the person giving the instructions would not have been the creator unless work-for-hire applies (the human carrying out the instructions would have been the creator-in-fact), and that therefore they aren’t in the Midjourney case either.
To be consistent with both the memo and Thaler, the theory seems like it has to be that Midjourney is the creator-in-fact, and therefore the human isn’t (and can’t get a direct copyright as the creator), but also that software can’t be hired in the work-for-hire sense and therefore the human can’t get the copyright that way either. Which seems odd! It seems to acknowledge that the software is the creator-in-fact, but then deny both making the software the creator-in-law (because not human) and making the user the creator-in-law via work-for-hire (because I’m-not-sure).
Some other countries are different and imho somewhat more sensible about this, as in the UK’s Copyright, Designs, and Patents Act, of which Section 178 explicitly talks about “computer-generated” works, meaning “that the work is generated by computer in circumstances such that there is no human author of the work”. That’s still imho a little sketchy (I continue to think that Kashtanova is in fact the human author of the images in Zarya), but at least it then provides that “In the case of a literary, dramatic, musical or artistic work which is computer-generated, the author shall be taken to be the person by whom the arrangements necessary for the creation of the work are undertaken.”
There’s still some room for doubt there, as for instance whether it’s Kashtanova or the Midjourney people or some combination who relevantly undertook the arrangements, but at least we aren’t in the position of saying that the author is a being that is not legally allowed to either be a creator, or confer creatorship to a human via work-for-hire.
In the case of the many, many currently-registered copyrights on images made with AI tools (including mine), it seems that if the copyright office is notified, or notices, that fact, they are likely to cancel / withdraw the registration. The theory will be that the registration materials were incorrect when they named the creator as the author of the work, without in any way informing the Copyright Office that an AI tool was used. I could, for instance, send the Copyright Office a note saying “oh by the way I hear that you want to know when AI tools are used, and in my case Midjourney was”, and then they might cancel my registration on their (imho mistaken) theory that I’m not really the author.
Since I believe their theory is mistaken, I’m not currently planning to do that. :)
If they discover it on their own hook and send me a letter telling me they’re withdrawing the registration, I will do whatever easy thing one can do to contest that, but I’m not going to like hire a lawyer or anything; life’s too short.
I’m very curious to see what others do; I would expect that Midjourney itself (assuming it’s big enough to have lawyers) will have their lawyers working on a response to this memo.
My copyrights on the Klara trilogy and Ice Dreams (casually announced here) are secure, as to the text and the image selection and arrangement and all, just not to the images per se. Which is fine. And I haven’t registered those anyway. :)
I should go back and add a note to all of my existing copyright weblog entries, pointing at this one; or, more sustainably, pointing at the entire “copyright” tag on the weblog here. Then I won’t have to keep updating it.
I’m quite happy I decided not to worry too much about this whole thing, and just make pretty pictures (see pretty picture of concerned purple aliens above).
Updates: as this is a developing topic (as opposed to my usual topics which are Timeless Truths of the Universe), you may want to check the copyright tag on the weblog here for later updates, if this post is more than a week or month old.
ceoln 