OpenAI: Where by “can” we mean “can’t”

Disclaimer: I work for Google, arguably a competitor of OpenAI; but these opinions are solely my own, and I don’t work in the AI area at Google or anything.

And I mean, oh come on!

So much AI “research” in these hype-heavy times is all bunk, and I suppose one shouldn’t expect OpenAI (“Open” heh heh) to be any different. But this pattern of:

1. Use an AI to try to do some interesting-sounding thing,
2. Evaluate how well it did by waving your hands around, or just by eyeballing it,
3. Declare victory,
4. Publish an “AI can do thing!!!!” paper that will get lots of media attention.

is just sooooo tiring. (See for instance the YouTuber in that prior post who showed their system producing a non-working tic-tac-toe game and saying “well, that worked!”.)

The one I’m facepalming about here was brought to my attention by my friend Steve, and omg: “Language models can explain neurons in language models“. They did sort of the obvious thing to try to get GPT-4 to make predictions about how a few selected “neurons” in GPT-2 behave for a few inputs. The key line for me in the paper is:

“Although the vast majority of our explanations score poorly, we believe we can now use ML techniques to further improve our ability to produce explanations.” 

— OpenAI

They say this because (they have been drinking too much of the Kool-Aid, and) they tried a few things to make the initial abysmal scores better, and those things made them slightly better, but still poor. They say in the (extremely brief) report that although it works badly now, it could be the case that doing it differently, or maybe doing more of it, might work better.

In any other field this would be laughed at (or politely desk-rejected with a “fantastic, please submit again once you find something that does actually work better”); but in the Wacky Wacky World of Large Language Models it goes on the website and gets cited in half a ton of headlines in the media.

And is it really honest to use “can” in a headline to mean “can, very very badly”? By that standard, I can predict the weather by flipping a coin. I didn’t say I could predict it accurately!

I suppose the LLM hype is better than the Crypto hype because fewer people are being bilked out of money (I guess?), but still…