ChatGPT is not a huge leap forward, it’s an expensive deal with the devil

Sometimes those who forget history are doomed to repeat it. For those of us with long memories, the current hubbub, not the hysteria, surrounding ChatGPT (and “generative AI” in general) sounds familiar.

We’ve been here before: January 1966, to be precise. That was the moment MIT computer scientist Joseph Weizenbaum introduced Eliza, who she would have called the world’s first chatbot if that term had existed at the time. Weizenbaum wrote the software (in a programming language intriguingly called MAD-SLIP) to demonstrate that communications between humans and computers were inevitably superficial. He did this by providing a text box in which one could exchange written communications with the machine. Inside the show was a script (named DOCTOR by Weizenbaum) that parodied the exchanges a patient might have with a psychotherapist practicing person-centered therapy pioneered by Carl Rogers. (The name of the show comes from Eliza Doolittle, the Cockney girl who was taught to “speak properly” in Shaw’s play Pygmalion.)

The way it works is that the program takes what you’ve typed and processes it to produce a response. Suppose you type (as I just did): “I’m a bit down about UK politics.” Eliza: “Do you think coming here will help you not get depressed?” I hope so”. Eliza: “You say that you wait for it for some special reason?” You get the point, but you can try it yourself: just go to

Weizenbaum wrote the program to show that while machines could ostensibly copy human behavior, it was actually like a magician pulling a rabbit out of a hat: an illusion. And once you know how the trick was done, Weizenbaum thought, it’s no longer an illusion. There was nothing secret about Eliza: if you read the code, then you could understand how it worked. What puzzled her creator was that even if people knew it was just a show, they seemed to take it seriously. There is a famous story about her secretary asking her to leave the room while she was having her “conversation” with Eliza. The people was absolutely ecstatic about it. (I saw this myself when I once ran it on a PC at my university open house and had to get people off the machine so others in the queue could try.)

The intriguing echo from Eliza thinking about ChatGPT is that people consider it magical even though they know how it works.

After Weizenbaum’s article on Eliza was published, some people (including some practicing psychiatrists) were quick to start saying that if a machine could do this kind of thing, who needed psychotherapists? Weizenbaum was as appalled by this as today’s educators and artists are by contemporary drooling over the tools of generative AI. For him, as one insightful commentator put it, “there was something about the relationship between a person and his therapist that was fundamentally an encounter between two human beings. In language sometimes reminiscent of Martin Buber’s ‘I and you’ formulation, Weizenbaum remained obsessed with the importance of interaction between human beings. In that sense, he was not only a distinguished computer scientist, but also a notable humanist.

This humanistic outrage fueled his lifelong opposition to the technological determinism of “artificial intelligence.” And reported his 1976 book, The power of the computer and human reasonwhich confirmed his role as a thorn in the side of the AI ​​crowd and ranks with Norbert Wiener Human use of humans by exposing the reservations of a tech-savvy about the direction of humanity’s journey towards “automating everything”.

The intriguing echo of Eliza’s thinking about ChatGPT is that people consider it magical even though they know how it works: as a “stochastic parrot” (in the words of Timnit Gebru, a well-known researcher) or as a machine for “high-speed plagiarism.” technology” (Noam Chomsky). But we really don’t know half of it yet, not the CO2 the emissions incurred in training its underlying language model or the carbon footprint of all those delighted interactions people have with it. EITHER, passed Chomsky, that the technology only exists because of his unauthorized appropriation of the creative work of millions of people who happened to be on the web? What is the business model behind these tools? Etc. Answer: we don’t know.

In one of his lectures, Weizenbaum pointed out that we are incessantly making Faustian deals with this technology. In such contracts, both parties get something: the devil gets the human soul; humans obtain the services that delight us. Sometimes compensation works for us, but with these things, if we finally decide that it doesn’t work, it will be too late. This is the deal that generative AI now brings to the table. Are we prepared for it?

what i’ve been reading

He New York Times’ Self-obsession is an excoriation political Jack Shafer column.

visions of hell
Ken Burns on His Most Important Film is an interview conducted by Baris Weiss on the Free Press website about American attitudes to the Holocaust.

monopoly rules
Understanding the antitrust case against Google is a good explanation from Matt Stoller at Substack of a really complex issue.

Leave a Reply

Your email address will not be published. Required fields are marked *