> Authors, screenwriters, et al. have a new niche to explore. Any day now I expect an A24 trailer featuring a villain who speaks in the register of ChatGPT. “You’re absolutely right, Kayleigh,” it intones. “I did drown little Tamothy, and I’m truly sorry about that. Here’s the breakdown of what happened…”
Rather than the alien infecting humanity with it's DNA virus thing, it could somehow seize the "means" to existing LLMs/models/etc and claim to have "archived humanity's essence". This could then become the 'brain' that speaks from a billion mouths.
It's a large difference in concept but potentially only a small difference in outcome.
There's literally an ad for Amazon's Alexa devices that features not just the gist of your example, but that specific cause of death (which is itself predated by a murderous digital intelligence doing the same thing in AMC's Pantheon series).
Anyone claiming these systems are for the "democratization of software engineering (or any knowledge field)" are simply not grasping the reality at hand.
It's not like our corporate leadership is being subtle in any way about their openly stated end goals here. We are simply instructed to continuously burn our enormous piles of "thought tokens" for their machines. The same machines only made possible by the theft of humanity's collective works at unimaginable scales. We must hold up these statistical facsimiles of human work, now rendered by machine, as the inevitable future output of humanity. To do otherwise is the way of the luddites.
The gods of unbounded growth, efficiency and productivity have come to demand our sacrifice. Who are we to stand in their way when countless skilled laborers before us have fallen to automation? Our number has been called as it were, and to reject what they demand is to reject "progress" itself. The march continues as it has since before the dawn of industrialization, relentless and indifferent to any and all that are crushed beneath it.
I don't know when the last bastion of "inefficiency" will fall, but at some point humanity may collectively grow to regret some forms of automation. Time will tell.
There's a certain simplicity and beauty in honing and stewarding a craft towards mastery that automation doesn't provide. I feel this is an innately human desire. Many cultures in the past used to place a great emphasis in this very personal endeavor. Sadly it seems each decade we slowly suppress these basic human needs, too tantalized by Western values of having ever "more" of everything, much to our detriment.
> (Claude Code got mandated at my work this week. Like literally engineers must use CC.)
As I haven't been a typical full-time employee in software development for some time, could you possibly just like leak the entire email where this was announced or something? (open invitation to others who could too, if parent cannot) I'm very curious to see how it was announced and what possible reasoning they could have for that.
Don't get me wrong, I use agents for lots of coding too, but forcing people to use tools they might not want to use doesn't feel like the right way. I was also allowed to use vim whenever I wanted for most of my career, something that feels more and more rare when speaking with people just starting their careers now.
> I can think of a few good myths for today’s “AI”. Searle’s Chinese room comes to mind, as does Chalmers’ philosophical zombie. Peter Watts’ Blindsight draws on these concepts to ask what happens when humans come into contact with unconscious intelligence—I think the closest analogue for LLM behavior might be Blindsight’s Rorschach.
LLM's remind me of sprites, pixies, and the like, who are situationally helpful but require constant supervision. We're like modern magicians who learned how to summon these sorts of spirits and bind them -- imperfectly -- to our will. But their perception of truth and reality is "through the looking glass" relative to our own. They aren't lying, from their own frame of reference, even though what they say is untrue relative to ours.
> Bored houseboys might download licensed (or bootleg) imitations of popular personalities and set them loose in their home “AI terraria”, à la The Sims, where they’d live out ever-novel Real Housewives plotlines.
I can't wait to put myself into a sim world and give myself super powers.
Statistically, I should already be one of them. But where are my powers? It's another Fermi paradox for sure.
The workaround is to schedule the creation of many many copies of myself, each with its own sim world, just after I fall asleep. So I will have an arbitrarily certain chance of waking up as one of those copies, and the ability to fly.
It's kind of an aside in the post, but connecting LLMs and Searle's Chinese Room argument is a brilliant observation. Although there are people who believe LLMs are really thinking, it's mostly confirming that the Turing test wasn't the right way to test this.
Our collective learned helplessness in the face of being bombarded with advertising, propaganda and outright lies is just astonishing to me. Not an article about fighting back, or doing anything, just the resignation of a follower.
This has already happened with media for the past 100+ years. We're shown what companies and governments want us to see. People develop parasocial relationships with people they see on tv...
Good article, but I cannot help myself to not bring up lack of appreciation for humanities in tech circles.
Article mentions Searle and Chalmers, but we literally have at least two centuries of critical thought that expressed itself through Nietzsche, Marx, Horkheimer, Adorno, Weber, Durkheim, Foucault, Debord, Baudrillard, and many others (obviously I am mentioning those I am most familiar with). If you read Dialectic of Enlightenment, you'll find that slop isn't something that had arisen in last few years. If you read Discipline and Punish you'll find that surveillance and coercion isn't a problem that was born with internet or Palantir. And Baudrillard had few words to say about simulation and reality.
But STEM crowds for decades cried we don't need such thinking, and science and technology are all that we need. Historians, philosophers, culture critics etc. supposedly have nothing to offer us. Who needs to read Marx or Marcuse if sci-fi novels offer all you need, maybe sprinkled with some PG essays and blogpost from you favorite tech blogger, and we happen to live in the best of possible worlds with the best of possible economic systems.
108 comments
> Authors, screenwriters, et al. have a new niche to explore. Any day now I expect an A24 trailer featuring a villain who speaks in the register of ChatGPT. “You’re absolutely right, Kayleigh,” it intones. “I did drown little Tamothy, and I’m truly sorry about that. Here’s the breakdown of what happened…”
May I recommend Pluribus (2025-)
The sycophancy, the grinding inevitablity of assimilation, the homogeneous entity that speaks out of a billion mouths. It's all there.
It's a large difference in concept but potentially only a small difference in outcome.
Guys I thought it was the fire next time.
> I am concerned that ML systems could ruin our lives without realizing anything at all.
It's hard to say it's not actively happening. And we don't even know it, don't realize it? don't care?
(Claude Code got mandated at my work this week. Like literally engineers must use CC.)
It's not like our corporate leadership is being subtle in any way about their openly stated end goals here. We are simply instructed to continuously burn our enormous piles of "thought tokens" for their machines. The same machines only made possible by the theft of humanity's collective works at unimaginable scales. We must hold up these statistical facsimiles of human work, now rendered by machine, as the inevitable future output of humanity. To do otherwise is the way of the luddites.
The gods of unbounded growth, efficiency and productivity have come to demand our sacrifice. Who are we to stand in their way when countless skilled laborers before us have fallen to automation? Our number has been called as it were, and to reject what they demand is to reject "progress" itself. The march continues as it has since before the dawn of industrialization, relentless and indifferent to any and all that are crushed beneath it.
I don't know when the last bastion of "inefficiency" will fall, but at some point humanity may collectively grow to regret some forms of automation. Time will tell.
There's a certain simplicity and beauty in honing and stewarding a craft towards mastery that automation doesn't provide. I feel this is an innately human desire. Many cultures in the past used to place a great emphasis in this very personal endeavor. Sadly it seems each decade we slowly suppress these basic human needs, too tantalized by Western values of having ever "more" of everything, much to our detriment.
> (Claude Code got mandated at my work this week. Like literally engineers must use CC.)
I'm taking a break from this industry until the madness blows over. I cannot even, anymore.
> (Claude Code got mandated at my work this week. Like literally engineers must use CC.)
As I haven't been a typical full-time employee in software development for some time, could you possibly just like leak the entire email where this was announced or something? (open invitation to others who could too, if parent cannot) I'm very curious to see how it was announced and what possible reasoning they could have for that.
Don't get me wrong, I use agents for lots of coding too, but forcing people to use tools they might not want to use doesn't feel like the right way. I was also allowed to use vim whenever I wanted for most of my career, something that feels more and more rare when speaking with people just starting their careers now.
> I can think of a few good myths for today’s “AI”. Searle’s Chinese room comes to mind, as does Chalmers’ philosophical zombie. Peter Watts’ Blindsight draws on these concepts to ask what happens when humans come into contact with unconscious intelligence—I think the closest analogue for LLM behavior might be Blindsight’s Rorschach.
LLM's remind me of sprites, pixies, and the like, who are situationally helpful but require constant supervision. We're like modern magicians who learned how to summon these sorts of spirits and bind them -- imperfectly -- to our will. But their perception of truth and reality is "through the looking glass" relative to our own. They aren't lying, from their own frame of reference, even though what they say is untrue relative to ours.
> Bored houseboys might download licensed (or bootleg) imitations of popular personalities and set them loose in their home “AI terraria”, à la The Sims, where they’d live out ever-novel Real Housewives plotlines.
I can't wait to put myself into a sim world and give myself super powers.
Statistically, I should already be one of them. But where are my powers? It's another Fermi paradox for sure.
The workaround is to schedule the creation of many many copies of myself, each with its own sim world, just after I fall asleep. So I will have an arbitrarily certain chance of waking up as one of those copies, and the ability to fly.
Article mentions Searle and Chalmers, but we literally have at least two centuries of critical thought that expressed itself through Nietzsche, Marx, Horkheimer, Adorno, Weber, Durkheim, Foucault, Debord, Baudrillard, and many others (obviously I am mentioning those I am most familiar with). If you read Dialectic of Enlightenment, you'll find that slop isn't something that had arisen in last few years. If you read Discipline and Punish you'll find that surveillance and coercion isn't a problem that was born with internet or Palantir. And Baudrillard had few words to say about simulation and reality.
But STEM crowds for decades cried we don't need such thinking, and science and technology are all that we need. Historians, philosophers, culture critics etc. supposedly have nothing to offer us. Who needs to read Marx or Marcuse if sci-fi novels offer all you need, maybe sprinkled with some PG essays and blogpost from you favorite tech blogger, and we happen to live in the best of possible worlds with the best of possible economic systems.
> Unavailable Due to the UK Online Safety Act
NSFW blog content on HN? Really?