David Sirota, who helped create “Don’t Look Up”, sometimes feels like we’re living inside of his movie. Does he share my belief that the looming planetary threat is rogue AI?
Sirota is an award-winning investigative journalist, bestselling author, and former speechwriter for Bernie Sanders. He was nominated for an Oscar for co-writing the story of Don’t Look Up.
Follow David’s work by reading The Lever: https://www.levernews.com/
Timestamps
00:00:00 — Cold Open
00:01:20 — Introducing David Sirota
00:04:34 — Why David Fights Against Power and the Concentration of Power
00:13:46 — From NAFTA to AI: The Warnings We Ignored
00:22:05 — How Big Will the AI “Jobpocalypse” Be?
00:25:28 — Superintelligence & the Parallel to Don’t Look Up
00:28:37 — What’s Your P(Doom)™?
00:31:44 — The Speed of the AI Threat
00:36:26 — Society Is Losing a Collective Capacity to Focus
00:38:34 — Is Climate Change David’s Biggest Existential Concern?
00:45:01 — David Reacts to Bernie Sanders’ Data Center Moratorium Proposal
00:49:11 — Can We Build The “Off Button”?
00:52:08 — “Don’t Look Up” x AGI Mashup
00:54:35 — Why There’s Still Hope
00:58:14 — Living in “Don’t Look Up”
00:59:46 — Wrap-Up: Where to Follow Major AI News
Links
The Lever, investigative news outlet — https://www.levernews.com/
David Sirota on X — https://x.com/davidsirota
David Sirota, Wikipedia — https://en.wikipedia.org/wiki/David_Sirota
Master Plan podcast — https://the.levernews.com/master-plan/
David Sirota, “Hostile Takeover” on Amazon — https://www.amazon.com/Hostile-Takeover-Corruption-Conquered-Government/dp/0307237354
The Three-Body Problem (novel), Wikipedia — https://en.wikipedia.org/wiki/The_Three-Body_Problem_(novel)
WarGames (1983 film), Wikipedia — https://en.wikipedia.org/wiki/WarGames
Adam McKay, Wikipedia — https://en.wikipedia.org/wiki/Adam_McKay
Watch Don’t Look Up — https://www.netflix.com/title/81252357
2023 White House Press Conference featuring Peter Doocy’s AI question —
Why experts fear superintelligent AI – and what we can do about it by ControlAI —
Bernie Sanders & AOC’s Data Center Moratorium Proposal —
AI Professor Stuart Russell on AGI’s Value —
Stargate Press Conference —
Fox News Stargate Interview with Sam Altman —
AI 2027 scenario — https://ai-2027.com/
Transcript
Cold Open
David Sirota 00:00:00
Every sci-fi movie starts with a scientist not being listened to.
Dr. Randall Mindy 00:00:05
There is a huge comet headed towards Earth.
Liron Shapira 00:00:09
I would claim to you, I’m in the position of the scientists from “Don’t Look Up.” We’re about to enter this age of superintelligence and find ourselves, before we know it, sharing the planet with this vastly superior force that we don’t really have a handle on, that can easily run away. I’m bringing you the message from Geoffrey Hinton, Yoshua Bengio.
David 00:00:27
So I’ve seen a lot of that. It absolutely is something that should be front of mind right now.
Peter Doocy 00:00:34
Literally everyone on Earth will die.
Karine Jean-Pierre 00:00:37
Your delivery, Peter, it’s quite something.
David 00:00:41
If we are talking about it at all, it’s the Trump administration trying to deregulate, preempt all kinds of regulation of AI. I think that’s insane. That is when I feel like I’m inside of our movie, “Don’t Look Up.”
Liron 00:00:55
David Sirota, what’s your P(Doom)?
David 00:00:57
There’s an optimism in me that wrestles with my pessimism all the time. I ultimately am not a doomer.
Dr. Randall Mindy 00:01:09
“Don’t Look Up.” “Don’t Look Up.”
Introducing David Sirota
Liron 00:01:20
Welcome to “Doom Debates.” My guest today is David Sirota, an award-winning investigative journalist, Oscar-nominated screenwriter, best-selling author, podcaster, and former presidential candidate speechwriter for Bernie Sanders.
His podcast on corruption in American politics, called “Master Plan,” has won the National Press Club Award and multiple other journalism honors. He is the founder of “The Lever,” a reader-supported investigative news outlet whose mission is to hold power accountable.
And most relevant to us at “Doom Debates,” he came up with the story for the film “Don’t Look Up.” The film follows two astronomers attempting to warn humanity about a civilization-ending comet.
It’s like “Dr. Strangelove” did in its day for nuclear war. “Don’t Look Up” has become our generation’s warning shot. Initially about climate change, and now for AI doomers like me, the film’s satire feels all too real.
So I’m excited to talk to David about AI in the media, how the left is reacting to powerful AI, and whether we can agree on a roadmap to avoid doom of all kinds. David Sirota, welcome to “Doom Debates.”
David 00:02:26
Thank you. Thanks for having me.
Liron 00:02:28
I got to kick it off by saying I am a “Don’t Look Up” fanboy. That’s definitely one of my top 10 lifetime movies. I just think you really nailed it, and you told a really compelling message.
David 00:02:40
Do you notice the background here? That’s the movie poster from the movie inside of the movie.
Liron 00:02:46
Oh, nice.
David 00:02:47
Remember, there was a movie that was going to come out on the day that the asteroid is going to hit the Earth, and it’s an asteroid hitting the Earth movie inside of the movie. It’s my favorite artifact from the movie.
And yes, it does feel like sometimes that we are living inside of the world that we depicted in “Don’t Look Up.” But as I always say, “Don’t Look Up” is not supposed to be a prophecy. It’s supposed to be cautionary. But I definitely don’t like the feeling that, as some people have put it, it feels like it’s becoming a documentary.
Why David Fights Against Power and the Concentration of Power
Liron 00:03:21
Yeah, exactly. And we’re going to show some funny juxtaposition clips that I prepared with producer Ori. But first, let’s talk a little bit about you. You’ve worked in politics, journalism, screenwriting. What do you see as your focus across your whole career?
David 00:03:36
I think what I’ve tried to do in my career is challenge assumptions and challenge power. I think broadly speaking, that’s been the through line in my career — trying to question concentrated power and challenge it and defeat it where it needs defeating.
Some people who’ve looked at my career said, “Oh, it’s a weird career path. You worked on campaigns, you worked for some politicians, and then you went into journalism, and then after many, many years in journalism, you then went back to the Bernie Sanders presidential campaign for a stint, and then you went back into journalism.” It seems like a lot of zigging and zagging.
And at one level it is, but at another level, it’s all about really trying to challenge the existing status quo. And the work I did in the movie, I think, is all about that too. It’s all consistent with that — trying to grab people by the proverbial lapels and try to get them to think or really, to look up.
From NAFTA to AI: The Warnings We Ignored
Liron 00:04:34
I think it’s the tagline of “The Lever” that you’re trying to hold power accountable, and I think you’re mentioning a concentration of power — some people have way more than others. And that definitely reminds me of Bernie Sanders’ platform, famously talking about the millionaires and billionaires.
Do you feel like the concentration is the issue, or just some people having very little of it? How do you think about that?
David 00:04:58
I think that concentrated power and concentrated wealth is essentially the central problem, if not the biggest problem that we face. What I think about power is that there need to be countervailing powers. Right now, things are out of whack, they’re out of balance because so few have so much power, as opposed to more of an equilibrium where power is more distributed, and there are countervailing powers that create a kind of balance.
Back when I think our country was working somewhat better — and I don’t idealize the past — but if you just think about the economy, there was a bigger, more prosperous middle class when you had both capital, corporations, billionaires, and the like, but you also had a powerful labor movement as a countervailing force. And so the fights between capital and labor created more of an equilibrium in the middle.
To create a more prosperous middle class in the middle of the 20th century. Wasn’t to say there weren’t very rich people back then. There were very rich people back then, but there were fewer insanely rich people back then and many more middle-class people back then. And I think it’s because power at that time was more distributed, it was less concentrated.
Liron 00:06:29
That’s an interesting perspective. I’m not sure I personally agree, but let me ask you it this way. Do you think that the average middle-class American, the median of the bell curve of income in the year 1950, would they be in a more empowered position than the median American today?
David 00:06:46
That’s a great question. It’s hard to answer exactly. I certainly think — and let’s be clear, there are other factors in American life. Back then, race was a huge factor in where you could live, where you could work. This is pre-Civil Rights Act America. It was almost an apartheid state.
But all things being equal, I certainly think middle-class people had more political representation back in the middle of the 20th century than they do today. I think today, big money so dominates our political process, so dominates our political discourse that middle-class and working-class people are barely at all represented in the government, barely represented in the civic and political discourse.
How Big Will the AI “Jobpocalypse” Be?
Liron 00:07:52
I’ll put a pin in that just because we have so many other more pressing topics to get to. But certainly, I feel like what you’re saying is arguably consistent with the left wing in modern politics. And do you object to me saying, “Yep, you’re a left-wing guy.” Do you identify as that?
David 00:08:09
Sort of yes and no, I guess. Yes, there are certain tenets that we’ve come to call left that have been my viewpoint. If we’re defining as left that I think the central problem is that power and wealth are concentrated — if that makes me on the left, I guess that makes me on the left.
I would say if you look at polls, I would say that puts me firmly in the center of American public opinion.
Liron 00:08:39
Hmm.
David 00:08:39
So I guess what I’m getting at is where I would say no is I think these terms don’t mean very much anymore. What you call left, I may not see as left. What I call left, you may not. So it’s not to be evasive, it’s more that I’m just more precise about — if you want to ask me where do I come down on this or that issue, I’m happy to tell you that. I just think the labels perhaps used to be more clarifying, and today I think they’re actually more obscuring. They make it actually harder to figure out what’s actually going on.
Liron 00:09:11
When you were working with Bernie Sanders, you go back with him pretty far. First you were working with him as his staff, and then you were a speechwriter in 2020, and I know he calls himself a Democratic socialist. Do you feel like that’s representative of your preferred policies today?
David 00:09:28
Again, these labels, the problem is that they’re changing all the time. I can tell you this. I believe that the central problem in American life is the concentration of wealth and power.
I believe that the public sector, that the government, has a much more assertive role to play in making sure that everyone in this society can access the basic core necessities of remaining alive as a human being. And those are pretty straightforward. You need medical care, you need food, you need shelter, and you probably need to be able to retire. And to compete in the economy, you need an education.
And I think that the idea that we as a society, as the richest society in the world, do not provide some of the bare minimum necessities to everyone is extremely bad for the society. It’s immoral. It’s also bad for the economy.
As an example, I don’t think it’s good for the economy that many Americans cannot afford medical care, or that if they have medical care, it is tied to their job, which means that they can’t necessarily go leave their current job to start a small business and the like because they are tied to their job through a corporate healthcare system. I think that’s very bad for the economy. It’s very bad for innovation. On top of things, it’s also very bad for people’s stress levels.
So that’s an example of where I can be precise about what I believe in, and I think that’s more revelatory about my beliefs than any one label or the other.
Liron 00:11:23
That’s very fair. I agree with most of what you said. I think most Americans would agree with most of what you said, so I get why you’re saying what you might call left wing seems like the majority. And I agree that when you state the platform like that, it does seem quite unobjectionable, and it probably is in the mainstream. So that’s very fair.
If we were to debate present-day politics, we could probably get into a discussion of, okay, well, how much free market dynamics can you inject?
David 00:11:49
I’ll admit it. For sure. My first book that I wrote was called “Hostile Takeover,” which is really about how to own American politics. You don’t just need cash in an envelope to pay politicians. You need to control the entire discourse.
And I certainly think that we live in a society where our politics — now I will use some of these words — our politics are way to the right of where the American public is on various issues, particularly economic issues. The “center of American politics,” quote unquote, if you put the exact precise center of something like the US Senate — the center of the US Senate, the most centrist senator in the US Senate is very, very far to the right of where the center of American public opinion is.
And so part of the reason these terms are difficult to use is because a “centrist,” quote unquote, a “moderate,” quote unquote, in the US Senate is a far-right conservative in the population at large. And I think that’s why these labels can be obscuring about where we actually are.
My view is that if we’re going to use those labels — for instance, the center, are you on the center, are you on the left, are you on the right — we shouldn’t be judging the center as where is the center of the US Senate, an institution full of millionaires, people who are backed by billionaire money. It’s a very select, specific population in the US Senate.
Where we should be using those labels, what is the center of American public opinion? And if we’re using those labels, I don’t consider myself on the left at all. I consider myself a centrist. I consider myself squarely in the center of American public opinion on most issues.
Superintelligence & the Parallel to Don’t Look Up
Liron 00:13:46
Okay, so present-day politics, I’d probably be more economic right than you. And I would have a focus more on making sure we get the fruits of capitalism, because I think capitalism has a lot to offer in our current human-based economy. But rather than even try to make that argument and get into the weeds, I think it’s more productive to just be like, “Okay, yeah, maybe, but what’s coming is this AI wave, and the AIs are going to take the economy and run away with it to a degree that even people like you who are pointing out inequality in the human world, that’s even going to dwarf that.”
So I can easily get on the same page as you being like, “Oh, there’s a hell of a lot of inequality coming,” and I think we might as well just look a few years ahead and focus on that. Is that fair?
David 00:14:28
Well, I’m glad we’re talking about this because I’m in the middle of writing something about this, and I have a lot of thoughts about where we’re going with AI, specifically when it comes to jobs, the so-called AI jobpocalypse. I think there’s a lot of panic out there. I understand where it’s coming from. I think the panic is legitimate.
But I think the panic tells us a lot about where we’ve been as a society. I think a lot of the panic is coming from the same part of the population that did not care about what happened to blue-collar workers during the last economic transition.
There were a lot of white-collar workers, information sector workers, suburbanites, winners of the original information economy, who voted for and cheered on the politicians that were touting neoliberalism, things like free trade. Who cares about the blue-collar workers who were watching their jobs go away during NAFTA?
People who portrayed people like Bernie Sanders as a Luddite for saying that throwing millions of blue-collar workers out of their job is unacceptable. Bernie Sanders — I was there on Capitol Hill when Bernie Sanders, in 2000, was portrayed as sort of a troglodyte, Luddite, primitive demagogue for saying, “Hey, before we do something like NAFTA, before we do the China trade deal that outsources and kills off millions of blue-collar workers’ jobs, maybe we should think about how to make that transition more acceptable and survivable for millions of American workers.” And he was essentially scoffed at by the elite of this country, by the media of this country.
And the voters, the voting base of many voters in both parties supported the politicians who were touting that neoliberalism. So all of that is to say, I think the panic now coming from the same part of the electorate, part of the society that scoffed at the economic destruction of the working class — I think where the panic comes from is that set of society knows what happened to those workers and now fears the same thing is going to happen to them.
And so that panic, I think, is very real, and it comes from knowing what happened in the past. And there’s sort of that old adage — first they came for so and so, and I didn’t speak out. Then they came for so and so, and I didn’t speak out, and now they’re coming for me.
And I think now all of a sudden, the white-collar class realizes that they are not immune from the same powers that destroyed America’s blue-collar working class. And so I think what that should forge is an awakening and an understanding that, okay, we really didn’t do right by blue-collar Americans in the last economic transition. We better find some solidarity with blue-collar America right now, some solidarity to understand that the problem is the people at the very, very tippy top willing to do to everybody what they’re planning to do via AI.
In terms of essentially killing off millions of jobs and nobody has any way to survive. Not to go on too long here, but when I read the articles about this many millions of jobs are going to be killed off, or Jeff Bezos wants to buy manufacturing companies, put in AI, and there will be fewer manufacturing jobs in the country — one part of me says, “Hey, I could imagine this being seen as a utopia,” in the sense that a utopian vision is people don’t have to do such back-breaking work anymore, and people can find other ways to do creative and constructive pursuits.
But that presumes an economy where we actually take care of people and make sure that the basic necessities of life are taken care of. Instead, what we’re talking about in America, because we never created those institutions — like other advanced economies did, by the way — instead, we’re talking about a dystopia. Throw out tens of millions of people from their jobs. They have no way to survive, while the tech oligarchs become even richer than they already are.
And just to conclude this, when I said we didn’t have to do it this way — if you look at a place like Sweden, as just one example, Sweden is more trade-exposed, their economy is, and their workers are, than American workers after NAFTA. But Sweden has put in place a set of policies to make sure that if a worker’s job is eliminated because of trade, their job gets offshored, the factory gets offshored, et cetera — that does not mean they’re suddenly going to be thrown out of their home or not have healthcare or not be able to retire. We need to catch up right now to where many of those advanced industrialized economies already are that created those institutions for the trade adjustment. We need those institutions for the AI adjustment.
What’s Your P(Doom)™?
Liron 00:22:05
So that’s a good spelling out of your position. I get that you’ve been sensitive to blue-collar workers and classes of people that have been losing their jobs in the past, and now it’s going to happen in the future. I don’t know if I would’ve agreed with you that much in the past that we have to worry about it too much. Maybe I’m one of the guilty parties who didn’t take Bernie Sanders seriously enough in the past, and maybe there were arguments in the past why the upsides outweigh the downsides. But I think at this point, you and I are on the same page about there’s this huge jobpocalypse coming up, correct?
David 00:20:51
Right. Exactly. Listen, I will say, I do have questions, technological questions, about how big that jobpocalypse is going to be. I do think that Big Tech is selling a story to investors, and I’m not sure that I 100% trust their story.
In other words, the people saying there’s going to be as huge a jobpocalypse as it seems have a vested interest in saying that. Because they’re saying, “Our technology is so amazing, you should be investing in it.” And yeah, it may cause all sorts of these dislocations. So I’m not sure I know how big the jobpocalypse will be because the people selling it are not trustworthy.
But presumably, there will be some transition. There will be some dislocation. And so I think unless we want what happened to blue-collar workers after NAFTA to happen to millions of white-collar workers right now, which I certainly don’t, then we better educate ourselves on what we did to those blue-collar workers and realize that to allow that to happen was absolutely wrong, as Bernie Sanders had been saying for decades, and start adjusting as quickly as possible.
Liron 00:22:13
Okay. Well, the next piece of the puzzle — you hinted at it — not understanding quite how big the wave is. So my own view as a technologist and somebody who’s been studying computer science — I’ve never worked in AI per se, but I’m in a good position as an analyst, basically.
I would tell you that it’s going to be extremely huge, the level of disruption, not even comparable to an economic revolution, but more comparable to a phase of the evolution of Earth. Not even life on Earth, because you can’t build an intelligence like this out of biological cells. To even categorize the kind of transformation that’s coming, I would say it’s a mind-expanding exercise.
David 00:22:56
Yeah. I certainly can allow my mind to go there. I guess what I would say, just as an example — I’m a journalist. There’s been a lot of talk about how the AI apocalypse, or the AI jobpocalypse, is going to destroy journalism.
My view is more nuanced on that, just in my own little world, my own industry that I’m in. I believe AI tools can be incredibly powerful tools for journalists. I believe that journalists who are currently in the business of simply packaging already existing information, in the business of punditry, in the business of hot takes, in the business of everything other than what we might call original journalism — actually talking to other humans, digging out actual revelations — if you are a journalist and you are not in that business, your job is AI-exposed. Your job may be made completely obsolete by AI.
And you know what? I don’t have a lot of sadness about that. I think right now in the media industry, we are immersed in a lot of noise that has very little value. And my hope is that if AI does create a transformation in the media industry, it will start preferencing the jobs that AI can never replicate, and that AI can be a tool for those journalists doing real human reporting to help that real human reporting be better.
Now, I don’t know how replicable what I’ve just outlined in the media industry — how that also applies to other industries. But I guess what I’m getting at is, when I say I don’t know how big the jobpocalypse will be, what I’m saying is I’m not sure how much of the jobpocalypse will be millions of jobs will go away, sort of on net, or how much of the jobpocalypse will be mitigated by some set of jobs will go away, creating more of a premium for a whole other set of jobs that we haven’t put enough of a societal value on.
The Speed of the AI Threat
Liron 00:25:28
Before we talk more about these details of what kind of jobs are going to get unemployed and maybe how do we redistribute wealth or what do we do — these are all interesting details, fascinating details — but in my mind, there is an even bigger elephant in the room, which we should get out there, which is superintelligence. Do you have any thoughts on this idea of vastly superhuman intelligence that may be coming down the road soon?
David 00:25:49
Yeah. Initially, obviously, I think it’s a terrifying thing. It’s exciting, but it’s also genuinely terrifying. It’s terrifying in the same way that the idea of communicating with an extraterrestrial superintelligence would be.
I think we know as humans — as in, I don’t know if I’d call us superintelligent, but we know as intelligent beings — we haven’t treated other species that well, to say the least. So I don’t think it’s much of a jump to worry about, hey, if there’s an equally or more intelligent being out there, whether it’s organic or whether it’s a machine, we know how we’ve behaved, so we could presume or at least be fearful that the other kind of intelligence will treat us just the same.
That’s pretty terrifying. There’s a great book, “The Three-Body Problem,” that kind of explored this idea. So superintelligence is terrifying at that level.
Again, I am not a technologist, so I don’t feel qualified to know how real the prospect of superintelligence actually is.
Society Is Losing a Collective Capacity to Focus
Liron 00:27:15
So I would claim to you, I’m in the position of the scientists from “Don’t Look Up.” I would claim to you that we’ve got this asteroid — or it was a comet, I believe, in the movie — approaching maybe not six months, maybe we have a few years, and we’re about to enter this age of superintelligence and find ourselves, before we know it, sharing the planet with this vastly superior force that we don’t really have a handle on, that can easily run away.
And we’re talking a few years away. And this isn’t just crazy Liron on “Doom Debates.” What I’m telling you right now, I’m bringing you the message from Geoffrey Hinton, Yoshua Bengio, the independent research institutes, a lot of people who work at the AI companies.
Narrator 00:26:56
Experts are warning AI could soon cause human extinction.
Geoffrey Hinton 00:28:01
I’m more concerned for whether my children and the children of my friends are going to survive.
Liron 00:28:05
I’m just transmitting the message of—
David 00:28:06
Yes, I’ve seen a lot of that directly from them. And it absolutely is something that should be front of mind right now. I think the fact that we’re barely talking about it in our politics, and if we are talking about it at all, it’s the Trump administration talking about trying to deregulate, preempt all kinds of regulation of AI. I think that’s insane. That is when I feel like I’m inside of our movie, “Don’t Look Up.”
Is Climate Change David’s Biggest Existential Concern?
Liron 00:28:37
Right. Exactly. So let me just ask you this then, because you’ve been epistemically modest. You’ve expressed epistemic humility, you’re not sure what the situation is. But there’s different degrees of uncertainty. So let me just ask you the number one question of the show. You ready for this?
David 00:28:53
Go for it. P(Doom). P(Doom). What’s your P(Doom)? What’s your P(Doom)? What’s your P(Doom)?
Liron 00:29:01
David Sirota, what’s your P(Doom)?
David 00:29:02
What? My P(Doom)? What does that mean?
Liron 00:29:04
It’s your probability that we’re doomed.
David 00:29:07
Oh, I see. Oh, boy. The problem is it’s weird. I’m an eternal optimist caught in a...
I put it this way. There’s an optimism in me that wrestles with my pessimism all the time. I ultimately am not a doomer, and I’ve never really asked myself why exactly I am not a full-on doomer.
And I think the answer — and I’m just speaking right off the cuff here — I think the answer is because maybe it’s as simple as somehow we’ve made it this far. We created a technology to blow up the entire world, and we somehow, in the last — what is it, I guess it’s 80 years — we haven’t actually blown up the entire world with nuclear weapons. And I think we certainly could have.
I think we’re making an unfortunately good effort to blow up the world—
Liron 00:30:23
Right.
David 00:30:24
With climate change.
Liron 00:30:25
That’s the obvious rejoinder — yes, we didn’t, but we kept getting close.
David 00:30:31
We kept getting close. And by the way, a more slow-motion process, although it’s accelerating now, as I said, has been with climate. We’re coming close to blowing up our own livable ecosystem.
Liron 00:30:39
And what does the world look like right before you blow it up? It looks like you kept getting close until eventually you did it.
David 00:30:45
That’s right. I tend to think that the sort of apocalypse scenario will be somewhat slower than we think. I’m not talking about decades here. I’m just talking about that the movie version of an apocalypse is it’s one instant, and I don’t think that’s the way it happens. I think it is more like the frog in the boiling water, and I think the water is pretty damn warm right now and getting warmer.
And I’m wondering whether we’re going to have that moment where the collective mind is focused. Listen, I also helped make the movie “Don’t Look Up” where the mind never gets focused, and that’s the doom scenario.
David Reacts to Bernie Sanders’ Data Center Moratorium Proposal
Liron 00:31:44
Have you heard of the arguments why we’re going to be screwed on the time side, where we don’t really have that reflective period? It just kind of comes on, like COVID, the exponential part.
David 00:31:53
Yeah, tell me.
Liron 00:31:53
So one analogy — this does a lot of work for me — is if you think about human intelligence coming on the scene in evolutionary time, the biological world wasn’t prepared to evolve to deal with the human brain. Organisms aren’t supposed to have this highly intelligent brain. They’re just supposed to stick to their niche and execute all of these adaptations to their niche.
And then humans come along and we’re like, “Hey, we don’t even need to evolve. You put us in another niche, we don’t need to evolve another thousand generations. We can just start cutting down this niche right now.” We can start engineering this niche to be a niche for us, not a niche for you. And so all of the animals were blindsided — and yes, some of them still live underground, but certainly the macrofauna are screwed, because they couldn’t react in evolutionary time.
And there’s a similar argument to how we are going to be like these slow-moving plants when AI is coming along. It’s going to be much smarter than us and much faster. It can organize itself and make things happen and do science and engineering just much faster. The timescale is going to be shockingly fast.
David 00:32:50
Yeah. Look, I believe it. I definitely believe it. I don’t understand why we are not listening to the warnings. It does feel like we’re in a sci-fi movie. And as I think it was Neil deGrasse Tyson said, every sci-fi movie starts with a scientist not being listened to. It’s basically every sci-fi movie.
Peter Doocy 00:33:21
There’s an expert from the Machine Intelligence Research Institute who says that if there is not an indefinite pause on AI development, this is a quote, “Literally everyone on Earth will die.”
David 00:33:33
I mean—
Karine Jean-Pierre 00:33:34
Your delivery, Peter, it’s quite something. We have seen the letter. We understand what their concerns are. We put our blueprint out back in October. I just don’t want to get ahead of our findings.
Can We Build The “Off Button”?
David 00:33:46
I don’t understand why we’re not listening to people inside of — I shouldn’t say I don’t understand. I do understand why. There’s a lot of money to be made in the short term. I think there’s now this national security argument being, “If we don’t build up our AI capacity, China will.” It’s all the old tropes, right? It’s greed, it’s fear, national security fears.
It’s a world that can’t work together, a world whose human species is factionalized among countries and governments and the like. So it really does feel like the classic sci-fi movie that we’ve been warned about.
Look, again, I’m not naive. I do think that we have edged closer. Look, I don’t know how old you are. I’m a child of the ‘80s. I actually was born in the mid-’70s. I remember growing up and the fears of an imminent nuclear war were extremely real. There were drills in schools.
There was a terrific movie called “WarGames,” which I don’t know if you’ve seen, the old ‘80s movie, which is sort of, by the way, about artificial intelligence. Sort of. If you could call the launch machine artificial intelligence getting out of control—
Liron 00:35:10
Yeah, I didn’t see that one. I think I managed to not see it. I saw a couple. I definitely saw “Dr. Strangelove.”
David 00:35:14
“WarGames” is a great movie. And it’s sort of about — just for those who haven’t seen it — it’s sort of about the US government deciding it can’t trust human beings to pull the trigger on a nuclear weapon because human beings will get sentimental. And so they plug the nuclear launch codes into an AI to make sure that if and when a launch needs to happen, the AI will order it and have no feelings about it. And it’s sort of a cautionary tale about that.
And that movie came out, and it scared the crap out of a lot of people at the top of the government, including President Ronald Reagan. And soon after, Congress passed a series of laws, the Computer Security Act, et cetera, et cetera, that actually started addressing the central fear in the film.
Now, we’re not living in the 1980s anymore. It’s hard to imagine a cultural product grabbing everybody by the lapels and shaking us into political awareness. We certainly tried with “Don’t Look Up.”
“Don’t Look Up” x AGI Mashup
David 00:36:26
I think where I get really doomer is not necessarily on the technological side. You’ve articulated a number of things on the technological side to be afraid of, and the scenarios are frightening. I think I already kind of know that.
I think what’s scary is that we’re this far along and I’m not sure we have the capacity anymore to focus the collective mind on a set of problems in the way perhaps we were able to in the 1980s.
I’m not idealizing the 1980s, but there was more of a collective consciousness, I think, before the internet information system and sort of the noisy mediascape where we can’t even agree anymore on a basic set of facts or on what are the big problems we need to address right now. That’s the scary part.
Liron 00:37:25
Yeah, that is a big part of the scary part, and you satirize that so well in “Don’t Look Up” — the famous scene where the scientists are coming to warn people on the news that this comet is coming—
David 00:37:36
Right.
Liron 00:37:36
—and the broadcasters just react like, “Oh, wow, yeah. So it’s going to blow up everybody’s house. It’s going to blow up my mother-in-law’s house,” and they—
David 00:37:43
Right.
Liron 00:37:43
—laughing about it and moving on to the next segment. They can’t — yeah, everything is ironic.
Jack Bremmer 00:37:47
As it’s damaging. Will it hit this one house in particular that’s right on the coast of New Jersey? It’s my ex-wife’s house. I need it to be hit. Can we make that happen?
Brie Evantee 00:37:55
Oh, you and Shelly have a great relationship.
David 00:37:58
“Don’t Look Up” is fundamentally not about — it’s not necessarily about climate change or the pandemic or AI. It’s really about whether we have a cognitive ability as a society to focus on what’s important or not.
And by the way, I don’t necessarily think we all have to agree on everything. In other words, I don’t think we all have to agree on what the right particular policy is. I think, unfortunately, we’re still at a point where we can’t even agree what the big problems are — what the central priorities should be.
Why There’s Still Hope
David 00:38:34
To my mind, I think if you’re honest with it — if you had a pyramid of important problems that are in front of us as a species — you’d put climate change, literally the livable ecosystem. We’re a tiny blue dot floating around in a vast vacuum. This spaceship needs our life support system to work. I think AI is obviously up there.
I think if you go down just to the United States, it is an existential problem for people who can’t afford medical care to stay alive. I’m just going down the list of what we say are the big problems that are existential, both as a collective society and as individuals. The things we need to remain alive.
And I think that the information ecosystem crisis, the attention economy crisis, is that we cannot even agree on what I just said. What I just said — there isn’t even consensus around those being three of the biggest problems.
Liron 00:39:56
Yes. And regarding what you just said, I did want to get a sense of your prioritization here. So let me ask you the question this way, because when I asked you your P(Doom), you said you’re not a doomer, but you’re open to the idea. So as a probability, it sounded like your P(Doom) is — I think it’s fair to say it’s maybe more than 10%, right?
David 00:40:12
Oh, yeah.
Liron 00:40:12
Okay. All right. So you’re in the double digits then, which is what I call the sane zone. I just feel like everybody should be somewhere in the double digits, personally.
So you’ve got this P(Doom). Let me just ask you this, though, in terms of prioritization. Let’s say the year is 2100. You get to peer into the future, and you see that, oh, in the year 2100, there’s no more humans, and there never will be. We screwed up the future so bad, there’s literally never going to be humans. Eventually, aliens are going to come by and conquer our section of the giant universe, of the light cone, as they say.
So if you knew that about 2100, what would be your best guess about which existential risk got us?
David 00:40:48
Boy. Well, I might’ve said AI a week ago or two weeks ago or three weeks ago. Now I’m genuinely worried even more imminently about nuclear war. I think the Iran situation has a possibility of unlocking a nuclear exchange, which would be awful.
And by the way, I don’t think it necessarily is one big giant flash of light. I think what I fear is that somebody, a government, the United States government, normalizes through — not a terrorist attack, but a direct, deliberate order to use nuclear weapons, which will then say to every other country, “This is now okay to use.”
We did it once or twice, actually, two nuclear bombs that we used many decades ago, and I think we should all feel lucky that that did not end up normalizing the use of nuclear weapons. What I fear now is if you do it again, you are saying to other countries, “Hey, if you got a big enough problem, you can use nuclear weapons.”
Liron 00:42:16
Yeah, no, I hear you. Nuclear weapons are absolutely up there. So is bioterrorism, biorisk. I think AI is in the big three, and I would submit to you that AI is just coming so fast — the chance that nuclear and bio is going to kill us in the next decade, bio is going pretty fast, so I don’t know, they’re substantial. But just with AI, I still think AI is head and shoulders above the others simply because if you just step back — have you read that scenario, “AI 2027?”
David 00:42:42
No, I haven’t.
Liron 00:42:43
Highly recommend it. It’s super famous for good reason, and the predictions have been very accurate so far. They wrote it in 2024, and they’ve pretty much perfectly predicted 2025 and the beginning of 2026. And also, one of the guys who wrote it did a really good job five years ago predicting the present. So these guys are literally super forecasters. They win all these forecasting tournaments.
And the scenario makes a ton of sense to me. And the way they got the scenario is they didn’t even inject clever ideas. They just stepped back and they’re like, “Okay, what are the strongest trends? What’s the least going out on a limb we could possibly do? We’ll just take trends and extrapolate and not claim to know anything.”
And if you just extrapolate trends on all of these benchmarks for AI, you just extrapolate that it’s wildly powerful, that it’s way more powerful than any human has ever been. And so far, that is coming to fruition. If you look at the latest benchmarks, I can tell you there’s been discontinuous changes where my own job as a software engineer, that I’ve been doing for 20 years — I now just have the AI do the software engineering. I literally just sit there and manage it and don’t write the code myself.
I don’t even know if you’re aware of how fast this AI has been scaling up into the superhuman regime. And this is why I’m telling you, we only have months left, maybe years, probably not decades. And that’s why I think the risk is just head and shoulders above the others.
David 00:43:59
Yeah. Listen, I respect your opinion. I have done my own reading on this, and I definitely think it is a very real and significant threat.
I think that, again, my P(Doom) — is that what you called it? P(Doom)?
Liron 00:44:25
Yeah. P(Doom).
David 00:44:25
Yeah. My P(Doom) is way higher than 10%. But I think the only thing I would say is that I do the work that I do because still somewhere, as jaded as I have become, I still do believe that we have a capacity to change. And maybe that makes me naive. Honestly, it may make me naive.
Liron 00:44:52
And my policy position is similar to the recent statement by your man, Bernie Sanders. He came out and he’s like, “Hey, we need a data center moratorium because—” ## Bernie Sanders and the Data Center Moratorium
David 00:45:01
For sure.
Liron 00:45:01
Yeah, there’s a big problem for jobs, and there’s a big problem for existential risk. So big props to that.
David 00:45:07
For sure.
Bernie Sanders 00:45:08
I want to get back to the point that some of the founders of this industry, people who know more about it than anybody else in the world, are telling us that within a few years it is likely that AI will be smarter than human beings, that human beings may lose control over AI, with possibly catastrophic impacts. How do you ignore those issues? You tell me.
Liron 00:45:32
I don’t know how related you were to this new development, but I take it you support it, right?
David 00:45:37
Look, I certainly think for many, many reasons, the build-out of data centers needs to, at minimum, absolute minimum, take a pause. The environmental impact, the energy impact, the job impact, and obviously the existential impact — take your pick of what reason you want to—
Liron 00:46:04
Existential, for sure, because I think those impacts are much bigger.
David 00:46:07
Right. Well, I guess my point is that even if you don’t believe in existential risks, you probably will believe that it’s not great that your electricity bill is going up. And I think, by the way, that’s why, in part, in polls, AI is so unpopular, and why big tech is suddenly so unpopular.
There was a poll I saw recently that showed that big tech is the most unpopular, untrustworthy set of institutions in the society now on a poll. So I think the public maybe at large can’t articulate all of these particular reasons, but has a good sense that something is going wrong.
Living in “Don’t Look Up”
Liron 00:47:00
Right. Now, since you’re on the same page as me saying, “Hey, P(Doom) seems high, and I’m optimistic that we can get our act together and do something.”
David 00:48:08
Right.
Liron 00:48:08
People like myself who have thought about what that something is, given the caliber of the problem — it’s a really bad problem, unfortunately.
Arguably, in my mind, it’s actually a much harder problem than climate change because the fix for climate change is we can just have more solar power. We don’t even give up that much. Climate change seems like a relatively easy problem. Unfortunately, the AI problem, on one hand, it’s easy — we just have to not build AI. The problem is that there’s no easy replacement. With fossil fuels, it’s like, “Eh, just geothermal.” There’s so many things you can do — solar, wind.
But here, there’s no replacement for this AI, man. I’m literally using AI in my job. It’s so freaking useful. And it’s only going to get more useful, and there’s just no replacement for it, so everybody’s going to want to build it. So in that sense, it’s hard. And militarily, there’s no replacement for it. It’s more valuable than oil. It’s honestly the most valuable thing that’s ever been created. The size of our economy — my best prediction is that if we don’t all die in the next decade or so, I think our economy will be 20 times bigger.
David 00:48:07
Yeah. I see that potential and that problem. It is a vexing problem. I also see that we have had some success. Again, I use the nuclear build-out. We have had some success putting serious and real guardrails on that technology so that we haven’t blown ourselves up.
I think where maybe I am — not a naive optimist, but at least hopeful, with my eyes open, not naively so — is that we can find ways to boundary what we are creating. Now, I also think that offers a lot of faith in humans, and there’s plenty of evidence that we should not have such faith in humans.
Wrap-Up: Where to Follow Major AI News
Liron 00:49:11
I just want to tell you what I think the ideal policy is, just so at least you know, and then you can always disagree.
David 00:49:15
Yeah, please.
Liron 00:49:15
I originally took it from Eliezer Yudkowsky, bestselling author of “If Anyone Builds It, Everyone Dies.” This was a bestseller a few months ago.
David 00:49:25
Yeah.
Liron 00:49:27
And I’ve been reading him for 20 years.
David 00:49:27
Yeah.
Liron 00:49:28
I think his argument is spot on, and I describe “Doomed Debates” as a Yudkowskian show. It comes from basically his same perspective. So he’s saying the next generation of AI that’s coming, we don’t know how crazy it’s going to get. Maybe it’ll be nice. So far we’ve gambled and we’ve won. Today’s AI, I believe, is net positive. Some people already disagree. I think it’s great so far, but I’m expecting a couple generations down the line to not be great because of what you were saying earlier — how did we treat the other species?
I do think we’re accidentally going to end up in that position, and we won’t be able to tweak it because we won’t even have the power. We’ll accidentally give up the power permanently. That’s what I see coming, that’s what Eliezer Yudkowsky sees coming.
So to get to the policy proposal, he’s saying we need an international treaty where first of all, there has to be an off button. We need to build the button at the very least, so that if we all agree things get crazier, a majority of us in the world agree things get crazy, then we agree — okay, press the off button now before it’s too late. An emergency stop button. At the very least, build that. And there’s all these other proposals related to that.
But the point is, you have to have a centralized treaty where it’s not a free-for-all, where anybody can press the big red button of summoning the demon, unleashing the next AI. What are your thoughts?
David 00:50:39
Yeah, again, to go back to the nuclear age, that reminds me of the phone that they installed between the US and the Soviets. There were a number of near-miss scenarios where we were truly on the brink of one person pushing one more button and a nuclear winter being upon the planet. And we had essentially a fail-safe button, or at least a process to pull us back from the brink, pull the plug, if you will.
The idea that we’re building these systems without prioritizing that — that is the single scariest part of the whole thing.
Now, some may say there’s no such thing as that. You actually can’t build that, that the superintelligence, by virtue of it being superintelligent, could get around any way for us to pull the plug. But I also — again, now you can call me technologically naive — I don’t think we’re there yet.
I do think there must be ways. There are very smart people in the world who must know how to build the off button. And I don’t think the off button is as simple as just pulling the plug out of the computer. I’m sure it’s somewhat complicated, but I do have faith that we’re still at a moment right now where that can be constructed.
The “Don’t Look Up” Mashup
Liron 00:52:08
Yep. All right. Well, heading toward the wrap-up here. Let’s talk about “Don’t Look Up” again. Like I said, love the movie. Here’s a clip I made, which is a mashup. This is from a couple of years ago.
These AI company CEOs are talking like your character, Peter Isherwell, right? Where he was like, “Oh, this comet’s coming. We got to mine the comet because there’s so much value in the comet.” And they’re allowed to talk like this while the comet is streaking over, and the most likely outcome is just, “Oh, it’s going to smash everybody. We’re not going to survive this.” And they’re talking like, “Man, look at all these riches that are coming.” So I thought this mashup was interesting. Take a look. You can see my tab, right?
David 00:52:45
Yep.
[Compilation plays]
“$15 quadrillion. So that’s the cash value, minimum cash value of AGI as a technology. So then you can see why we’re investing relatively large amounts of money. Stargate — so put that name down in your books. The ones making the decisions are the CEOs of the companies. I’m thrilled we get to do this in the United States of America. A couple of them have said that there’s a significant chance, like 10 or 25% chance of human extinction.”
“When the other metals are factored in, the comet actually contains almost $140 trillion worth of assets. Fuck 40 trilly. So they are deciding to play Russian roulette with the entire human race without our permission. What do these trillions of dollars even matter if we’re all going to die from the impact of this comet? I was about to ask the same question.”
“Oh, no. What if we’re rich and we’re safe? For the people who are concerned about AI, what do you say to them that there are rails to make sure that it doesn’t go out of bounds? I have enormous faith we’ll figure it out.”
“And when these treasures from Heaven are claimed, poverty as we know it, multitudes of problems are just going to become relics of the past. We will see diseases get cured at an unprecedented rate. You could do more space travel, mining asteroids maybe becomes feasible. No shortage of goods and services for anyone on Earth. Increase the size of the pie, extend the human lifespan. We will be amazed at how quickly we’re curing this cancer and that one, and heart disease. And humanity is going to stride through the pillars of Boaz and Jachin, naked into the glory of a golden age.”
“And the bad case, and I think this is important to say, is lights out for all of us.”
Why There’s Still Hope
Liron 00:54:35
Does that seem accurate to you?
David 00:54:38
Yeah. That scene in the White House where Dr. Mindy is saying, “What does $32 trillion matter if all of us die?”
All credit to Adam McKay — I helped come up with the story, he wrote the screenplay. That scene is an amazing scene. It’s an incredibly powerful scene, and it’s honestly hilarious, but it’s also incredibly terrifying.
Because it’s satirizing the way these conversations are happening right now, and the conversations that are happening right now are not satire. They’re extremely real. And what’s incredible is that more people aren’t saying, “Hey, what are we doing?”
The self-awareness — that there is no self-awareness. And I think I don’t know where this goes, but I think it’s important, just to give you a sense of why I’m still hopeful.
I want to go back to those polls I mentioned. We talked a lot about how the public has trouble focusing, how we’re in this information miasma, this attention economy, and it’s hard to get people to agree on what’s important. Okay, that’s horrifying, scary, and depressing.
I think where I find hope is that when you look at those polls that show people think something has gone very wrong with big tech and with AI, that at least says to me at a kind of perhaps impulse level, the reptilian brain level, lots of people understand that something is going wrong.
And so even inside of the “we can’t specifically focus on the exact right set of issues or set of solutions,” I do think where we can be optimistic is that even inside of this confusion machine, the general public does have a general sense that this is going in the wrong direction.
And that, I think, is the potential to build off of. That, I think, means that there are lots and lots and lots of people in this country who vote for our elected officials, who then make those decisions. There are lots of people who can hear the messages that we’re talking about, whose brains are open to that.
Liron 00:57:13
Great.
David 00:57:13
And I think it’s on all sides of the political spectrum, which is really very, very good news.
Liron 00:57:19
I totally agree. And that is, in fact, what surveys show — AI is highly unpopular, which is so funny because I remember a lot of the people in the technology sector, which are my friends. That’s where I come from. I’m a technology guy, but I’ve broken ranks with a lot of the technologists being like, “Ah, I agree this is really cool, but you guys are about to destroy the world, unfortunately.”
And I have broken with the technologists, but I haven’t broken with the scientists, I haven’t broken with some of the tech leaders. So it’s an interesting factional war going on. But a lot of these people have been trying to gaslight, being like, “You guys are astroturfing. People don’t hate AI that much.” But you’re telling me as a politics guy, “No, people aren’t happy with it,” correct?
David 00:57:56
No, it’s real. It is absolutely real. We have finally advanced beyond the sort of techno-utopian optimism, and we are now into techno-skepticism, and I think that is a healthy transition.
Living in “Don’t Look Up”
Liron 00:58:14
Yeah. So going back to “Don’t Look Up,” the movie was so meaningful to me because I like movies when I can suspend my disbelief because I feel like they’re realistic. And obviously there’s little tweaks of the science — you didn’t 100% perfectly get the science — but even the idea that a comet is coming and is super dangerous and going to kill everybody, you got that part right.
The reason it’s so realistic to me is just this idea that, okay, here’s an existential threat, and we’re just going to run around and be kind of stupid about it, and most people are just going to do stupid things in reaction to it. And then that day is going to come, and it’s just going to be game over. That’s really going to happen.
That is a causally close event. There’s a simple sequence of cause and effect that can just get you right there, and the movie goes all the way. It depicts that happening, spoiler alert.
I just thought that was so poignant because I personally think we’re living in “Don’t Look Up” right now, and I think this literally could be our last — or our lack of lives — in 10 years. I think we could reach that same game over state, much like what your movie depicts, and that is pretty crazy. So thank you for depicting that.
David 00:59:15
And thanks for saying that, and thank you for trying to sound the alarm. As I’ve said before, I don’t think our movie was prophecy. Our movie was designed to be cautionary. I don’t think we’re all at the dinner table at the end scene just yet.
I do think that we may end up at that dinner table, and the point is to try to — in all of whether it’s journalism, whether it’s what you’re doing, whether it’s movies — is to try to prevent us from ending up at that dinner table.
Call to Action and Closing
Liron 00:59:46
Yep. All right, so my audience, a lot of people have a high P(Doom). My own P(Doom) is 50%. I’ve talked to many people where it’s higher. If they are resonating with your message, what’s your call to action for them? Or where can they find your work online?
David 01:00:01
You can find our work, our reporting at The Lever. It’s levernews.com. We actually report a lot on AI policy and politics.
I do think there’s a very important debate going on right now about whether to preempt states from even regulating AI. I think the fact that the Trump administration is trying to do that is insane. I think it’s incredibly good news that when the Congress tried to pass that, they were about to pass it, it became very, very public, and when the Senate had to take a public vote on it, it was voted down overwhelmingly.
Which shows you that even the Senate, a sort of corrupt institution owned by big tech — when things got into public, even those senators couldn’t find the shamelessness to vote for it. So keeping that pressure on to allow as many regulatory bodies to regulate this, I think it is a messy fight, but it is an incredibly important fight.
If the Congress and the federal government tries to preempt all regulations of this technology, that is one step closer to the doomsday scenario. So these fights — they may seem messy and esoteric and arcane and partisan, et cetera — they are really important, and we cover them aggressively.
Liron 01:01:21
All right. David Sirota, please keep using your writing powers for good. Thanks so much for coming on Doomed Debates.
David 01:01:27
Thank you. Thanks so much for having me.
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
