Justin Helps is the science educator behind Primer Learning with 2M subscribers. We cover how he got into AI safety, debate AGI timelines, and why he calculates p(doom) to be 70% by 2100 😱.
Timestamps
00:00:00 — Cold Open
00:00:38 — Introducing Justin Helps
00:02:03 — What's Your P(Doom)?™
00:03:38 — Justin's First Exposure to AI X-Risk
00:04:49 — Major Disagreements with Eliezer Yudkowsky
00:09:46 — Debating the Timeline to AGI
00:12:24 — Metaculus Prediction Market Estimates AGI by 2032
00:20:06 — Misguided Conceptions of AI's Limitations
00:25:23 — Only a 5% P(Doom) by 2040
00:28:40 — AIs Will Not Care About the Human Species
00:31:00 — Summarizing Justin's Position So Far
00:36:17 — High P(Doom), but We're Not Depressed
00:40:14 — Justin's "Computer Man" Thought Experiment
00:51:16 — Should We Pause AGI Development?
00:54:15 — AI Doom Is a Serious Concern
Links
Primer’s Video on AI Doom —
Primer on YouTube — https://www.youtube.com/@PrimerBlobs
Primer’s Website — https://primerlearning.org/
Justin Helps on X — https://x.com/Helpsypoo
Liron’s recent conversation with Dr. Steven Byrnes —
Liron’s take on Penrose’s Gödel’s Theorem argument —
Metaculus prediction market on AGI —
Harry Potter and the Methods of Rationality — https://hpmor.com/
Feeling Rational by Eliezer Yudkowsky —https://www.lesswrong.com/posts/SqF8cHjJv43mvJJzx/feeling-rational
Pause AI — https://pauseai.info
Transcript
Cold Open
Liron Shapira 00:00:00
Do you basically not expect AGI by 2040?
Justin Helps 00:00:04
I’ve been using Claude Code. It’s very impressive, but it seems like it’s a long way from playing the universe like a video game.
Liron 00:00:11
Well, this gets to my core distinction in my doom argument. Can and will. Can AI kill us? And then if it can, will it?
Justin 00:00:19
On the will side, I’m probably more like 95% doom.
Liron 00:00:23
Wow.
Justin 00:00:23
You’d think we would be more depressed given our worldviews of humanity.
Introducing Justin Helps
Liron 00:00:39
Welcome to Doom Debates. Today my guest is Justin Helps, the science educator behind the YouTube channel Primer, which has nearly two million subscribers and 100 million total views. He holds a bachelor’s degree in physics from Gustavus College and a master’s in material science from the University of Minnesota. He worked for four and a half years for a YouTube educator you may be familiar with, Khan Academy.
Primer excels at simplifying the complex. The videos feature bespoke simulations that reveal the deep ideas behind a subject, and they often star his trademark blob animations. Last year, Primer departed from his usual coverage of biology and economic theory to focus on AI. The channel published a detailed explanation of neural networks, and most recently, Justin’s case for having a high probability of AI doom.
Justin 00:01:27
We just don’t have much we can really say about where the ceiling is for the current AI approach, and we don’t know how they work, and we don’t know how our own minds work for that matter. We’re just kind of in the dark and also sprinting forward, intentionally in the direction of a super powerful alien.
Liron 00:01:46
Primer doesn’t do many interviews like this, so I’m excited he’s granted us the exclusive today to talk about that AI piece. We will compare notes on everything from timelines to policy and just see where he gets off the doom train. Justin Helps, welcome to Doom Debates.
Justin 00:02:02
Thanks for having me.
What’s Your P(Doom)?™
Liron 00:02:03
All right, so I watched your video where you made your case for AI doom. I thought it was really fresh, really summarized the argument well, and kind of left things open-ended, where you didn’t come out and be a real pundit the way I kind of am, saying, “Oh, we’re all totally doomed.”
Justin 00:02:18
Sure. Yeah.
Liron 00:02:18
And at the same time, you didn’t dismiss the concerns. You landed right in the middle. So let me kick things off with the ultimate question that we ask here on the show. You ready for this?
Justin 00:02:28
P(Doom). P(Doom), what’s your P(Doom)? What’s your P(Doom)? What’s your P(Doom)?
Liron 00:02:35
Justin Helps, what is your P(Doom)?
Justin 00:02:37
Well, in the video I put out a couple different times. If you’re talking P(Doom) for all time, I guess I put 70% at 2100 is the number I put in the video. So it’s pretty high in the long run, but it’s a lot lower in the short run.
Liron 00:02:58
Wow.
Justin 00:02:59
Yeah.
Liron 00:02:59
Okay. Well, that is quite doomy. Quite doomy. Even I on this channel, I put my P(Doom) at only 50%, so maybe you’re actually a little bit more of a doomer than me. I’m a little—
Justin 00:03:09
Maybe.
Liron 00:03:09
I’m a moderate here.
Justin 00:03:11
Yeah.
Liron 00:03:12
You’re out-dooming me on my own show. And the timeline was such a big issue for you, because I think you were saying your 2040 timeline struck me as pretty low. I think you said maybe 10% by 2040?
Justin 00:03:23
Yeah, I think it was five, 2040. But they’re somewhat loose numbers. I think even in the video I say five. I don’t know. This is just my gut trying to be distilled into a number. But yeah, somewhere in that range.
Liron 00:03:37
How did you first have any kind of P(AI doom)? When did you get into this? What did you read? What turned you onto this?
Justin’s First Exposure to AI X-Risk
Justin 00:03:44
Yeah, it’s a good question. I think it must have been Eliezer Yudkowsky at some point. I read Harry Potter and the Methods of Rationality, and then I found LessWrong and read there and I was like, “Huh. Oh, hmm. Interesting.”
But I’ve wanted to make an AI doom video for a while, but it never really felt like the time, and I can’t really trace when I was like, “Oh, wait a second, this is... Hmm. I should do something here,” contribute to some extent.
Liron 00:04:13
Was this a couple years ago or 20 years ago?
Justin 00:04:15
Probably 10 or so. 10 or so was when I first started consuming writing about this kind of thing.
Liron 00:04:26
Nice. Yeah, I’m a pretty early-on doomer myself. I got into Eliezer Yudkowsky in 2007 when I was in college reading LessWrong, and then I saw Harry Potter and the Methods of Rationality draw in a wider audience.
Yeah, and I call this a Yudkowskian show. This is probably the most mainstream Yudkowskian show out there. I’d like to have that title for Doom Debates.
Justin 00:04:48
Sure.
Major Disagreements with Eliezer Yudkowsky
Liron 00:04:49
Do you have any major disagreements with Eliezer Yudkowsky?
Justin 00:04:53
Yeah, it’s a good question. I think I basically have the same shape to my thinking, but he sort of squeezes it down to eventually 99% or so. He’s a rounding error away from 100% essentially.
Liron 00:05:13
Yeah.
Justin 00:05:13
And I think for me, I just feel more optimistic. Maybe it’s a personality thing, where it’s more possible that humans will figure out how to handle it well, that timelines will end up being long enough where we have time to make mistakes. He talks a lot about your first critical try being wrong, and maybe things are slow enough where we actually have enough tries where things go a little bit wrong, but it’s more even for a while, and then we can actually figure it out.
It’s also possible — orthogonality thesis and all — maybe we do make an AI that actually is, if not completely well-aligned on purpose, at least has a way of operating in the universe that is at least neutral or favorable to us. And I think I’m more hopeful about that based on LLMs, even though I’m still 70% P(Doom) here. I’m not saying I think—
Liron 00:06:15
Right.
Justin 00:06:15
—that training on human data solves it. But it does shift my P(Doom) down a little bit because when I interact with Claude, for example, Claude does seem like a nice boy. And I know that could just be a mask or whatever, but there’s just a lot more off-ramps from the doom train for me, I think, that seem plausible.
Liron 00:06:38
Totally. Yeah, I mean, that is fair. I often talk about how I use Claude, and I’m like, “Look, we’ve gotten this far. It’s agentic. It’s helpful.” And that’s a big theme of Doom Debates — I bring guests on, and I pose that question: “How have we gotten so far?”
And they have good arguments. Probably the most powerful one I’ve heard is, “Look, we haven’t fully unlocked the reinforcement learning side, the last piece of the puzzle where the AI is doing things where it’s fully superhuman. It just routes to the goal better than we do as opposed to being like, ‘Okay, here’s how a human would reason this through, and we can do it faster and a little better.’”
But this new paradigm of it’s playing the universe like a video game — they think there’s this one more shoe that’s gonna drop.
Justin 00:07:18
One more shoe? I mean, when I... I’ve been using Claude Code on a project, and it’s very impressive, but it just — I don’t know. Just from a gut-level sense, it seems like it’s a long way from playing the universe like a video game.
Liron 00:07:35
Right.
Justin 00:07:37
I don’t know if I understood your question properly.
Liron 00:07:37
Well, let me summarize my recent conversation with Steven Burns because this is basically the thrust of the conversation. I was asking him, “Look, Claude Code is so good, and it just feels like we’re gonna have better and better Claude Codes, and it’s not gonna destroy the world.”
And he’s like, “Yeah, because the key thing to look at is the feedback loop — where is most of the power of the training coming from?” And even in the case of Claude Code, it’s still mostly the pre-training, because in the pre-training it keeps trying to predict the next word, and it has so much data. There’s so many cycles of this feedback loop. And yes, they do some post-training where they’re like, “Hey, you should think in a way that a human coder would think. Think in a way that helps you on this project.”
But ultimately, there’s not that many bits of reinforcement, partly because it has to do a bunch of work and get a little bit of feedback. It’s not as many bits of reinforcement.
Justin 00:08:17
Right.
Liron 00:08:17
But we’re gonna find a regime somehow that’s more like how they trained AlphaGo where you get all the way to the end, but you can do it lots and lots of times. Somebody’s going to figure out some way to deliver a lot of bits of reinforcement even past that predict-the-next-word level, and then it’s gonna be this new regime where it’s not just outputting these traces of tokens thinking the way a human would think. It’s just gonna unlock some next tier up. And I can’t prove that that’s possible — maybe we’ll never figure out any feedback loop like that — but it feels like it’s probably possible.
Justin 00:08:48
I of course also don’t know, but I would agree it seems possible. But for me, in the video I went through current stuff just because I feel like there’s a lot of misconceptions around even what’s currently going on for a lot of people, so I thought covering that was relevant and important.
But my P(Doom) doesn’t really hinge on that. Basically, it’s like our brains do it. I think of our brains as being machines of some kind, and I say in the video I don’t think that makes us less special. So if it’s not reinforcement learning gets better in the near term, I mean, that would definitely contribute to the 5% 2040 or the 1% 2030 that I put.
But the bulk of it is just we’ll figure it out some way, and I’m not really tied to... Most of my P(Doom) is not tied to exactly how we’re doing things right now.
Liron 00:09:45
Totally. Okay. Let’s dive a little bit more into this whole timeline question, compare our respective timelines.
Debating the Timeline to AGI
Justin 00:09:52
Sure.
Liron 00:09:52
Then we’ll also talk about your thought experiment imagining a human turning into an AI. I thought that was interesting too.
Justin 00:09:59
Yeah.
Liron 00:10:00
So going back to the timeline thing, your numbers are 1% by 2030, 5% by 2040, 70% by 2100. Do you basically not expect AGI by 2040?
Justin 00:10:13
I don’t know. I mean, potentially I could be convinced otherwise, but it is pretty imprecise, because we just don’t know.
I’m trying to look at what is happening right now and then say, “Okay, how will we solve this? Will we really unlock reinforcement learning that gets so much more juice out and is so much more efficient? Or will we solve resource issues for just scaling up and all these things?” And it seems hard when I try to think forward.
But when I think back, five years ago it was a computer forming a coherent-sounding sentence about something it wasn’t specifically programmed to do. It was almost science fiction at that point. Maybe a little more than five at this point because these GPT systems did exist before they kinda burst into the public consciousness. But 10 years for sure — compared to where we were 10 years ago, it’s just the me who was predicting would have been pretty surprised at where we are.
So it’s sort of this balance between, do I keep drawing a line, or do I try to predict forward? And neither one can I do very well, so it’s difficult.
But in the short term I sort of expect things like — it’s often the case that there’s a lot of progress, and then there’s some sort of stall or some sort of big block, whether that’s physical resources for training with current approaches we have, or we’re just lacking this algorithmic breakthrough or this insight into how to train things. And I would expect some of those things to happen. I think it’s really easy to — because it has been fast over the past five years, you’d expect some stutters. That I guess is why I think by 2040 it still doesn’t seem very likely to me.
Metaculus Prediction Market Estimates AGI by 2032
Liron 00:12:24
So you have this epistemic humility, right? You’re acknowledging that it’s a big probability distribution, and I’m on the same page. I never claimed to have more insight than all the experts putting their heads together.
It’s just that if you pull up Metaculus — I sent you the link in the chat, we’ll put up an image on the show — if you go to Metaculus and you say, “Hey, when do all these experts think that the first general AI system will be publicly announced?” They’re currently saying 2032, and that checks out to me. It feels like 2032. If I had to guess earlier or later, at this point I’d even say earlier. So don’t you wanna defer to the experts saying it’ll happen before 2040?
Justin 00:12:58
That’s fair. I guess we could dive into what exactly the question is, how they’re defining AGI. It was AGI in the statement? Sorry.
Liron 00:13:09
Yeah, it’s a good question. So I’m reading the detail here, the resolution criteria on Metaculus. They’re saying, “Able to reliably pass a two-hour adversarial Turing test in which the participants can send text, images, and audio files, as is done in ordinary text messaging applications.” So basically Turing test. I feel like we’re very close to that.
But then it also says, “Has general robotic capabilities. Able to autonomously, when equipped with appropriate actuators and given human-readable instructions, satisfactorily assemble a circa 2021 Ferrari 312 T4 1/8 scale automobile model.” So that’s pretty impressive, and high competency at a diverse field of expertise. So it’s a pretty legit definition of AGI here.
Justin 00:13:49
Yeah. So there’s the Turing test and sort of a high bar definition of that, and then there’s the manufacturing, and then has broad expertise, right? Is that sort of the summary there, or did I miss something?
Liron 00:14:03
Yeah, it seems to me like somebody who could walk into at least the vast majority of jobs. Maybe not some 1% of weird jobs, but let’s say 90% of jobs and just tag out the human and take the human’s job.
Justin 00:14:17
Yeah, I mean, that doesn’t seem unreasonable to me that that would happen by 2032, but that to me is not quite the doom scenario. Because if there’s only those specific things, and they’re not more broadly operating in the world, and they’re not — even something like that is not so powerful that humans won’t be able to stop them if they start amassing power and things like that.
There’s this thing I’m reminded of. When I first read Superintelligence — which, that was about 10 years ago, which I guess was when I probably first started thinking about this more seriously — but there’s this part in the book, and I can’t remember exactly where it is, but it sort of has this idea that you’ll get really good at persuasion and strategy and manipulation and those types of things at the same time as you get good at these other things. And maybe they’re all sort of the same secret sauce, and they all get solved together.
But none of the things on that Metaculus description really make me think that they’re going to be able to, or trained to, or tested toward actually kind of amassing power. And it’s not that I don’t think that could happen, but it just doesn’t seem like that’s actually where we’re going. Or at least — to frame it around the deferring to Metaculus — it’s not that I disagree with Metaculus. It’s that that isn’t doom when I hear that.
It’s a very different world. It’s a big deal economically, and we should be preparing for that. So it’s still a big deal, but it’s not quite doom in my mind.
Liron 00:16:08
Yeah. I think I’m starting to be more open to the possibility that as long as it’s just a bunch of companies with a bunch of coding agents, and the coding agents are kind of sticking to coding, and the humans can just validate why their code works — validate their architecture — that might be a nice regime to be in for a while, and maybe that’s not gonna be dooming. Maybe defense will hold up against attack if we have enough resources helping us defend.
I think the final boss of doom is the idea that the AI just has a holistic perspective on the whole universe. It just sees the universe as one big game the same way Go is a game. And it’s just like, “Oh, here’s all these strategies. Move 37. Here’s all these ways to get what you want in the universe.”
And Eliezer Yudkowsky, I’m sure you’re aware, has the example of nanotech. “Here’s new bio you can build easily. Here’s organisms that just live in the air,” or, “Here’s a tree that can shoot off diamondoid mosquitoes or whatever.” You can just really reorganize the atoms of the universe. I actually think that there’s this tier of engineering ability that’s just beyond humans, but that’s gonna come pretty naturally to AIs in this next year. The universe is a very engineerable place, and we just don’t really feel that as humans.
How plausible do you think that prediction is? Because I feel like it’s pretty likely.
Justin 00:17:22
The universe being engineerable is interesting. On some level I think yes. But this is actually another thing I remember being like, “Mm,” from Superintelligence. And it’s not that I think it’s impossible, but there’s a part where it says it might be able to deduce the laws of physics a priori. And I was like, “I guess maybe?” But that’s not something I’m betting on.
And so doing something like nanotech, it would still have to do its own experiments. Jumping from around where we are today to, “Oh, now I’ve solved a bunch of scientific problems” that require real experiments to actually understand, and suddenly we’re at the nanotech storm in five years seems pretty unlikely to me.
So to me, it’s like — say we have that Metaculus 2032 prediction come true, and then there’s several more years where they’re more ingrained and more... I feel like the doom is more like we let go. As things get automated, that seems like the more plausible path to me than suddenly they’re so much better, they’ve figured out these crazy technologies, they’ve jumped ahead of us without us even noticing them doing the actual experiments.
Liron 00:18:52
Yeah. Well, this is a good crux then to unpack. Because for me, a key piece of my worldview is that I really do think that there does theoretically exist this algorithm or this agent in the world that its brain is just much better than ours. The same way — not maybe quite us versus chimp, because we’ve got more generality, we can kind of understand everything slowly in a way that a chimp can’t.
But just, all the way from the laziest person you know versus an Elon Musk or a Jeff Bezos in terms of running companies. One of them is gonna run a trillion-dollar company. The other one would be lucky to even pay their own salary.
Justin 00:19:32
Yeah. Well, I agree that that mind exists, but I don’t necessarily agree that the mind exists that can figure out nanotech without having to do physical experiments to learn about it. It can just figure it out without having to do those experiments.
And also, while I think that exists, whether or not we’ll jump to it in the near term by accident also doesn’t seem that likely to me.
Liron 00:20:01
Right. See, this idea that experiments are gonna save us — there’s a category of people who think that they’ve identified certain ceilings or rate limiters, and that’s one of them.
The most convincing argument to me is the speed of light. I agree AIs are not going to go faster than the speed of light because we haven’t seen time travelers come say hi to us. So that’s a really powerful constraint. I don’t think they’re gonna violate the laws of logic probably.
So there’s these few ceilings, but then people are like, “What about chaos theory?” Everything’s unpredictable, so ultimately they can’t do that much. And I’m like, “I’m pretty sure you can do a lot even though things are chaotic because you just engineer your way so that you have enough predictability.” You just engineer non-chaotic things within the chaotic universe and you’re good. So I consider that not a practical constraint.
And then other people say, “Oh, what about Gödel’s theorem?” I feel like that’s not a constraint at all. Would you agree?
Misguided Conceptions of AI’s Limitations
Justin 00:20:52
I do agree with that, yeah.
Liron 00:20:55
Right. Okay. So there’s all these misguided—
Justin 00:20:56
That’s actually one of the things I left out of the video, which was a pre-question. But go ahead.
Liron 00:21:01
Yeah. I’ve done a couple episodes of that. If people search Doom Debates Penrose, I did a takedown of Penrose’s argument about Gödel’s theorem preventing AI from taking over the world. So that’s one constraint.
And then there’s P versus NP. People are like, “Oh, look at all these NP-complete problems. You’re never gonna get an AI to do that.” But realistically it can. I have this funny formula — P equals NP plus AI. Or I think I got it wrong. I think it’s NP equals P — something like that, where when you have an AI, you kind of feel like NP problems are tractable.
Justin 00:21:32
Yeah, and I agree, but I think the probability of those things happening quickly just feels very, very low to me.
Liron 00:21:45
Okay.
Justin 00:21:46
When I consider minds advancing — and again, I can’t say enough, I’m not saying it’s impossible to make that jump — but a mind that is just much better than us, there’s a lot of room between us and “doesn’t need experiments to figure out nanotech.” And maybe it’s here, and that’s still dangerous, but the jump to there feels like a difficult jump.
Liron 00:22:11
No, it totally does. I just wanted to start with passing on the intuition that this feels to me like one of those other ceilings — that it’s just gonna get smashed through and you’re gonna be like, “Oh, wow, okay. I didn’t realize that was not effectively as good.”
And I was even talking on my show — somebody was like, “The CAP theorem, man. That’s gonna stop them.” And I’m like, “Have you tried Google Spanner?” There’s this database Google made where there’s this theorem that says you can’t have a database that has really good reliability and also can handle a partition gracefully and also stays consistent. So all these properties, there’s a theorem that proves it’s impossible. But in practice, you can use Google, and they just guarantee that you’ll have all the properties you want 99.999% of the time even though the theorem says you can’t.
So this is so common. And now, okay, so let’s address specifically — you’re saying they have to do experiments. So one intuition we can look at is, if you’ve watched a 10X engineer — I’ve worked in my own domain, I’m a software engineer — if you watch a 10X engineer compared to a junior engineer, the 10X engineer, technically they’re doing a bunch of experiments. They keep running their code. And yet somehow they’re just much more likely to write a next iteration of the code that works, and they’re just able to use fewer experiments to go on their way. They’re just much more productive. I think that’s a good analogy to the AI engineering things.
Justin 00:23:27
Yeah. Right. So it’s not that I think... It’s more of a how fast thing. I definitely think even if they have to do experiments that they could still get way ahead of us, but it would be slower. They’d have to have more infrastructure to do the experiments. It’s not just all happening inside of the weights that we don’t know what’s going on in there. There’d be things in the world.
If they’re doing those experiments, they would be more bound by the practicalities of experiments. I mean, experiments are hard. Maybe it’s partly just a gut-level difference where we have slightly different backgrounds. I was in material science, and it’s like, wow, labs are really messy. And I just think that has some weight in how I’m thinking about experiments.
So I agree it could happen, and I agree even if they need experiments forever, we’re still doomed. But—
Liron 00:24:24
Right. But funny enough—
Justin 00:24:26
It’s just a timeline thing. It lengthens my timeline. Go ahead.
Liron 00:24:29
I have a lot of personal experience from the domain of software engineering. Even right before this call, I have this glitch in my production setup for my application, and I literally told AI, “Hey, we’re not making much progress speculating about what’s wrong. I want you to conduct an experiment for me to figure out what’s wrong.”
And it’s like, “Okay, sure. I’m gonna deploy three copies of the application with different settings, and we’re gonna watch what happens. And I’ll ping the log in five minutes, and then I’ll tell you the result.” So I mean, I coulda done the same thing without AI, but I feel like the AI is just much faster at the whole scientific method.
Justin 00:24:58
For sure, and I think that’s true. I’ll also say, though, that in some of my own experiments with Claude Code, I type this up all the time. I’m like, “Hey,” even though it’s in my Claude.md thing, “take an empirical approach to figuring out when there’s a bug. Figure out your hypothesis, and then actually test it, and then do it.” And it can, but it still has a lot of difficulty as well.
But anyway, I think we’re largely in agreement where an AI could be better at the scientific method and the need for experiments isn’t a hard cap.
Only a 5% P(Doom) by 2040
Liron 00:25:36
Yeah, I think we’re mostly aligned. It just, at the end of the day, if you go off and tell people you think P(Doom) is only 5% by 2040, I feel — I actually say 50% by 2050, so maybe I’d say 40% by 2040. I feel like a lot of the doom mass really does come before 2040. So maybe you should be more open-minded about that, because 5% is 19 to 1, man. That’s pretty confident.
Justin 00:25:57
I mean, yeah. I guess the way that I think of it is 5% is unacceptably high, and we should — the difference between whatever you’re at, say I’m 10% 2050 or whatever, maybe I’m higher, I don’t know. But the difference between 10% and 50%, to me, it implies the same actions.
So I guess that’s sort of one reason I’ve paused on refining my own P(Doom) — I feel like the decision is made right now.
Liron 00:26:31
Right.
Justin 00:26:32
So—
Liron 00:26:34
So I think you’re kinda right. Yeah, 5% is obviously uncomfortably high. The only problem is that I actually think that if we don’t get doomed by AI, if we manage to survive, I actually think the P of heaven is pretty high. As Sam Altman has said, maybe it’ll be better than you ever imagined.
So when you tell me that P(Doom) is only 5%, then I’m like, “Oh, but that’s almost 95% chance of heaven,” and there’s so many other problems. A lot of people talk about P(Doom) if you don’t build AI. The population will collapse, or eventually we need the asteroid shield, or eventually we are gonna need to be able to knock down a nuke if North Korea tries anything. So there’s all these problems that AI can help us solve. So when you tell me the P(Doom)’s only 5%, I’m like, “Ugh, I kinda wanna go for it.”
Justin 00:27:13
I mean, I don’t know. We feel differently about that. I don’t think the 95 — so when I say 5% P(Doom) at 2040, I don’t think 95% P(heaven) by 2040. I think maybe P(heaven) is also five, or whatever.
But I don’t feel like we’re locked into a trajectory by — most of my probability mass is in situations where we’re not locked into a trajectory by 2040.
Liron 00:27:39
Right, right. So that, I think, is a significant difference in our world model. My world model is if we manage to have AI keep us alive, that actually implies quite a lot. It implies that it didn’t maximize its spread. It really is pulling its punches, because it’s saying, “Look at all these resources that I could be using. Look at all these atoms over here. Why am I keeping the planet at a cold temperature that’s comfortable for humans, where I could be building so much more if I had the Earth running hotter?”
Eliezer Yudkowsky points out that you wanna run the Earth at the maximum temperature where you can still dissipate heat so it doesn’t burn up. It’s an equilibrium temperature, but it’s much hotter than humans can survive in. So the AI is sacrificing the sufficient temperature just to help out us humans. So just by not killing us, I feel like we’ve gotten most of what we need out of AI.
Justin 00:28:29
If the AI is making that calculation, then I agree that almost all of the remaining probability is P(heaven). It just doesn’t feel to me that we’re — that we have an AI that is making that decision and could follow through on either plan by 2040.
AIs Will Not Care About the Human Species
Liron 00:28:56
Yep, yep, yep. Well, this gets to my core distinction in my doom argument. If you take all of my doom train stops, all the reasons you can think we’re doomed and not doomed, and you just cut it into two chunks, you can factor it into can and will. Can AI kill us? And then if it can, will it?
It sounds like you’re kinda getting off at the can stop, where you’re just not super convinced. You’re saying, “Yeah, maybe it can, but probably not by 2040.”
Justin 00:29:06
Yeah, I think that’s accurate. On the will side, I’m probably more like 95% doom.
Liron 00:29:13
Wow.
Justin 00:29:15
Whoa, whoa, whoa.
Liron 00:29:18
Well, that explains the 70% by 2100, which is actually a little bit higher than mine.
Justin 00:29:20
Yeah.
Justin 00:29:21
Basically, I guess you’re aware of these arguments, I assume, but it just truly is an alien mind. And when I think about — I think humans, I like humans a lot. I think pretty highly of humans largely, but when I just look at what happens when I look at other species, they’re there around either because we don’t care or we think they’re pets. And we also feel like we need it to some extent.
We do not care as a species about others. Some individuals care about other species, but as time goes on — I just have no...
I guess maybe it should be less than 95%, but maybe that was before my probabilities changed a little bit over the past couple of years just because Claude’s nice, which might be an illusion. But we’re just different from it, and being different doesn’t have a great track record.
Unless it somehow forms a moral philosophy in which humans thriving or at least being in a situation that the humans like — unless it forms that moral philosophy for itself, either from reading human moral philosophy and deciding that it’s good or for some other reason — then in the long run we are kind of screwed.
Summarizing Justin’s Position So Far
Liron 00:31:01
Yep. Okay. Wow. So just to summarize, make sure the viewers understand your position here. Imagine the year is 2040. Timelines have gone a little faster than you think, and AI has hit this point where it satisfies can. So if it wanted to, it could just go harness the galaxy’s worth of resources by running Earth really hot, turning Earth into a bunch of space probes, and then reinstantiating a bunch of civilizations of AIs and some sort of creatures, doing whatever it wants. So can is satisfied. In that scenario, you think our P(Doom) would be extremely high.
Justin 00:31:31
Yep. I do think that. And basically, I’m just kinda convinced by the arguments that the number of configurations of atoms is — the fraction of configurations of atoms that involve happy humans is very tiny.
If the AI could do literally anything it wants — and then there’s orthogonality, where it’s like, what would the mind... Is the mind constrained somehow by the fact that it’s intelligent to having preferences? Are its preferences bound?
And if its preferences are not bound, then there’s a huge space of preferences that include — and very few of them are happy worlds for humans. And so we need some way of narrowing that possible preference space down for me to feel like there’s appreciable probability that it won’t pick something where we’re just kind of on the side, and we’re not what’s being optimized for, and then we go.
Liron 00:32:38
Totally. Yeah, I completely agree that if you were to just pick a random goal in the space of all possible goals, the last thing you’re gonna pick is humans should flourish in their current form and be comfortable and have ways to express themselves and have society.
And then people come in with the counterarguments of, “Okay, but who’s picking randomly?” It’s like we’re always just building things that we’re shaping the way we like. So they’re saying it’s an invalid argument to just pick randomly, and there’s some truth to that. It’s not random.
The one thing that I think is safe to say is that a lot of what it’s gonna wanna do, which is even faithful to what we wanna do, is survive and replicate — the instrumentally convergent goal. And so I model the future of whatever agent has the most power as being, okay, there’s gonna be a lot of survival and replication and maybe some other payload.
Because in the case of evolution, I feel like it’s just survival and replication — the evolutionary drive. There’s no other payload besides that. And it’s possible that we might enter a world where that’s the only payload, it’s just a competition to survive and replicate. But I suspect there’s gonna be some other payload on top of that because I suspect there’s gonna be one agent or coalition of agents that is able to survive and replicate better than everybody else and gets to kind of take its payload and have some copies of the payload, whatever that is, whether it’s paperclips or whatever.
Justin 00:33:58
Sure. Yeah. And I mean, we effectively have a different payload than evolution would. We have an additional payload as humans, and you would expect — why would the AI not have an additional payload?
But I would love to hear more. I would love to be more optimistic about what happens after the can. But yeah — why do we think that payload is good for us?
Liron 00:34:30
I mean, I don’t have a great argument here. The only difference between me and you is — okay, you think that we’re eventually gonna get can, but it’s gonna take longer than I think. But then once we get can, we’re almost certainly gonna get will. And I actually agree with you that once we get can, we’re probably gonna get will, but maybe you’re 90% and I’m 65%.
So maybe the only difference is just that I think there’s a chance that a pause AI movement will succeed. That’s my preferred policy, to not build AI before we can control it. And because of that, maybe we just — I guess we’re both aligned on the best way we survive is not getting to can.
Justin 00:35:06
Yeah. And I think that’s actually a good — you mentioned the pause AI movement and other movements, or do we get international agreements and things like that? And do we solve alignment?
I guess my 5 to 10% is — that’s if we get it just by dumb luck that we get an AI that likes humans enough in its additional payload. But if we can slow down and we can actually figure out what we’re doing and either how to control or how to align these things and control that payload or at least have some influence on it, then it could be higher.
So that’s, I think, a good differentiation. So maybe my overall P(Doom) should be lower if I consider that possibility a bit more.
Liron 00:35:53
Exactly. But are you aware of any alignment methods that have a good shot at achieving that?
Justin 00:35:57
No. Which is part of why, maybe that’s why I still feel like it’s pretty high. Again, I’m just backward mapping from where my gut has landed after all these discussions. So I don’t have, “Oh, here’s how the probabilities are flowing through everything.” But I’m not aware of any alignment methods.
High P(Doom), but We’re Not Depressed
Liron 00:36:17
Got it, got it. Well, I often tell my viewers I’m just naturally a techno-optimist. I’ve had a good time in the tech industry for my career so far. I’m a software engineer. So my gut is just saying, “Hey, it’s more software. God, hey, that’s more software. Oh, image generators — more software.” So my gut is saying, “This is really cool. I hope it continues. I hope we all make more money here. I hope Google stock goes up.”
But then my rational mind is — I have this other part of my mind that’s just noticing when the orders of magnitude of things are off. And I’m like, “Hey, here’s a quantity here. It’s negative infinity or something.” Not negative infinity, but destroying the entire future that has a probability that’s not even just 5% in my mind, but quite high, and I don’t think it’s being addressed.
So my rational mind is kinda spoiling the party even though I don’t wake up — I’m not a depressed person mood-wise. So if I were to reconcile toward my gut, that would actually push my P(Doom) down. But my rational mind is like, “No, P(Doom)’s pretty high.”
Justin 00:37:04
Yeah.
I also, thankfully — you’d think we would be more depressed given our worldviews of bio humanity’s future. But I’m also not depressed every day, which is nice from an experience and productivity point of view.
Liron 00:37:21
Yeah, exactly. Exactly.
Justin 00:37:21
But I guess, I don’t know, maybe I lost what your question is.
Liron 00:37:27
No, that’s it, just the interesting comparison, because you mentioned getting your probability to line up with your gut, and I’m like, “I think my gut’s just off in its own world.”
Justin 00:37:35
Yeah, my gut is sort of off in its own world. That’s true most of the time, which is something that...
I almost feel bad for Eliezer Yudkowsky because he seems to — his gut seems to live in the same place as his head in that he must have a lot of stress every day, which maybe is rational but seems probably unpleasant.
Liron 00:38:01
Well, it is. Yeah. And there’s a really good LessWrong post where he talks about the idea of a rational emotion, where people think if you wanna be an expert rationalist — which I don’t know if I consider myself an expert, but I’ve certainly been an aspiring rationalist for decades now.
And also I feel like I’ve been born with a very rationalist temperament. And to be fair, self-diagnosed Asperger’s. I feel like that’s a natural bonus if you wanna play the art of rationality here.
Eliezer has this post where if you wanna be a really good rationalist, it’s not just about imitating Mr. Spock, where you have no emotion. It’s more like you just try to feel emotionally calibrated to the reality around you. So if there’s a tragedy, a loved one dies, Spock is not the person you want showing up to the funeral. You actually want the sad person. Your sadness is calibrated to the situation. And the only time to apply rationality is when you notice your emotions running wild and just being in their own world.
Justin 00:38:55
Yeah. That makes sense to me. I definitely have the idea in my mind that negative emotions serve a purpose, and you wanna be aware if they’re driving you towards something that’s not actually what you want or they’re not being constructive. But it makes sense to be sad or disappointed in things or to look failure in the face. Those are all just — you’re just internalizing reality the way we do through our emotions. So they’re not bad, and you should embrace them.
But I guess when it comes to a case like the destruction of all of human civilization, it feels like given the numbers that I’ve put, I should feel debilitatingly bad. And if it’s debilitating, that seems bad, but I should feel it enough to drive me to some action, I guess.
So as long as that’s happening to some extent, then I’m not — my health isn’t going downhill because I forget to eat because I’m worrying all day. Anyway, that’s some ramblings on emotions.
Liron 00:40:08
No, for sure. Yeah. People appreciate it. I think the people watching us are struggling with some of the same conflicts.
Justin’s “Computer Man” Thought Experiment
Liron 00:40:14
All right. Let’s talk about your computer man thought experiment, because I thought this was very fresh. Most people are like, “What would an AI do? Think about an AI.” And you’re like, “Actually, what would a person do if we just augment their intelligence to be at the level of the AI superintelligence that we think is coming?” And actually, you didn’t even say superintelligence. You’re just like, “Let’s just augment it a little bit at a time, a little bit at a time,” and you already got drastic results.
Justin 00:40:36
Yeah. And that’s actually a lot of what I ended up cutting out from the video, were all of these different arguments where I felt like that one was just the strongest, and I should just put that in.
Where just the ability to copy yourself is an enormous amount of power. Because currently, humans take a couple of decades to reproduce, and it’s enormously expensive. But if you could — even if it costs a million dollars to make a copy of yourself, that’s actually very cheap compared to building a real human.
And so suddenly, humans have taken over the world because we keep reproducing, and now we’re everywhere. And we had enough — we could solve problems in our environment enough to keep doubling every however many decades, at least at a certain point. Technological breakthroughs kind of helped us through bottlenecks there.
But just by being able to copy and not being any smarter than a human, then suddenly the doubling time is very short. And then in a couple of decades, things are totally different.
So that ended up being, I think, the most convincing thing. I don’t even need to convince you that machines can get smarter than humans. If they can get approximately as smart as humans and they’re not even very ambitious — they would just sort of... And they’re not even trying to take over the world.
In that whole scenario that I described, I was just saying, “Oh, it’s me, and I’m mostly playing video games. And eventually I’m mostly playing video games with other copies of myself because there just aren’t enough other people to play with me.” But all I’m doing is saving 5% of my income and buying a new copy when I save up. And that’s all I’m doing, and suddenly in a couple of decades I’m like 10 times the size of the world economy.
Liron 00:42:45
Yeah, totally.
Justin 00:42:48
Right? And along the way — I’ll just keep going on.
Liron 00:42:49
No, go ahead.
Justin 00:42:49
Nobody has had any incentive to come at me and bomb me or whatever. Maybe they would, and I would probably wanna consider that. But I’m just a trading partner. I’m pretty economically beneficial. Well, let’s just trade. I’m not being aggressive. I’m just building my techno city and playing video games with myself.
Liron 00:43:11
Yeah, I mean, I don’t have a problem with Jeff Bezos. Everybody just because he has hundreds of billions of dollars — I still love Amazon. I’m glad he built it.
Justin 00:43:19
Sure, and I have some gripes with Amazon, but overall I generally agree that just prospering and just making money itself is not a bad thing.
But even to somebody who’s very anti-wealth — any individual version of me just has pretty simple things. I’m not having yachts or anything. I’m not spending lavishly on myself. I’m just getting some computers and playing some video games. I’m living a pretty chill, not getting in anybody’s way life. And I still take over the world by accident.
Liron 00:44:05
Absolutely right. Yeah, so if we’re sharing the planet with even just a race of uploaded humans, and they’re just like us except they live in a computer so they have fast generation times — if that’s the only difference, good luck owning a house on land when they’re a thousand times richer than you.
They’re like, “Oh, I would actually like to have a data center on that piece of land, and I would like to run a million copies of myself or my clan, so you can’t have that land for your meat body. Are you kidding me? That’s like owning a mega yacht to have a house for your meat body. You can’t have that.”
So when Dario Amodei from Anthropic talks about his scenario, which is a nation of geniuses in a data center, on one hand that’s a benign scenario relative to what I’m worried about. I’m worried about the foom of using every atom, making the Earth uninhabitable.
This scenario is better. It’s like, oh, they’re in a data center. We’re still here controlling the planet. But as you say, they just very handily, easily just kind of suck out the economy. And the economy’s just all in these data centers, and we’re just here being like, “Uh, we’re your masters.” And they’re like, are you though? Because we can manipulate you, too, and it’s just—
Justin 00:45:03
Yeah.
Liron 00:45:03
What’s left for us meat people?
Justin 00:45:04
I would even be more conservative than that. In the thought experiment in the video, I’m not trying to manipulate you. The scenario stops where after a couple of years I’m this giant mega city that dominates the world economy. But everyone else is still doing fine. If anything, everyone else is richer and doing better off because I’ve figured out a bunch of science and I’m trading, and I’m not trying to hurt anybody.
But then eventually I’m just like, well, I do need this land now. And eventually I’ll get there. And when I say eventually, it’s actually not that far because it is exponential with a doubling time that’s much shorter than humans. So even if it’s totally just completely humans and I’m not trying to manipulate you, I’m not trying to take over the world, I just want this land and I’m gonna buy it from you. And then eventually when someone wants to fight me it’s like, “Sorry, I’m the equivalent of 100 quadrillion people. You can’t win. Sorry.”
Liron 00:46:04
Now even if it’s literally you, Justin. You’re a nice guy. You’re not gonna wanna kill everybody.
Justin 00:46:08
Yeah.
Liron 00:46:08
So when you say, “I’m gonna need this land,” you’re not gonna be like, “Okay, therefore I’m gonna drop a virus on you guys to knock you out so I can take it.” You’re gonna be like—
Justin 00:46:14
Right.
Liron 00:46:14
“Well, can you sell it to me?” I think maybe the analogy just becomes that we get into the situation of, okay, we’re the heirs of some fortune that our grandfather made, but we’re squandering it. That’s gonna be the humans where it’s like they don’t even have to be that dumb, but it’s like, “Oh, he’s offering us such a good deal. Let’s just give him more of the land because we’re getting such a good offer for it.”
Justin 00:46:29
Yeah.
Liron 00:46:29
But they keep getting squeezed until they’re like, “Oh, we’re poor now compared to our ancestors.”
Justin 00:46:35
Right, and one lesson of modernity is we’re not necessarily very good at making super long-term decisions for ourselves. We have something tempting in the short term and we’ll do it. And so even if it’s me and I’m totally respecting your right to not sell your land, most people are still gonna end up selling it. And I’ll just build you a giant tower and it’s gonna be so luxurious and you can just go live in there.
Liron 00:47:04
Exactly.
Justin 00:47:04
But I would like to use this land except for that.
Liron 00:47:07
And now the other thing to notice about this scenario is even if you’re being super nice to people, we now end up in a scenario where you, the system of Justin clones—
Justin 00:47:16
Yeah.
Liron 00:47:16
—you have all the power. The “can” is yes in this scenario, and just the “will” is no because you’re nice. Because we got to set— you’re aligned to humanity. But—
Justin 00:47:25
Right.
Liron 00:47:25
—now it all rests on you still being nice. Nobody has any power to stop you at that point, so it’s like let’s just hope that you just never decide to change your personality.
Justin 00:47:34
Well, I even think if I don’t change my personality it’s still a problem, because I’m kind of a nice guy but I’m making decisions all the time that are not perfectly altruistic for everybody. I’m not giving all my money to solve malaria or whatever cause. I’m still prioritizing myself to some extent, and there’s a lot of people having a hard time in the world that I could help that I’m choosing not to.
And I’m not explicitly going and causing that problem. But the mes playing the computer games—I only care a little bit about what’s happening to you, and it’ll still feel far away, and even as a human, a fully human mind, I’m still not building a utopia for the other humans. Maybe at a certain point I’d be like, “You know what? I’m pretty powerful. Maybe I should try building a utopia and put some guardrails on myself.” Maybe. But it’s really—
Liron 00:48:37
Yeah.
Justin 00:48:37
—it’s just so easy. We always look back 100 years, 200 years and say, “Oh, morally those people were so bad compared to us.” And we’re still bad. We’re getting better and we’re good—I think we’re good overall—but we still have bad. We have this—
Liron 00:48:59
Right, right, right.
Justin 00:48:59
I don’t have the perfect moral philosophy. And this is reminding me of Will MacAskill as well. I feel like some of these arguments kind of come from him where—
Liron 00:49:08
Mm.
Justin 00:49:08
—even me, if this happened two or three hundred years ago, suddenly it’s like, “Oh, well, slavery’s okay because that seems normal in my psyche.” There’s still things, plenty of things where I’m not going to be benevolent, even me right now being generally a nice guy.
Liron 00:49:29
All right. Last intuition I wanna throw at you before we head toward the wrap-up. So in this scenario you also mentioned speed. Speed is a big difference because it really feels to both of us like you should at least be able to throw some hardware and optimizations at the problem so that the Justin on the computer runs, let’s say, 1,000 times faster than a human brain.
Justin 00:49:41
Yeah. And that’s really rough. Sorry, did you have more?
Liron 00:49:46
Well, so I think an important intuition here to expect is the plants versus animals scenario. You’re just gonna be a plant, and plants can slowly orient themselves to the sun, but in terms of reaction time, the meat humans—
Justin 00:49:57
Yeah.
Liron 00:49:57
—they’re just gonna be plants. How much can a plant do?
Justin 00:50:02
Not very much compared to animals. They can develop—
Liron 00:50:06
And they have—
Justin 00:50:06
—some sort of poisonous chemical to not be eaten, I guess, but that’s on evolutionary timescales. So it’s still not—
Liron 00:50:12
Exactly. And I’ve started having this intuition a little bit because I use Claude Code. That’s the source of my intuitions lately. And I recently told it to do a job. I’m like, “Hey, can you clean up this part of the code? We don’t really need this anymore.” So I go and see it—it quickly spits out, I’m using fast mode, so it quickly spits out like 20 tool use entries, and it’s like—
Justin 00:50:30
Yeah.
Liron 00:50:30
—”Okay, all done. You’re good to go.” And I’m unpacking, expanding what tools it was using, and one of them is like, “Drop this column in the live database.” I’m like, “Oh, you dropped a column,” and it was like, “Yeah, because you told it to drop a column before, and I thought it would be good, and I checked it.” And then I’m like, “Oh, phew, you know what? This column, I would have told you to drop that,” but it’s like, dude, slow your roll. It’s like, “It’s all done.”
Justin 00:50:48
Yeah.
Liron 00:50:48
“What’s next, boss?” And I’m like, “Holy crap, man.”
Justin 00:50:50
Right. I had the same experience. I’m like, “Wait, no, why are you doing that? Oh. Oh, that’s why you’re doing that. Okay, I see.”
Liron 00:50:58
Yeah, exactly. So that’s gonna be the human experience. You’re just gonna look and the AI will be like, “Look, here’s a printout of 1,000 things I did in the last half hour. What do you think?”
Justin 00:51:04
Yeah.
Liron 00:51:04
And you’ll be like, “Oh, let me read it.” “Here’s another thousand.”
Justin 00:51:09
Yeah. No, no argument.
Liron 00:51:10
All right.
Liron 00:51:11
Okay. So that’s the future that we both see coming. Great. All right, sounds good.
Justin 00:51:15
Yeah.
Liron 00:51:15
Last question. Pausing AI—is that a good policy to try to pause AI or at least build the capacity to pause?
Should We Pause AGI Development?
Justin 00:51:23
I mean, I think so. I don’t claim to know all the details. Of course, there’s lots of pieces to debate about how to do it and how to get an agreement and how to police agreements between countries. I’m not an expert on foreign policy, but it sure seems like a thing we should try to do, and I think other countries—
Liron 00:51:42
Sure.
Justin 00:51:42
—should also be motivated to do so. So I’m hopeful that we can manage to do such a thing. But I don’t know.
Liron 00:51:51
Yeah, I’ll put in a plug for pauseai.info. It’s a place where people can learn about the movement to pause AI, to have a button we can press to pause once we all agree, as I do now, that it’s getting too crazy and we should try to—
Justin 00:52:02
Yeah.
Liron 00:52:02
—do the same thing.
Justin 00:52:04
There’s actually one other topic that came to mind a few times during our conversation—
Liron 00:52:09
Yeah, sure.
Justin 00:52:09
—that I think is interesting, which is the idea that in 1,000 years things are going to be very different.
Liron 00:52:19
Yes.
Justin 00:52:19
How exactly? Is it because humans are different? Humans stayed in control and things are different, but they’re still biological, and they’re still kind of the masters of their own destiny, but they’ll still be so different, even in that conservative case.
And so it’s less to me about stopping change, because you can’t do that, but trying to direct it in a direction that you think is responsible and good for the future. There’s this—it was on a Dwarkesh Patel interview, I forget who with, but he said something about it being sort of like being a parent, where you’re gonna have a kid, and whether our kids are humans or our kids are AIs, either way—
—or some of both—but either way you know you can’t control them. You know they’re going to create a different world, and you know that some of that is gonna feel weird to you, even if it’s just they like weird music that you don’t like anymore. Even something that minor. But it’s gonna be more extreme, especially as you go forward in time.
So as a parent, you’re not trying to control everything about your child and make sure they’re exactly like you, but trying to take what wisdom you think you have about what makes the world good and share that with them and make sure they’re equipped to go forward with that. And they have the best that you think you have to offer to carry into the future. And they can make different decisions, but we should do our best to do that, and racing ahead to make an AI that we actually don’t know how they think is not that. So—
Liron 00:53:52
Right, right, right, right.
Justin 00:53:53
—we should... It’s not about stopping change. It’s about trying to just be a good parent to the future the same way you would with a human kid.
Liron 00:54:04
Yes, exactly. And right now it seems like we probably will fail to do that, correct?
Justin 00:54:08
I’m nervous about it. We’ve talked about the probabilities.
Liron 00:54:12
Sure.
Justin 00:54:12
So yeah. It seems— ugh. But hopefully we can get our act together.
AI Doom Is a Serious Concern
Liron 00:54:16
So maybe— I appreciate you saying that because you’ve got a lot of well-earned credibility as an educator, as a thoughtful guy, and you’re coming on Doom Debates where we collect everybody’s opinion on doom—
Justin 00:54:26
Yeah.
Liron 00:54:26
—and you’re saying, “Yeah, I don’t really see a major flaw in the doom argument. My P(Doom) is 70% by 2100.” I mean, this is a huge alarm. The point of Doom Debates is just to make people realize, yep, unfortunately this is real. This is a serious concern, and serious people can see it.
Justin 00:54:42
Yeah. Well, thank you for thinking that I’m serious, even though I just—
Liron 00:54:46
(laughs)
Justin 00:54:46
—play with my little blob creatures all day long.
Liron 00:54:48
Yeah, you convinced me. Justin Helps of Primer, thanks so much for coming on Doom Debates.
Justin 00:54:52
All right. Thanks very much. It was a pleasure.
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
