Devin Elliot is a former pro snowboarder turned software engineer who has logged thousands of hours building AI systems. His P(Doom) is a flat ⚫. He argues that worrying about an AI takeover is as irrational as fearing your car will sprout wings and fly away.
We spar over the hard limits of current models: Devin insists LLMs are hitting a wall, relying entirely on external software “wrappers” to feign intelligence. I push back, arguing that raw models are already demonstrating native reasoning and algorithmic capabilities.
Devin also argues for decentralization by claiming that nuclear proliferation is safer than centralized control.
We end on a massive timeline split: I see superintelligence in a decade, while he believes we’re a thousand years away from being able to “grow” computers that are truly intelligence.
Timestamps
00:00:00 — Episode Preview
00:01:03 — Intro: Snowboarder to Coder
00:03:30 — “I Do Not Have a P(Doom)”
00:06:47 — Nuclear Proliferation & Centralized Control
00:10:11 — The “Spotify Quality” House Analogy
00:17:15 — Ideal Geopolitics: Decentralized Power
00:25:22 — Why AI Can’t “Fly Away”
00:28:20 — The Long Addition Test: Native or Tool?
00:38:26 — Is Non-Determinism a Feature or a Bug?
00:52:01 — The Impossibility of Mind Uploading
00:57:46 — “Growing” Computers from Cells
01:02:52 — Timelines: 10 Years vs. 1,000 Years
01:11:40 — “Plastic Bag Ghosts” & Builder Intuition
01:13:17 — Summary of the Debate
01:15:30 — Closing Thoughts
Links
Devin’s Twitter — https://x.com/devinjelliot
Transcript
Episode Preview
Liron Shapira 00:00:00
Devin Elliot, who was a professional competition snowboarder for ten years.
Devin Elliot 00:00:04
I do not have a P(Doom). I find it nearly impossible to figure out how to even make that happen.
Liron 00:00:09
You’re so confident you won’t even give it like 5%?
Devin 00:00:12
But I’ve been doing these extrapolations for longer than you guys have been thinking about it.
Liron 00:00:16
I mean, did you predict we were about to pass the Turing test? Didn’t that surprise you?
Devin 00:00:20
The point is the LLM can’t do a bunch of the things that people are imbuing it with being capable of doing. The person on the ground knows what works, knows what doesn’t work.
Liron 00:00:33
Many such people who are on the ground working on this stuff, not even AI doomers, AI optimists, many of these people are saying, “Hey, we’re getting close to superintelligence.” So, you wanna start your sentence with, “If you’re on the ground working on this stuff,” the end of that sentence goes?
Devin 00:00:46
No. I think it’s super easy to look at why those guys are conflicted. And I’ve got 15,000 hours of working with LLMs directly, but I hit the wall with them every fricking day. And I’m telling you, these things cannot go further.
Introducing Devin: Snowboarder to Coder
Liron 00:01:03
Welcome to Doom Debates. My guest, Devin Elliot, is an autodidact. He’s self-taught in many fields. Starting with dropping out of high school, he was a professional competition snowboarder for ten years. He’s run his own house painting company, he’s run businesses in applied chemistry, and he’s also a self-taught software engineer who’s built various systems as an independent contractor for various companies in AI and crypto.
He’s very much a lifelong learner, self-taught builder, and also an active Twitter user, which is where I met him recently. We started off the cuff debating AI doom, and that’s why I invited him on the show, and you’re going to hear that debate today. Devin Elliot, welcome to Doom Debates.
Devin 00:01:54
Thank you. I’m excited to be here.
Liron 00:01:56
So this is really getting back to the roots of Doom Debates, because I’ve had the show going now for a little over a year, and the genesis of Doom Debates was actually me arguing on Twitter and getting frustrated that people were being wrong about AI doom. And then I’d be like, “Okay, let’s take this to video. Let’s take this into the arena.” And I haven’t done one of these in a while, so thanks for bringing me back to my roots.
Devin 00:02:15
Yeah, for sure. I’m just excited someone wants to playfully debate these things, and in a good-hearted way.
Liron 00:02:22
Exactly right. I think it’s fun and they always go in different directions, which is why I felt it was worthwhile to have a show because it’s not gonna be the same debate over and over again. I actually don’t know where we’re going to focus on, where the route is gonna lead, which stop in the doom train you’re going to get off on.
So, we shall see. To kick things off, you know that I think that there’s a high likelihood of being killed by AI in our generation or in our kids’ generation and extinguishing humanity’s future. What do you think is going to happen that you feel like is very opposed to what I think?
Devin 00:02:52
Every tool comes with sharp edges. And to me, this is a tool, and it’s a tool to learn how to use, to become sophisticated in using. To get to this area where it’s this runaway freight train, I think there’s some impossibilities in between us and that.
Liron 00:03:10
So high level, I think there’s a high chance that humanity is going to go extinct in the next generation or two, and you think that is a very unlikely possibility, correct?
Devin 00:03:20
Yeah, yeah.
Liron 00:03:20
All right, if you don’t mind making it a little more precise, the big question is?
Devin 00:03:25
P(Doom). What’s your P(Doom)?
Liron 00:03:30
Devin Elliott, what is your P(Doom)?
Devin 00:03:33
I do not have a P(Doom). Like, it’s just a zero number. I guess the way that I think about it is, I think about P(Doom) as sort of the answer to the wrong question. There’s plenty of ways we annihilate ourselves. I don’t think it’s from this version of an AI doing that.
Nuclear Proliferation and Stagnation
Liron 00:03:51
Right. So when you say it’s zero, you’re willing to give other risks more than zero chance? Like for example, the risk of nuclear war in a given decade. What would you give that risk?
Devin 00:04:00
I’m not a political scientist. I don’t have a good number on what nuclear is, but I stand on the same side equivalently, which is I don’t think we should be slowing progress on nuclear either. I think that was a mistake that was made. I think what Brett Hall said on your last interview, which was interesting, which I tend to agree with, which is stagnation leads towards destruction. That’s the path that we need to avoid.
I think we’re on a type of path of stagnation, and I think nuclear we stagnated. And I think what happens is we derail the efforts rather than appropriately retard the efforts, which was the intent. And I think that most of these things take care of themselves in a lot of ways. I don’t mean that to be flippant or to dismiss the utility in it. It’s more to say that you create your own self-fulfilling prophecy by focusing on all of the wrong things for an inordinate amount of energy.
You can’t just talk about a runaway AGI and then have that be as easy as building it. The difficulty in building that is immense. The difficulty in building nuclear reactors is rather immense. These big things that we try and slow down, the derailment comes because we lose the focus and the context on what the actual problems are at hand, and we don’t get any sort of real way to put attention on solving those problems. Instead, we’re solving these side quests as if they’re the main quests. And I don’t think that’s the right way to do it.
Liron 00:05:40
Okay, so going back to my question, you do acknowledge that there are various catastrophic risks. Like, you do acknowledge that nuclear extinction is a risk, right? You’re not quantifying it, but it’s worth at least calling a significant risk. Correct?
Devin 00:05:55
Yeah. I mean, probabilistic distributions are in play for some realm of this. So it’s not zero, but it’s also not meaningful.
Liron 00:06:04
Right. Now, you mentioned this claim that it always works out, meaning if everybody just kinda muddles through and does their best, it always adds up to the best equilibrium. You were kinda making that point, right?
Devin 00:06:17
I think there’s an angle to say that. And it’s sort of like I don’t sit there and choose the best path for light. It falls into that natural algorithm because it’s got an order to it that it has to flow through and follow. And if you go back and try and analyze, did the light take the best path? The answer is yes, always.
Liron 00:06:34
Okay. Well, in the case of nuclear proliferation, don’t you think that the natural course would just be more and more nations spinning up a nuclear program unless there was centralized coordination to make rules about when that’s allowed and when it’s not allowed?
Devin 00:06:47
I don’t think it’s as easy as that. It’s not as trivial as those statements. You get game theory mechanics that show up too, so you start getting less and less likely to destroy each other with those things.
Liron 00:06:58
So let’s take the recent case of Iran, right? Like, they were racing toward a nuke, and now they got set back because Israel and the U.S. attacked them. I would be scared if we lived in a world where that check wasn’t in place, right? The centralized check. Do you agree that this is an example of a success story of a centralized check of power?
Devin 00:07:17
I would say the example of that exists within a regime that’s already decided that derailment is the path that we’re gonna go down, and once you’re within that regime, your optionality is now constrained to that regime unless you’re trying to break out of that regime. We’re not trying to break out of that regime right now.
Liron 00:07:37
So you’re saying the world is already made up of these high power countries, and you would have liked to roll back the clock and not even have a country like the U.S.?
Devin 00:07:45
Yeah, I think if I rolled back the clock, the argument would be to approach this with the intent to build functional systems correctly and that regime that we would enter and the optionality and the sets of how to handle those would look entirely different. If you’re asking about today, it’s like today I might take out that plane.
Liron 00:08:05
Okay. I’m just trying to understand. So okay, roll the clock back. You’re saying the geopolitical world order should just look so different? Like, there shouldn’t be like a superpower country? Like, I don’t even know what you’re suggesting here.
Devin 00:08:15
No, I don’t know that I’m making any sort of grand arching statement on that. I’m saying that at one point, we had a choice globally on how we were gonna handle nuclear and we chose a path, and I can’t sit there and describe what I think the actual counterfactual of the world looks like. I don’t know what that counterfactual looks like.
Liron 00:08:39
It’s just that you seem to be suggesting this principle where if you let local actors do what’s best for them, if everybody’s kind of out for themselves keeping an eye on everybody else, that’s the best way to get an equilibrium outcome. Just this decentralized kinda capitalism style, free market style. Like, that will always add up to a good outcome.
I should say I do agree with this, you know, 95% of the time. I am generally libertarian. I am generally capitalist. I think there’s a lot of power in this approach, but I can’t help noticing that there’s that last 5% of very important exceptions—exceptions that prove the rule. And I think not letting Iran get a nuclear bomb, or maybe you wish that North Korea didn’t have access to the nuclear bomb. Like, don’t you think there’s cases where a centralized kind of world police has to stop countries from nuclear proliferation?
Devin 00:09:40
I do agree that those scenarios show up once you’ve entered the regime that says that this is how we’re gonna operate. I don’t think that you should enter that regime at all. I get that we wanna pick the things that are the most catastrophic in mass, singularly, but if we look at construction, for example, housing, that can mass kill people if not done right. If standards aren’t held, people die.
One way to think about this: Are you in a high rise? Are you in a house or something like that?
Liron 00:10:11
In a second floor of a house.
Devin 00:10:12
Okay. Imagine you’re in a house right now like you are, two stories tall, but the quality of that house is today’s current quality… Let’s say it’s “Spotify quality” build. Do you feel safe in that house or do you think that house might kill you in the middle of the night?
Liron 00:10:27
Spotify works well.
Devin 00:10:29
Does it work well enough to make sure that you’re always alive?
Liron 00:10:34
I mean, I’m gonna say no, but the actual percentage of times that people opening Spotify don’t have a functional experience is probably less than at least one in a thousand. But sure, let’s say the correct number for Spotify is one in 10,000. Do I want my house to collapse once every 10,000 years? No, I guess I need more like once in a million.
Devin 00:10:51
Yeah. So you get the direction that I’m going with this. Which is like the quality of our building construction has standards and expectations that far exceed that of the stuff that is supposedly our best build stuff, which is software. And that important difference didn’t necessarily arise from this insane… “If everyone doesn’t build this way exactly, everyone dies.” But the truth is, in mass in a city, on an earthquake, if we don’t follow a set of standards, you might wipe out an entire city. And we’ve had these things happen in the past.
Liron 00:11:27
Yeah, exactly. Okay, so you believe… So you’re not like a 100% free market libertarian. Now, of course, there are also libertarian solutions to this. So the libertarian solution would be independent certification bodies that people can pay to evaluate these buildings. You still don’t need the government. But I’m just trying to understand your argument here. So your argument is you shouldn’t go full libertarian, right? You should have standards in some fields.
Devin 00:11:50
Correct. Never go full lib.
Liron 00:11:52
I thought you were trying to make this grand principle. Of how decentralizing is always gonna work out. What’s your claim here?
Devin 00:12:00
I think there’s heuristics that I’ve seen that are consistent across the board where if you just… Effectively, if you could just get everyone to follow this path, your outcomes are going to be fine. If I focus on quality and I hold a standard—I’m not gonna build dog shit. And in this case, that would be equivalent to an AI that takes off, in my mind, because you haven’t built a good system.
Like, we could make this very concrete about AI specifically if we want to. The ability for AI to run out in front of itself is effectively zero because if it doesn’t have access or if it doesn’t have token, it stops. Period, end of story. And it is not trivial to try and get those feedback loops to perpetuate. And moreover, you run into walls that require keys or money.
Centralized Risk Regulation
Liron 00:12:56
I was trying to factor the disagreement, though. ‘Cause we were off talking about systems of government. I like the nuclear proliferation example because I’m going to claim that AI is going to be an existential threat and you’re gonna push back. But conditionally, if I can get you to buy that claim, don’t you think that then would hypothetically put AI in the same category as nuclear proliferation? And aren’t you happy that there’s centralized control to stop nuclear proliferation? And hypothetically, if AI were a similar existential danger, wouldn’t you still want a similar type of solution, a centralized control solution?
Devin 00:13:32
I don’t think there’s a runaway potential with AI.
Liron 00:13:36
Right, so again, I’m asking a hypothetical question. This isn’t the part of the debate where I wanna argue whether or not AI is an existential risk. This is the part of the debate where I say, shouldn’t existential risks have centralized control as a way to prevent an independent actor from causing them?
Devin 00:13:52
No.
Liron 00:13:54
So, okay. So you’re unhappy with how we try to stop countries from getting nuclear bombs. You wish we would just let them?
Devin 00:14:00
I understand the approach that you’re taking, but it’s leaving out a key function, which is you don’t have to make choices like that unless you exist within a regime that has already made a certain set of choices.
Liron 00:14:14
Okay, but don’t we actually exist within that regime, and it’s hard to change? Like, we exist in a regime where the United States exists, right?
Devin 00:14:19
Yeah. We don’t for AI and we do for nuclear. So, my positions between the two then become balanced on what is practical and realistic.
Liron 00:14:28
When you say we don’t for AI, is it just because you’re not accepting the hypothetical that AI is an existential risk?
Devin 00:14:35
The current regime of the world has a mechanism in a box for nuclear, and they do not have a mechanism in a box for AI. I don’t like the mechanism in the box that the world has, but I can’t play around that. I can’t work with that.
Liron 00:14:51
So, can you just humor me? Just do you think that whatever existential risks actually exist should be regulated the way we try to regulate nuclear proliferation?
Devin 00:15:12
I do not like the way we regulate existential risks. I do not think we have people that understand risk correctly trying to measure risk. Just plain and simple. I don’t think most people understand risk.
Liron 00:15:26
Are you saying that you’re unhappy with what’s going on with efforts to stop nuclear proliferation or no?
Devin 00:15:30
I’m more upset at the box, but for the sake of moving past the box, then I’ll say sure.
Liron 00:15:37
So, you don’t like that the US bombed Iran’s nuclear program?
Devin 00:15:41
I mean, I’ll say this one. If we… I already know which direction you’re on this one. And I’m happy to take the same fire. I’m fucking stoked that they bombed Iran.
Liron 00:15:51
Okay, so, but I thought you were just saying how you’re unhappy with how we try to stop nuclear proliferation. I’m just trying to get your position here.
Devin 00:15:58
I understand that there is a dichotomy there that’s hard to reconcile. I think both things can be true. I disagree that we should be in this regime of existential threat from nuclear the way we are. Since we are, I disagree with certain actors getting a disproportionate access to that power.
So, I think it’s the power dynamics. The power dynamics that are created by entering a regime of centralized risk assessment turns into one that has lots of cancerous lumps on them. And one of them is having to make choices like this and say, “take out someone else’s nuclear facility.” And you won’t get a lot of consensus.
Decentralized Infrastructure & “d/acc”
Liron 00:16:53
Okay. Well, now I’m just curious. So, imagine you get to rewind history. You can just design the evolution of modern geopolitics from scratch. What do you want geopolitics to look like in terms of nuclear proliferation control? You said that you think that it’s bad that we got into this regime where the US is a superpower. So what would you hope for if you got to be God and rewrite this?
Devin 00:17:15
Small scale, regional nuclear facilities. I’m not talking statewide, like small decentralized. If I’m a maxi on anything, it’s like the concept of what decentralized infrastructure can do and how it should be used.
Liron 00:17:33
So you like Vitalik’s d/acc?
Devin 00:17:36
I did like that.
Liron 00:17:38
Mm-hmm. Yeah, so when you say, “Hey, my ideal is not to have a superpower like the US, it’s just to have a bunch of like really small governing, you know, city-states or whatever.” That’s your ideal world?
Devin 00:17:49
Not either. And this gets more confusing ‘cause I don’t think nature represents that as being the most effective solution either. You’ve got for example, the bacterial world has more variety and more diverse capabilities, but that diversity makes it so that it’s exceedingly difficult for them to team up and create any sort of centralization, which in turn becomes hyper relevant.
Liron 00:18:26
Okay, sure. In this ideal world, right? When you have a bunch of city-states playing out this scenario. I think it’s pretty likely that like a hundred different governing entities around the world will have their own nuclear programs. Let’s say it gets even easier and easier to make these nukes.
Devin 00:18:43
It has to be sooner if we want AI to scale up, so it’s gotta be soon.
Liron 00:18:46
Right. And forget about AI. I’m still just talking about nukes. Because nukes to me are such a good example of this insanely powerful thing that physics gives us that throws a wrench in the normal way that we like to organize humanity. Capitalism and libertarianism served us so well. But the existence of nuclear energy throws a wrench in this idea that you can solve everything with capitalism because you really don’t wanna have nuclear proliferation.
Like in this ideal world that you’re describing when there’s like so many governing bodies, I feel like 100 plus are going to have nukes and I just don’t see how we maintain mutually assured destruction when that many people have nukes. You really want that much nuclear proliferation?
Devin 00:19:34
I just feel like we’ve repeatedly demonstrated throughout history what happens when you’ve got disproportionate power or when you have well-distributed power, and so distribute the power. It’s a better scenario. We are less disruptive in those formats.
Liron 00:19:51
So just to be a little specific though, so you imagine that what’s going to happen is that nobody will ever dare to explode a nuke in this scenario where 100 plus governing bodies have nukes. Just nobody will ever explode one?
Devin 00:20:02
I don’t think in those absolutes like that. I think what the world that has distributed nuclear capacity has a much different relationship with materials. I just don’t think it’s reasonable to play the mind games where you go, “Well, what if this…” And then we negate all of the things we know to be true so that we can manufacture the scenario that it’s either you agree with all destruction or you agree that everything is utopia.
Liron 00:20:29
Wait, hold on. You’re talking about me manufacturing a scenario? Again, I’m just asking what you think a good outcome is. This is entirely Devin’s scenario right now.
Devin 00:20:38
Well, so my scenario is you don’t have to sweat a gigantic uncontrolled nuclear detonation in a world where we have distributed access and ownership over nuclear technologies. I think we’ve got a world in that case that has material science that looks entirely different and our capabilities are different.
Liron 00:20:58
So in your ideal world, I’m just asking do you think that a nuclear bomb would ever get exploded or no? And you said in your world there’s gonna be 100 entities that have nuclear bombs, right?
Devin 00:21:07
We’ll go with no.
Liron 00:21:08
Presumably going to be skirmishes, correct? You don’t think any of those skirmishes are going to escalate toward one being like, “Okay, you’re dead.” And keep in mind that here on Earth we definitely have countries that hate one another enough that if they had a nuke, they’ve been very explicit that they would love to use it.
Devin 00:21:24
Yeah. And if all of their neighbors who hate them had those nukes, they wouldn’t let them get there.
Liron 00:21:34
Yeah. I mean, I’m biased on this one because I’m Israeli. But I think that Israel’s nukes wouldn’t be enough to stop Hamas in Palestine from launching whatever nuke they had at Israel and then just letting the chips fall.
Devin 00:21:48
Sure. But that scenario that you’re imagining is missing where the whole rest of the world also has those capacities.
Liron 00:21:56
Right. But so what? I mean, mutually assured destruction doesn’t actually stop terrorists from launching nukes.
Devin 00:22:01
Yeah. I think these are bad world models that you’re working with. I think that is the wrong way to think about risk.
Liron 00:22:10
Back to clarifying what you’re claiming. You’re claiming that in your ideal world where you just have a bunch of little nation-states, and 100 of them plus have massive nuclear proliferation. In that world, you just think it’s unthinkable that anybody would dare ever explode any nuke.
Devin 00:22:25
I just think that’s a false scenario to consider.
Liron 00:22:28
And when you say false, you’re just saying it’s just the humans with their fingers on the button. You just never imagine anybody pressing the button ever.
Devin 00:22:34
What time is it on Mars right now? I think that it’s equivalent to that question. It has no relevance. So the answer that you come to has zero relevance to us. So that’s what I think of this scenario.
Liron 00:22:54
Well, it’s an answer about your ideal world. Like you’re advocating for this world and I’m just asking an important question about your world. Specifically the question of do you think your world will involve nuclear explosions or just a really good equilibrium of lots of people pointing nukes at each other and nobody ever pulling the trigger. And you see that as such a robust equilibrium that you’re questioning why I would even question it.
Devin 00:23:14
Yeah. I think that’s the closest we’re gonna get to an agreement on it. ‘Cause I don’t think that world that you’re saying is my world is my world at all.
Liron 00:23:26
But you can clarify. I’m asking you what happens in your world.
Devin 00:23:31
In my world we just keep building stuff and we keep working on how to work through the requirements to build things correctly because that’s what we’ve always done and it does always work. It hasn’t not worked.
AI Risk: The “Car Flying Away”
Liron 00:23:44
Okay. Well, I agree we might as well stop this branch of the conversation, and then there’s this other interesting branch which is, is AI even an existential risk? Because if AI isn’t an existential risk, then everything I just talked to you about about ideal governance is irrelevant.
Devin 00:24:07
Right.
Liron 00:24:14
I think that our intelligence is the reason why we have power over the fate of the world and there’s really no other animal or plant that really gets a vote. Similarly, I think that when you have a smarter than human intelligence, its capacity to think and control the future will vastly exceed ours.
And unless we maintain this connection where it’s always listening to us, always obeying us… Unless we maintain this, what I see as a tenuous link, I think that it’ll go rogue. And it just has to happen once, and there’s no undo button. And then it’s going to do whatever its initial programming kind of implies that it should eventually do. Like, you can’t update the software basically.
And you know crypto, right? And that is sometimes a problem with crypto, right, that you can’t update the software? I think we’re going to see that problem too with AI because if it doesn’t wanna be updated, and it’s too late to update it, and it’s smarter than us, I see that as a very likely scenario. Don’t you feel like I’m already talking about something that’s at least a couple percent likely?
Devin 00:25:23
I don’t. I think that’s like talking about my car flying away right now.
Liron 00:25:28
Okay. Well, I mean, with your car flying away, the problem is just it doesn’t have wings. It’s like the moment that it lifts off the ground, there’s just no force that could counteract gravity. But like, with AI, once it gets away from humanity, what do you think is that force that’s going to get it back in humanity’s control?
Devin 00:25:45
The AI doesn’t have wings, so just like my car doesn’t have wings.
Liron 00:25:51
Right, but what’s gravity? Gravity would be analogous to humans trying to tell it what to do again?
Devin 00:25:56
Let’s start from here on the LLM for a second. Right now, to get any intelligence out of an LLM, we have to wrap it with extra code. You take the trained LLM, which will accept prompts and output data, but all of us use them after a set of additional layers of software that are not LLM software. This is just external code that is then doing things to try and either feed stuff back, feed stuff in, work with it.
Liron 00:26:32
I mean, that’s somewhat true. I feel like if you just gave me a raw LLM, the reasoning that it’s doing, you could argue that at this point it’s a post-trained LLM, but it’s still an LLM. It’s not invoking like a harness or whatever.
Devin 00:26:47
It’s post-trained, but the thing is, is it can’t go anywhere, and it can’t even engage with you in a useful enough conversation without some additional layers. So we can’t get past step one.
Liron 00:27:01
That seems like it’s going too far. I think this isn’t a super critical point, but you can go really far in terms of being useful with an LLM. But maybe the point you’re trying to make is like, okay, well any time that it starts acting like an agent, like taking actions in your workflow, either it kind of dies out and it can’t work for many minutes at a time without messing up, or it can but that’s because it’s got this other non-LLM piece of the architecture.
Devin 00:27:26
Yeah, yeah. That’s fine ‘cause my point is more that the LLM intrinsically has no ability to get to run away anything because… Here’s a simple example. The LLM needs a calculator, right? The LLM doesn’t have an internal calculator.
The Calculator & Long Addition Debate
Liron 00:27:45
I’ve been surprised at how much math it can do. I feel like there’s an analogy between like a human trying to do math on the fly, doing Fermi estimates, and then I feel like it pulls out a calculator in similar occasions to when a human pulls out a calculator.
Devin 00:28:01
Right, and the important things is it needs the calculator which is not in the LLM, right?
Liron 00:28:06
Yeah. I think modern LLMs can do long addition. I don’t think they do it perfectly. Let’s say like five digit plus five digit addition. My rough sense is they’ll get it right like half the time, which is pretty good. That’s like better than a first grader.
Devin 00:28:20
Okay, but we through just our brains can get to what? 10 to the minus 43 and 10 to the minus 33 for scale and time? The calculator getting out five decimals… The point is the LLM can’t do a bunch of the things that people are imbuing it with being capable of doing to get into the future where it runs away.
For it to run away, it always requires the human to build an additional set of conditions, feedback loops and systems in order to support its effort. And all of that effort always runs into walls which is the boundary of the network that I have control and access over.
Liron 00:29:10
Okay, well, that’s an interesting claim to unpack. I will say this, I just tested GPT-5 even without thinking mode turning on, and I said, “Do long addition 38,239 plus 29,344,” which I don’t think that exact number was in its training set. And it did actually say, “Okay, we’re gonna add it digit by digit. We’re gonna carry as needed.” It managed to carry one of the digits and then it got the correct answer, 67,583, which I checked on a real calculator.
So, if you were trying to use that as an example of like a barrier of like, oh, it can’t really reason or follow algorithms. So it certainly passed that barrier but you’re saying it’s going to slam into some other barrier.
Devin 00:29:52
No, you’re still making my point. GPT-5 is not a raw LLM. You are using the output of the LLM married to internal software that OpenAI has written with the intent to make it capable of even doing that sort of stuff.
Liron 00:30:08
No, no, no. The output I got just now, there… It was entirely from the LLM.
Devin 00:30:13
Maybe we need to define some things here. When you go through a software interface from OpenAI, you might be selecting the model but you are not touching the bare metal of that LLM model. You are going through a pipeline of context orchestration that is being managed by a series of sophisticated architectural components.
Liron 00:30:41
Here, I’ll do it again.
Devin 00:30:41
This is not a raw LLM. GPT-5 is not just pure output with no extra software magic on the backend. There is non-LLM based code making it so that you can get to these outcomes.
Liron 00:30:55
So my understanding is because it didn’t invoke thinking mode, my understanding is it really did just run the same feedforward loop with all the layers, like token by token, to get the output we’re seeing.
Devin 00:31:05
All right, then I have Arizona oceanfront property to sell you.
Liron 00:31:09
[laughs] Okay. Well, can you clarify then? When I said, “Do long addition,” what’s shown on the screen right now? When it said the word, “let’s,” don’t you think that was just an LLM next token?
Devin 00:31:21
No, I think it’s grabbing tools and additional software packages in the background and loading them in. And they’re probably cached in Redis, so you’ve got sub-millisecond response times, and you would never know the difference here.
Liron 00:31:33
This is what I wrote. I said, “Do long addition [numbers] no tools.” I wrote that. I pressed enter. It didn’t activate anything. It immediately spit out the token “lets,” as in “Let’s add carefully.” So I’m just curious, in your mind, this response, “let’s add carefully,” do you think there was some system that’s not an LLM that said, “Yeah, let’s output the phrase ‘let’s add carefully’”?
Devin 00:31:53
No, I think the LLM output, “Let’s add carefully digit by digit.” And then the second agent that was involved in the loop said, “Oh, the user is probably looking for this.” And then probably 57 other agents in the loop said, “What do you think?” And all that stuff happened because it’s local, insanely fast, and the output is, “You got a calculator.” The agent got a calculator. It took that data. It runs deterministically through the calculator. The agent gets passed the attribute value that then gets displayed and rendered.
Liron 00:32:26
I see what you’re saying, but I think you’re just wrong in terms of how this particular tool works. Because over here where it started writing, “Nine plus four equals 13” because it’s adding the digits on the right. Do you think that there was some separate input-output tool that is not an LLM, like a calculator service inside of OpenAI’s cloud, and you think they fed the nine and the four into the calculator service to get 13?
Devin 00:32:54
One thousand percent I think that.
Liron 00:32:56
But you’re not even granting that an LLM can do nine plus four. I mean, that’s in the data.
Devin 00:32:59
So here’s the important thing about how LLMs work. When an LLM takes in your words, it turns those into tokens. So it’s not looking at the words, it’s looking at the tokens. Now, numbers are important here. The numbers are also being tokenized. So there is not a token of one and a token of two… et cetera, that are then mathematically accurately represented within that model. You’ve removed that context.
Liron 00:33:49
Okay, hold on. Can we do another example? I completely disagree. Look, one plus one. I literally told ChatGPT-5, “One plus one.”
Devin 00:33:55
But this isn’t a thing that we can disagree on. This is just how LLMs work. So an LLM doesn’t take in a number and map it to another number. It maps it to patterns in words and it regurgitates those. That’s why all of this extra effort had to show up to figure out how to get it to work with numbers.
Liron 00:34:20
Okay, but Devin, I can type one plus one into GPT-2, right? The model from 2018, and I think I’ll get two.
Devin 00:34:30
Yeah, and they’ve got tools around them now also. So if you put in a… if you go back to original GPT-2 and do math, it will not do basic math. Period, end of story.
Liron 00:34:48
But I certainly know that GPT-3 would tell you that one plus one equals two. And GPT-3 was still before OpenAI was even a consumer products company. And I’m telling you, GPT-3 or DeepSeek—you can run DeepSeek locally on your computer. There’s no fancy tools there. DeepSeek is going to tell you that one plus one equals two. So I think you’re really overreaching here telling me that one plus one isn’t something an LLM can do.
Devin 00:35:09
All right. I haven’t said it as concretely as one plus one equals two it can’t do. I said it doesn’t do math in the way you think it does. And so if you understand how it does math, then you wouldn’t be surprised when it gets something like two correct, but then it can’t do something with three, four, five digits, or even two digits. Like GPT-3 did not do simple multiplication very well.
Liron 00:35:38
I’m not claiming it does. I only brought up GPT-2 just to tell you that when you have an input like one plus one, there really is no need to send it to a separate tool.
Devin 00:35:52
Yes, but that’s still sort of an irrelevant point.
Liron 00:35:54
Well, the relevance is this. Look at what it originally said to my answer when I asked it the long addition question, what did it do? First, it broke it up into digit by digit addition, the exact what humans would do when you teach a kid the addition algorithm. The LLM knows the addition algorithm. And fortunately for me, just like a human child would, I know the one-digit sums. I know what nine plus four is. So it’s just doing what a human would do. It broke it up into small digits and then it ran the long addition algorithm, and these are all LLM capabilities.
Devin 00:36:24
Yeah, they’re not, but so let me flip this. Do you think that this interface is telling you the truth? Do you think that OpenAI, Sam Altman are telling you the truth in how this works?
Liron 00:36:38
Yes, because two reasons. Number one, they’re often very explicit when they invoke a tool. So you’re claiming that they’re like secretly invoking tools that they’re not showing. The other reason is I remember seeing similar things in GPT-3 before OpenAI was even a consumer company. Number three, I’m pretty sure this is also what you get when you run DeepSeek locally. And nobody thinks the open source DeepSeek project also has these other external tools.
Devin 00:37:06
I guess I don’t know what to say. The model you’ve got for how they work is incorrect.
Liron 00:37:13
Okay, so DeepSeek is gonna say something different? Why don’t we just test our prediction? You’re telling me if I go load a copy of DeepSeek on my computer, it’s gonna look nothing like this.
Devin 00:37:21
Yeah. Go load a copy of DeepSeek on your computer and do some logic-based deterministic problem set.
Liron 00:37:29
So this exact thing that we saw… doing long addition. You’re saying DeepSeek is gonna totally choke at long addition. That’s what you’re saying.
Devin 00:37:34
No. You’re reframing the way you use my words on the reframe is exactly how a girlfriend does it for me, which is to say you are extracting out of it the things you thought I said, but not actually the things that I did say. So I’m saying that an LLM is a non-deterministic system, and if you try and expect to get deterministic results out of it, you’re going to fail.
When you don’t fail, what you’re experiencing is software that is trying to overcome the magic and overcome the problems that LLMs can’t do these things. In terms of all of the research that’s come out, no one has figured out how to bridge this gap to create non-deterministic things, to have deterministic outcomes all the time, except by way of tools which do a spectacular job of doing that.
Determinism vs. Non-Determinism
Liron 00:38:41
So my understanding is when you set the temperature to zero, you can get a deterministic output. I heard Thinking Machines, Mira Murati’s new company, they claim to just take out even that non-determinism. So I’ve always thought that this claim of LLMs being non-deterministic was like such a clueless claim. You know, A16Z has been really big on saying LLMs are non-deterministic. “Look, we invented non-deterministic computing.”
Non-determinism has always been like pretty… You know, we’ve always had pseudo-random number generators. LLMs did not invent non-determinism or randomness. Sure, you can get different variations of your answers, but if you want the same answer every time, you can get the same answer every time. So I’m completely stumped by this focus on non-determinism.
Devin 00:39:40
An ant is non-deterministic, a dog is non-deterministic, and you are non-deterministic, and we do not get the same outcomes from all three of those.
Liron 00:39:48
I mean, I’m deterministic when I do long addition. Or 99.9% accurate if you give me long addition problems.
Devin 00:39:57
No. Okay. But you’re not perfect either.
Liron 00:40:01
Well, I mean, I can get an LLM with temperature zero that’s going to do long addition even better than me.
Devin 00:40:03
Okay. And I can do you one better. I could just make a tool. Like there’s literally no point to try and engineer and go through these insane machinations to try and get your AI to hop through a hyper-specific gate when the non-deterministic potential of it is what gives it quite a bit of power. And the way you shape it is to put it into a deterministic playground.
Liron 00:40:32
I do wanna stay for the record that claim you just made that non-determinism gives it power, I think is extremely misguided.
Devin 00:40:38
I know you think that, and I know all the other engineers out there who say and show that AI isn’t working also think that, and I can show you in code why you’re wrong and I can demonstrate it with tools that work, why I am correct, because I’m building with these things. The theory and the research is not the same as on the ground experience when it comes to building the things and making them work.
Liron 00:41:37
Okay. The reason I focused on a long division example is because I actually see it as very telling when I see LLMs successfully do long addition without invoking a calculator tool. Because even though they could bust out a calculator, the fact that they’re such broad tools and one of the things that they can do is follow an algorithm like the addition algorithm or compiling code. The fact that LLMs can do that even with some loss, much like a human, to me is actually a very important thing to know about LLMs—that they already have a handle on true reasoning.
Devin 00:42:43
They are. But I hit the wall with them every fricking day, and I’m telling you, these things cannot go further than like… They can’t even make it out my driveway without me pushing it out there. I used eight billion tokens this last month. The volume of work that I’m doing on a daily basis with these things all revolves around their failure modes. All I do is run into their failure modes and then build systems to improve and manage for those failure modes. If I could get it to do as natively as the things that you’re saying or believe that they’re doing, that would be spectacular. They don’t do that.
Liron 00:43:36
I mean, I agree that there’s some boundary of what they can’t do yet, and that’s what you’re going to find yourself doing up until the point that the boundary exceeds your own abilities, of course.
Devin 00:43:47
Where does the non-determinism become the weak point for you? You said you don’t see the power in it, and you’re pointing out to A16Z, which kinda makes me like icky that I got associated with them.
Liron 00:44:02
So you just gave an A16Z talking point. I mean, Marc Andreessen loves to say… Sometimes the guy is clearly wrong. He was wrong when he said, “Web3 is the new web.” Similarly, he’s saying that non-deterministic AI is the revolution. He’s actually correct that AI is a revolution. But to make a big deal out of the non-determinism piece, that is definitely the least interesting piece that you could point to.
Devin 00:44:28
And see, that’s the part where I take issue with. That is exactly the capability that I rely on. So, if I had a biggest fear, my fear is that someone would remove the non-determinism from the LLM because it will no longer be the thing that we think it is. It will no longer be capable.
Liron 00:44:54
Let me just be very clear, okay? Do you run the same prompt multiple times, the exact same prompt, and you really just need a different answer for the same prompt?
Devin 00:45:04
Okay, I have never reused the same prompt twice. I don’t save prompts. I don’t premeditate prompts.
Liron 00:45:10
Okay, so you don’t even know if it’s non-deterministic because the definition of non-determinism is giving you multiple answers for the same prompt.
Devin 00:45:16
I’m very certain of that. I think what I’m trying to point out is that the way that everyone using LLMs right now, are using it wrong. You don’t need to reuse prompts.
Liron 00:45:33
Okay. So, why do you like the non-determinism? ‘Cause you just said you don’t reuse prompts.
Devin 00:45:36
Because it allows more flexibility for edge cases internally. So, for example, I could create a regex parser. I could also make a deterministic boundary with constraints and say, “LLM, pretend you’re regex and parse this thing.” And that non-deterministic capability is far more robust than if I build out a regex parser because the regex parser is going to break every time something doesn’t meet exactly. And the LLM is gonna go, “Well, I know it’s supposed to parse this thing, but that thing’s not quite right, but it is close to this.”
Liron 00:46:41
So, you’re not using non-determinism in the sense that people normally associate, like, “Oh, I need a probability distribution over what it’s going to do.” You really don’t care about that aspect. What you care about is the unstructuredness of it, right? Which is really a different concept than non-determinism.
Devin 00:46:59
It… Yes, but it’s manifest as non-determinism. And if you take it away, it will break the same concept.
Liron 00:47:06
Okay, I don’t think I agree that it’s manifest as non-determinism because, again, if I just set your temperature to zero, meaning the same tokens always follow the same series of tokens—
Devin 00:47:15
Won’t work the same. When you change the temperature to zero, what you do is you completely change the light cone of where it’s looking at its distribution.
Liron 00:47:28
But for all you know, your temperature has been set to zero and you admit you wouldn’t have noticed because you never run the same prompt twice.
Devin 00:47:34
That’s a weird claim to make. I would totally know and I adjust the temperatures with purpose.
Liron 00:47:39
That’s true. Even if you run a prompt once, you might notice that running ten different prompts you’re getting such a standard average answer to all my ten prompts. So, you might even notice even without running the same prompts.
Devin 00:48:01
Yeah. I mean, I can tell the second there’s a change or an update. The patterns of behavior adjust.
Liron 00:48:27
To me, it’s just a profound point because I think there’s this general point in computer science where randomness doesn’t create value. Like, the only time when you really need randomness is like in cryptography where you’re trying to psych out your attacker. But in the case of answering your question where there is no attacker, there is no need for intentional confusion. You don’t get smarter from introducing randomness or non-determinism to anything. And so when I hear Marc Andreessen say, “Wow, non-deterministic AI,” I’m like, “What are you talking about? That’s not interesting.”
Devin 00:49:21
I think it’s simpler than that. The non-determinism really is, as you said, I’m just not necessarily gonna get the exact same answer every single time, and I need that adjustment because I need it to not be so concrete because I intend to give it the context that it needs to give me similar answers. And I don’t want that predetermined in there because I need the flexibility to put it into whatever box I want to.
AI Progress & Bio-Computing
Liron 00:49:49
All right. So, you’re so unimpressed with the limitations of AI capabilities, and I’m saying even if I accept that, let’s say I even grant your claim, don’t you still think that we’re going to keep making better and better AIs and we’re going to keep going until we make an AI that’s just better on every dimension than the human brain?
Devin 00:50:30
I don’t. I see a distinct difference between the underlying LLM’s potential and then the software wrapped around it. I think the expressions that we will see of brilliance that come from these are going to come from the software that’s wrapped around it.
Liron 00:50:57
So you’re kinda saying there’s no such thing as high general intelligence.
Devin 00:51:01
Yeah, because the model itself is fundamentally limited by its potential. The takeoff that we’ve had so far, 70 years leading up to the point where we suddenly get an LLM that actually works. We’re in a completely new regime and paradigm now. We are now on the plateau on the other side in a new paradigm, and that new paradigm has an exponentially larger surface area than the one we just came out of. And that surface area is technical understanding that we’re entirely missing.
I don’t buy the general intelligence thing. I think AGI is the answer to a bad question. What I think is gonna happen and where the brilliance shows up is the stuff that we’re gonna wrap it around, but it’s so fundamentally limited.
Liron 00:52:01
Okay. Let’s try this hypothetical. We upload human brains. Don’t you think at some point in the next century there’s going to be like scans of human brains in computers and we can simulate them? And don’t you think within a century or so we’ll get ‘em running faster than a human brain runs?
Devin 00:52:23
Uh, I’m in line with Federico Faggin’s thinking on this. I do not think it’s fundamentally possible to upload a human brain because I do not think a human is their brain. I think the essence of what we are is much more… it’s a technically different set of concepts.
Liron 00:52:46
Okay. What are you that’s not just your brain?
Devin 00:52:48
Well, I’m a cell at the end of the day.
Liron 00:52:51
Yeah, but the cell is just scaffolding for the brain.
Devin 00:52:56
Yeah, but then you’ve got the gut-brain. Like, you’ve got DNA that the research is showing is some form of fractal antenna. You’ve got a hell of a lot of technical details that are very important and unanswered about what it means for a human to be thinking. We are not just the brain. We are the whole set of information going through it.
Like, we would get closer to potential if you said, “I’m not gonna upload your brain. I’m gonna upload all of the information from your virome, your biome, and your brain.” That’s a little more interesting.
Liron 00:53:36
I think a human has contingent parts. Like, think about like an airplane versus a bird. At the end of the day, there’s this property called lift and power. There’s these certain building blocks that are actually important to building flying machines and birds have like a healthy dose of them, but when you build an airplane if somebody’s like, “Oh, come on, man, you got like the beak? You got like the legs?” Not all of these bird things correspond to the building blocks of flight. Similarly with intelligence, does the AI really need a gut-brain? Is that really the essence of intelligence?
Devin 00:54:19
Yeah, and my time in sports tells me that is so wrong. If we know each other and you have had a bad meal the night before, I know before you’ve said anything.
Liron 00:54:39
Right, but isn’t that just because my stomach will be hurting?
Devin 00:54:42
You are an expression of the entire system you are in. You are not just isolated to whatever is thinking between your ears.
Liron 00:54:50
I feel like you’re taking a very holistic perspective. Mostly my performance in math will be determined by my studying and the intelligence that I came in with. The meal I had last night is just not going to be a major factor.
Devin 00:55:10
Your meat space is a certain amount of potential for sure, but it is not the whole thing that shows up.
Liron 00:55:22
So you really… Okay, so the gut-brain. Imagine I was just able to pull out the entire nervous system, brain stem complete with like all the nerves that go into every part of your body. Don’t you think that like all of the nerves of a human including the neurons constitute the human’s intelligence or do you have to go farther?
Devin 00:55:38
I generally think you gotta go farther. It’s provably not enough.
Liron 00:55:47
I mean, what if we just wire up all the nerves into electrical stimulation? Like, do you really need other physical effects besides electrically simulating the nerve system?
Devin 00:55:56
I think if we go like… No one enjoys the reductionist argument for some reason. But the fundamental differences between what an LLM is made of and what we are made of are so… They’re just entire worlds apart.
Liron 00:56:24
Right. But I’m not even talking about LLMs though. I was trying to make an easy hypothetical of like, okay, just imagine an uploaded human and you’re not even willing to imagine an uploaded human.
Devin 00:56:39
Well, because I think it’s a bad line of thought because it’s not plausible. We will never see an uploaded brain. Like, if you want a prediction, we will never see an uploaded brain.
Liron 00:56:56
Are we ever going to see a module of a hundred neurons that’s artificial? You know, like the same way that they’ll have like artificial organs.
Devin 00:57:07
Oh, I’m sure we could do some weird shit. But I mean, there’s a what? How many trillions of neurons delta between those things? I can’t upload my brain into a computer until the computer is made of something that is not computer.
Liron 00:57:36
Well, what does it have to be made out of?
Devin 00:57:42
Well, look, Eliezer… In chapter nine of Ascension or something like that, he was talking about the advancement of bio-computers and stuff like that.
Liron 00:57:51
Oh, oh. There’s a chapter nine in If Anyone Builds It, Everyone Dies? Chapter called Ascension.
Devin 00:58:02
Yeah. And he’s talking about nanobots and stuff in there. To get to some of these things, like he used the term growth, right? We grow these computers. We will not grow anything unless it comes from a cell. Doesn’t matter how you want to call it. So, we gotta have some control over the cell to be able to grow things. Like that’s how things grow. We have no examples of anything else.
Liron 00:58:40
I mean, doesn’t a computer virus grow?
Devin 00:58:43
It copies, but I don’t know if growth… Growth is a difference than expansion, right? I start with a single cell and I become what’s standing in front of you. That’s growth. If you copy me, we expand.
Liron 00:58:59
What about a computer virus that just coordinates bigger and bigger server farms?
Devin 00:59:05
It’s just expanding in scope and scale, but there’s no growth going on. It’s not becoming a new shape. It’s not extending its capabilities. It’s not applying some learning. It’s not giving birth to new things. It’s copying and it’s expanding.
Liron 01:00:03
All right. So you think human intelligence is made out of something besides electrical signaling maps between neurons, correct?
Devin 01:00:10
I think it’s pretty likely that it’s at least powered by a proton hopping on the back of an electron at every 10 angstroms. And that the inside and the bottom of my LLM is just a gate. I know reduction is kind of not…
Liron 01:00:25
So you think that if we were to simulate on like the atomic level a human brain inside of a computer, and the computer was like a very different substrate. But if it was running an atom level physical simulation, then you do feel confident that it would very slowly and painstakingly simulate human intelligence, correct?
Devin 01:00:48
I think if you can get down to an atomic level and in theory you were playing like programming all the way up, then yeah. Then maybe the answers that I would have would be entirely different.
Liron 01:00:59
It sounds like you feel like there’s some lower layer of the stack that we really need to model. Like, you really need to model what hormones are there above and beyond the way that they electrically influence neuron firing.
Devin 01:01:19
Yes. We can go with the holistic.
Timelines: 10 vs 1000 Years
Liron 01:01:27
Okay. And that makes you think it’s gonna take longer to simulate human brains. So, don’t you think that’s eventually coming?
Devin 01:01:40
Right. I think we’re potentially capable of quite a bit more than we’ve accomplished so far. So, perhaps we’re growing things from cells in a million years from now or a thousand years.
Liron 01:01:51
So there’s gonna come a day when humanity is going to be about to create… I think we’re like very close to it, but it sounds like the only difference between me and you is I think we’re like a few years away, and you think we’re like, I don’t know, a century away. But we both think that there’s this threshold coming.
Devin 01:02:12
Okay. I think we’re a thousand years away from growing computers.
Liron 01:02:19
Okay. But remember the famous anecdote, right? The Wright brothers were saying… I think Orville Wright was saying like, “Man cannot fly for like a thousand years.” And then like two years later, he made the first heavier-than-air flight himself successfully. So, don’t you think that even though you’re claiming right now, “Yeah, it’s gonna take a thousand years,” don’t you think that it might actually just take a few years?
Devin 01:02:42
I mostly don’t.
Liron 01:02:44
Okay. Mostly don’t. Don’t you think there’s like a few percent chance that it might just take a few years? I’ll give you a few percent chance it might take a thousand years.
Devin 01:02:52
I think anything less than 100 years feels highly suspect. I wouldn’t bet on anything in 10 years.
Liron 01:03:00
Okay, highly suspect, but can you give me 5%?
Devin 01:03:04
In my experience being on cutting edge fields, anything predicted within one to two years shows up within the one to two years. Anything that’s predicted two to five years takes at least 10 years. And anything where someone says five years, 20 years down the road, you still won’t see any changes.
Liron 01:03:29
Okay, I think it’s coming in one to two years. What do you say to that?
Devin 01:03:33
I think in this case, I think it sounds a lot more like U2. And I think a lot more like it’s 10 plus years minimum. And I think that simply because the surface area of technological learning that we have to do is so big. And we might be on an exponential learning curve, but if your space is exponentially larger, you need all of that time.
Liron 01:04:01
Okay. So, let’s say we get to that time, whether it’s in one year or a thousand years. We get to that time when we have a smarter-than-human intelligence, faster than you. Don’t you think that there’s a pretty large risk of losing control of a system like that?
Devin 01:04:30
I still simply know because even just like the basic current limitations. If there was 100,000 users out there like me, we’d be underestimating potential power capacity to even provide that back by 200%.
Liron 01:05:00
So the direction you’re going with this is you’re just saying that “Yeah, in a thousand years, this super smart system might exist, but I’m confident that it’ll be like so power hungry.” Is that basically what you’re saying?
Devin 01:05:10
Everything is so power hungry, we can’t even get over the next step. We have no potential to reach right now. None.
Liron 01:05:20
Okay. Even a thousand years from now, right? We’re not gonna figure out how to not make it power hungry a thousand years from now?
Devin 01:05:26
I have no concerns about that future. And it’s all entirely moderated by the fact that we don’t even have the potential to take the next step right now, let alone the hundred steps required to get to that.
Liron 01:05:41
I’m just asking you what you think we have when you get it. I know you’re saying, “Well, I think that time is far away, so I don’t wanna talk about it.” But can’t we just talk about it hypothetically? Because here’s the thing, one day, something might change your mind about the timeline. And when that day comes, isn’t it useful that we will have already discussed what happens hypothetically?
Devin 01:06:13
I just don’t know… Like, my head doesn’t live in that space ever. In the space of impractical and impossible, I don’t go to. I try and find the edge of what’s practical and plausible.
Liron 01:06:30
Yeah, sure. So you’re basically saying, “Look, I’ve already gotten off the doom train. I’m just so unmotivated to think about a hypothetical future where something exists that’s smarter than the human brain. So like, whatever happens then, happens. We’ll discuss it in a thousand years.”
Devin 01:06:44
It’s not that I’m unmotivated. I’m entirely motivated to find that. Like, I wanna find a disaster ‘cause I don’t want to walk into a disaster. I don’t wanna be the one that built… I wanna build the thing that everyone… If someone builds it, everyone dies, right? I don’t wanna build that.
Liron 01:07:04
You don’t wanna build that. Okay.
Devin 01:07:04
I 100% want to build the thing Eliezer thinks he’s scared of, but I don’t want to build the thing that kills everyone, right? And I don’t think that’s really that difficult because I find it nearly impossible to figure out how to even make that happen.
Liron 01:07:20
Right. So I think I correctly summarized you as just saying like, you are so confident that we’re not going to get smarter-than-human AI anytime soon. And because of that, you just think that all of the dangers of smarter-than-human AI are just not even worth talking about because it’s so far in the future.
Devin 01:07:40
I guess so. Yeah.
Who to Trust? Builders vs. Doomers
Liron 01:07:42
Yeah, I mean I think that’s… we can wrap on that crux. I understand why you might have some reasons making you think it’s less than 50% chance. But when you’re so confident you won’t even give it like 5%, to me, that’s weird because we have such a track record of making breakthrough after breakthrough. I just feel like a basic extrapolation should make you worry that smarter-than-human AI is coming.
Devin 01:08:37
But I’ve been doing these extrapolations for longer than you guys have been thinking about it. I started building these scaled-up systems more than a decade ago. I’ve done all the math on the processors.
Liron 01:08:49
I mean, did you predict we were about to pass the Turing test? Didn’t that surprise you?
Devin 01:08:53
I… You can go back and look at my track record. I had a million devices that were distributed globally around the world, shipping data all over the place, building systems that people said were never gonna be useful.
Liron 01:09:17
So you’re saying progress can happen fast, right? That supports my argument.
Devin 01:09:21
I think progress happens fast within a relative potential. I’ve experienced quite a few people in my life who are risk-averse, and I know what that thinking looks like. But I think if you’re on the ground building the stuff and at the cutting edge… I don’t mean going to piece together some APIs with LEGOs. I mean trying to build these systems that do these things. I want the system for myself that is fully autonomous. And yet, it’s extremely difficult and the scale and the scope going on the decades of trajectory and planning are still very far off from a lot of them.
Liron 01:10:18
Okay, I’m not really liking how you said “if you’re on the ground working the stuff,” because I get that you’re talking about yourself. But if you actually look at the set of people who are, quote… “on the ground working on this stuff,” many of these people are saying, “Hey, we’re getting close to superintelligence.” So you wanna start your sentence with, “If you’re on the ground working on this stuff,” I think the end of that sentence goes, “There’s a 50% chance you’re warning everybody that superintelligence is near.”
Devin 01:10:53
No, I think it’s super easy to look at why those guys are conflicted. And then I also think it’s super easy to see that historically, the people that are the researchers that build the stuff are not the ones that actually understand how to use the things correctly or what their real potential is.
Liron 01:11:11
Okay, so in your mind, the class of people who we should trust are people who are on the ground building the stuff, but haven’t joined a company that’s on the ground building the stuff. Which is yourself. You’re the best authority.
Devin 01:11:24
I’m certainly in the top 1% of how these things work. I mean, I’ve got 15,000 hours of working with LLMs directly unmitigated by any other companies. And I don’t think I’m the only one. I think there’s a bunch of us. I actually think the other people like me are being ignored and hilariously so.
But I had a CTO once, he used to call it “plastic bag ghosts,” and what he meant was if you put a kitten in a room in a plastic bag, the kitten gets into the plastic bag and it starts freaking out because the bag is like alive and like the world’s ending and chaos ensues. There’s a lot of plastic bag ghosts out there.
And when you don’t have hands… Here’s the part where the building to me becomes relevant because it’s hands on around the third rail. The engineer laying the rail, the electric rail that the train’s running on, they’re the ones that understand the real risks, not the person in the boardroom talking about it, not the person in the research lab thinking about the next one. The person on the ground knows what works, knows what doesn’t work, and has to touch near the third rail. So they’re the ones that are getting electrocuted or not.
I just think that the hardest thing in tech is that you’ve got people playing with tools that are mostly invisible that can do powerful things and most these people don’t have a good grounding in what risk really is and how to work with it. And I think that’s expressed in the words that they use most of the time.
Conclusion
Liron 01:13:11
Okay, well, we can wrap on that. Let me recap some of the major points that we hit and then tell me if I missed anything.
All right, so first we talked about policy and this general idea of should you divide the world into these independent, decentralized, independently governing bodies. Then we pivoted because you said, “Well, regardless that’s not even something we have to discuss because AI is so far from being an existential threat.”
So then we talked about do we know about the limitations of LLMs, and I seem more bullish on how far LLM-based technology can go compared to you. But ultimately, that didn’t matter that much because I just claimed that AI systems will get better and better. I just claim that they’re going to keep improving on benchmarks.
And then we talked about do you think that AI is close to surpassing human intelligence? And you gave a very emphatic no. You threw out a thousand years as the amount of time you think it’ll take. I think it’s more like five years, maybe 10.
And then we talked about who we should trust. You know, like who’s really on the ground having a good opinion about this stuff, and you think that people like you, who have the 15,000 hours of experience are unanimous as long as they don’t have corrupt incentives, they’ll all tell you that there’s nothing to worry about. Right? I feel like those are the major points we hit.
Devin 01:15:08
I would push back on most of those as being caricatures of this stuff. But I don’t know. It’s fine. It’s a fine characterization at the same time.
Liron 01:15:23
Okay, I’ll give you the last word, yeah. Uh, so you can make a closing statement and leave viewers a summary impression of your view.
Devin 01:15:30
I think what AI’s gonna bring in will be amazing. I think a bunch of it is gonna suck before it actually works awesome. I think most of the underlying models themselves don’t have that much more room, at least for now. They require new innovations ‘cause you’ve already gotten 60, 70% of the juice out of that. That doesn’t at all touch the software that being packaged on top of it.
I think this is a lot like discovering electromagnetism. It starts with a stick that’s spinning circles in the sand, and then you realize we can build motors. I think we’ve got a distance before we get there. I think wherever there is that we get, we’re all gonna be fantastically smarter than we were before. And whatever version of brilliance that is, it’s certainly more than we’ve got now.
Liron 01:16:20
Nice, man. All right. Well, we obviously disagree. I think our opinions are kind of light years away. Maybe one of us will update. Maybe something will make me think that we’re getting into an AI winter. Maybe something will make you think that the timelines are actually fast. But I just want to thank you for coming in, hashing it out. You know, you were totally game to answer all the questions, and I appreciate that. You know, I think this show is facilitating good discourse. And thank you for being a part of it. Devin Elliott.
Devin 01:16:58
All right. Thanks for having me. Really appreciate this.
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
