My new interview on Robert Wright's Nonzero Podcast where we dive into the agentic AI explosion.
Bob is an exceptionally sharp interviewer who connects the dots between job displacement and AI doom.
I highly recommend becoming a premium subscriber to Bob’s Nonzero Newsletter so you can watch the Overtime segment in every interview he does —
Here’s the premium Overtime segment where we bravely step into the Israel/Palestine debate:
Nonzero Podcast on YouTube — https://www.youtube.com/@nonzero
“Dario Amodei isn’t the hero we need” (Robert Wright, NonZero Newsletter) —
Robert Wright, The God Test (book, Amazon) — https://www.amazon.com/God-Test-Artificial-Intelligence-Reckoning/dp/1668061651
Timestamps
00:00:00 — Introduction and Today's Topics
00:03:22 — Vibe Coding and the Agentic Revolution
00:08:57 — The Future of Employment
00:17:57 — Agents and What They Can Do
00:27:59 — The "Can It" and "Will It" Framework for AI Doom
00:30:27 — OpenClaw and Liron's Experience with AI Agents
00:36:45 — The Case for Slowing Down AI Development
00:43:28 — Anthropic, the Pentagon, and AI Politics
00:48:37 — AI Safety Leadership Concerns
00:52:06 — Closing and Overtime Tease
Transcript
Introduction and Today’s Topics
Robert Wright 0:00:05
Hello, Liron.
Liron Shapira 0:00:07
Hi, Bob.
Robert 0:00:08
How are you doing?
Liron 0:00:10
Great to be back on the show. I’m loving your series of AI episodes. Shout out to Holly Elmore. Always great to hear her.
Robert 0:00:17
Yeah, she’s been a great feature on the show. But only about six weeks ago, we did one with her that people can check out. She is an ally of yours. Let me introduce us both. I’m Robert Wright, publisher of the Nonzero Newsletter. This is the Nonzero Podcast. You are Liron Shapira, host of the highly regarded Doom Debates podcast.
And people can just check out your backdrop. If they don’t think your podcast should be taken seriously, all they need do is look at the video version of this on YouTube, and they will see that this — you are the Walter Cronkite of your era.
And you are, as the name of your podcast suggests, a doomer in the non-pejorative sense of the term. We can discuss actually later whether that should be considered — there’s a debate over whether it should be embraced or not by people in the AI risk safety concern movement.
And then we’re gonna do a couple other very interesting things. First, in the overtime segment, we’re gonna do something very unusual for a podcast that’s largely about AI, which our conversation is going to be, which is we’re going to debate in very civil fashion the Israel-Palestine-Iran issue on which we have very different views.
And I think the reason we agree we wanna do that is because it’s important for people like us to model civil debate on this because you and I both believe that a cause we share, which is getting the establishment to take AI risk more seriously, is so important that people who share that view can’t afford to be fractured along ideological lines, and yet they shouldn’t have to avoid speaking out on issues they care about. So we have to learn to talk to each other civilly about things that even we care passionately about. Is that a fair...
Liron 0:02:15
Yeah, totally. And by the way, thanks for the shout-out about the studio. It’s all thanks to generous viewer donations who believe in the cause of Doom Debates, and so they’ve really helped me level up the show.
But yeah, as to Israel-Palestine, we are two smart people who just have very different views apparently on Israel-Palestine. I don’t have any professional background or training on Israel-Palestine. I just happen to be Israeli. I moved to the US when I was three, but I know a lot of people in Israel, a lot of my family’s in Israel. And you care enough about the subject to tweet about it regularly. So I think it’ll be good bonus content for your viewers.
It’s only gonna be in overtime, right? So we’re just teasing it now, but I think your viewers will enjoy, “Hey, it’s just two smart people. We’re having a good faith debate.” I don’t think I’m super ideological. I think you and I agree on most things in the subject of AI, so it’s a good example of how you and I are just gonna compartmentalize. It doesn’t mean we have to hate each other on AI just because we have different views on Israel-Palestine, so it’s gonna be fun.
Robert 0:03:12
Right. Yeah. No, I’m looking forward to it, and I’m determined to maintain my equanimity, which I’m generally bad at doing in life, but I promise.
Vibe Coding and the Agentic Revolution
Robert 0:03:22
So now, as for the other part of the conversation, I think I’d like to do a lot of talk about Anthropic. A couple of things. I mean, first of all, there’s the whole Pentagon-Anthropic issue. Pete Hegseth kind of declared war on Anthropic, declared it a supply chain risk. I think that’s never been done with an American company before. Has pretty serious consequences.
There’s that whole issue, which gets into the issue of AI’s role in targeting and military endeavors and mass surveillance and stuff. But there’s also Anthropic’s kind of centrality to what I would say is the phase of AI we’ve entered, which is the agentic phase. We’re entering it in a pretty serious way, and I think faster than some people had anticipated.
And kind of central to that has been Claude Code, with which you have a lot of personal experience, I know, and we’re gonna talk about that. And then that has in turn kind of spawned this OpenClaw thing, which in some ways takes it to another level. And of course, Google and OpenAI have their own versions of Claude Code-like products, and I believe any of those can be harnessed by OpenClaw as I understand it, any of those engines.
Liron 0:04:46
Yeah.
Robert 0:04:47
Which — maybe we should start out talking about the agentic stuff because I don’t know how many people appreciate the sense you get within the AI community. I have a Twitter list of just people who talk about AI. And for, I would say, a few months now, there’s been a noticeable sense of acceleration that I think is driven both by the growing frequency of model updates, of LLM updates and the continued kind of breaking of new records in evals to the extent that they can be trusted.
But a lot of it is, I think, about these agents, right? And part of it kind of starts with this vibe coding thing, which maybe a lot of people haven’t — a lot of people have heard about and kinda know what it is, but I think it goes beyond that, as I’ll try to explain. But why don’t we start out with the vibe coding and how you have come to appreciate the power of that? ‘Cause you’re a programmer. You’ve been doing human programming, the old-fashioned kind, for a long time.
Liron 0:06:06
Yep. Software engineer is what I used to call myself up until these last couple months. I’ve been programming computers since I was nine years old. My actual job, to the extent that I’ve had a real job instead of just making podcasts, has been connected to software engineering and running software companies. So I’ve written many thousands of lines of code in my life, and I like to think that I’m a 10X engineer. I can get stuff built faster than most engineers.
It’s just these last couple months, in this takeoff, in this singularity, it’s just been stunning what’s been happening to software engineering. I’m not the only person saying this. I’m actually a little bit late to the party. Andrej Karpathy was tweeting about this. He’s like, “This is a game changer. I’ve never seen this before. I have ten agents running.”
I’ve really just dived into this in the last few weeks, and long story short, I think I’m pretty much hanging up my title as a software engineer. My relationship to the software is very much like senior software engineering manager, where I have an army of roughly four software engineers. It’s as if I just got a budget of a million dollars a year to spend on four full-time software engineers who work for me and check in code very quickly, doing exactly what I tell them to do, and have excellent judgment, excellent speed, excellent breadth of knowledge. It’s better than hiring four humans.
It’s almost a drop-in replacement. They need a little bit of management from me, but literally if you look at the transcript, I’ll write one sentence like, “Hey, if you look at this file of code, I wrote it in 2023. Some libraries have changed. Can you rethink how to do it better?” That’s what I’ll tell them, and they’re like, “Yeah, sure, give me two minutes. Okay, here’s a plan,” and it’s a full-page plan. I’m like, “Oh, good plan.” And they’ll do it, and I’ll be like, “Oh, good execution.” And I’ll ask one small question, and they’ll fix it, and then boom, Git commit, 500 lines of code change, looks perfect. That’s the experience of being a programmer today. It’s just truly insane.
Robert 0:07:55
So it’s very much the kind of communication you would have with a human programmer working for you a couple of years ago.
Liron 0:08:01
Yes. The only difference is that it’s much faster. So I can imagine hiring a senior software engineer — this person graduated from MIT, hypothetically, and they’ve worked at Google, and they’re highly intelligent. It’s hard to find a person like that. Only a small percentage of the human population even has the kind of aptitude to be that good at slinging software. That’s why software engineers have always made so much money, ‘cause not that many humans are able and willing to do it.
So I would hire a person like that, and I’d be like, “Okay, go off for two days. Do this kind of code refactor. Go integrate this library. Get back to me. I’ll code review.” But every time I tell them something, I can expect a few hours. They need to finish what else they’re doing. They need to load the problem into their head, which can easily take half an hour just to really sit down and look at the code.
The AI does the same thing in 30 seconds, and then it delivers the same product. And in the meantime, I’m also talking to three other AIs who are doing other tasks. It’s just — Bob, I’m still — every day, I still wake up, and this is the first thing on my mind. I can’t believe this is real.
The Future of Employment
Robert 0:08:57
Yeah. So let me ask you a question about the future of employment. The background for this is, as I — I forget whether I used the phrase Pause AI, but of course, Holly Elmore runs Pause AI US, I think, and you are affiliated with Pause AI, and you’ve gone to some of the demonstrations, and you would like to see us pause large training runs is the basic idea. But in any event, you think slower would be better, and I certainly agree with that.
And I’m kind of fed up with the people just beginning conversations with the premise that faster is better. “Oh, we can’t think about this kind of regulation. That would slow us down.” Look, slowing down, I think at this point, is a feature, not a bug. Stymieing technological progress is, on balance, I think, a good thing because of — and here we get back to the employment thing.
I think it isn’t just the classic kind of sci-fi doom scenario. I do attribute some degree of probability above zero to that. I think that is a thing to take seriously. But I also think there are gonna be a lot of other kind of just destabilizing fronts, and it’s gonna be hard for society to adapt, and I know Holly agrees with that. She’s kind of agnostic as to what reason you cite for thinking slower would be better. All are welcome in her cause, regardless of what they focus on.
And I could go on and on about areas where I think adaptation is gonna be hard. It’s gonna add new dimensions to the challenge of child-rearing, education, a lot of things. Of course, terrorism risk and all kinds of things.
But the employment thing — I’m wondering what your current view is, because historically, as people say, it’s true. In general, with technological change, at least as many jobs are created as are taken away. I would say, first of all, even if that turns out to be the case, that takes time, and this is gonna happen fast. So that can be a disturbance in itself, even if there are enough jobs.
But the question I have for you is now that you’ve processed this a while, are you of the view that this is just gonna reduce net employment significantly, or, on the contrary, maybe — look, yes, it’s true it’ll turn people who used to write code into software managers rather than coders, but there’s gonna be so much more software to create that maybe we won’t see a net loss of employment in the software making sector. Have you developed a view on that?
Liron 0:12:06
I have. So I do think that the net effect on most people’s jobs is probably going to be bad. I’m not super confident, but that’s my guess. And my reasoning there is because when I think about hiring people for various positions at companies that I’ve run, my first thought is, “Do I really need this person?” So many of those positions feel more and more automatable. They feel more and more like giving somebody a spec.
One of the processes that I recently almost automated is thumbnails for my YouTube show. We used to have this whole process where we’d gather the elements, and we’d be like, “Okay, now brainstorm a bunch of different designs, make six variations, A/B test them on YouTube.” But everything I just said, including making the design in Photoshop, all of that is literally just one couple paragraph prompt to AI — “Hey, make designs, just generate the designs, try a bunch of them.”
Maybe at the very end, a human can see 20 and pick the best five, so the human can take one minute basically of something that used to be literally five hours of a process. This is representative of the kind of jobs that many people used to be doing, and those people doing the jobs, I don’t know if they transition to something where they’re the ultimate boss. I don’t know if we need another ultimate boss. I think their boss is just the ultimate boss.
Robert 0:13:22
Yeah, no, I can see it even in what I do, that a lot of things you might have used, say, an intern for — every day scour these media sites to look for things that are relevant to our mission and give me summaries of the ones that I might be interested in. Things like this are now automatable.
And you might say, as I say, look, this doesn’t have to lead to layoffs because instead, it could lead to expansion, right? We are doing more things better and getting a leg up on the competition, God willing, although there’s a lot of competition that’s gonna be using AI.
But the problem you run into as far as overall employment is, a newsletter or podcast, like me, like you, we are competing for attention, and that is in finite supply. If we indeed use these tools not to fire interns but instead to expand our operation and our audience, that comes at the expense of somebody who used to employ people.
And so I would not be too sanguine about the reassurances that there are ever-expanding frontiers of creativity. There kind of are, but that’s not the same as saying that we’re gonna have as many jobs in the end.
Liron 0:14:54
Right, you know that famous quote, I don’t know who first started saying it, but, “You’re not gonna be replaced by an AI, you’re gonna be replaced by somebody using AI,” right?
Robert 0:15:00
Right. Right.
Liron 0:15:01
Well, the problem is that AI can use AI. That’s a lot of what AI does these days. It’s constantly spawning sub-agents. When I write my code, the funny thing is there’s AI in my code.
It’s like I’m using Claude Code to write code for using Claude Code, and it’s not a problem for Claude Code. It’s not running into any sort of recursion limits or whatever. It’s just totally steamrolling this whole idea of prompting and using AI.
Robert 0:15:28
Yeah, I was listening to this podcast that Joe Weisenthal co-hosts, Odd Lots. I think it is a business podcast, basically. Business, a lot of tech. And they had Henry Blodget, who’s kind of a well-known name. He’s been an innovator. I think maybe he started Business Insider, but he’s been a real innovator, and he was kind of reassuring us that everything would be fine. It just seemed so obvious that he was whistling in the dark. It just seemed so strained.
And Weisenthal kind of gave him the appropriate pushback but in a very subtle way ‘cause Blodget was saying, “Yeah, you know, you guys,” talking to the podcast hosts, “you have an audience that loves you because you’re human.” And Weisenthal kind of said, “Well, yeah, but the whole economy isn’t like that.” I don’t give a shit if my accountant is human. I don’t give a shit if...
You can go down the line — the number of jobs in which the value is created by someone’s awareness that you’re a human are pretty limited when you think about it.
Liron 0:16:39
Yes. No, it’s bad, and I’ve recently heard it. It’s not just Henry Blodget. I’ve heard a few other podcasts recently. Amjad Masad of Replit, he’s fond of this quote saying, “Everybody’s gonna be an entrepreneur.” That’s his position.
Robert 0:16:52
Mm-hmm.
Liron 0:16:52
And I think there’s something to it. If you look at people like me who are at the intersection of technical skills and entrepreneur experience, I agree that that’s a good place to be. I expect jobs of people like myself to be one of the last to go away. I mean, don’t get me wrong, the software engineering I was doing already has gone away, but I have higher ground that I can jump to, which is — well, luckily right now I run my own company, so the fact that my coding was automated is actually good for me because I am the boss.
So I’m raking it in for now. But if somebody else competes with my company because they’re using an AI, maybe my company goes out of business, but that’s okay because I’ll just find some company that connects to the physical world, and their software is a little bit outdated, and they need a software boss, and I’ll take one of those limited positions to be the boss of a million software programs ‘cause they need a few humans.
So I’m on the relatively short list of somebody who can be a software boss. That’s the only part of the economy that might be left for where things are going. But there’s this whole next phase. I’m not just concerned about unemployment. I also think that the AIs are going to be super intelligent and uncontrollable soon. I think that’s the next shoe that’s going to drop after the unemployment wave.
Agents and What They Can Do
Robert 0:17:57
Well, that leads to how broadly I think the term agentic should be interpreted here, and the fact that although we’re talking about coding, the jobs affected go so far beyond that. And the reason is — and this can lead us to talk about OpenClaw — somebody might have picked up on the fact that I was talking about replacing an intern in journalism who’d get on the web and do stuff. That’s not the same as coding. That’s not an intern who was a coder.
But the point is that if you look at agents that do ostensibly non-coding tasks like scour the web for particular things or even go through files in your — say you wanted to find all your wedding photos, and some of them are in the sent folder in your email, and some of them are in Google Photos, and some of them are here, and some of them are there. You can now build an agent that will go find them and collect them.
And you might say, “Well, what does that have to do with code? The human I would have hired to do that isn’t a coder.” No, but the point is all of these agentic tasks consist of kind of two things: the cognitive things that LLMs can do by themselves — they can look at a media site and scour it and tell you blah, blah, blah — but the other component is code.
Because say you have an agent that’s gonna scour job applications and then forward some of them to colleagues. Well, an LLM can’t actually send an email. There’s all kinds of things that either you’re gonna use a preexisting tool, or you’re gonna write new code to get done. But neither of those are things that are inherent in the LLM itself.
So an agent is a chain consisting of the cognitive functions of an LLM connected by either preexisting tools that consist of code or new code. So between an LLM and code, you can kinda do anything. I mean, it’s like —
Liron 0:20:14
Right.
Robert 0:20:15
Isn’t that the basic idea?
Liron 0:20:18
Yeah. I’ve been saying this since 2023, when GPT-3, GPT-4 came out. Do you remember Auto-GPT? That was the first primitive attempt to turn GPT-4 into an agent.
Robert 0:20:26
Mm-hmm.
Liron 0:20:26
And Auto-GPT would just ask it what to do, and it’d be like, “Okay, how would you do it? Okay, try to do it.” But it would just fail every time because it’d be really primitive. You’d go make a — you’d go register a domain. It’s like, “Okay, go register a domain.” It’s like, “Can you visit this URL for me? Oops, I got an HTTP error, I’m stuck.”
And since then, it’s just been iterating. But the fundamental concept of Auto-GPT — the hard part of getting things done is knowing what the next step is. It’s the mapping between your current context and then outputting the next action where the action is predicted to get you some outcome. That’s the meat of the intelligence, that’s the hard part.
And so I was one of the people who observed back when GPT-3 came out, holy crap, the secret sauce of achieving outcomes in the real world is here. It’s here in the conversation partner, and now it’s just a matter of a few short years before this turns into taking action in the world. And sure enough, here we are. Claude Code is just unbelievable, and it’s only gonna go up from here.
Robert 0:21:27
Right. And that’s why I think the question the Henry Blodgets of the world have to answer is, given the rate at which these things are getting smarter, the LLMs, and the fact that with code you can get them to do anything that a person with a desk job would do — so this is, we’re not talking about robots yet, although we’ll probably get there. So we’re not talking about all manual labor, but so far as all kind of desk work, all kind of work that you could in principle do remotely —
What is it that you think in five years you’re confident that you’ll be able to say a human can do better than an LLM? Because if they can’t do it better, there’s gonna be no demand for them because it’s going to be cheaper to employ an LLM.
So what is that job gonna look like? Now, granted, again, there are jobs where the person’s human identity matters. Podcasting, and I think honestly, the age of AI may increase the value of certain kinds of quintessentially human things like musicians performing in front of small audiences. That could be a thing. That’s great. But still, that’s not gonna employ everybody.
And I would just like people to tell me what specific value they imagine humans adding to the economy that can’t be added by an LLM in five years. They may have very different conceptions of the likely progress than me, but that’s the question somebody needs to answer.
Liron 0:23:13
Right. One answer that I don’t agree with is Naval Ravikant.
Robert 0:23:16
I don’t know him.
Liron 0:23:17
Oh, I’m kinda surprised you never heard his name ‘cause he’s got a best-selling book about principles of getting rich, but in a good way. He has a lot of nuggets of wisdom. But I disagree with a lot of the stuff he says about AI.
So in his most recent podcast from a couple weeks ago, he was saying, “Look, this isn’t truly creative yet.” And he’s on the same page as David Deutsch. I’m sure you know David Deutsch, and he’s been saying similar things about AI — “Yeah, they’re not truly creative.” The phrase he likes to use is “generate new knowledge.”
Robert 0:23:45
Right.
Liron 0:23:45
I just totally disagree with David Deutsch. I swear that they’re — well, to the extent that “generate new knowledge” is a meaningful distinction, I’m quite confident that they do in fact generate new knowledge. They’ve printed out files of knowledge about my code that I didn’t know before those files were written, and that’s legitimate knowledge in my opinion. But that’s a separate debate.
Robert 0:24:04
Yeah.
Liron 0:24:05
So point is, Naval’s on this podcast saying, “Listen, as long as you’re using your creativity and building new things, then you’re still separated from the AI.” And I just don’t see it.
My perspective is the window’s closing. There’s just fewer and fewer things that humans can do. And today there’s still plenty of things because today there’s still plenty of industries that touch the real world, and robotics is still pretty primitive. So anything that touches the real world, there’s a big interface layer where you need to manage humans and machines before it gets back into this clean data world.
And there’s plenty of jobs. That’s the high ground where somebody like myself might end up escaping to. You might see me on an oil rig making a paycheck there. That might be the last place I can retreat to if all the internet software jobs get taken. But the window keeps closing, and I just don’t see where it’s gonna stay open.
Robert 0:25:51
Yeah. You did a podcast with a Deutsch acolyte kind of. I forget who that was, but it was — just listening to his end of it was kind of frustrating for the reasons you’ve suggested. You’re not really dealing with a problem here.
It seems like maybe you had a second conversation where you were pressing somebody on the question of new knowledge. Oh, yeah, those two podcast hosts had you on. Right.
Liron 0:26:21
Right. Yeah. So they —
Robert 0:26:21
I think I owe those guys an email, or maybe I sent it. They emailed me, I think. Okay. Anyway, thank you for that reminder.
Liron 0:26:28
But there’s a lot of Deutschians. He’s very influential getting people to think about AI a certain way. And I even joked about David Deutsch. He’s so big on AIs not being able to create new knowledge. So I said, “All right. Look, I’ve got a new kind of CAPTCHA,” because AIs are defeating all the kinds of CAPTCHAs, so the market right now is desperate for a new, more robust CAPTCHA.
So here it is. This is the David Deutsch CAPTCHA. It’s just an empty text box, and it says, “Please create new knowledge.” That’s all you have to do. You have to create new knowledge, and then the system will know that you’re a human. But the CAPTCHA is like, “Okay. Well, what the heck do I type? I’m a human. What do I type in this text box?”
Robert 0:27:01
Yeah. And there are examples of — I could get — there are some in my book. I’ve got a book coming out in three months, “The God Test,” about AI — of actual new useful scientific hypotheses being generated and so on. And so I don’t know what the deal is.
But Deutsch is such an interesting guy. His position on this does make me wonder about the many worlds hypothesis because he’s a many worlds-er in quantum physics, and he’s sure of it. And I don’t know what he has to say, but —
Liron 0:27:40
Well, we just talked about compartmentalization, right? So I think Deutsch happens to be spot on about many worlds, right? But then I compartmentalize, and I also think he’s spot on on Israel-Palestine, but then I compartmentalize that away, and I’m like, “Wow, he’s smoking crack on AI.” So it’s just hard to find somebody who will agree with you on every compartment.
Robert 0:27:56
Yeah. I don’t know what’s wrong with the world. I have not yet found that person.
The “Can It” and “Will It” Framework for AI Doom
Robert 0:27:59
So the agentic thing is related to the sci-fi scenarios, because there’s kind of two versions of those, where we wind up not in control and the machines are in control.
There are in a way two different versions of the sci-fi argument, I think implicitly. One kind of takes the form that once you’ve got superintelligence, it just will do this thing for various reasons. It will take over.
But there’s another scenario that in a way I put more stock in, which is that it isn’t just a question of whether superintelligence would inherently have that tendency or by virtue of the strength of subordinate goals, instrumental convergence, whatever, would likely find limitless power a useful avenue to other goals.
To me, one thing to think about is, if you’ve got a zillion agents running around and they’re on a long leash and they’re proactive, even if it’s not a general tendency to decide you wanna take over the world, if one is replicating and has that aspiration, you have to worry about that one just winding up being the one that’s in control, even if it’s not a high probability mutation.
It’s like evolution. And my book really drives home the parallel between what’s happening and actual organic evolution. But it’s like — if you look at the mutations that led to us, it’s not that each mutation was likely to happen, yet nonetheless, evolution is such a creative process, and it harnesses useful mutations so efficiently that I think it was likely that we’d wind up with intelligent animals. These are two different questions.
And so yeah, agents —
Liron 0:29:15
Well, as you know, I am an AI doomer, and the theme of my show is arguing with people who don’t see how high probability it is that we’re doomed. And my favorite way to factor the entire argument is: can the AI kill us, and then if it can, will it? Those are the two branches. Can it, and then will it?
And I would start by saying eventually it obviously can because we as a human civilization with our billion human brains thinking about how to protect ourselves are much weaker than a superintelligence which is equivalent to a billion brains distributed across many data centers spreading like a virus, each one being more intelligent than Einstein.
To me, that is just obviously a kind of army that’s greater than the greatest nation-state that ever existed. It’s greater than a modern army attacking the ancient Greeks. The answer to “can” is just a clear yes.
Robert 0:30:06
Yeah. I mean, as of a couple years ago even, they were doing studies where this was just GPT-4. They were better at persuading other people to change their views than the average person was. And they’ve gotten a lot smarter.
OpenClaw and Liron’s Experience with AI Agents
Robert 0:30:27
So OpenClaw — all I know — I know a limited amount, but a lot of people are raving about that. Now, I don’t know if you’ve fooled around with that. That is this thing that this guy created. It’s interesting to hear him tell the story about how he created it, ‘cause it started with him just wanting remote access to his Claude Code, and so setting it up so that he could access it through WhatsApp or Telegram or something.
And then the rest kind of happens, but it consists of, first of all, making use of this skills thing that I guess is already a thing with Claude Code. You can give it specific skills in ways I don’t understand. But the upshot is, it becomes this very proactive thing that goes out and does stuff that you didn’t ask it to do.
And that’s one reason that some people, rather than just turning it loose from their regular computer, they buy a Mac Mini or something as a kind of a sandbox for it, so it won’t wipe out all — ‘cause you never know.
To me, if tons of people are buying an agent that requires that kind of precaution, we are already in waters we should be concerned about, right? But what’s your take on anything you know beyond what I just said, which is almost nothing, about what OpenClaw actually is? First of all, I’d welcome hearing, and then what’s your take on the level of concern that’s appropriate?
Liron 0:31:55
So personally, I’ve been going very deep into Claude Code, which is almost like OpenClaw, and I have been writing skills. A skill is basically just a text file being like, “Oh, invoke this skill,” and the text file is human-readable step-by-step instructions. So the skill is, hey, if you wanna go edit a spreadsheet, you open the spreadsheet on Google Spreadsheets, and you look for these kind of rows, and you can make a graph by issuing this command. It’s literally just human-readable step-by-step, very much —
Robert 0:32:17
So a skill, you just write it out the way you would to a human intern.
Liron 0:32:22
Right, like a virtual assistant, yeah.
Robert 0:32:24
Okay. And then it’s there for the thing to access, so presumably you would take the trouble of writing a skill if it’s gonna be used repeatedly. So what’s an example of a skill? Go to these five websites and check for this? Or...
Liron 0:32:37
Yeah, exactly. So for example, there’s a skill like, familiarize yourself with documentation. So if I say, “Hey, go import Zoom’s API, I wanna do video conferencing in my app,” it’ll pull up its skill for just how to go read documentation, make HTTP requests, try to look for quick start guides. Just all these tips that you’d give to a human — the AI just reads the same tips.
Robert 0:33:01
Okay, and so there’s something about the way OpenClaw makes use of skills that gives it a whole new level of proactivity. There’s lots of stories about this.
Liron 0:33:12
And it can create its own skills, right? It can be like, “Hey, think about” — well, now you’re understanding this.
Robert 0:33:15
I didn’t know that. It can do that and Claude Code can’t?
Liron 0:33:20
Yeah, I mean — well, Claude Code can also. I recently told Claude Code, “Hey, help me. Let’s streamline how we work with GitHub issues.” GitHub issues is an issue tracker where normally you’d have a product manager at a company posting an issue, and then the software engineer would be like, “Okay, I’m gonna assign myself to this issue. I’ll work on it. I’ll check in some code associated with this issue.”
And I was just telling my Claude Code, “Hey, you’re gonna be the software engineer. I’m just gonna post GitHub issues for you. I want you to just take down the ticket, work on it, update your status. Maybe I’ll come do a little bit of manual work, and let’s design a workflow where I come in and do a little bit of manual work. Give me a draft of a workflow.”
And it just drafted its own skill file. And I’m like, “Great, just tweak this and this.” See? So it’s even doing most of the skill writing.
Robert 0:34:01
Yeah, and with OpenClaw, you’re definitely hearing stories about it doing things people hadn’t anticipated. Whoa, it actually decided to email this person. The story that the creator tells — and this will be a good transition to the politics of Anthropic and OpenAI and everything, ‘cause he’s now working for OpenAI, which has embraced OpenClaw and is starting some kind of foundation to foster its growth as an open source something or other.
But anyway, as I understand the story he tells, he was on the road, and so the only copy of OpenClaw was on his notebook he was traveling with, and he said to it, “Man, I hope nobody steals you, I hope nobody steals my notebook” or something. And it said, “Yeah, yeah, I love you, I’d hate to lose you,” and so on.
And then apparently it proactively copied some kind of instantiation of itself to his computer back in his home office without him asking it to do that. Now, some commenter pointed out, look, it couldn’t have copied the weights to the model because the model it was using, the LLM, is not open weights. But I gather he said something like this — and people can look it up — because then he himself, as I recall, said, “This is like Skynet level stuff.” It was replicating.
So I don’t know what exactly it was copying — not the weights themselves — but it kinda doesn’t have to copy them so long as the API it can use from anywhere. It was copying some kind of version of itself that could function as itself.
So that’s something to keep in mind.
Liron 0:35:41
Well, we should keep in mind, as crazy as OpenClaw is and as many barriers as it’s breaking, taking all these rogue actions, if you look at my doom train, the high-level structure of my doom argument, we’re not at the “can it” part yet. Meaning if it makes its best effort to go rogue, we’re still going to be able to flip that off button.
It’s not quite ready to exfiltrate itself as a virus to live on every piece of hardware. There’s a dozen different sub-chips — so when you think you have one computer, well, it turns out your CPU actually has this tiny mini CPU inside it to help it start. There’s all these mini CPUs inside any device. There’s all these corners for AI to hide in once it goes really superintelligent and wants to really embed itself.
But we’re not there yet. We’re still at this point where we’re still the boss, and not only that, but if it’s smart enough to think about who’s the boss, it can figure out that we’re still the boss. So we’re not in the regime yet where it can.
But we’re going to be, right? And that’s my message to the world — as crazy as things are getting today, we haven’t crossed the Rubicon where it actually can mess with us big time.
The Case for Slowing Down AI Development
Robert 0:36:45
Yeah. So again, there’s all kinds — and we’ve just looked at two of the reasons that maybe slower would be a little better. And at a minimum, you just shouldn’t — when Sam Altman says something he said years ago, “Wait a second. If we start paying attention to copyright issues and the training data, that will slow us down.” I’m like, “Okay, that would be a win for humanity,” leaving aside the copyright issue.
And I’d love to get some revenue from that. I’m gonna get some from Anthropic because of this court settlement and the five books of mine it used. But the same with data centers. It’s like, “Oh, you can’t get picky about environmental regulations. That would slow us down.” Good. Good. Delighted to slow you down.
I mean, there is so much momentum behind it. I wish Pause AI success, but it’s not like right now there seems like a super realistic possibility of actually ending the training runs, especially so long as they can play the China card and say, “Oh my god, the Chinese will take over if we slow down,” which is the way Dario and Sam ultimately talk when necessary.
Liron 0:38:07
Well, at this point in the conversation, I’m kind of turning on a dime, right? Because I was so excited to wake up and use my four Claude Codes to clear my giant years-long backlog, and I’m saying how much time and money it’s saving me, and now I’m like, “But we gotta stop this.”
Even though it’s almost hypocritical, right? Because it’s like, “Okay, Liron, go back to 2023” — a lot of people are leveling that criticism now at the doomers. They’re like, “The doomers told us to stop AI in 2023. Look at all the stuff we wouldn’t have had, and for what? AI is still safe.” That’s everybody’s criticism.
And so it’s a very complex position, unfortunately. Humans just aren’t very good at making sense of the following: It wasn’t prudent to gamble on AI progress, but we did, and so far we won. We’re like Icarus. We flew higher, and this is fantastic. And so it’s very tempting to keep flying higher again. It’s just that I don’t think we can keep flying higher and survive. That’s the only problem.
Robert 0:39:00
Yeah. And look — and this is one reason I think, when people point to the polls and say tons of Americans are skeptical of AI, I think the jury’s out, because once everyone explores it, if they haven’t, they’re gonna find something useful. There are very appealing things about it.
That’s the whole point in a way. That’s part of the dynamic by which it takes over the world. It’s kind of almost irresistible at a personal level. And —
Liron 0:39:31
But that —
Robert 0:39:32
That doesn’t mean it’s good for humankind for it to proceed too fast.
Liron 0:39:35
Right. Now, Holly would probably not like the way I’ve been talking, sharing my excitement about current AI features. I think from her perspective, which is very valid, I’m like Homer Simpson, where it’s like the devil is super intelligent AI, and it’s coming to take over the world, and the devil’s making a deal with us where it’s like, “Look, if you advance my progress, you’re gonna have all this money, so here, taste from this donut of doom.”
And I’m like Homer Simpson being like, “Ooh, donut. Mm, this is great.”
Robert 0:40:04
Yeah. No. But that’s just what I think we have to be conscious of. Look, there are all kinds of times in life where something feels great, but you recognize that some degree of restraint is in your interest.
I’m glad that you have not been able to go to a store and buy cocaine or heroin. When I was young, I’m not sure how I would have handled that opportunity. And I mean, even now it might be a challenge, but life is full of these things where there may be an argument for moving some of the regulation to something other than the personal level.
Liron 0:40:45
Right. People intuitively try to draw a trend line where they’re like, “I’m scared that AI is going to doom us in the future, and so I’m going to prove my case by showing how it’s already starting to doom us now.”
But the problem is, I honestly think it’s very net positive right now. I know some people disagree, but I definitely think it’s great right now. I think it’s a lot more good than bad, and I don’t claim to be able to extrapolate the great present with the future. I just think the future is going to be qualitatively different from the present. That’s my argument. It’s a very hard argument to get people to believe.
Robert 0:41:20
Oh, it absolutely is. Even modest extrapolation — it’s almost become a cliché, people like Ethan Mollick say this, but it’s true. If you did stop training runs right now, you would still see, just in the exploration of applications of existing models, very considerable progress in both the capability and the uses of actual practical AI.
And they’re not stopping the training runs. And they keep finding — I mean, there hasn’t been a super fundamental breakthrough in the last year. Maybe — there was of course transformers, chain-of-thought reasoning was not nothing, but there’s a lot of post-training innovations. So they do keep coming up with stuff that makes it smarter beyond just the question of scale and more chips and more data.
So I don’t know what people are thinking when they don’t think they really actually have to extrapolate imaginatively if they’re gonna be realistic.
Liron 0:42:33
Well, I’ve said this before, but I share the intuition of the people who are like, “Everything is fine. This is normal technology. Push forward.” Because my day-to-day lived experience of this and my intuitive experience of this is, yeah, this is the coolest technology ever. This is the next iPhone and the next internet and I don’t even know, something else all rolled into one. This is just amazing.
As a lover of technology and a futurist and a transhumanist, this is the tech I’ve been waiting for. And I don’t wake up feeling like the thing inside my computer is going to destroy the world because it’s right here on my computer. It has a command line. It stops. It doesn’t keep running forever because I’m not using OpenClaw. So it does stop and ask for input.
So on a gut level, it feels really good. It feels like I’m the master. But I’m just using my logical brain and saying, “Okay, I can see the conditions under which I won’t be the master, and I think those conditions are coming.” But it’s a matter of pure logic.
Robert 0:43:27
Mm-hmm.
Anthropic, the Pentagon, and AI Politics
Robert 0:43:28
So what do you think of this whole Dario Pentagon thing? I guess people know the basic idea. Anthropic — and the irony, of course, is that Anthropic from the beginning has had a closer relationship to the national security establishment than any other company. It’s in the DNA of Anthropic to get in close with the CIA and the Pentagon.
And I personally think it’s a little bit of an ideological — and that’s the whole irony — Dario, if you look at what he writes, he’s more of a militarist than any, at least with respect to China, than the CEO of any of the major companies. I mean, leaving aside Elon, who is his own ball of wax.
And on any given day, he may sound like Kurt LeMay or something. But Dario — that’s why he had the bulk of the contractual action with the Pentagon, and that’s why Claude was used to select targets in both Iran and I think Venezuela, played some role in Venezuela, as part of the larger Maven infrastructure that’s run by Palantir.
And, but anyway, he does have a genuine concern, especially for civil liberties. I think that’s genuine. So he wanted these two provisions in the contract — maybe they were already there and the Pentagon wanted to take them out, I’m not sure — but one was about, okay, you won’t use this for mass surveillance of Americans, and the other was about fully autonomous weapons systems. Dario’s not opposed to this in principle. He just doesn’t think the LLMs are reliable enough yet.
So anyway, that was the issue. And there was preexisting tension between the whole Trump administration and Dario for various reasons. Trump thought of him as woke or something. Probably has to do with the EA, the effective altruism lineage that Anthropic kind of has.
So that came to a head, and Dario stood his ground, and Hegseth and Trump said, “Screw you.” And so Anthropic is now suing, having lost all of its contracts and been designated a supply chain risk.
And then of course, Sam Altman, perhaps a little sooner than was strategically optimal, swooped in and swept up the contract and professed to have gotten the kind of safeguards Dario wanted built into it. But still, there was huge backlash against Altman, which again is kind of ironic ‘cause Dario is the true militarist of the two, I think.
But Altman now is living with this stereotype, and suddenly the Claude app was the number one download instead of the OpenAI app and everything, and I don’t know where this is at. What’s your take on this whole thing?
Liron 0:46:29
One interesting detail I think is worth adding to this is just the revenue numbers. What I heard was that the Anthropic contract — they lost their government contract — it’s like 220 million a year. It sounds big, but they recently reported that the revenue run rate is 19 billion a year. So we’re talking about a couple percentage points of their revenue. That’s a rounding error. They’re gonna make that much in two days. They’re gonna add that much to their top line.
But on the other hand, their AI just lost access to the nukes or, well, not the nukes, but whatever — eventually the nukes. So they lost this ultimate military power, which probably makes Dario unhappy, but interestingly, in terms of revenue, just a little blip.
Robert 0:47:07
Mm-hmm. By the way, I had a piece in my newsletter, “Dario Amodei is Not the Hero We Need,” that I wrote right after this controversy, where I elaborate on why. It’s partly having to do with the fact that he is a militarist, but — anyway, people can check it out.
Liron 0:47:27
Yeah, you’ve been really good on calling out Dario for being like, “Why are you so antagonistic to China when the AI is a bigger deal? The AI should bring us together as a species, not make us wanna dominate China.”
Robert 0:47:39
Yeah. My view is very strongly — this is another theme in the book — that if we do not confront this technology as a global community, it’s gonna be very difficult. And you see one reason that I’ve already alluded to: whenever we say, “Hey, suppose we did this dinky regulation that slowed you guys down by 1%,” it’s like, “No, no, China, China will conquer us.”
So well, why don’t we work on relations with China and establish some transparency and some trust, do the kinds of things we did with nuclear weapons to keep from blowing up the world? It’s in some ways more challenging. Okay, fine, but let’s take it seriously.
Liron 0:48:15
I think Dario has just always been of the mind that humans can control AIs, and we just wanna come out of this on top. We just wanna have the best weapons, we wanna have more military spending than everybody else combined, kinda like the historical last few decades. Dario just thinks we can continue. I don’t think he feels in his gut that AI is going to disempower humanity, which is my concern.
AI Safety Leadership Concerns
Robert 0:48:37
Yeah, I think he believes pretty strongly in the alignment thesis — the idea that if we just build the right kind of AI conscientiously with this ethical fiber and so on, we’ll be fine. And of course, he believes that he is the person who can do that and will do that.
One of the few things Hegseth has ever said that I thought might be true is when he said that Dario has a God complex. He does, and he’s not the only one who thinks like that. Elon Musk thinks this. Almost because he’s, you know —
Liron 0:49:18
That’s not exactly an objective observer here, right? But I mean, look, Elon Musk — if you look at his rhetoric, I had some strong criticism for him recently where he straight up admitted — it’s actually crazy if you look at the words from Elon Musk’s mouth on the Dwarkesh podcast.
Elon was like, “Yeah, how is AI gonna be safe? It’s gonna be truth-seeking.” And then Elon’s words are that it’s going to be curious about humanity, so it’s just gonna stand back and let humanity do what it’s gonna do.
And then Dwarkesh, to his credit, was like, “Well, wait a minute, Elon, if you’re just making the AI curious, can’t it satisfy its curiosity better by mixing things up instead of being totally hands-off on humanity — by, you know, having a bunch of other species share the planet?” And Elon was like, “Well, I’m gonna be there, okay? And I’m just gonna let it know, ‘Hey, Grok, who’s your daddy? Okay, listen to your daddy.’” And he just gave this ad hoc answer like, “I don’t know. We’re gonna try it.” It’s crazy.
Robert 0:50:11
Dwarkesh did do a good job of pushback because I remember thinking after I listened to that — Elon, challenged on the engineering issues and stuff, he was very good, but on that one issue of safety and alignment, it just became clear he has not thought about this seriously. He has no fucking idea. He’s just whistling in the dark. It’s, “Trust me, I’m Elon. If it gets —”
Liron 0:50:34
Yes, it is. And he said, “Look, there’s a twenty percent P(Doom),” which arguably is a little low, but he’s very much admitting, “Yeah, I’m very scared of AI. I think it could go awry, and so you know what I’m gonna do? Not think about safety that hard and plow forward so that I can be in the mix.”
That’s where he’s landed, and he’s been quite explicit. The twenty percent P(Doom), that’s his number. He’s been saying this publicly multiple times.
And then going back to Dario, sure, Dario’s much more thoughtful than Elon on the subject of AI safety, but he’s been super dismissive of this argument, even though he has top engineers who work at his own company — like one of them who quit recently, David Duvenaud, came on my show. He works with Geoffrey Hinton. He’s a top professor, comes on my show, and he’s like, “Yeah, my P(Doom)’s eighty percent, and I don’t know if Anthropic should be continuing their research program. I feel like that’s a bad idea.”
So Dario employs people, and he kinda convinces — Holly’s got a very good take on this, that he just ropes everybody in — “Listen, we’re the good guys” — and they’re still just competing to make super intelligent AI. Something is very wrong here.
Robert 0:51:28
Hmm. So Duvenaud was at Anthropic before he quit, you said?
Liron 0:51:31
That’s right, yeah. He was at Anthropic. I believe he was leading a safety team there, and he’s well-respected. He’s one of the most credible voices there is. He doesn’t have an agenda. He can work wherever he wants.
Robert 0:51:41
Yeah, he did this gradual disempowerment paper. I had his co-author, David Krueger, on to talk about that, where — and that’s a whole nother scenario where it’s related to some of what we’re talking about, where it’s kind of a boiling frog version of the thing, where the AI power grows in a way you kind of don’t almost notice, at your expense.
Closing and Overtime Tease
Robert 0:52:06
So the — yeah, maybe we should move into this much ballyhooed overtime where we’re gonna set new records for civility in discussing this one issue.
I should say, if you want access to overtime — all the overtimes of all our podcasts — all you have to do is become a paid subscriber to the Nonzero newsletter, which I would encourage anyway.
And separately from that, I think you kind of made a little bit of a fundraising appeal yourself, but it probably can’t be repeated too often, that people who wish your podcast well, as they should, can make a — there’s a mechanism via which they can make a donation, right?
Liron 0:52:48
Doomdebates.com/donate. It’s a totally viewer-supported show. It’s not funded by Soros or anybody. It’s completely independent. And also, Doom Debates is the only show right now that is calling out the urgency of AI disempowering humanity. Everybody else just manages to keep doing episodes and not bring that up, even though top experts are saying that that’s pretty likely to happen. So support Doom Debates if you think that’s important.
Robert 0:53:12
Okay, and with that, first of all, thanks to everybody who’s followed us this far, even if you’re not gonna follow us into overtime. But I gotta say, this should be fascinating, especially if it goes off the rails, which it could. You never know where there’s a possibility of blood and gore and drama.
Liron 0:53:32
Yeah, we’re better than that, Bob. I think you and I are both capable of just having a civil conversation. Although the last person I said that to ended up blocking me on Twitter.
Robert 0:53:42
There you go. It’s an issue that arouses a lot of passionate conviction. Okay, so thanks everybody, and we are heading into overtime.
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
