Don't listen to Marc Andreessen about AI

The case for maintaining basic standards of discourse

Marc Andreessen’s recent essay, Why AI Will Save the World, doesn’t meet basic standards of discourse. Claiming AI will continue to be safe and net positive is his right, but the way he’s gone about making that claim has been undermining the quality of conversation.

I’m not writing this because I’m an AI doomer and I disagree with Marc’s views (though I am and I do). I’m writing because his logically-flimsy arguments, his shallow and contradictory mental models, his gratuitous character assassinations, and his question-dodging, given his status as a public intellectual and role model that many aspire to imitate, have reached the point where someone has to step outside the object-level AI risk debate and blow the referee’s whistle on Marc’s poor “sportsmanship” in this discourse.

I don’t hold any credentials as a judge of logically-valid argumentation (besides Asperger’s). But neither does anyone else, which is why our public discourse is dysfunctional. Posts and comments from randoms like you and me are one of the few ways we can referee some of the more egregious violations.

Bulverism

Marc indulges in constant Bulverism. He spends much time labeling and psychoanalyzing the people who disagree with him, instead of focusing on the substance of why he thinks their object-level claims are wrong and his are right.

In his essay, he accuses a large fraction of AI doomers of being “bootleggers”, which he explains means “self-interested opportunists who stand to financially profit” from defending the claim that AI existential risk is a serious worry. He even makes this blanket statement:

If you are paid a salary or receive grants to foster AI panic…you are probably a Bootlegger.

Has it occurred to Marc that there may be a population of folks who understand the importance of working on AI existential risk mitigation, but aren’t independently wealthy? And that these particular types of folks often have technical skills that require them to turn down higher-paying job offers?

I’m not here to provide an ad-hominem defense to counter Marc’s ad-hominem attacks. I’m just pointing out how Marc has decided to Bulveristicly assert, without evidence, that most people working in the field of AI safety are self-interested opportunists.

He’s so enamored with psychoanalyzing the population of AI-existential-risk-concerned folks, that he befuddles Sam Harris with this nonsensical theory about how AI existential risk is a stand-in for war against Nazis:

Maybe Marc’s Bulverist claim about AI and Nazis is an accurate description of somebody somewhere. I don’t know or care. Speaking for myself, I couldn’t care less about Nazism or Communism when I worry about the uncontrollable consequences of a computer system whose proficiency with the task of mapping arbitrary outcomes in the physical universe causally backward to strategic actions exceeds that of the combined human race.

Marc also claims that AI doomers are a “tiny minority” who “bully” people like him:

Since he knew about the recent Statement on AI Risk with hundreds of prominent signatories from across business and academia, that was a disingenuous framing on his part, or at best just inaccurate and pathetic.

He’s probably also seen the thousands of prominent signatories on the FLI 6-Month AI Pause Letter, and various survey results like 42% of CEOs say AI could destroy humanity in five to ten years and over half of AI experts give at least 5% odds of AI causing human extinction. It shouldn’t be hard for him to just stop inaccurately dissing his opponents and focus on the logic of his own position.

“Category Error”

Let’s start with a passage from the essay where Marc is 100% objectively wrong, as a matter of pure logic.

Marc writes:

My view is that the idea that AI will decide to literally kill humanity is a profound category error. AI is not a living being that has been primed by billions of years of evolution to participate in the battle for the survival of the fittest, as animals are, and as we are. It is math – code – computers, built by people, owned by people, used by people, controlled by people. The idea that it will at some point develop a mind of its own and decide that it has motivations that lead it to try to kill us is a superstitious handwave.

The flaw in Marc’s claim was noticed immediately by various readers:

The absolute worst argument against AI risk (or any AGI related things) is that “it’s just math”. Like what exactly does he think is happening in your brain that can’t be described in terms of math?
—@ravisparikh

Saying that AI can’t be dangerous because it’s just math and code is like saying tigers can’t hurt you because they’re just a clump of biochemical reactions. Of course it’s just math!
—Dwarkesh Patel

The idea that AI will decide to literally kill humanity is not a category error. Here are examples of what actual category errors look like:

• “The number two is blue.”

• “Which of the buildings at Oxford is the university?”

• “The theory of relativity is under the table.”

As these examples illustrate, a category error is a logically-incoherent proposition that’s neither true nor false. But “AI will decide to kill humanity” is a logically-coherent claim that will eventually turn out to be either true or false. That’s why Marc himself goes on to argue that it’s false.

If you argue that AI won’t kill humanity, while simultaneously arguing that “AI will kill humanity” is a category error, your logic is mistaken. Period. End of story.

Should we give Marc the benefit of the doubt that he didn’t mean to intentionally use the technical term “category error” imprecisely in his essay? No:

1. Marc linked to the Wikipedia explanation of the term in his essay, implying that he understands the technical meaning of “category error” and is really classifying “AI will decide to kill humanity” as such.

2. Marc repeated the same “category error” language in various interviews in the weeks after his essay (e.g. the video clip above).

3. Marc hasn’t retracted the claim or shown any sign of correcting himself on this point in the month after posting his essay.

Let’s pause to reflect on this state of affairs:

Marc Andreessen, a highly respected public intellectual / thought leader, has been stubbornly repeating a claim that’s 100% objectively wrong as a matter of simple logic.

Why dwell on this point?

First, because it’s the canary in the coal mine that shows how vulnerable and exploitable our discourse is. It shows there’s no referee capable of blowing the whistle to elicit a retraction or disqualification from any player who makes a clear-cut logical foul.

Second, because if Marc doesn’t feel any pressure or incentive to correct a simple logical error exposed in his essay, it means we’re likely to find many more basic errors in his work…

Orthogonality Thesis

Marc dismisses the Orthogonality Thesis, the assertion that it’s theoretically possible for a superintelligent AI to have any set of values and goals. He thinks something like an amoral paperclip-maximizing AI can’t plausibly ever exist.

Marc’s “argument” is made of two logically-flimsy pieces:

1. A goal like making paperclips is “stupid” (begging the question)

2. The existence of moral-sounding LLMs “disproves” the thesis

Listen for yourself:

A self-driving car can be pointed at any destination. So can a nuclear missile. If Marc wants to argue that AIs can’t be pointed toward any goal, what kind of argument is “today’s AIs talk as if they value what we value”?

The specific version of AI that Marc played with, gpt-4-0314, isn’t the one & only AI that can exist. (Hell, it’s not even the only version of GPT-4. There’s also the disturbingly amoral gpt-4-early.) Even granting that Marc witnessed a particular AI that cares about human morality, why is he saying it “disproves” an existence claim that ranges over the space of all possible AIs?

In another interview, Marc refers to an AI with destructive values as “having a bug”, pointing out that a superintelligent AI would be able to fix its own bug:

The reason Marc is able to laugh about how an AI would notice that its destructive behavior is a “bug” is that he pre-assumes that the Orthogonality Thesis is false (begs the question). He pre-assumes that there’s no such thing as a superintelligent AI that cares about a destructive goal.

How about if we don’t pre-assume that the Orthogonality Thesis is false? In that case, it’s obvious that an AI accidentally programmed with destructive values wouldn’t see itself as having any bugs that require fixing. On the contrary, it’d likely see attempts to “fix” it as malicious attacks against the integrity of its values.

But regardless of whether the Orthogonality Thesis is true or false (I happen to believe it’s true), I just want to highlight the meta-level, discourse-level point that Marc is substituting a logically-inadequate question-begging argument where a substantive one ought to go—and people are letting him get away with it.

Superficial mental model

What if Marc is just bad at making his points logically-valid, but he still has some kind of wisdom to share with us about what makes AI safe? What if he has some kind of insightful model in his head that he can communicate to us better if we approach it with a different line of questioning, without directly mentioning the Orthogonality Thesis?

Unfortunately, that’s not the case. Marc doesn’t have any coherent mental model of why AI is safe. The proof is this fun clip wherein:

1. Marc says AI is safe because it wasn’t created by the evolutionary process that created humans, so it doesn’t have humans-like aggression and survival instincts.

2. But he also says AI is a finely-detailed mirror image of humanity.

3. Sam Harris deftly catches the contradiction.

4. Marc, caught deer-in-the-headlights by a serious challenge from a non-sycophantic interviewer, turns and runs to the safety of tangential claims.

Besides the emperor-has-no-clothes superficiality of Marc’s mental model, the other thing to take away from this clip is the pathetic inappropriateness of Marc’s behavior in the aftermath of this rare moment when he allowed himself to be publicly challenged. Someone with the integrity and competence to participate in AI x-risk discourse would acknowledge the apparent contradiction highlighted by Sam, and either concede the point or give a more nuanced non-contradictory account of his mental model—rather than try (unsuccessfully) to pivot and save face.

Casual disregard for nightmare scenarios

Marc has repeatedly criticized and mocked Eliezer Yudkowsky’s proposal to have large-scale AI model training regulated by a multi-government agency empowered with the necessary force to deal with cases of non-compliance. In Marc’s mind, the idea of a missile strike enforcing a regulation on a noncompliant data center is a (1) nightmare scenario of authoritarian ideals winning over freedom and technological progress, and (2) logistically impractical.

Fine. It’s understandable why Marc wouldn’t want humanity to compromise on the values of peace and freedom unless it’s an absolutely necessary sacrifice. He can make and defend that claim. But when he turns around and cheerfully dismisses the horror of a rogue unaligned AI scenario, that’s when he crosses the line into low-quality discourse.

When asked about a scenario where unaligned AI has spread itself like a virus across millions of internet-connected devices, he says our ability to EMP-bomb ourselves back to the pre-internet age will keep us safe.

(We’re a long way from “category error”, Dorothy.)

If Marc’s contingency plan is to demolish the internet-enabled operations of hospitals, food supply chains, and a large fraction of total economic activity, in a scenario where we’re fighting a losing battle to put a superintelligent-AI genie back in the bottle, then he’s already granting there’s cause to make some amount of sacrifice in the interest of rogue AI scenario prevention.

Eliezer’s proposal is a bad scenario intentionally designed to be the lesser of two evils when compared to the nightmare of rogue superintelligent AI. Marc’s proposal for how to deal with a rogue superintelligent AI is a worse scenario that he’s talking about as if it’s not that bad. Again, low-quality discourse on Marc’s part.

Wishful thinking isn’t a principle

Marc also endorses a logically-flimsy claim that his partner Martin Casado frames as if it’s some kind of well-known principle:

If you have a tool that’s arbitrarily powerful, that actually doesn’t change equilibrium states. You could have something that does arbitrarily badly, but then you would just create something that does arbitrary good.

Ok guys, what if someone discovers a recipe that lets anyone build a 500-megaton nuclear bomb in their kitchen? A single explosion of such a bomb would permanently destroy human civilization. What’s the logical necessity of “creating something that does arbitrary good” to restore equilibrium after such a discovery? There is none. There’s just a teenager who pushes the nuclear detonator, and then our time on Earth is done.

Sure, we’ve survived a few decades of nuclear brinksmanship, but it’s impossible for us to build comparably effective missile defense systems, and so we remain vulnerable to someone pushing a button in Russia to launch one of many stockpiled ICBMs at us, each of which has a yield literally 1,000x larger than the explosion that killed 50% of the population of Hiroshima.

This kind of low-quality discourse from public intellectuals on high-stakes topics is irresponsible. If a16z’s logically-flimsy “equilibrium principle” isn’t true about AI, but we act like it is, we’ll all die.

Blocking thoughtful critics

Dwarkesh Patel, a popular podcaster who interviewed Marc earlier this year, wrote a point-by-point rebuttal of Marc’s AI safety claims. Dwarkesh’s post was widely praised and shared on Twitter. Marc reacted by blocking Dwarkesh.

Albert Wenger, a VC known for posting thoughtful takes, criticized Marc’s essay for being overly dismissive of AI’s existential threat. Marc reacted by blocking Albert.

Blocking people on Twitter is a valid exercise of personal freedom. But when:

• You’re a public intellectual

• Someone is making substantive counterarguments in good faith

• You can see that their arguments have widespread support from smart people on social media

• You block them

• You don’t engage with their counterarguments

• You do a media tour where you repeat your original arguments without any update or refinement, like how it’s all just a category error

Then your behavior isn’t up to the minimum standards of discourse expected from a public intellectual.

Conclusion

The next time you read a new essay from a public intellectual, instead of gauging how you feel about it overall, ask yourself two separate questions:

1. Is the writer meeting the most basic standards of logical argumentation and discourse etiquette?

2. Do you agree with their conclusions?

If the answer to #1 is no, I hope you’ll call it out as such, as I’m doing in this post, and separate that from your object-level opinion on #2.

For instance, if you agree with Marc’s conclusions that near-term AI existential risk is low, or at least low enough that the benefits of rapid progress will exceed the risk-weighted costs, fine. But don’t let him get away with promulgating low-quality discourse for your side.

There are plenty of public intellectuals whose AI safety views are as diametrically opposed to my own as Marc’s are, yet I don’t take any issue with their quality of argumentation or etiquette of discourse: Robin Hanson, Scott Aaronson, and Sam Altman, to name a few. I also have no desire to question their character or motives. I just disagree with their claims, and I focus on why I don’t find their arguments sufficiently convincing.

That’s how we do it in the post-Enlightenment age. It’s not hard, Marc.

---

If you liked this post, consider:

• Subscribing to my Substack

• Following me on Twitter @liron

• Reading my Medium blog

---

P.S. This behavior isn’t new (see: Web3)

Back in 2011, when Marc’s contributions to the discourse were solid essays like Software Is Eating The World, there was a lot to like about the guy.

But in the last few years, as his firm a16z took in $7.6B of capital to make a disastrous bet on “Web3”, while charging LPs an estimated $1B in management fees for the privilege, he’s been putting out a stream of disingenuous and logically-invalid arguments.

For those who didn’t follow Marc’s Web3 debacle, I’ve kept the receipts:

• His firm’s public presentations are shockingly disingenuous gaslighting

• His crypto right-hand-man’s documented claims are logically-flimsy

• His firm led over $100M of investment into a Web3 “game” that was clearly architected as a Ponzi scheme and played out as such. See my Twitter thread.

And Marc, despite running the world’s largest crypto investment firm, couldn’t name a single Web3 use case when asked by Tyler Cowen last year:

It makes sense that he’s maintaining the same years-long lack of logic and behavioral integrity in his push to rebrand himself as an AI champion. I was honestly hoping he’d hold himself to a higher standard on a topic where the stakes are much higher—letting everyone die vs. just hustling a bunch of LPs and retail investors—but it’s not surprising to be let down.

P.P.S. Balaji Srinivasan, another public intellectual who used to be Marc’s partner at a16z, also fails to meet basic standards of discourse.