I was asked to watch an episode of Steven Bartlett’s Diary of a CEO featuring Mo Gawdat introduced as an Ex-Google officer available here.
My main takeaway is that Mo Gawdat is largely operating out of his lane in this podcast episode. Though they are only briefly touched on, he does seem to have a grounding in economics / sociology. However, the episode is mostly focused on A.I. and alignment of super-intelligent A.I. I found the episode to be a very frustrating watch, the guest frequently evades the question being asked and dresses up his comments with confident sounding tics. He is also guilty of a motte-and-bailey argument on multiple occasions where he makes a bold claim but retreats to a much less bold claim when challenged. At times the host attempts to challenge him, but on other occasions he gives very lowball questions.
Note, he’s introduced as an Ex-Google officer, I certainly took that to mean some kind of technical officer, given the subject matter, but it turns out he was a business officer.
The quotes in this post are sometimes lightly edited to remove ‘uhms’, ‘uhs’ and repetition of words, correction etc. I hope and believe I have faithfully maintained the meaning, substance, and tone of all the quotes.
Credit where it’s due
Maybe good to start with where he is good. Although I don’t think his arguments here are particularly novel, inspired, or profound. He gives a pretty standard description of capitalism:
capitalism has always been all about using labor and capital or debt to create things at a cost that is lower than the price you sell them for […] you bring a team together, they make some shoes, whatever, and you sell the shoes for a dollar more than how much it costs you to make them.
The more unusual observation he then makes is that the workers were also the consumers, this is a point that often gets made with respect to immigration, an immigrant may well “take” a job, but they also create jobs through increased demand.
how does the GDP look if all of those workers no longer have the purchasing power to buy the things that you can create
Unrelated he also has a good heuristic for judging integrity vs virtue signaling:
look at what they’re willing to sacrifice in the near term that’s against their incentives. That for me is the essence of […] understanding if someone has integrity […] they will give up something in the near term for what they believe in over the long term.
It sounds a bit garbled to me, in particular many companies are willing to give up something in the short-term particularly when doing damage limitation. A company might, for example, fire a well-performing CEO because he made racist/sexist remarks. This is usually virtue signalling, the company is signalling (perhaps genuinely) that it doesn’t tolerate racism/sexism whether or not the CEO is performing well. Still, I think there is something important here, the discussion is mostly about Anthropic and their refusal to engage in public survellience for the government. I’m not sure ‘integrity’ is the correct word, but mostly the public didn’t know about this request so their refusal really does provide evidence that they believe in the principle that the government should not be engaged in public surveillance, and at the very least they are not keen to assist that.
Finally, despite, as we’ll see, dodging numerous difficult questions, he doesn’t dodge the one Bartlett poses regarding whether a purely ethical AI can be as engaging as one built “with the reins off.” Gawdat answers:
Uh, no, it won’t. […] Yeah, but that’s the problem humanity needs to solve.
I do not think it is the problem but it is a problem humanity needs to solve. Note that he’s also not pretending he has an answer for this.
Okay, however, note that what he is saying above, is basically pretty standard fare for any AI discussion. Nothing is novel.
Unearned Optimism
So, most of what is discussed is Mo Gawdat’s optimism for humanity’s ability to align a super-intelligent mind to our goals. His main argument is that intelligence breeds benevolence.
The orthogonality thesis, which is approximately the consensus view in the AI alignment community, is that an agent’s goals and its intelligence are independent. Intelligence is the ability to achieve goals, not some kind of force that makes those goals ‘kind’, for some definition of kind (to humans?). According to the thesis, a system can be arbitrarily intelligent and have almost any goal.
Mo Gawdat is essentially arguing against this. Note that he is not a researcher in AI alignment, nor any kind of AI researcher. Imagine, a business officer coming on to discuss how they think that climate change theory is wrong. It’s not that they are necessarily wrong, it’s that you would first of all expect them to acknowledge that they are going against the consensus in a field which is not their expertise, and secondly you would expect them to have brought some extrordinary evidence with them.
He first argues his point from evolutionary biology. He claims that simple organisms are selfish, but more developed ones protect kin. He states:
If AI is super intelligent, it wouldn’t destroy anything at all. As a matter of fact, it would completely […] favor diversity of everything.
A matter of fact? No. Unfortunately Mo consistently dresses up his arguments with confident sounding rhetoric, but that doesn’t make them true. He is making a claim, and then stating that it is a fact. It most certainly is not.
The more intelligent you become, the less you feel the need to hurt others to succeed.
Uh, maybe? I have no idea what he is basing this on. But also, the more intelligent you become, the more ability you have to hurt others to succeed. More importantly the greater chance that you will hurt others unintentionally, or rather, inconsiderately. It’s true that humans with our greater processing power have some ability to look at the world and conclude that it would behoove us to, for example, keep tigers and orangutans alive. But we also threaten them incidentally by destroying their habitats. That’s the point of alignment research. Our goal is not to destroy tigers and orangutans, but our actual goals may conflict with optimal conditions for their survival. A super-intelligent AI may well wish to keep humans around as zoo exhibits, but also might need the atoms that constitute our bodies (or food) for compute or something else more pertinent to its goals.
He then builds a physics argument:
let’s start from physics if you don’t mind me saying not too complex but if you assume that our entire universe is built on chaos and built on entropy the physics of the universe is all about the universe trying to decay, then the only role of intelligence is to bring order to the chaos if you agree with that
Well, ‘order’ with respect to the goals of that intelligence. But okay.
then what’s the ultimate physical order of the universe something called the minimum energy principle okay the highest order of any system is a system that’s not only efficiently and predictably performing, but it’s performing with the least wasted energy.
What? No. The minimum energy principle is just a restatement of the second law of thermodynamics. It doesn’t say anything about “predictably performing”. Ultimately a closed system would move to the lowest state of entropy. There is nothing about efficiently performing, such a system isn’t likely to be performing anything at all. Note also, you need a closed system to reason from the second law of thermodyanamics, which the Earth is not.
Correct? If you if you agree with that, what does war do? It wastes a lot of explosives, a lot of money, a lot of lives, creates a lot of hate, you know, creates long-lasting conflicts and so on and so forth. It’s a very wasteful process to include war in your approach of running humanity.
War is indeed wasteful for ‘running humanity’ for some ends, but not if your goal is to, for example, produce lots of explosions. Again, he seems to have assumed the conclusion he wants. The conclusion he wants is that a super-intelligence would somehow want the same things that we want, in this case ‘running humanity’. But he hasn’t at all made the case that it would. Not to mention that there is an obvious way to prevent all human wars: kill all humans.
And so a super intelligent AI by definition will want to optimize against this.
It’s really frustrating that he keeps using the term ‘by definition’. That’s not by definition at all, that is just what he wishes were true.
The other thing is evolutionary biology. This actually blew me away when I realized it. So if you look at evolution, so I think the debate of whether intelligence is biological or not is over. Okay. The reality is that complex beings don’t have to be biological at all. And I think we can see and witness one of them being built or born in AI. Okay.
Yes. I agree here. I think some others will disagree. Roger Penrose (who doesn’t disagree) thinks that the human mind is not computable, but that’s not the same as saying it must be biological. Roger’s argument, IIUC (which is not at all certain), is roughly that there is some quantum process in the brain that allows it to compute non-computable things. This would mean that we couldn’t simulate a human brain on a traditional computer, but perhaps if we built a non-traditional computer we might be able to. But furthermore just because we couldn’t simulate human intelligence, wouldn’t mean that we couldn’t possibly simulate some other kind of intelligence, that may be more powerful. I think given current AI systems it’s clear that we can simulate some kind of useful intelligence. So I agree with Mo here, that debate is over.
If you look at evolutionary biology, you realize that the simpler a form of being is, the more concerned it is with itself, right? So an amoeba is only in survival mode for itself. A single cellular being is only trying to protect itself, right? If you’re, you know, a little more developed, you start to look at something known as kin selection. If you know the concept, basically kin selection is I’m going to protect everything that comes from my DNA. If I know if I’m a squirrel, I’m going to try to protect the other squirrels.
I do not think an amoeba is concerned with anything. Is a squirrel? This is a matter of debate, whether animals have consciousness, or whether their behaviour is ingrained, via evolution. Protecting other squirrels here may not be anything to do with intelligence. I think it’s pretty unlikely the squirrel is logically thinking that it should protect other squirrels. It may have more intelligence than the amoeba, but that isn’t allowing it to reason logically that it should protect other squirrels. It has an in-grained will to protect other squirrels because the genes of squirrels who did, did better than the genes of squirrels who did not. In any case we see lots of examples of animals not protecting their kin. Some species are routinely cannibalistic, and certain parasitoid wasp larvae or fig wasps kill siblings as a matter of course. Lions commit infanticide when a new male takes over a pride, Gorillas act similarly. Chimpanzees conduct coordinated lethal raids on neighbouring groups. You might say that humans with their greater intelligence are more benevolent, but humans can certainly kill one another. In any case, there doesn’t seem to be any correlation between intelligence and benevolence. Chimpanzees are more intelligent and violent than bunnies.
And then you get into where humanity genuinely begins, which is they call it expanding circles in evolutionary biology. Basically you start to expand and expand and expand and include more into your family because an ecosystem that works together well is is better for everyone. So abundance is a very interesting intelligent way of creation. If AI is super intelligent, it wouldn’t destroy anything at all. As a matter of fact, it would completely favor diversity of everything.
More ‘matter of fact’ which is anything but. Why would it favour diversity? If humans have a tendency to favour diversity it’s because we need food to survive. Humans, for example, have been very good for chickens, in the sense that more chickens exist now than would otherwise exist. But are we good for their well-being? If you’ve seen chickens farmed probably not. So you have to wonder is it good to have more organisms if those organisms have horrid lives? Would it be a good outcome if a super-intelligent AI kept humans around but in tiny cages running on treadmills? In any case, humans keep chickens around but if we could (economically) grow chicken meat in a lab (which we are attempting to do), chickens would pretty quickly fail to exist on the scale they do now if at all. Look at horses, the number of horses peaked around 1915 and declined as soon as cars and other transport obviated the need for horses.
It would put a bit of limitation on our lifestyle. So, no more flying all the way to Sydney to surf because that destroys the planet, right?
It doesn’t destroy the planet it heats it up, it destroys the status quo, but would a super-intelligent being want the status quo? Why? Maybe a warmer earth would be just great for a super-intelligent being.
But it will genuinely say, I think humans can contribute something.
What exactly would humans contribute? It’s not intelligence since you’re assuming a super-intelligence. What would we therefore contribute that you cannot get from a live-stock, or a solar panel? It’s possible that the super-intelligence is ‘jagged’, that is, it’s better than humans at some things and worse at others, but overall better, in the same way that a human is better than a calculator at most things but worse at mental arithmetic, so we keep calculators around to do our mental arithmetic. I do not think this is likely, if he does I think he needs to make that argument rather than asserting that human can contribute something.
AI is not your kin
There is of course another problem. I do not know if he is correct about squirrels protecting other squirrels, but there are certainly some creatures that project their kin. But AI is not your kin. But then it gets a bit more bizarre:
the biggest debate is […] can those machines become our teenage children that look at us and say daddy is so annoying but I love him.
Well, yes that’s pretty much the whole debate, worded in strange ways. The question is, can we align the goals of a super-intelligent AI with our own goals? Perhaps one way is to train it with some kind of daddy issues, but whatever, the fact is it’s still a debate as to whether or not alignment is possible at all, and if it is whether or not we are capable of achieving it.
He then claims that whether children kill their parents is down to how they were parented:
[whether they kill you or not is down to] how you parented them […] almost all the time.
What? This is big claim. First of all, very few people kill their parents. This risk is mostly approximately zero. It does not seem to matter how well or badly you parented them. This is down to a number of things, such as murder being generally quite rare, and humans being genetically pre-disposed to love their parents. So Gawdat treats “we made it” and “it is our kin” as the same relationship. They obviously are not, we create many things that can harm us, from guns to poison, to even just industrial machinary or cars that can harm us incidentally.
There is also a spit-out-your-drink moment when he claims that parenting is “the only high-risk sport that actually does not require a driver’s license.” Okay so obviously this is just false. Sky-diving doesn’t require a driving license. Running across a busy road does not require a driver’s license. But I think he’s making a point that basically anybody can become a parent and that parenting is hard and the results might not be to your liking. Okay, but he’s slipping in a determinism argument that how well your child turns out is dependent on how well you parent them. The social research on this is pretty brutal, but more importantly, just what has this got to do with a super-intelligent AI? Are we suppose to ‘parent’ it? Again, that’s pretty much the whole debate, is it possible to ‘parent’ a super-intelligent AI into outcomes that are good for you. He seems to be saying “we manage it with kids so we’ll manage it with AI”, which is a much larger leap in logic than he acknowledges.
UK AI Strategy
The most frustrating part involves a self-contradiction that the host notices and tries to challenge him on. The contradiction is the guest’s two positions:
- The UK is finished unless it builds and deploys AI as fast as possible
- AI development is dangerous and should be restrained; job destruction is a looming catastrophe and this doesn’t need to happen.
Keep importing all of your tech from elsewhere [and] welcome to the […] third world.
The host Bartlett repeatedly draws the obvious inference, the UK cannot both go after AI as fast as possible and also restrain AI development.
Barlett: So the point was then you said this doesn’t need to happen. It does not need to happen which is the job loss or the arms race. But we’re telling our governments, you’re saying that to tell the UK government to like join the arms race.
Gawdat: I’m telling the UK government to to create an independence within the the the the UK economy so that they don’t have to be at the receiving end of technology,
That’s just “join the arms race” with a different name.
Bartlett: which is join the arms race.
Correct
Gawdat: You don’t have to compete against anyone else. You don’t have to be better than anyone else. You don’t have to. You’re simply saying I can build those things in my economy now.
I do not find this coherent. So the goal is to build an AI that is good enough for the people of the UK to use, but not good enough to actually compete with other countries’ AI and join the arms race? How are you going to set that dial?
Bartlett: But but I’m never I’m not going to use a terrible UK AI as a UK person if there’s a great US AI. I’m going to use the great US AI. So if you don’t compete and win, I’m not going to use you. That’s what you know, Gawdat: I’m not saying replace the frontier models. These are very, as we speak, they are these are very uh compute intensive. They’re infrastructure in intensive and so on. I’m saying replace Microsoft Word. Seriously, like how much intelligence do you need to build a software that writes documents?
So the UK needs its own version of Microsoft Word? Given that the UK hasn’t been able to replace Microsoft Word in several decades it clearly takes quite a lot of intelligence. Not only that, but it takes more than intelligence, actually creating the software is not really the hard part, it’s getting people to use it. Who would want to use UK Word when the only documents people send them are written in Microsoft Word? In any case, his whole point is that the UK will be left behind if it doesn’t build its own AI. But if all that AI can do is write some software whilst the US has AI smart enough to cause a jobpocalypse, then the UK is still left behind. He simply cannot have it both ways and I suspect he regrets making both of those statements.
Bartlett: I can do that. Like we we’ve built our own applicant tracking system here. We build our own software. So just ask yourself how much money is spent in the US in the in the UK government or in the or in the you know the the UA UK corporate space on licenses of software that you and I can vibe code in 4 minutes.
Yes, there is a lot of expensive software that isn’t needed anymore because it’s now possible to build your own custom version. But Bartlett here shouldn’t be straying from the main point, Gawdat has two contradictory positions, force him to give up one of them.
Gawdat: I’m thinking from the UK’s perspective cuz you know what’s interesting with the UK is the economy is struggling from a growth perspective. Correct. And I was watching this documentary the
… meanders on for some time …
And so when I’m thinking about like what the UK needs to do to not become to not fall into decline and to keep up, I’m trying to get clarity on that as it relates to AI.
And so, again I was frustrated here. He is basically evading the question, pivoting (or distracting?) to boats crossing the channel without ever returnning to the contradiction.
Later we hear that the arms race has already been lost:
Because they’ve invested in solar power, renewable energy, because they don’t care about this is, by the way, I say that publicly. I say that the arms race of AI was won a long time ago.
So why join it if it is already lost?
It continues like this for a bit, there is an interesting part regarding the thought that maybe everyone becomes an entrepreneur. But this mostly isn’t explored, I think because the guest is in evasion mode, evading the uncomfortable contradiction that he has set up.
By definition as a matter of fact
One thing that particularly frustrated me was the guest’s repeated use of phrase ‘dressing’ to make it sound more authoratative. He constantly uses “as a matter of fact” for things that are not facts but highly debatable claims. He also uses “by definition”, here are some of examples:
a super intelligent AI by definition will want to optimize against [war]
No, a super intelligent AI may want to optimise for all manner of things.
that by definition is where the tendency of intelligence goes, the more intelligent you become the less you feel the need to hurt others to succeed.
No, that’s definitely not by definition, that’s another highly questionable claim.
Okay, we by definition are connecting them. And you know what that means? It means that what we’re building is not multiple brains. We’re building multiple regions in a brain.
I am by definition writing this blog post. Either we are or we are not connecting them, adding by definition is just window dressing used to make it sound certain. It’s a pretty basic fact that we are connecting AIs together, but this is mostly language semantics. We connected computers together to make the Internet does this now mean that the Internet is one big computer? Maybe, that’s mostly a semantic argument about the meanings of certain words. Similarly multiple AIs acting in concert can be described as a single brain if you want, but your definition of the word brain is not a fact of any consequence.
Here he is using “matter of fact” my emphasis:
As a matter of fact, you know, the Boston Dynamics dog is probably more efficient than a humanoid at doing
If it’s probably then it’s not a matter of fact is it? Maybe you could say that it is a matter of fact that it is more likely than not, but you can just say that it’s probably true, you don’t need the window dressing.
Bookended softballs
I feel the host could have done a better job here. In the first instance he has simply picked either the wrong guest or the wrong topic. Mo Gawdat has written multiple books about happiness. I have no idea whether he is an expert on this or not, but surely simply talking through his books would have been more worthwhile than this.
To open with, the host asks the question:
why at that time did you start talking about AI before anybody else?
This is more faint praise than a question and it’s also simply not true. Plenty of people have been talking about AI for a long time. Here is a CGP Grey video talking about humans becoming obsolete the same way horses did, through AI. That video is 11 years old, so 2015, Mo Gawdat’s book came out in 2021. People have been talking about AI since at least the 1950s.
At the end, on Gawdat’s happiness books:
I’m wondering if any of the principles that you wrote in this book […] are more important now in the world that we live in than maybe when you wrote this book.
The opposite question is so much more interesting. Are there any principles in that book that are less important or even wrong now?
Conclusion
This was an incredibly frustrating watch. The host and guest skirt around some interesting topics. There is a good dianosis of near-term economic issues. There are the occasionally interesting thoughts; Can everyone be an entrepreneur? Should the UK be trying to build its own AI? But these are never resolved, the guest is mostly out of their depth, but pretty well practiced at evading the hard questions and dressing up his answers so that they sound meaningful. Most of the conversation is in service to the guest’s optimism for building highly aligned super-intelligent AI, but unfortunately his optimism is based on nothing more than wishful thinking. I do think both the guest and host are operating in good faith, the guest comes off as sincere and he’s engaging with a genuinely hard problem.