ToKCast

Ep 213: ToKCast in Auckland - live

This is a recording (with a brief introduction first) of a keynote address I gave to open the 2024 "Naturalistic Decision Making Association" conference. People from business, government and academia came together for 3 days to talk about how to make better decisions under pressure. It was an opportunity for me to share the work of David Deutsch and Karl Popper with everyone from people working in international militaries and government defence organisations, through to leaders in business and university academics and students. Regular listeners will notice this is an adaptation of another recent episode - but I think this live version is better as I say in my introduction.

Duration:: 1h 8m
Broadcast on:: 11 Jul 2024
Audio Format:: mp3

Welcome to Topcast and really to a cheat episode on my part. This is a recording of sorts of the talk I gave in Auckland recently about decision making. And as I always say whilst errors are my own, this talk is based in large part on what I have learned about the topic from David Deutsch in the beginning of Infinity. In particular chapter 13 from that book titled Choices. But also chapter 1, The Reach of Explanations. David's own TED talks on this topic and well the entire book really. And you'll see that I'm aping a lot of his material. And I wanted to say that he upfront, although I do of course in the talk credit him by name and secondarily Karl Popper for what I say. As I do with much of my content I've been pointing out recently, incidentally, that there's a reason for making a big deal about giving credit where credit is due. And it's not, I'll emphasize again, some attempt at modesty false or otherwise. It's so that people who hear these ideas from let's say me have a place to go to, to find out more if they want to. So I am saying in this talk here that I credit David Deutsch and his major work at the beginning of Infinity so that people can go off and have a look, look him up and look up the book to learn more. That's why credit exists so far as I'm concerned. And that's why it's right not to plagiarize. So all of that said I was in New Zealand for a few days spreading the words her to speak and listening to other ideas about decision making. The good people from this organization, the naturalistic decision making association invited me to give the opening keynote to their conference this year, 2024. And I noticed that their attendees were from business, academia and the government. Interestingly, lots of defence and military people, even from my own country were there. And I met people from the Singapore Air Force and other militaries from around the world that I might not ever have encountered otherwise perhaps. All there to listen and to share ideas about how to make decisions better, especially under pressure. So there were many cognitive science types there and psychologists, that kind of thing. People from across the world, my part of the world but also from the United States, Europe, Asia, South Korea, I noticed. And as I say from a lot wide variety of backgrounds. So it was good to get a lot of positive feedback from people who had never heard the name David Deutsch before or Karl Popper, much less anything about this optimistic worldview that I was giving them a taste of. So as you watch this for any fans of top cast, you will see that as much as it is about decision making ostensibly, it's really a soft introduction me sneaking in the epistemology of Popper and Deutsch in their real epistemology and real optimism in David Deutsch's sense. And of course, therefore, criticism of the prevailing pessimism of our times. And yes, it's strongly based on a recent top cast episode that I have already released, but I find this live version more refined than that other version that was pre-recorded. I might at this point also take the opportunity to thank the organizers of the conference and Brian Moon in particular for inviting me across the ditch as we say here in Australia when we travel to New Zealand. And obligatory warning here about the audio quality, it's not going to sound exactly like this. I should have worn the mic that was there or stood more closely to it or something like that, but I was playing things to the live audience that I was speaking to rather than the recording in this particular case. But anyway, it's possible so far as I'm concerned and you'll notice that there is a mixture of slides and video as we go along. Well, enough from me. Over to Brett Hall in Auckland to deliver the rest. Thank you so much, Brian, and good morning, everyone. Brian mentioned hostile. I don't want to use that word. I would say that I'm going to speak to you about the day. I hope Doug Aylton likes with your naturalistic decision-making. I think that we're in a period of a kind of anti-human pessimism over all of you. And what naturalistic decision-making from what I'm aware of comes from the same perspective that I do, which is that human beings are crucially important too. Not only the future of civilisation, but quite potentially the entire cosmos. We are the changemakers because we are creators. And that is going to be a central theme throughout what I'm going to speak to you about this morning. I mentioned the word rational up there. Now, rational to me means little more than our search for good explanations of the world. And we come to a good explanation via method of detecting errors in what we already know and correcting those errors over time. I mentioned also this idea of the future because in large part being able to make a decision is an attempt to guess at what the future might be. Because after all, you want to make a decision that is going to lead to the best outcome, not right now, but at some point down the track. We've always had people who have told us that they know the future, soothsayers and fortune tellers that all cultures still do have such people. These people have appeals to the supernatural, supernatural ways of thinking, secret knowledge that the lame person has been denied. The titles may have changed these days, but has the possibility of knowing the future changed? Alexander the Great, so it was said, went to visit the oracle of Delphi. He wanted to know from her if he was going to conquer the world. He wanted to know the future, and by extension, the future of the world. And so the story goes, the oracle at first, when Alexander turned up outside of her cave, outside of her cave, she refused to come out of the cave. So he sent some soldiers in, dragged her out, kicking and screaming, and then brought before him, surrounded by a small army. She said, "Oh, of course, Sire, of course you are going to conquer the world. You are invincible." He wanted to know the future, and she had, shall we say, deep motivation to tell him what he wanted to hear. How much of things change now? She cannot have known, but with a little prodding, let's say, she was all too willing to tell him what he wanted to hear, and we might compare leaders today, desperate to know the forecasts of a number of people, what their future is going to hold, and what the motivations behind people are who are willing to tell us what the future is going to be. But I want to tell you that there is, in fact, much we cannot know for very good reasons, scientific and epistemological. And much of what we cannot know is about what is yet to happen, prediction, prophecy. These are two ways of speaking about the future. And I want to say to you that between them is a chasm of science, reason, rationality broadly, which cannot possibly be bridged. And I will tease out what the precise differences between these are as we move through my talk this morning. Because there is a difference between knowing, which is to say possessing a good explanation, like, for example, and I'm going to use this example over and over again. I've noticed outside walking through the Department of Engineering here, there's a number of equations that are appearing on whiteboards. I love that kind of thing, because as a physicist, when you are given an equation and a simple object like a ball, which I was to drop to the ground, then I can tell you down to the millisecond, when that ball will hit the ground, if only you give me the height of the ball, that is our prediction. But prophecy is something else. Prophecy ignores how knowledge yet to be created is yet to affect the world. In short, it is ignoring what choices or decisions people, both individually and collectively, whether it's governments or businesses, will make. And for deep reasons I will come to, those choices cannot be predicted before they are made. We speak choice, we mean decision. So in a real way, decision making, it's very hard to have what the organisation and its conferences about is about decision creating or choice creating. But while it might be true in most cases that we cannot know the future, we can still have a commitment to rationality, to science, to mathematics, to epistemology, and yes, even to intuition, our own private sense that something is or isn't a good idea. Whether we can articulate precisely why that is the case or not, can be indispensable as a guide on what to do. And I understand that others will be speaking about intuition at some point throughout this conference. But none of this changes the fact that many of us still want to know what the future will hold. We are desperate to know about the uncertain future. Our decisions may, moment to moment, from the minute to the momentous, depend upon us, guessing at the future, hoping or dreading as the case might be. But the problem is that while we know a lot, our knowledge is always finite. But our ignorance, that's infinite, and will always remain infinite. Because the more we discover, the more we find out, we do not know. I'm old enough to remember that back in 1992 as a kid interested in astronomy, we didn't yet know that there were planets orbiting stars beyond the sun. We hadn't found one yet. But in that year, we finally did. So the old debates were over. Was our solar system unique throughout the universe as being a place where there was a planet going around a star? Or were there other planets going around other stars? Most people interested in astronomy thought it must be the case that we are not unique. But once we had found that one, once we had made that discovery of the observation of that first exoplanet so-called, we ruled out the counter-clane. And suddenly we knew that there must be trillions upon trillions of other planets out there. We do not yet know the locations of, we're still ignorant of that. But because the laws of physics are universal, they apply everywhere and at all times and govern things like the formation of planets. We were able to postulate that these planets existed, but we had an infinity of ignorance about where they were, what they were like, and more besides that they contained life. One discovery opened a window on a new infinity of ignorance. And astronomers ever since right now are engaged in the active project rushing to try and find ever more and more Earth-like planets. Do we know are out there? We're just ignorant of where they are. We explain what we see these few exoplanets that we detect now and again with sophisticated telescopes. We explain them in terms of what we do not see the universal laws of physics. No one sees a law of physics. We conjecture it. We create knowledge about the underlying reality. We explain the scene in terms of the unseen. The scene planets in terms of a ocean of trillions of more that we are yet to actually see. Our ignorance of the world is woven into the laws of physics. The laws of physics bound our capacity to know in advance what is going to happen. Just moments from now, much less months or years from now. And this is not just the problem of intractability, a fancy term for our incapacity to calculate what is going to happen to complex systems. When you go to a casino and you roll a dice, the reason why the outcome is unknown is technically down to intractability. And all of those particles and those collisions and forces that come to impact a particular system is too difficult for even the most powerful supercomputers to model precisely what is going to happen. But there's something deeper even than this. And that is born out of what we are as human beings, creative entities. And I mean that very literally. We create things, something wasn't there before, and it is there now. And that's not a mere recombination of existing knowledge, although it can be that too. But there is no denying that people generate newness in this world. And fundamental physics, powerful though it might be, able to predict things in prescribed places at prescribed times, cannot predict the products of creativity. No physicists, no calculator, no supercomputer, can predict what a two-year-old will do with those blocks in their bedroom. That's not just intractability, something else, something special. The philosopher Karl Popper and his intellectual descendant David Deutscher incidentally invented the theory of quantum computation. Spawning a multi-billion-dollar global industry right now where almost every country has some institute or other devoted to building one of these quantum computers hasn't happened yet. We're working towards it, but that's the gentleman that invented the theory, the underlying theory, the Alan Turing about times if you like. These two gentlemen have written and spoken extensively on what it is that we know from my perspective about knowledge, how it is tested against reality and what its limitations are. And that is the domain of epistemology. What do you know about how you know affects your own psychology? It underpins how science works, mathematics history, and all of our academic disciplines. And the day-to-day we encounter the world in solving our problems. So in what follows I credit both Popper and Deutsch up front for inspiring much of what I say to you about the possibility of prediction in an age of uncertainty, especially as people become uncomfortable with uncertainty. And I do not want people to be uncomfortable with uncertainty, but rather I want them to appreciate the hazards of certainty, for to be certain, to be truly certain about something as being true or as absolutely going to happen is to embrace dogma. And people tend to go to wars over dogma, metaphorical or real. So we want to resist certainty. Knowledge of the kind I'm talking to you about, conjectural knowledge, plain, old, uncertain, good explanations, which are not certain, is wonder. That's what we're not certain to. And so as you might guess, my area is originally physics and physics is, as I've already hinted to that, preeminent, shall we say, among the sciences for making highly precise predictions. That's not all it does. Its job is not just to predict stuff, but famously if you give me the equation and the height at which you are holding some ball and you drop, but I will tell you down to the millisecond exactly when it will hit the ground, when dropped, that's a prediction. And it gives rise to what is sometimes termed in other fields, physics envy. A pejorative phrase, because for one thing, as I say, physics is not just about prediction, it's about explanation. Incidentally, chemistry does just as well as with predictions. You take hydrochloric acid next to with sodium chloride and you will get, next to with sodium hydroxide other and you will get sodium chloride and water, but no one speaks of chemistry energy. In any case, science in the main is not about prediction. It's about explanations. The distinction matters far more than most seem to understand. What we are doing day to day is trying to explain the world to ourselves so that we have a deeper understanding of it, rather than trying to predict what's going to happen next. And sometimes, our very explanations of the world, especially in science, lead us to understand that we cannot predict certain things even in principle. And I want to suggest to you today, there are three reasons that we cannot predict certain things. One I've already mentioned, the issue of intractability, the literal imprintable beyond our capacity of our computers, no matter how powerful of our, to calculate what's going to happen. Number two, which I'll come to presently, bounds placed by the laws of physics. And three, which I've also hinted at, acts of genuine creativity, in particular, the act of a human being in creating a new explanation. When we can't make precise predictions in some domain or other, this is no value judgment. After all, the very thing I'm talking to you about right now, the epistemology, how knowledge is created, what it can do in its limitations, can't predict much of anything. But I find it extremely worthwhile, it's highly valuable. At even in physics, these three pre-hop provisions against our capacity to predict, also apply, and so therefore must inform rational decision-making, the decision-making of any kind. So when, as I say, and I will, some things are inherently unpredictable, this does not mean they are forever incomprehensible. And ultimately, that is what we need and understanding, a good explanation. But let me focus on number two there for a moment, because this is a fun one. Why it is that we cannot predict the future as a matter of fundamental science, fundamental physics. We have known about these things called photons, particles of light, since 1905, when Albert Einstein explained what is known as the photoelectric effect. Einstein was awarded the Nobel Prize in 1921, not for his work on relativity, for which he is most famous, but for rather for his discovery of the fact that light travels as packets of energy, particles in other words. We can take half-silver mirrors, so-called, because they have half the silver or regular mirrors, and instead of when you look at it, seeing a lovely, almost perfect reflection, one sees only half the light bouncing off the mirror. The rest goes through, which is transmitted, as light tends to through glass. In fact, the half-silver mirror is also known in some applications as one-way glass. It is the sort of thing you see in those detective movies in an interrogation room, like this. But the lighting inside the room, it is bright, so the people in there can be seen, but on the outside, beyond the half-silver mirror, the people in the other room cannot be seen. This is the half-silver mirror. What is that got to do with anything? We can take single particles of light, photons, shine them at half-silver mirrors. I said before, when you do shine light at a half-silver mirror, half is transmitted, and half is reflected. Half bounces off, half goes through. Okay, well, let's turn down the intensity of our light. Turn the intensity down until we have just one particle of light traveling towards our half-silver mirror, as illustrated over here. It is the famous experiment where it is part of a broader piece of apparatus that is actually used in quantum computation, but we are narrowing it down to where we simply have this simple system of a half-silver mirror and a single photon traveling towards it. And as I say, the half-silver mirror allows 50% transmission of the light and 50% reflection of the light. So here is our not merely a thought experiment, but a real-life experiment that can be done in the laboratory. You fire one photon at this half-silver mirror. What does it do? Make a prediction. It is no use saying, well, 50% of the time it goes through and 50% of the time bounces off. That's true. That's what the laws of physics say. But if your very life depends upon you knowing whether the photon that you fire goes through or bounces off, you're out of luck. We cannot say that the heart of the laws of physics is subjective unpredictability. I say subjective because you don't know. The laws of physics themselves are objectively deterministic. It's absolutely determined that repeating the experiment over and over again, the data will converge on 50% going through and 50% bouncing off. But again, that's of no use whatsoever. If you are interested in your photon, that you're firing now, which may determine what you're going to do next for whatever reason. From your point of view, you just do not know what you are going to observe until you've done the experiment. Not only don't you know what you're going to see, the very laws of physics say it's impossible to know. There are many such examples in this area of what is now known as quantum physics. The point is there are some systems so simple, single particles and little mirrors, that we cannot even know what will happen with certainty. What will happen with the particle as to whether it will bounce off or be transmitted. Now to speak in terms of the motion of particles when discussing something like the topic of our conference here, decision making is to commit the fallacy of reductionism. There's not anything that I'm saying here is false or entirely irrelevant, but it misses the point. You don't usually base important life decisions on whether a particle of life will bounce off mirror or not. But we are talking decision making about guessing at the future to make the decision that leads to a better future wanting to be able to predict to know rather than simply be ignorant. So we should want to know when prediction is at least possible and when it's not. When we make a choice, we are choosing among options. Which options? Well, the ones we know about obviously, you're in a foreign country, walk down the street, you're open if you've not seen it before, you will see McDonald's and you'll see KFC. They are on the table so to speak. They're among the options about what your next meal might include. But there are also options you are yet to discover or better yet, create, wander into a random grocery store, find some exotic spices, meats and vegetables and concoct something that you've never made before and perhaps has never even existed before in the history of the universe. And this idea of knowing and creating new knowledge is the domain of a epistemology. What I'm talking to you about right now and that's where I'm going with this. And this is the good news because you cannot predict the content of knowledge yet to be created. That's a good thing because it means the future is genuinely open, open to getting better. We cannot know what the content of the next theory yet to be created will be. If we could predict the content of, let's say, the next scientific theory, then we'd have a prediction of what is in the next scientific theory to be discovered, let's say, next year now. In other words, we'd know it now. See the contradiction. Every physicist, for example, would love to be able to improve on Einstein. If anyone in this world thinks they can predict stuff, it's physicists. So if it's true they're so good at prediction, why are they just predicting or calculating their way to whatever might replace Einstein's general relativity? Well, it's because they cannot, physically cannot. We cannot predict the content of theories yet to be creative. If they could do that, they'd already have the replacement and presumably the Nobel Prize. The truth is that knowledge is hard one. And this applies also to something that will come back to the large language models, chapter 15 in the life. Many people, if you were listening to them in the tech world, think that they are going to be able to wrap up all of mathematics or all of science or all of physics in some way and they'll be calculating their way to a final theory of physics. But as I say, these are acts of creativity or calculation. Something remarkable goes on when creating knowledge and it's all there in the creating part. Actual creation goes on in the minds of people even when some try to deny this. Those people animated by the existence of the large language models, chapter 18 and its rivals, which are all the rage right now, are saying that, in a sense, people are just like large language models of a kind, recombining things already stored in their brain in new ways. Sounds reasonable, completely forced. Recombination is not creativity, however much the two at times might resemble each other. Sure, recombination can give novelty, the illusion of creativity, for taking all the information across the world and putting it into a database that exceeds the size of any database ever concocted until now and having a clever algorithm recombine that information in novel ways is impressive. Sure, absolutely astonishingly impressive, super impressive and super useful. But it is not creativity of the kind I am talking about. Chat GPT and similar AI are deriving things from their library. Derivations are not creations strictly. An infant child on the other hand or any person for that matter is achieving more remarkable things not by referring to vast libraries but rather given very scant information. They guess at the nature of reality with just a few little clues about it and they are curious about what they see and observe, prodding it or prompting it to use the jargon of today. Chat GPT 4 or 40, whatever we're up to right now, wonderful as it is, has never demonstrated curiosity about any. It is a perfectly obedient chatbot. Now I don't know if you know any real people. They aren't in the main perfectly obedient but they are curious so we have a chasm of difference there. The AI is judged to the extent that it fulfills its task, that it obeys the instructions given to it by the user or the programmer. This is not the way we judge other people. They don't need to be perfectly obedient and in fact have the quality of having the possibility of disobeying us. An AI that was to disobey us will be thrown in the bin as having malfunctions. There is also the difference between chat GPT, large language models, as I say, recombining information in interesting ways, ways we've not seen before, that's novelty. But that versus knowledge creation, something that wasn't there before in any library anywhere that we have created in our minds. It is knowledge creation I want to say that underpins human decision making. Knowledge creation is what separates a person from the tools that people use, whether those tools be pocket calculators or large language models. I'm talking a lot about knowledge but to understand this thing knowledge creation, this mystery that is at the heart of what it is to be a person. Perhaps we first need to hone in on what knowledge is. My view of knowledge or poppers view of knowledge or Deutsch's view of knowledge. Traditionally, in over many years and if you've ever sat through a philosophy epistemology class, they will tell you, coming all the way from Plato and the ancient Greeks, that knowledge is a form of justified true belief, something counts as knowledge if it's a justified true belief. Sometimes they say, well, the justification doesn't have to be perfect, it could be more or less strong, it could be justified to some extent, people say, probably justified, justified to a certain level of credibility or to credence, and some people want to say they can quote the level of confidence they have in the justification. That's come to be known by the term Bayesianism. I will come back to that. Let's just deal with straightforward justified true belief, what are known as get year problems, and this first get year problem is due to Bertrand Russell. What time is it? We've got Big Ben over there, so you're wandering around London, you've forgotten your wristwatch, your mobile phones run out of battery, you need to know what time it is. In his daytime, the sun is shining and you look up and there you see Big Ben, little hand and big hand both pointed at 12. What time is it, midday? Do you know it's all that? Well, Russell asked the question, what if I told you that by remarkable unfortunate coincidence, that clock has been showing midday for the last two weeks, it's been broken. So now, do you know it's midday? What if I further remarkable coincidence that as you look up, it just happens to be midday? Well, now it's true that it's midday, but you're not justified in believing that it's midday. What does that got to do with anything? Well, this whole idea that knowledge is justified true belief is problematic given a get-a-year case like that. Now, I myself prefer real-life examples over thought experiments. That kind of thing is a quirky little way of imagining that this definition of knowledge can go wrong. But a real-life example that can refute this notion of what knowledge is and I will come to what knowledge really is telling you what it isn't, is to consider Newtonian gravity. Newtonian gravity is used in all sorts of ways to this day. It's used in calculating rocket trajectories and the tides. Historically, it's been used in calculating the positions of the planets throughout the solar system that they orbit the sun. This there on the screen is Newton's so-called universal law of gravitation, simply elegant. And as it turns out, completely, strictly speaking false. First, the F stands for force and we now know there is no such force of gravity. We know that there is no force of gravity because, as I say, historically, it was used to calculate the position of all the orbits of the planets throughout the solar system. However, towards the end of the 19th century, beginning of the earlier 20th century, it was realized that there was one planet for which it never quite worked. When the astronomers pointed their telescopes sky to look to see where Mercury was, according to the predictions of Newton, it just wasn't in the right place. Many hypotheses were put forward. Was there another planet that we couldn't see on the other side of the sun perturbing the orbit of Mercury? That couldn't be found. Unsurprisingly, it took a genius to actually solve this question. And the problem was known as the procession of the orbit of Mercury. And as Mercury went around the sun, the long axis tended to move in a way that Newton's theory simply could not account for. In the genius that's offered, Albert Einstein, he created an entirely new theory of gravity, general relativity, created, I say, and not discovered for reasons I'll come to. He created an explanation, a scientific theory, that there was no force, but rather it was the curvature of spacetime. Now his equation, shall we say, looks a little more complicated than Newton's, and it is. But it does allow for more precise predictions. It gets Mercury's orbit correct while Newton's does not. So that rules out Newton's theory of gravity as being the best explanation of what gravity is. In favour of Einstein's general relativity, Newton's theory is falsified by observation or experiment, if you'd like. So yes, Karl Popper was right on that point. That is how science works. You take two fields, you perform an experiment ruling one out, and that's called a crucial test. Yes, the test can go wrong. In 2011, for example, at the Large Hadron Collider, many people may have heard this story, the particle accelerator, the largest one in the world at CERN in Switzerland, there were particles called neutrinos that were observed by instruments traveling faster than the speed of light, completely in violation of Einstein's theory of relativity. People said at the time in breathless headlines, Einstein proved wrong. That happens periodically by the way. Of course, no such thing had happened because there was no alternative theory against which Einstein's theory could be proved wrong. What would we replace it with? Not Newton's, we already knew that one was wrong, and it wasn't just the fact that Newton wasn't able to get Mercury's orbit right, it wasn't able to do things like predict the amount by which light was bent during a solar eclipse from stars traveling behind the sun, that's known as Eddington's experiment. And there was far more besides that general relativity was able to do that Newton's theory was not able to accomplish. Yet in the Large Hadron Collider experiment, where they observed this thing apparently in violation of Einstein's general relativity, namely particles traveling faster than light, we may have heard that law, nothing can travel faster than light. True, we didn't have two good explanations to decide between. We had one. Or rather, we had to make a decision. Was it Einstein's general theory of relativity that was wrong? Or, if you were forced to place a bet, that someone made a mistake with the experiment. There's a genuine decision, and lots of physicists actually said, Einstein must be wrong, we've got to find a new theory. Now, Einstein may be wrong, but it didn't come down to that experiment, because as it turned out, someone had disconnected a cable somewhere or another, and the calculations were wrong, and so accounting for that turned down after all, that neutrinos did indeed travel slower than both ones of light. Einstein vindicated once again, but it wasn't just about predictions. As I say, Einstein's theory not only makes more precise predictions, that's one thing, but it does something far more important in my view, and this is what all of science and all of human creative knowledge does more generally. It explains stuff. It tells us what exists and what it does. So in the case of Einstein's general theory of relativity, what does it say exists? Well, it says that what exists is a fabric of spacetime, and what it does, bends and weaves whenever there is mass present. Space tells matter how to move, matter tells space how to curve. Let's have a great general relative as John Wheeler put it. There is no force because no force is needed. Technically in science, a force is mediated these days, we know, by particles called bosons, but there is no boson here. No particle is needed to achieve this. So why does it happen? Why does space bend in the presence of mass? Well, at this point, we can invoke another philosopher, Ludwig Wittgenstein, who said, "Our spade is turned," by which he meant we can dig no deeper at this point. We just don't know the theory is silent on that, but it does point to the fact that we will need a better theory at some point. We'll always need a better theory. The quest for knowledge is infinite. We can always ask why that theory, but any case, at the moment, Einstein's theory is the best explanation of gravity so far. But what then for Newton's theory, do we just throw it out and chuck it in the bin? Well, no, as I say, it's used for all sorts of things. Even if it does say there is a force of gravity and kids in high school are still taught Newton's theory. If we're in university, it's what Newton's theory. Why? Why am I taught it if it's known to be false? Even if it's strictly speaking, also gets the prediction wrong. It's not as bad as some random guess of a man on the street or an ancient Greek, but it gets things wrong. But it still counts as knowledge. Why? Well, it's because we have to reconsider what knowledge actually consists of. And it's not justified true belief because belief, by the way, puts us in a mind of something subjective about our psychology, purely about our psychology. And I want us to take us out of our minds when it comes to what knowledge is. Knowledge is information that solves a problem. There's one approach. And none of these are going to be definitions, but rather approaches to what might be regarded as, features of this thing called knowledge. And this thing called knowledge is, I say, cosmically significant. It is the thing that will build the structures that will change planet Earth and perhaps ultimately solve a system, the galaxy, and more besides, if we have an optimistic view of the world. Information that solves a problem is useful, a very different definition of what useful is. You're accomplishing something. You're solving a problem. And because it solves a problem and because it's useful, what would you do with something like that? Well, you copy it. It's the information that tends to get copied. And as David Deutscher so eloquently put this in his major work, the beginning of infinity, it is information that once instantiated in a physical substrate tends to cause itself to remain. It's almost as if knowledge has a life of itself. His colleague and work researcher into constructive theory, Kiara Maletto, says that knowledge is resilient information. It weathers the storms. And she read a book called the Science of Pan and Cart and has the most remarkable example there of why it is that knowledge is the most resilient thing that we've yet discovered in yearbook, the thing that persists at the time. And all we need to do to understand that is to consider certain forms of bacteria or archaea that have histories of three billion years. Their DNA strands have remained relatively unchanged in copy generation after generation for three billion plus years. No rock on planet Earth has survived that long, that long since weathered away. But knowledge of that kind, evolutionary knowledge resilient. And our knowledge, so on of our civilization persists, will be the thing that gets copied and will be resilient off into potentially, hopefully an infinite future. But here I've mentioned information many times and implied knowledge is the kind of information. So then well, what's information? Well, that's a physics question. And I'm not going to go into that right now. I'll stick to what concerns us. Knowledge is useful information. So there's one sort, but there's also useless information, useful information is the stuff that gets discarded by the errors. So a composer of music fills up their waste paper basket with all the attempts that do not meet their criteria for what is harmonious or melodic, let's say, that gives them some information about what does not work in constructing what does work, constructing knowledge of a better composition. And Einstein didn't get general relativity right on the first go either. He got information from his mistakes. He guessed at what was really going on, as we all do, and most of the time, we make a mistake, no surprise, we're wrong. So you must decide between what is useless information and what is actual objective knowledge, by which I mean a good explanation. And by good explanation, I mean hard to vary. And by hard to vary, I mean every part of the explanation is very difficult to change or even impossible to change, because each part of the explanation that we're talking about has causal power. And you cannot change it even slightly without ruining the explanation. That simple example here might be, our explanation of matter comes down to the fact that the stuff is made out of atoms. Patterns are made out of smaller particles, still one of which is the electron. The electron has a very precise charge and a very precise mass. You can't arbitrarily change what the mass in the charge of that electron is. It's hard to vary. We can't just swap out the electron for any other subatomic particle that you might like. Hard to vary. The only change we know of, for example, in the hard to vary explanation that is Newton's theory of gravity, which is technically known as an inverse square law, is to throw it out all together and to replace it with Einstein's general relativity. It's not you saying, well, well, let's just change it to an inverse 2.1 law and inverse Q2. That won't do. It will ruin the explanation and we'll get the predictions completely wrong. What we are doing as a civilization is searching for ever better, hard to vary explanations. And the enlightenment may as well be defined as humanity's search for good explanations. How? Via a tradition of criticism. Tradition has universally been regarded hitherto as a way of maintaining the same thing over time. All cultures have traditions. Those things that persist throughout time. But the enlightenment is the first time that we occupy a tradition where we have a tradition of change. We expect change. People are unhappy if the next iPhone isn't as good as the previous one, that we aren't seeing rapid change. We expect rapid change. We have now this remarkable situation that we're in, a tradition of criticism. And that is how we improve our explanations. We seek out our best knowledge, criticize them, find flaws and hopefully remedy those flaws. Criticism aimed at finding flaws in existing knowledge. For what purpose? For finding something better for improving things, making progress. And within that context, science and experimental testability, they're just special cases. A special kind of criticism is the scientific experiment that applies in science to scientific theories. And when a scientific theory is testable, which is a necessary but not sufficient, not sufficient criterion for a theory to be scientific, then we know we have some knowledge. Incidentally, it's necessary but not sufficient because if the doctor is just diagnosed you with COVID and then says to you, "Here's a bunch of roses. Eat them all. That will cure your COVID." That's a testable theory. But it's not scientific. Why? No good, hard, a very explanation of companyism. And it could be easily varied. And the doctor may very well say after you've eaten your bunch of roses and fail to be cured of your COVID. Well, the bunch of roses was too big or too small and so on. So hard to vary explanations. The very thing that I'm suggesting to you what the enlightenment can be defined by is to be contrasted with easy to vary explanations. And what is an easy to vary explanation? The magic. Anytime something happens anywhere for which you don't yet know an answer, just say, I explain it by magic. Magic did that thing. Don't know how thunderstorms work? It's magic. It's an all-purpose explanation that can be varied from phenomena to phenomena. Never actually explaining anything because ultimately it's contentless. You can swap out magic primuricals if you like or sorcery, easily varying the word to describe what the supernatural forces at work are doing, manipulating the world behind the scenes as it were. They are bad explanations. And I don't, I mean that technically, I don't just mean false. I mean bad in terms of easy to vary. If you're unsure how thunderstorms work, you might well say the God of thunder thaw is the cause of thunderstorms. But if you're satisfied with that, you may as well be satisfied with the Hindu God Indra causing the storm, who incidentally is the God of thunder and Hindu. Gods of thunder are easy to vary. Saying magic or miracles or gods is no explanation. We want to know how, and we want to know how to test our explanations, how to sift, as it were, the good explanations from the not so good. Now there are rare cases where we have two good explanations to choose from. But it is rare, rare than most people think. You're lucky in science sometimes to have one explanation. The whole point of science is we're at the frontier of knowledge. We don't know and therefore we have no explanation whatsoever. So we're lucky to have one. And if we have two, we're extremely fortunate. Other times, red day to day life, you may have many options on the table, as I've mentioned before, what to have for dinner. All options may seem good to you, but sometimes nothing seems to be appealing. So what do you do then? Well, you're not necessarily forced to do anything, because we are not machines. We are people. People are not just mechanisms following deterministic laws, being forced this way and that by electrical signals or some such modulo what neuroscientists might say. We're not done computers. We're something more. We're creation engines. We create. We bring new things into the world, as I've said, new options that were not there before. If there is no good option on the table, we create one. Before anyone discovered nuclear physics, for example, no one could choose to have nuclear energy as an energy source. Where I'm from in Australia right now, there is a huge debate animating the political and business class. Should we build a nuclear reactor for the first time? Never mind the answer. The fact that we can ask the question is proof positive. Something knowledge has been created. It's a question that could not have been asked in 1900. We do not know in fine-grained detail how it is that we create the knowledge that we do. It's part of the central mystery of what it is to be a person. A person is able to generate something that wasn't there before, but is there now. They are a creative entity. But we don't yet know how they do it. This is the distinction to be made between AI and AGI. Objectively, in terms of science, as a matter of science, the scientists can come together and say, why is it that this thing the human brain can create, and in particular, can create it? Explanatory knowledge. If we could figure that out, then we would have this thing called AGI, artificial general intelligence, a computer system that would be a person, a silicon person, but we don't have it yet. But also, that's the objective side. Subjectively, as a matter of your own conscious thinking about what you're doing, you're speaking a sentence during a regular conversation, and you don't know how you get to the end of it. The words just flow. You're creating on the spot. An idea pops into your head. That's you. That's you doing that thing. You have the idea. You're creating it. But how? You don't know. Someone asks you, how did you come up with that? You can tell yourself a just so story, but ultimately, you are creating your creative thing, the mystery at the heart of what a person is. And yet, listen to some technologists try it now. Some people do really think that we're on the way to solving this question about creativity, because after all, isn't chat GPT, and the various other large language models and chat bots that are out there, a sign that were on the road to creating this thing called AGI, a computer that would be a person. The rather abrupt appearance of the large language model in just the last few years was a surprise to many, a remarkable piece of software that manages to achieve a level of language proficiency in an otherwise dumb computer that almost no one saw coming so soon. One moment, it was very obvious that computers were failing the Turing test, and the next moment, well, there it was having passable conversations. So, okay, it's not perfect. chat GPT is not perfect. We can still tell it's not a person, but it is a great leap on what existed before, and there will be greater leaps still. What explains this remarkable increase in functionality? Is it about to achieve what we have? Is it a stepping stone on the road to becoming an actual thinking creative person? There are two parts to a large language model, which is what chat GPT is, the first of the large part. chat GPT 404 has loaded into it the largest library of information ever gathered together in one place on the planet. Most of the books that have ever been published, almost the complete literary works of human socialization, loaded onto its hard drives. So, that's the large part of the large language model, the LLM, and the M, well, the model is able to recombine all of that information in ways people may never have seen before, but not randomly, in ways that are genuinely new and interesting, or novel, I should say, but that's from a God's eye perspective. What it's doing is taking a library larger than there's ever been conceived before, and using an algorithm known as the transformer architecture. The transformer architecture, I won't attempt an explanation of it right now. If anyone interested in, I went looking some years ago for exactly how this thing works, and the coders understand how the transformer architecture which underpins the large language model works, but they have great difficulty explaining it in plain language, so I've attempted that year. But the point is, it is an obedient system, perfectly obedient, it will respond to the prompts given to you, it will not self-prompt, it does not have interests, it's not creating explanations, even if it does generate novelty. Again, what's the difference? Novelty is just something not seen before, and if your special language calculator has access to the entire corpus of human knowledge and clever rules about how to recombine it in a sensible way, you will get novelty, a limerick that's not been seen before, yet saying, a seemingly new script for an ever-before-made episode of Seinfeld, but prepare to be unimpressed with the level of creativity when you ask it to do such things, because there is none really, it's derivative, it's recombination, because you're drawing from inside a finite library, however large that library is, but people can step outside of the library, we can think creatively, our creations also may involve recombinations, but they also have something else too, some other spark of originality, a true innovation, and as I say, the key things people can disobey, they have the capacity for disobedience, they can look at all otherwise good explanations on offer and reject each of them one by one and say no, no, none of that, I don't like any of it, I'm going to make something better, all of what I can see before me is flawed by my standards, a computer doesn't do that, but why do people, well, just for fun might be one reason, but another could be what it's animating us here and now, decision making, what should I do now, what should I do next, how can I know what will work out and what might fail, but that is a special case of us wanting to solve our problems, as Karl Popper said, all life is problem-solving in the hope that we can make the future a better place, so we want to be able to predict, to know what the future holds, and these days, we even have entire professions devoted to it, future interests, there's something called a forecasters or super forecasters, because I guess soothsayer has some stigma attached to it now, they're claiming to know the future even when they cannot, they claim to have good explanations, and if they do, then perhaps they can make a prediction, they keep on saying, as a physicist, if you give me a ball and I drop it to the ground, I can predict when it was hit the floor, if I'm on level eight of an apartment block and I hold a ball out the window and I have a computer in the very laws of physics enabling me to predict down to the millisecond where it's going to hit the ground, say it takes 2.2 seconds for the think it's done, I drop it from level eight, all of that calculation, all of that reliance upon the very laws of physics themselves, is rendered completely useless if on level four Jeremy reaches out his hand and catches the ball before it ever hits the ground, the inherently unpredictable actions of a person have intervened to change the prediction that I made, and that's for something as simple as dropping a ball out the window of a building, never mind forecasting off into the deep future, much of what people do is inherently unpredictable, so people who claim to know the distant future are not predicting prophesying, and prophets tend in the direction of pessimism, why? Because many of us can imagine how bad the world will get, the problems that we're going to encounter, but it's rather more difficult to imagine the solutions that will be created to the problems imagined. If a thing is creative, it's not predictable, and if it's predictable, it's not creative. Trying to predict, for example, what the population of planet Earth will be in 100 years assumes we know what choices people will make in five years, and then in 50 in summer, what light extension technologies might be invented, what natural disasters might occur, what cultural shifts happen, how trends might change. In other words, a forecast about the future population of the Earth is not a prediction, it's a prophecy, it cannot be known because it depends upon what problems people yet to encounter are going to encounter, that's unknown, and what solutions they might propose to overcome those problems, that's doubly unknown, and therefore prophets tend in the direction of pessimism. They cannot see the solutions ahead of time, and yet people have a deep, deep need to know what the future is going to hold, and governments and businesses will demand that experts help them in guessing of what the future will be like, and then we have a new generation, a culture of prophets coming up, appearing in the media, and it is important sometimes to at least have some informed idea about the future, but notice this, that the people who speak of the distant future, the present-day prophets have two things in common, one, they claim to be predicted, and two, the future is not going to be good, some claim for example, that artificial intelligence today, that tends the end times, that the AI will become super intelligent and regard us as a threat and kill us all, they are the AI humans, some claim the next virus to escape a laboratory or a wet market will be impervious to our attempts to curtail it, and the population will collapse under the pressure of a spreading packaging, these are the disease doomers, others say that they are concerned about the fact that as in my field, because comets and asteroids have hit the earth from the past and it's going to happen again soon in the future and we won't have enough technology or power to do anything about it, those are the cosmological doomers, some are concerned we're not going to do anything to solve the problem of climate change in time, those are the environmental doomers, others are concerned that political institutions are going awry too fast under the sway of social media, civilizational doomers, Elon Musk and many others recently have been talking about the fact that the declining birth rates, especially throughout the West, portends the extinction of people not too far into the future, baby doomers, I could go on with seismological problems with super volcanoes and famine and nuclear apocalypse pestilence and locusts, the list is longer than anyone could possibly encapsulate in a short lecture, it's a horror story out there, if you pay too close attention to some of these people, people who guess at the future, again, are very good at imagining the problems we are going to have, but they're rather worse at imagining the solution to those problems and yet we are problem solvers, it's what we do as creators, it's the solutions that are the hard part, the solutions are the scientific theories among other things, the great feats of engineering, as I say general relativity was a solution that solved the problem of why Mercury orbited the sun in the way that it did, but it went on to produce so much more, everyone who has a smartphone now has programmed into it into the maps, Google maps or whatever you happen to use, general relativistic equations, literally saving lives of people daily who would otherwise have gotten hopelessly lost, doomers are not predicting but prophesying, claiming to know the content of future explanations and in the main they are saying to us, no solution will come in time, we are doomed, well near enough to do that unless we are taxed far more heavily or have rights taken away or have the technology itself controlled and regulated by a central authority that we are going to be doomed, so listen to them and make your decisions now, given the worst is going to happen, the AI will take up, you're going to get a deadly virus, the sea levels will rise while democracies fall, trust someone else who is going to protect you from the doom, have you ever been in a courtroom, people there talk about weighing the evidence, but how much does evidence weigh, and I don't just mean that to be clever, the fact is that we don't make decisions by weighing evidence, we make decisions neither by weighing evidence or on the balance of probabilities but by explaining the evidence before us, forming a single good explanation, if this is not mere semantics, this is an important understanding of how knowledge itself is formed in the relationship between knowledge and evidence, evidence the thing that we encounter that we seek to explain, there are alternatives out there and I did say I'd come back quickly to what is known as Bayesianism, it's all the rage right now and it stands in contrast to the kind of epistemology I've just been speaking to you right now, it says that we can assign probabilities to the options before us and calculate which is most likely, but as I have already said, it's very, very rare to have many options on the table, Newton versus Einstein, you're lucky to sometimes have one much less two and during that debate the decision as to which of these theories of gravity was actually the one we were going to go for did not come down to facing bets or a matter of probability but a crucial test and experiment decisively ruling out one, Newton's theory never contained, never was true, so it wasn't probably true, it contains something true about gravity but we can't quantify that. Bayesian theorem here has, you don't need to know about exactly how it works but PA for example is the prior probability, so when you roll a dice, we know the prior probability is one in six of rolling a three or any other number for an ideal dice by the way and no dice is ideal, they are physical objects, they're not mathematically platonic ideals but that's in the case where you actually know beforehand what the possible outcomes will be but what is the probability of, for example, there being an earthquake here tomorrow, this is apparently what Bayes' theorem is useful for us to form decisions about, we don't know and looking at past trends is not going to indicate future performance or future movements of the earth, it's the case may be, in that situation it's not a matter of probability but a matter of good explanations, so build your structures in the case of Auckland knowing that earthquakes can happen, do happen and will happen, things don't probably happen, they either happen or they don't happen, rational decision making is not a matter of weighing the evidence, it's a matter of explaining, decision making is not about calculating probabilities, it's choice creation, we can create an option better than all the others known and we can choose it, that is rational decision making, an act of creation, not calculation, it is a matter of creating understanding, understanding of good explanations, this does not mean the good explanation must be true, it just means it must be good to you, you're choosing this and not that because to your satisfaction in your mind you can say you've considered all of the other options and refuted them, ruled them out decisively and only one remains, the one you act on and you have created in your mind as being the best, that's your choice, it's the rational thing to do, so you create the choices you make, as you sit here today you may very well be thinking to yourself what am I going to be doing for the remainder of my time in Auckland if you're a visitor, even tomorrow you don't know in large part although you can kind of guess, we don't know in the privacy of our own minds what's going to happen tomorrow, what opportunities and challenges will crop up over the next 24 hours and that's you in your own mind trying to predict your own behaviour and your own thoughts just for tomorrow, that thing which you know better than anyone else, so what hope do any of us have in predicting what someone else will do, let alone entire civilisations over the course of months and years, so don't tend in the direction of pessimism, be careful of the soothsayers, prophets and forecasters, don't prophesy, the rational thing to do is to understand our circumstance now and when you do that then you can decide and not among fixed options but also from that which you create, the future is genuinely open and therefore the rational thing is to be optimistic about our situation because problems while inevitable are soluble, thank you. [Applause] [Music]