AI Researcher Explains Why Breakthrough Innovation Cannot Be Planned (Novelty Search Theory)

Kenneth Stanley Novelty Search [00:00:00] I don't know if like what level of artificial intelligence will happen, but there will be progress certainly and it may be very, very big progress. If there can be, there probably will be eventually. And so the question is, you know, how, what should we do about it now? Like, like I don't think the answer is to give up. I am Jeremy Ash, a retirement community CEO, living in Eureka, Illinois. And you are listening to the Vance Crow podcast.

Vance Crowe [00:00:30] Welcome back to the podcast. I'm glad you're here. Today we talk with a man named Dr. Kenneth Stanley who wrote a book, why Greatness Cannot Be Planned, the Myth of the Objective. And when I read that book about six or seven years ago, it changed the trajectory of my life. And for many years Dr. Stanley and I have been in contact over email. I'd write him something that I thought and I would always be delighted when I would hear back from him. But now that the podcast is going and it's strong and it's got all sorts of robustness and big conversations going on around it, I decided that I would reach out to Dr. Stanley and invite him onto the podcast. It turns out that he has left his role as a professor at the University of Central Florida and is now a research manager at Open ai, an artificial intelligence company that is really pushing the boundaries on how we can get computers to evolve and learn on their own. And so what you will hear in this conversation is an in depth but completely approachable conversation around what is artificial intelligence? Where is it going and how do we know if we have control over it? But after we get through some of these topics, we start diving into what Dr. Stanley knows from a personal side. We talk about raising kids, we talk about how hi, what his understanding of evolution and artificial intelligence and curiosity and the novelty search have impacted his life and what he thinks it can do for other people.

Vance Crowe [00:02:04] This is a fantastic conversation and I am delighted to bring it to you if you're enjoying these types of conversations. You've probably heard me mention the Articulate Ventures network and just last episode I mentioned a Dunbar number. We know that the network is valuable because we've developed a community and we have a culture that we're trying to make possible to grow. And we know that if the numbers grow too quickly, we won't be able to keep what makes it special intact. So we have decided to limit the membership to 45 people from the last time I mentioned it till January 1st and we've already had a bunch of people sign up. So if you've been thinking about it and you've been wondering is this the right community for me, when should I do this? I would recommend doing it now because we are going to fill up and I'd love to have you there. So if you'd like to learn more, go to network Articulate Ventures where you'll find all sorts of interesting people, activities, and a way to find a shelter from the tragedy of the Commons that is the rest of social media. So without further ado, we are gonna head into this fascinating conversation about artificial intelligence, the novelty search, and a conversation with a very interesting man, Dr. Kenneth Stanley. Ken Stanley, welcome to the podcast.

Kenneth Stanley Novelty Search [00:03:28] Thank you Vance. Great to be here

Vance Crowe [00:03:30] Man, I have never looked forward to an interview more than I'm looking forward to this one right here. And that is in no way an exaggeration because your book Novelty Search, why Greatness Cannot Be Planned sent me in a totally different trajectory in my life. And I would say there is no end to the number of things that changed because I read that book. So I have so much I want to talk with you about.

Kenneth Stanley Novelty Search [00:03:56] Ah, well that is exciting to hear and and definitely makes me, makes me really glad to be here to have this opportunity.

Vance Crowe [00:04:03] So let's start off for a lay audience that has never heard of this concept, novelty search, how do you open this, this very complicated idea up to people that it's a totally new idea?

Kenneth Stanley Novelty Search [00:04:16] Sure. So novelty search is actually the name of an algorithm, but I don't want that to be distracting. So algorithm and computer science and stuff like that. 'cause the interesting about this algorithm, whereas most algorithms are kind of obscure things that only people working computers would think about. This algorithm actually had implications for like how you might live your life or how we might think about changing things the way that we do things in society. And that's what's I think, unique about it and probably why, why, why you're thinking this could be, this could be an interesting show. And so novelty search was about a realization that we had, which we, we, we found out because of doing experiments with it with certain kinds of algorithms that sometimes the best way to achieve something is to not be trying to achieve it. So I guess that's takes a little bit to digest. So just to say that again, like sometimes like it's better to actually not be trying to achieve the thing that you ultimately hope to achieve if you want to achieve it. And so this is a very counterintuitive thing. It almost sounds like zen or something like that. But we found it to be true in a lot of experiments at first with computers and later we saw how this connects to actually doing things in the real world. But to, to actually connect it to the, to the computer experiments, just to make a, a brief introduction of what actually is about it basically is saying what we're gonna do since we're not gonna actually try to do something specific, instead we're gonna just try to do something new or novel, which is why it's called Novelty search.

Kenneth Stanley Novelty Search [00:05:54] And what we're gonna always be trying to do every step of the way is to try to get more different from what we've done before than we are right now. And what this should do then is it should lead to us making discoveries of things, things that we wouldn't have realized are possible if we had just been trying to move directly towards some objective. And the funny thing is, often we might fail if we tried to move towards a specific specific objective because it might be really complicated and hard, but if we were doing novelly search, we actually might end up solving some problems. Maybe there are problems we didn't expect to solve but that we would not have been able to solve if we were trying to solve them.

Vance Crowe [00:06:34] Yeah, it's one of those things that when you start to break it apart, at first you're like, well then how can you use this to get anything done? Because if I don't have an objective and I don't know what the other side of the river looks like, then what does that even mean for how I should move forward?

Kenneth Stanley Novelty Search [00:06:49] Yeah, yeah. So, and that, that's a common question about this. And so the first thing that, to answer that question I wanna do is just clarify that we're talking about really ambitious, really blue sky types of achievement here. Like how am I gonna do something that has never been done before? So like when we're talk, if we were talking about more modest things like I need to get breakfast this morning, like then this doesn't make any sense at all or even less modest than that, but something like, I we're gonna upgrade this software to the next version, like maybe for that, like that should be an objective and we'll just do it 'cause we basically know what the steps are. But if we're talking about things that no one has ever done before, then the problem is that we have no idea what the stepping stones are on the road to actually getting that thing done. And so it makes sense if you think about it, even though at first it sounds very con counterintuitive, but it makes sense that because I don't know what the steps are that I need to go through to get to this very unusual endpoint, then maybe what I should be doing is finding stepping stones, like things that might actually be useful for getting somewhere new rather than actually trying to get to this point. Because I actually have no idea, I have no roadmap towards how to get there.

Vance Crowe [00:08:06] So when you think about those big blue sky ideas, like I the, some of the ones that were interesting from the book are like, how do you make a robot walk? Or how do you teach a robot to figure its way out of a maze? But then how does that actually apply,

Kenneth Stanley Novelty Search [00:08:22] Right? So those are examples where we actually ran this computer program or algorithm to try to solve those problems using this principle. So the program was actually not trying to make the robot walk. Instead what it was doing was it was saying, okay, here's a robot and I'm going to try to control it to do something it hasn't done before. And what was interesting was that even though the concept of walking is nowhere in that formulation, eventually it just stumbles onto the idea of walking anyway. Because if it keeps on trying to do new things, eventually to continue to do new things, you have to actually start walking. And that's just to, to make a, a, a, a, a general statement about a more, a more general kind of principle, which isn't just about algorithms, which is that like many things are like this in life in general and also in the history of discovery, like human discovery or human invention. So like one, so getting away from like computer programs and teaching robots to do things like something, so like a concrete example from history would be sort of like the invention of the computer. Like if you wanted to create a computer and it was the year like 1800 or something like that. Like that would be, you have no idea what to do first. What should we do if we wanna build a computer? Well, what's funny is that people were during that century actually playing with vacuum tubes and they were getting more and more sophisticated with those vacuum tubes and, but they weren't doing it because they were trying to make a computer. They were interested in these vacuum tubes for totally independent reasons.

Kenneth Stanley Novelty Search [00:09:57] And yet eventually it turned out that you needed vacuum tubes to build the first computer. So the thing that was needed to make the computer was built by people who were not trying to build computers. And this principle just occurs over and over and over again in the history of invention and discovery. And in fact, what's interesting, even more crazy about this, if you really think about it, if we had gone back and found the vacuum tube scientists who are like looking at these vacuum tubes and exploring them, and we had said to them, you know, like, you're kind of wasting your intelligence here. Like maybe what you should be doing is trying to build a computer. Like why are you playing with these electrical experiments when we could be building this amazing thing that would like totally revolutionize the world? And if they had actually taken that up, we wouldn't have vacuum tubes, but then we wouldn't have computers either. So it was like, it was important for them not to be trying to build computers in order for us to eventually build computers, which is just a more real world kind of illustration of this principle.

Vance Crowe [00:10:55] So I, when I read through your papers, you start talking about how this compares with evolution or co-evolution and these different concepts. And it's an interesting point because evolution, when you, when you backwards engineer it, when you look at like, hey, where'd it go? You say, oh, it looks like it was intending that the, that you were gonna get these biped, you know, apes and then they would, they would turn into humans. But that's not actually the intention wasn't there, it was like each evolution gave it some level of competitive advantage and then that competitive advantage then was allowed to flourish. Is that the same thing? Or how do you, how do you make that bridge?

Kenneth Stanley Novelty Search [00:11:38] Yeah, there's, there's a really strong connection to evolution with this novelty search kind of thinking, which is this, the realization really is that these, these kinds of achievements that are like really blue Skype kinds of achievements, like a computer or something like that, they generally happen because of long chains of innovation that had nothing to do with that final discovery. So like the vacuum tubes, like they weren't, people weren't seeking computers when they were seeking vacuum tubes. And actually we see this principle in evolution, maybe most starkly that the stepping stone organisms that led to some, some, some of the most amazing discoveries like say the human brain or human intelligence, there was no way that you could predict that this would lead to that. And in fact, the reason that this appeared had nothing to do with that final thing. So like, to give a more specific example from evolution, there's, there's an ancestor of ours which is very, very, very far back, but it's a flatworm. And in fact I believe that the flatworm is the first time that we see bilateral symmetry like in a lineage in, in, in, in the tree of life. And so this is interesting because it, it appears like in hindsight that like you needed to get bilateral symmetry somehow to get to humans. So like you needed bilateral symmetry to get Shakespeare, but like if you were looking forward, that would be preposterous. You wouldn't say, well we need to get bilateral symmetry, like of all the, maybe we should be administering IQ test or something, but not going for geometric symmetry. Like that doesn't make any sense.

Kenneth Stanley Novelty Search [00:13:09] But the thing is, if we had thought that way and said, oh well, well maybe we were like a master breeder, like back in the, in the, in the days of Flatworms, we would just like kill off the flatworms, be like, this has absolutely nothing to do with what we're interested in here. And yet, so the reason though that the flatworms were preserved is 'cause it didn't care about human intelligences at that time. It wasn't trying to do that. And in fact that's kind of the explanation for why all of the amazing things in nature exist because none of them were the goal, which is a really interesting thing to think about. 'cause we do tend to think about processes of discoveries being goal oriented. But if you think about the greatest process discovery at all, which is probably the evolution of all of living nature, like this one run of something that produced everything that exists that's alive, that was not trying to produce any single thing that ever actually came into existence. And that's actually the explanation for why they exist. Because if we were trying to produce any specific thing, then all the things that led to that thing we would've eliminated because there's no way we could predict that they actually lead to that thing. So it's very important basically to keep an open mind.

Vance Crowe [00:14:13] You know, I've, I've been wanting to ask you this question about have you read or heard of this book called The Three Body Problem?

Kenneth Stanley Novelty Search [00:14:23] I did. I read that someone told me I should read that. Yeah,

Vance Crowe [00:14:27] It seems like it's a book that's like perfect for you. But then as I'm thinking about it, there's a key function in the book that is actually the opposite of what you think because without giving it away any of the details, it's essentially an alien race that is facing the way that these three stars are completely unpredictable as to when they will rise on this, on this planet at the same time. And if you have one star, it's a stable period. If you have two, it burns up everything. If you have three, it's complete chaos. And they keep running a simulation saying, what could we do to get past this? And, but they're trying like in, I think this is your phrase, like a fitness landscape where they're, where they just keep going for like what would be the one choice we could make that would be different And they have to just keep running it again and again and again because they can't figure out how to beat the three body problem.

Kenneth Stanley Novelty Search [00:15:19] Yeah, that's interesting that it, there is a connection there. Yeah. And other people have pointed that out to me. Yeah. Which is why Yeah.

Vance Crowe [00:15:27] So as if, as a researcher, so you're in artificial intelligence, you were at the University of Central Florida and now you're at open ai. When you start applying this to your own career, does this make you have like mental breakdowns now because it's like you want, you wanna have, you know, human beings, we wanna have progress, we wanna know where we're going. How do, how does, how does somebody like you then begin to integrate this into your life?

Kenneth Stanley Novelty Search [00:15:54] Well it's, it's true that it, it sounds, it sounds counterintuitive, but the, the thing about this, this kind of idea, this principle is that it always seems counterintuitive, but then if you really think about it actually makes sense. And so like the way that I think about it is that instead of like if I was trying to do something like I guess trying to create AI that would be very objectively driven, that would not be like a novelty search. And then I would basically need to say there's some measurement of progress. Like here's what I'm trying to do on the road to this final far off achievement that I'm trying to get to. But, but I can approach that differently because I mean probably that's not a great way to do things in general just because it is, it is such an ambitious thing, it's kind of blue sky that we may not know what the stepping stones are. And so another way of thinking about this is instead of trying to make progress, you can try to open up playgrounds like new places where tons of low hanging fruit might exist, which then lead us to gaining huge amounts of insight that can lead us to another step. So by playground I kind of mean stepping stone, but I kind of, I call it a playground 'cause it's like a place where you can play around and try new things. And so I think I try to seek out like new playgrounds, like something that is like involve so much possibility that it would just be really interesting to explore there as opposed to, oh, this is better than something else. Which is a more kind of objective view of things.

Kenneth Stanley Novelty Search [00:17:25] And, and so it's really about being keyed in on what's interesting at any given time. And that word constantly comes up when talking about this, this novelty search stuff. Is the word interesting or interestingness? It's like that's another kind of way, maybe a more better way than saying novelties saying what's interesting, like I do things because they're interesting, not because they lead towards something where I know where they're leading. 'cause often things are interesting and I don't know where they lead and that's why they're interesting and I'm willing to pursue things that are interesting. And I also believe that that's not a random statement. Like some people are like, well it's just totally random, you just like it. You know what, who's to say that this is actually useful? But I think that if you think something is interesting and it's in an area where you know some things like you're not completely ignorant, then probably, you know, we should take you somewhat seriously. Like you have an instinct. That's one thing humans are good at. We have like a nose for the interesting and we can follow those intuitions and see where they lead.

Vance Crowe [00:18:24] When, when I read your book, I had not dove deeply at all into philosophy. I mean, I'd taken a few classes in college and I just kind of stumbled around, but in the last few years I've done a bunch of reading into Nietzche and, and Gerta and Carl Young. And I would say that as I read what they're talking about and as far as being interested or finding your Damon, it all appears to me that they are saying the life well lived is the one where you allow your curiosity, your the thing to capture you and then become inhabited by it in some way. And I don't even think they mean it in some like cosmic religious way. They're basically saying there is something deep inside of you, that voice that says you should go over there. Even when everybody tells you, Hey, that's a, that's a bad idea. You shouldn't go over there. And there was one day when I was sitting around with my buddies and we were talking about novelty search and it cracked into my mind just like suddenly novelty search is the damon in some, in some way.

Kenneth Stanley Novelty Search [00:19:31] Yeah. That that is an interesting insight. It does seem compatible with that idea. And I, I know philosophers have said some, some related things. I think we have a quote, a quote from a couple of them in our book actually, that, that we used like to, to just represent the, the ideas and novelty searches that they're not, they're not actually completely new. I mean, people have had thoughts like this for a long time that there's this kind of liberating way to free yourself and just pursue what you feel deep inside is the right thing to pursue. And there may not be an objective justification, like why are you doing this? Like why are you going to art school? Like when you could just become a doctor, you know, but, but you just feel deep inside. Like that's just what is I meant to do? I just feel like I should do this. Which I, I mentioned that since I visited an art school and met all these artists and they were telling me about this and how the book made them feel better because they couldn't objectively justify like to their parents or their teachers, like, why are they going to art school? And so I I think, you know, the what's diff what's new here, it's, I think for a long time, you know, maybe millennia people have had this feeling like that you can, there's a kind of way to liberate yourself where you just follow your passion. But I think what's new here is that what we're saying is there is a principle justification for that. Like it's not just some kind of fluffy, well I'm not saying that they're being fluffy, these philosophers are

Vance Crowe [00:20:56] Sophisticated, but

Kenneth Stanley Novelty Search [00:20:58] It's not just philosophy. Like we've got experimental evidence of why this does lead to more interesting discoveries. You know, because of the fact that we don't know where the stepping stones lead. Somebody needs to go down those unknown stepping stones and find out where they lead. And it's true, they're not always gonna lead somewhere good. I mean that's, that's the price of exploration. But on the other hand, the price of not exploring is that you'll never find all those amazing things that are out there that we don't know the stepping stones to lead to them. And so it is, it is the new thing that maybe the new insight here is that yes, this is a, this is a appealing philosophy, but it's also principle to actually follow the philosophy. It's a way, it is the way that we make great discoveries and therefore you should not just feel good for yourself, but you should feel good that you're justified in actually doing it.

Vance Crowe [00:21:48] Your book was a, a very odd blend of being, you know, computer science ish, but then also kind of self-help would be a little bit too fluffy in and of itself. But did you but who are you intending would read that book when you wrote it?

Kenneth Stanley Novelty Search [00:22:05] That's a good, yeah, that's a really good insight. The book is like that. And yeah, this was a real struggle in writing the book like that we, we wanted to reach like a really broad audience, but I was also worried about watering down the argument. Like there's a scientific argument in there, which is based on these computer experiments and that's like very computer sciencey. And the thing is like what I, I realized that the, that the ultimate conclusion is like very self helpy in a way, but that on its own, it's, it's just like, just like a one of a million self-help kind of books. I mean it's just gonna be like, you know, go follow your passion and do what you really wanna do. Like good for you. Here's a pat on the back with this book. But I wanted it to be like a very, a serious scientific justifications of the book in effect would be like a weapon. Like you could really, 'cause there's a lot of people who won't let you follow this kind of path. Like in practice, which is one of the reasons we wrote the book. Like my perception is that society, like our society and our culture is totally about objectives. Like from when you're just a little kid. Like we're driven by metrics and objectives like measurements and whether you're going on the right path. And then we know where you're supposed to get when you get to the end point and every step of the way you have to justify yourself. You're justifying yourself to your boss, to your teachers, to the people who provide you funding, whoever it is. It's all objectively driven. And so I think that that is to me is the status quo. And the book is supposed to be a, a weapon to argue against the status quo.

Kenneth Stanley Novelty Search [00:23:37] And so for people who feel like things need to change in their organization, in their school, in their discipline, like if you think think things need to change or even in your family, like it wouldn't, I don't think it would be effective just to have a self-help book that makes you feel better about yourself. I think you want something which is really, really grounded in serious scientific evidence and arguments. And then you can use it and you can say, look, you should look at this book. This has a very serious set of evidence that justifies the choices that I'm saying we should really make, why we should change as an institution. And then when going through that reasoning, I realized that it comes out to this bizarre amalgam of things. It's like this self-help slash like science book slash computer science and I and Joe

Vance Crowe [00:24:25] That pissed every, it must have pissed everybody off, right? There's no, there's no publisher that was like,

Kenneth Stanley Novelty Search [00:24:29] Yeah, exactly. We, we really struggled. I mean we, we had an agent for a while who was like, kind of works in the, like the popular publishing like, like regular publishing industry and you know, he was, he was suggesting things to water things down that would make it more appealing to like mainstream publishers. But then I was like, I don't, I felt like that's like selling out in a way. I mean nothing against the agent. I mean he was a good agent. I mean he had good, good suggestions if he wanted to do that, but I just couldn't bring myself to to, to water it down because I just wanted it to be like the definitive argument. So it, it turned out to be a not very well positioned from a like a marketing perspective, you know, it's like, it, it's not really appealing to anyone in particular, any demographic really. It just is the argument that I wanted to make that we wanted to make Joel and I, and, and we tried to make it as like palatable as we could, but, but obviously there was some compromise there,

Vance Crowe [00:25:23] Man, to hear you describe it as a weapon, it makes me so happy because that is exactly how I use that

Kenneth Stanley Novelty Search [00:25:30] Because

Vance Crowe [00:25:30] I was in a situation, so I was working for Monsanto, which, you know, to the wider world, they think the, the Monsanto is an evil corporation. We want nothing to do with anything that they, they have to say or do or anything. And the company itself is like, we have brought forward technology that has revolutionized things. We, we have, we have scientific endeavors here that would blow you away. We have even Nobel prize winning scientists that have discovered things. Why does everybody hate us? So the company was totally embattled when they hired me, which was the only reason they would do something like that. And I got in there and very quickly found out that if I was going to play the game by the rules that everyone else played the game by, I was certainly going to fail. And when I came across your book, because I was looking for a language to describe why the strategies that they were using and that their PR company was telling them they should use, but just doing them harder or doing them with more money or doing them faster was not going to work. And once I had that and I had this concept of you can hit a local optima where it doesn't matter that if you keep going, unless you're willing to go down and and search for something that is not as efficient or as effective, then we're not gonna be able to make anything happen. And I used your book to describe why we should do a Reddit a MA and like at first the whole corporation was like, no, absolutely not. Fortune 500 companies don't do this.

Vance Crowe [00:27:03] And a long story short, like I got one put together and we did a science Reddit a MA and from that it was, it was run at about a month apart from when our Super Bowl ad ran. And it turns out that the Reddit a MA captured more people's attention for longer than a Super Bowl ad that they had run. And so it was like demonstrable bang, like, okay, now we can do something that's totally different. And when they asked me why do you wanna do this? Or what's gonna be the outcome? I'm like, I have no idea. I think we'll find somebody on Reddit that will think what we're doing is interesting. And all of these other things started cascading from that. And so we were able to reverse the polarity where they, where at first they were like, what we should do is pay to go get on stages. Let's sponsor conferences and then we'll get five minutes to say our piece.

Kenneth Stanley Novelty Search [00:27:53] And

Vance Crowe [00:27:53] Instead we, we got into the position where, hey, if the world thinks that you are North Korea, then you don't have to pay to get on stages. You just have to say, I'm willing to come talk and I'll answer any questions you have. And they're like, oh, here you go. We've got a chair right here. We've got lots of questions for you. And so for me, when I had the the thoughts from your book, it, it gave me a language to be able to describe to a scientific business community why we should try these things. And for a long time until I eventually went over the edge, I I was able to do things that, that they never would've let me do had I not had the language of that book.

Kenneth Stanley Novelty Search [00:28:33] Wow. That's a great story. I mean that's, that's gratifying to hear that that's what the book is for. Yeah, it's, it's it's supposed to get empower people to do that kind of thing.

Vance Crowe [00:28:43] So there's a bunch of like more abstract things that I want to talk about and I'm glad that we've been able to kind of talk about your book and kind of this concept of novelty search. But before we go too far, like who are you Ken Stanley? Like we, we, I never let you introduce yourself or what kinda work you do, I just jumped right in.

Kenneth Stanley Novelty Search [00:29:03] Yeah, so I, I mean my main work is in artificial intelligence and I, I've been interested in AI since I was a little kid, like around eight years old. And I was interested in, I was interested in, I wanted, when I was eight I wanted the computer basically to have a conversation with me at that time. And I wanted to kind of program a chatbot is really what I wanted to do. I I could program a chat bot, like I used basic 'cause that was a computer language on my Commodore 64 computer. But it, the problem was it was boring. Like, 'cause I knew what it was gonna say. So I was like, that was really kind of disappointing, you know, say, Hey, what's your name? I'd say, Ken, I'd say, hi Ken. At first it's like, wow, I got it say actually it's responding to me and saying hi. But then after a while it's pretty boring 'cause it always says that. And so I think like I just, even back then I realized I wanted to be surprised. Like I wanted to be surprised and like when as I moved forward in artificial intelligence research, I, I think I always was like leaning towards, I wanted to build a system that would do something I didn't expect, which is so related, you know, to like novelty search and open-endedness and the kinds of things discussed in the book. I guess that it's not a coincidence. I mean even though I, I wouldn't have been able to say anything like this book back then, or, or even like, you know, when I was in my twenties. But like I, I was just mo I guess there's something about my personality that is very open-ended leaning and doesn't like being told that I have an objective.

Kenneth Stanley Novelty Search [00:30:39] I guess I, I'm a little rebellious. I think I think I should be able to do things just 'cause they're interesting. I, I get really annoyed when people want like more justification than that. And so somehow this is, this is somehow suiting my personality, but it's kind of surprising 'cause I never was trying to, it is just like a coincidence that like we, we, we eventually discovered some, some kinds of technologies that, that seemed to justify my way of thinking Anyway, maybe it's not actually a coincidence.

Vance Crowe [00:31:10] When was the first time you that a computer did something that surprised you and you had to go and say like, whoa, how did you do that?

Kenneth Stanley Novelty Search [00:31:19] That's a good question. It's probably like experiments with what you might call artificial evolution or they call art evolutionary algorithms. I started getting interested in those in grad school and it was at the University of Texas at Austin. And in those you're, you're basically doing like breeding on like artificial brains that are controlling things like characters in, in some, like you could think of it like a video game and, and then quickly things start happening that you don't necessarily expect, which is probably why I was so drawn to them. Like, 'cause they, they tend to produce surprising behaviors that you wouldn't expect. A lot of times they're like loophole exploitation. Like you'll find out that there was something in the game that was like a way to cheat and you didn't even realize it. And then it'll just quickly lock in on that and find that. And that's, that's not really what you wanted. But I found it kind of satisfying 'cause it's so surprising. It's like, wow, this thing is thinking. It's like it came up with something I never would've thought of. So yeah, there, and there's actually recently I think Joel led a paper that has like dozens of authors, like my co-author Joel Layman from, from the book. But it was a paper about like the surprising creativity of digital evolution, I think it was called something like that. But it has like a huge like laundry list of surprising things that happened in all these different experiments that different scientists did. So it just, it's a kind of technology that tends to produce a lot of surprises, just like evolution on Earth is very full of surprises.

Vance Crowe [00:32:51] Yeah. And like evolution on earth when you start breaking it open, when I read Matt Ridley's book, the Red Queen, where you start figuring out like what were the competitive strategies that allowed something to, to, and they, and it's, it's funny to even call them strategies, right? Because strategies implies that they knew what they were doing in some way, right? It's it's, it's almost like water seeping into the cracks that were there that, that nobody even realized the, the cracks were there that the water could get in that deeply.

Kenneth Stanley Novelty Search [00:33:22] Yeah, yeah.

Vance Crowe [00:33:22] But like when you start looking at the, the different way the branches that have come off in that kind of fractal radiation concept, there's, it's, it's, it's an, it's an endless pit. You could just fall down inside of those different evolutionary strategies forever.

Kenneth Stanley Novelty Search [00:33:39] Yeah, yeah, yeah. I like the water seeping down metaphor that, that's similar to, I mean, I sometimes think of it as like spilled milk. So it's actually a similar metaphor. It's like, you know, if you spill milk on a table, it will just go everywhere. It can go eventually unless there's a barrier. But otherwise it'll go around the barriers. And so that's kind of more what I think it's like basically what evolution is doing is just showing everything that every possible way there is to make a life or to make a living. And all the things that could be eventually will be if you wait long enough for the milk to spill. And so it's, yeah, it's, it's revealing, it's like a, a revelation of like the space, the surf space of what's possible gradually over eons. Like it's just gonna reveal what's possible and that's, that's an open-ended process. Well,

Vance Crowe [00:34:28] And weeds are a fascinating and insects as well when you're talking about crops because as soon as you come up with a solution to be able to stop them, then, then they find some way out of it. So, you know, I oftentimes when I would go out and talk about pesticides and you would say, well, okay, if you don't use pesticides, let's say you just send human beings out there and those human beings pick the bugs off of the plants eventually, and it doesn't take very many generations, maybe two years, you start finding that the insects will begin to look more and more like the plants that they're at because those are the ones that the human beings didn't grab. So they had some slight variation of green that made it so it looked like a leaf so you couldn't spot it. And and weeds do that exact same thing. As soon as you find something that can spot 'em, then they're gonna find a way to wrap around that in

Kenneth Stanley Novelty Search [00:35:19] Some

Vance Crowe [00:35:20] Interesting, unique way.

Kenneth Stanley Novelty Search [00:35:21] Yeah, that's the milk spilling around the belt, the barriers there. Yeah. That's, that's interesting.

Vance Crowe [00:35:28] So when people hear this concept of artificial intelligence, I think most of them think of it as like either weird sci-fi or a a a legitimate threat to human beings. Like they're going to outthink us. And so it's a little bit like, you know, letting loose a nuclear bomb or something into society. How do you think about the, the beyond your curiosity, are you worried about letting something loose that will eventually transcend us?

Kenneth Stanley Novelty Search [00:35:58] Yeah, I think we, we, we should be worried about it. It's, it's, it can have really good effects. It can have very negative effects. It's, it's a, it's really the bottom line of it is it's just like completely impossible to predict. Like we have no real clue what it's gonna do to, to us. And so, but, but it, the thing is it's probably going to happen. Like that's unfortunately the case. Like I don't know if like what level of artificial intelligence will happen, but there will be progress certainly, and it may be very, very big progress. If there can be, there probably will be eventually. And so the question is, you know, how, what should we do about it now? Like, like I don't think the answer is to give up because the problem is that not everyone will give up. Like some people will always be pursuing it no matter what. So if the only people who give up are people who are like worried like maybe something bad will happen, then the only people working on it are people who don't care if something bad happens. And that seems like a really bad idea. So I do, I do like I've, when I've thought about it for myself, like, you know, should I not be involved in this? 'cause something bad might happen, but then I'm like, but, but if, if the people who are worried about something bad happening all just decide not to be involved, then who does that leave? It's just people who don't care about that. So I do think that that we should pursue the, the potential good outcomes as, as participants in the process of, of, of making advances in AI and hope that we can then help to sway like the overall community, people working in this to, to move in the directions that are safer and more beneficial to humanity.

Kenneth Stanley Novelty Search [00:37:42] Because there, there probably are a lot of benefits that we can have and, and, but there's gonna always be some uncertainty

Vance Crowe [00:37:49] When you're sitting around with your friends and you're having a highly open conversation. What are the dangers that you guys have elucidated so far?

Kenneth Stanley Novelty Search [00:37:58] Well, I guess that the, the more near term dangers probably have are more economic. I mean, it has to do with things like jobs and job replacement or loss of loss of jobs. And then in the long term you have like, or like radical re designs of the entire society. I mean that could happen like if AI can do anything that a human can do. And then you have like terminator like scenarios and stuff, which I think are maybe not as, they seem a little bit too far off I think to be, to be immediately worrying like that seems like the most popular focus of like science fiction is like Terminator style stuff. But I don't know that it's that likely of, of a real worry. But it, it is something, at least you wanna be aware that like if you don't want ais to try to do something to hurt us. So yeah, there's a whole spectrum of bad things that, that are worth considering. And I think like that there's, I mean the best thing to do is to really seriously try to engage with these bad outcomes. At the same time as doing scientific research to advance the technology we have to do everything. Actually OpenAI does try to do these, this, I mean they really try to think about these things too. The place, the new place that I'm at right now.

Kenneth Stanley Novelty Search [00:39:31] So

Vance Crowe [00:39:31] What is open Aim

Kenneth Stanley Novelty Search [00:39:34] OpenAI is, is a company that is researching artificial intelligence and is probably, it's probably fair to say, it's one of the leaders in that. And so their interest is to actually have the benefits of this technology go back to humanity. So that's what they're, at least that's what they're trying to do.

Vance Crowe [00:39:59] So I think one of the weird challenges about AI and the dangers of it is that it could be like, you know, the metaphor when they say the cells double every, you know, minute and when does the glass, when is the glass half full? You know? And then the next time it doubles, now it's all the way full. So it's like you don't really see it coming, but because it's exponential growth, it's like happens in an instant. And that's the part that to me is, is a little bit freaky about ai. It's almost like you could get to a point where you don't realize you've gone to the point of no return and now it's now it's increasing its ability so quickly that we can't even fathom it.

Kenneth Stanley Novelty Search [00:40:41] Yeah, that is, that is a a, a kind of puzzling aspect of it is this like potential to just kind of jump out at us when we may not think it's as close as it is. And that means we have to be, we have to be ready to anticipate that and we have to try to estimate how far that is by understanding how these kind of exponential processes work and then be out ahead of it hopefully. And before that actually happens, I mean obviously part of it happening is us, like we we're actually the ones making things so we can try to slow ourselves down, but you can only do that so much. 'cause again, there's, there's always somebody else who's willing to jump in and actually do the thing. Just 'cause you stopped doing it doesn't mean someone else is stopping. So it is a bit of a race I think to get out ahead of this. But we don't know when it's really gonna happen, obviously. I mean still it's still still speculation.

Vance Crowe [00:41:37] And so when you think about the positive sides of ai, what comes to mind as, as the major benefits?

Kenneth Stanley Novelty Search [00:41:44] Well I think one thing is it can replace a lot of the most dangerous types of pro jobs that human beings have to undertake. And so maybe all these places where humans risk their lives in, in theory machines could be doing it instead. And that's nice. It could be, it could help with ensuring that certain kinds of resources are always plentiful for us. If there are things that, where different parts of the world are under-resourced in different ways, maybe in terms of food, you're more of an expert than me, but AI can come in there and actually provide the labor and the logistics to actually make things available that need to be available. And so also I think in creative endeavors that we can imagine, and this maybe is less essential or important to, to our wellbeing, but it is still important. I mean, creativity is a big part of like who we are. Like if you think of art and music and stuff like that and it can, I think what it can do is it can allow us or regular people to be more of a part of the production process as opposed to just consumption. 'cause most of us just consume this kind of stuff. But we can unlock I think a lot of our kind of implicit preferences about like what appeals to us and what doesn't appeal to us with AI and make many, many people into artists that aren't right now into really good artists.

Kenneth Stanley Novelty Search [00:43:25] And that can be fun and change like the whole landscape of like creativity.

Vance Crowe [00:43:30] Wow, that is super interesting. I I did not expect that. So what, what do you mean like, I mean it's not gonna make people piano players. What, what, what will it do instead?

Kenneth Stanley Novelty Search [00:43:40] Well, I think it, it might, well it, I guess what I think is, there's two things. Like first is you, there is, there's a difference between implicit and explicit knowledge. Like explicit is like the ability to actually make something that you want to make. Like you wanna make a really nice picture like, you know, explicitly how to do it. And, and most people lack that. We don't know how to do that except for like really good artists, but implicit knowledge, like we all have like, almost anybody can look at a picture and say whether it's good. And so that means that there's a lot more, you have a lot more ability with respect to art than you actually can explicitly express. 'cause you can't really explain like it, why, why is this a really good picture? I mean, if you could then maybe you would just draw it ultimately, but, but you can actually, but you can judge it. And so I think AI has the potential to flip this implicit to explicit. Like, because there is that knowledge inside you. And so if we could get it out of you elicit it in a way that then it could be converted into actual explicit stuff like art, then we could actually create the things that you think are good, that really are about you. They're actually a form of self-expression. The other weird thing is just that you might have some explicit abilities, but there you don't have the right context to actually express them. So like you might be able to hum a nice tune, you know, but, but like you don't know what to do with that. Like may, maybe you could even come up with a new tune I think.

Kenneth Stanley Novelty Search [00:45:13] I think some, I don't know how many people could do that, but I can do that. I can, I can just hum it completely new tune and, but I'm not a musician. But someday just hum it into the AI and it will create the piano and it will, it'll give you the whole band it, we'll create a whole symphony around it and it'll fix all the rhythm and everything that's imperfect about what you did and it'll be your creation. Now if everybody could do that, I mean, could you imagine like the, the the gigantic like typhoon of just new art and music that we coming out constantly, like everybody's implicit abilities would be just like amplified. It would totally change the landscape of like all, everything.

Vance Crowe [00:45:51] Well this reminds me of a study that was done one time about, about art. So they brought a group of subjects into the room and they showed them two pieces of art and one was abstract and clearly somebody had spent a lot of time on it and it was, you know, interesting. And the other one was like one of those kind of hang in there, kitty kind of obvious pieces. And they said, okay. They, they broke 'em into two groups and the, to the first group they said you can have either painting, but you have to explain why you chose the painting. And it turns out that a huge percentage, something like 80% of the people that had to explain why they were gonna choose one painting over the other chose the hang in their kitty. But if, if you were in the group where you didn't have to explain, you just got to pick up your painting and leave, people chose the abstract one. Now that wasn't all that interesting until you fast forward about six months and then they come back and they say, Hey, what did you do with that painting? And the people that chose the hang in their kitty, because they had to explain what, which art they were choosing, almost all of them had either put it in a closet or down in their basement or given it to a friend. Whereas the people that had chosen the abstract one, almost all of them had hung it up somewhere. And there's something like really important about allowing people to feel something or absorb it without being able to describe it. That that altered what their, what their preferences were and then ultimately how much they, they wanted to be around that piece of art.

Kenneth Stanley Novelty Search [00:47:28] Yeah, that, that is very interesting. That's, you know, it shows something about, I guess it's about stepping stones, like somehow that that abstracting leads to like more interesting follow up thoughts than, than just a simple thing. And I guess that resonates with people as long as it's framed the right way. Like where they, they realize that they are allowed to do that. And so yeah, I think, I think AI can help to like kind of explore those, those aspects of people. So AI is not just about ai, I mean it's also about us. I mean it's, and it's interaction with us can help us to learn about ourselves and bring out more of our abilities. And that will be, I think eventually that will be a big thing. That will be amazing, I would imagine. Yeah.

Vance Crowe [00:48:18] Something I, a game that I play with my friends is I'll ask them to picture room that we've both been in or or a space that we've both been in and, and I'll say, okay, draw that space for me. And there's an interesting thing that happens and I've only ever seen two branches break off. There may be more, but I'm interested if I were to ask you, hey, pick out a room, describe for me how you would draw that room. What, how would you think about drawing it?

Kenneth Stanley Novelty Search [00:48:48] If, if we would pick, sorry, actually part of that cut out. Sorry about that.

Vance Crowe [00:48:53] So if, if you and I were both to know some space, some, some room or some building, and I said to you, Ken, I want you to draw for me that space so that I can see it too. How would you go about drawing that space

Kenneth Stanley Novelty Search [00:49:07] If we both knew if, if both of us knew a space? Yeah, I guess how would I draw the space? So I, I guess I would, I would try to draw the landmarks that are most likely to help you remember which room I'm talking about or thinking about. Would

Vance Crowe [00:49:30] It look more like a blueprint or would it look more like a painting? Like would you be in the subjective experience or would you be over top of it looking down?

Kenneth Stanley Novelty Search [00:49:41] That's a good question. I guess I would, maybe I'd go for the subjective for me. So, because I don't know if you would recognize right away the overhead view. 'cause that's not the way we experience things. So I would, I think I'd go with subjective.

Vance Crowe [00:50:05] That's interesting. And I'm the same way. So it, it's interesting because I think there's really only two branches. I've only ever heard people describe it. So a lot of my friends who are much more on the technical space, my wife is an aerospace engineer, my buddies are data scientists, right? They draw it from that blueprint perspective and they get the spatial things correct. You know, they, they, they know about how large the room is and how large the walls are relative to each other. And me, I can't do that. And if somebody draws one from above for me to look at a blueprint, I cannot tell you what that room will look like. I could build it because I, I, I worked with blueprints and I learned how to do building, but it will be a surprise to me how that building will look.

Kenneth Stanley Novelty Search [00:50:49] Hmm.

Vance Crowe [00:50:49] But if I were to draw it, I have to actually walk into the room in my mind and I have to look around and then I see things. And so the walls are not the correct dimensions. The space is not correct, but the things in the room are super high fidelity. In fact, if, if we had shared it, I'd be able to tell you what it smelled like or

Kenneth Stanley Novelty Search [00:51:10] What

Vance Crowe [00:51:10] It sounded like in that room. And you're one of the few people I I really expected you to say the blueprint view.

Kenneth Stanley Novelty Search [00:51:16] Huh. And so what does it tell you then that, what does it mean about me or somebody to take the, the subjective view?

Vance Crowe [00:51:24] Well, I think that it has something to do with the way people record memories. Right? And, and I think it also says something about, and not always, but the, the way that you experience things like books or the way that you experience maybe other pieces of art or, or any, any sort of experience, right? Like do you view it from the zoomed out lens or is it your own personal experience? I don't, I don't, I don't know that that's necessarily true, but it's something that I find infinitely interesting. Who chooses which, which path?

Kenneth Stanley Novelty Search [00:51:57] Yeah, I mean it, it, it probably does tell you something about people's view of the world or how they think about things. Yeah. Maybe, maybe it is partly a subjective versus objective type of distinction like that Yeah, I'm, I'm, I I don't describe things objectively. Ultimately I just think about the experience itself.

Vance Crowe [00:52:19] Man, that is shocking. Which is personal to me, shocking to me. I really, really would've expected you to be more of the objective. And it's probably some bias that I have towards thinking that people in the computer science are, you know, typically more on the highly objective, you know, tone, tune out as many of the, the, the, as much as the noise as you can, which often feels like the subjective experience.

Kenneth Stanley Novelty Search [00:52:43] Well, they may be, I mean, maybe most people are in computer science. But that's actually, that's an interesting point because, you know, the, the book, our book, which is called Why Greatness Cannot Be Planned, but the subtitle is the Myth of the objective. And so it's a big argument not just against like planning, but against objectives. And now there might be a, so at first it might seem like this is a different kind of objective. Like, okay, there's an objective in the sense I'm trying to achieve something, so that's my objective versus an objective view of reality. But actually think those two things are very, very similar to each other, very connected to each other. So like people who really like objectives like pursuing an objective also like to think objectively. 'cause like objective thinking is very like metric driven. It's like you're trying to accurately reproduce something in a way that's measurable so you can like prove it to somebody. Whereas like a subjective view of reality is, is much less metric driven. It's just basically, well that's the way it is to me. It's more of like an artistic view of reality in a way. Like it just, I'm trying to give you a sense of what it was like to, from my experience and, and so when I think of something's interesting, it's completely subjective. Like I don't have to prove to you. It's interesting 'cause I'm just saying it's interesting to me. I know we don't all share the same interests and that's my subjective viewpoint. And I think that this kind of subjectivity, it, this is interesting when you talk about computer scientists and you say, are they, they might tend to be objective. 'cause sometimes I've thought like there's a missing view of computer science or at least artificial intelligence, which doesn't get talked about that much.

Kenneth Stanley Novelty Search [00:54:19] 'cause it, you think of it as a science, which is to think of it more as an art. Actually I wrote a paper on this too at one point. But when I was thinking about this, this issue that like am I, when I, if I create an algorithm, like a novelty search, am I, am I actually behaving as a scientist or is this more of a work of art? And it's not clear to me why we assume this is science. Like we all, we're very, we feel very proud of being scientists. So nobody really would want to say, I'm not a scientist. Like that would not be a stamp of pride. But the thing is like, if, if you really think about it creating an artifice, like an artifact, I mean it's called artificial intelligence. You're talking about artifice, like creating an artifice is what art does, right? I mean if you paint a picture of an apple, nobody thinks it's a real apple. There's no, there's no like debate or like, we don't try to measure and check to see is that actually an apple? Like does it actually work like an apple? It's just a picture of an apple. It's really about your subjective experience of apples is what you're doing. So if I create an algorithm which is somehow supposed to resonate with some kind of process that you see in nature, why should, why should that necessarily be objectively validated? Like that just is an impression of something that I experience in nature. And so, so I think art, art algorithms can be, can be subjective experiences that are basically, it's, there's only a small group of people that actually appreciate it 'cause they have to understand the algorithm. But it could be, it could just be like a work of art that you're just trying to express a feeling about something like a process in nature.

Vance Crowe [00:55:56] I mean, man, that really hits home on so many levels for me, not least of which is I have no point to a conversation. Right. And I think like one of the things that my friends and even my wife struggle with is they think like, if I'm going to start this conversation, small talk is painful because there is no point to it. I don't, I'm not trying to get something out of it. In fact, I'm trying to get past that point in order that I can begin the process of figuring out why it is that we're communicating. Whereas for me, I don't even think of it as small talk. For me it's like we've just hit a new search space and the the beginning part of it is for me to, to discover something that you're gonna say that I didn't anticipate. And as soon as I hear you say something I didn't expect, bam, that's where I want to go. That's the's the playground, so to speak,

Kenneth Stanley Novelty Search [00:56:45] Right? Yeah, that's, I mean that is kind of a novelty search. And then you're just trying to find something interesting, something new, move the conversation to a new region. And that's true that someone who's very objectively inclined would maybe be hesitant or unsure where you're going or why are we going in this direction? Or what are we trying to do? I mean, objective, people who are thinking objectively are worried about that. Like where are things going? They're always worried about that. They're trying to get to an objective point. And I'm not trying to caricature people. 'cause I think diff the same person can be objective in one situation and less so in another. So it's not like people are just stereotypes or they're always thinking objectively. But it's often the case that people are thinking that way and, and then you know, that we miss this opportunity just to explore and, you know, conversation is one, one way we can explore.

Vance Crowe [00:57:39] So tell me about, you know, what, what do you do when you're not working, right? Like what, what is your search space for things that are fun or your, the subjective experiences that you enjoy the most?

Kenneth Stanley Novelty Search [00:57:55] Yeah, I guess I, well I mean, in reality right now it's mostly just taking care of kids. So, and that is a form of exploration, so I definitely actually enjoying it.

Vance Crowe [00:58:09] How old are your kids?

Kenneth Stanley Novelty Search [00:58:11] One and six.

Vance Crowe [00:58:12] Okay. Wow. So you have a very, those are totally different frames of life.

Kenneth Stanley Novelty Search [00:58:19] Yeah, they are very different. Yeah. Yeah, it's true. It's funny to see 'em interacting with each other. 'cause yeah, the one, the 1-year-old really wants to be doing what the 6-year-old is doing, but he doesn't understand at all why or what's really going on. So yeah, there's, well, so kids are a huge place to, to learn and explore and, and, and you don't have to have an objective, that's for sure. I like to really get a lot out of interacting with kids. But I like, yeah, I like things that make me feel a certain way. So like I, for some reason I like things that are tall. I like tall buildings and like tall mountains and stuff like that. I don't know why. And I have no objective. So I just, if I can look at them and I feel good. So I like try to get near things like that and look at them.

Vance Crowe [00:59:07] Have you ever been to Tulsa, Oklahoma?

Kenneth Stanley Novelty Search [00:59:09] No, I haven't been there,

Vance Crowe [00:59:10] No. So I like, I love tall buildings as well and you know, you have the experience of being in Manhattan where you're surrounded not just by the tall buildings, but all the noise and all of the, the chaos that's around it. But when you go to Tulsa, it's like you've just stumbled into an Iron Rand novel and there's very few cars and you know, that these big beautiful ornate buildings are actually empty, but you stand in their shadows all of the time because like in every direction there are the, and it was built during the era of oil and so there's these grandiose buildings and there'd be no reason to tear them down, although there's no reason for them to be full. But it's one of those places that when I go to it, I've been there twice. I, I mean like, I could just go wander around in that space because of the way that it makes me feel. It's, it's a truly profound experience.

Kenneth Stanley Novelty Search [01:00:02] It does. I've never heard that scripture of Tulsa, so I'll have to, yeah, hopefully someday when, when travel is actually something people do, I can actually try that.

Vance Crowe [01:00:11] Well, and it's interesting and like, it's actually a really weird juxtaposition because it's also where all these train lines come in. So you can go into these spaces where they're totally wide open

Kenneth Stanley Novelty Search [01:00:22] And

Vance Crowe [01:00:22] Then you walk another 500 yards, maybe half a mile, and then all of a sudden you're in the middle of all these buildings. So it, it is a, it is an experience. Yeah,

Kenneth Stanley Novelty Search [01:00:31] That sounds cool. I have to check out Tulsa.

Vance Crowe [01:00:37] And so Ken, you're, you're, you're kind of on the edge of, of, of artificial intelligence and, and what's going to happen in the future. Does it feel like the edge to you when you're going to work every day? Does it feel like, hey, I'm, I'm surfing on this edge of chaos?

Kenneth Stanley Novelty Search [01:00:57] Yeah, I think that I do. There's some feeling of the cutting edge or like where change is happening or like right at the right, right at the border where we don't know what's on the other side. I mean, that's, that's there a lot and yeah, that's fun. I guess that, that is exploration. I mean, you wanna be near that border to, to find something new.

Vance Crowe [01:01:24] I think that that's what children has been for me. So I actually just had a daughter just a little over two months ago. And for me it's, I I came to the realization that if you're, if you're expecting that you can be paying attention to something else. And then when the interesting thing happens, you can turn your attention to it. So like scrolling through Twitter for example, and then be like, Hey, when you smile, then I'll pay attention to you, you miss actually all of the thing about it. That's interesting, right? You're watching this little baby flap their arms as they're trying to connect. I'm excited, but how do I then move my hands and arms around in that thing? And it's been, I I was not expecting parenthood to be this edge of chaos and it 100% is for me. And I realized that the only way it feels like surfing is if I put everything else down and pay attention to this thing. And yeah, that's been a fun experience for

Kenneth Stanley Novelty Search [01:02:22] Me. Yeah. And, and congratulations. That's and that's true. The, the parenting is, or, or kids, I mean they're just basically something is profound about it. So it's true that like, it's very short and you could miss it if you're paging through Twitter or something like that. Although sometimes it's hard to pay attention 'cause like, you know, honestly you can, it can sometimes be boring too, but then you'll miss these moments that are really, really interesting and special too. And yeah, I was, I was always really, especially with my, my first kid, I was like super fascinated. Like it just what a baby is. 'cause it's just so strange and hard to relate to what it, what the baby is experiencing. Like it's just for it, everything is totally new and it is almost like not human in a way. Like it doesn't share any of the shared experience of, of humanity and, and it doesn't know anything. Or, or in the case case he doesn't know anything. So like what, yeah, what, that's just really interesting to watch. Like how does that turn into a human? It's going down some really long path of, which I would say is kind of a novelty search.

Vance Crowe [01:03:37] Oh, it, it's a hundred percent a novelty search as far, you know, like how do I get the, the adults to do the thing that I want when I don't even have language to process what it is that I want? The only thing you really have, you know, the observation that I have about like, when you're going to give it a bath, right? Like it gets a little bit cold and then freaks out because its tolerances are so low that the, that the millions of years of evolution have told it. If you get outside of these tolerances freak out because you know, to get any, you can't, you can't remember on your own. So you're required to force other people to pay attention to it.

Kenneth Stanley Novelty Search [01:04:13] Yeah, yeah, yeah. And then there's like strange things like where does, how does it, well, I mean maybe at two months they wouldn't have, but how do they have a sense of humor when they're like six months old or something? Like, like they think peek peekaboo is funny, but like why, how did they get this idea that that's funny? Like it doesn't make any sense. Where does that follow from? It's like sense of humor is just so innate or something. It's like it's, you're like born with a sense of humor it seems like.

Vance Crowe [01:04:40] Yeah. And then, and that's an interesting thing, like, because I always, I mean, I'd been around little kids but never when it was mine, so I had to pay attention to it so closely. But the, the, the fact that smiling isn't there from day one, right? Like it comes a month and a half in and then it's only intermittent and then you're trying to, but like there's all these things that you think of as like, oh, that's just what it is to be human. And maybe it is, but it wasn't there to begin with. They, there was some wiring that had to take place and some things that had to connect and then all of a sudden it's there and it wasn't taught that. So was it always there, it's like a very difficult concept to wrap your mind around.

Kenneth Stanley Novelty Search [01:05:19] Yeah, I guess we don't really know. And it's, yeah, the, the concept of what's innate is, is really slippery. 'cause it's, it's like some combination of what's there from the beginning and then, and then inevitability, like inevitability doesn't get talked about that much. But like some things, it's not like, it's not like an objective driven process where it's like you, if you start here and I keep on measuring, I make sure that you go there. It's more like if I, if you start here and then you just keep exploring like you will inevitably hit this no matter what you do. And that, that tends to be true on the early stuff. Like when you're starting out on a journey, like the later stuff tends not to be inevitable. Like it's not inevitable that you're gonna be hosting like a, a podcast. Like that's not inevitable, but it is inevitable that you're gonna walk somehow. 'cause way earlier in the journey it seems like all like, just about everybody hits walking, like when they're, but you could look at it as an objective thing. Like, okay, it was supposed to walk so it learned how to walk. But I think maybe not, I think like the, the baby isn't necessarily thinking, oh how do I walk? Like it's, it's gaining stepping stones just by trying things. It doesn't know what it's leading to. It has no idea it's gonna eventually be walking.

Vance Crowe [01:06:32] Yeah. And it's different I think if you're the second child and you can see, I see a behavior that this other child has that I want to get. But the fact that the first one figured out how to walk when they didn't even know that that was a part of their, you know, future vocabulary

Kenneth Stanley Novelty Search [01:06:47] Or the Yeah,

Vance Crowe [01:06:48] Go ahead. I

Kenneth Stanley Novelty Search [01:06:49] Mean I was thinking like the, the things that lead to walking, like why did it, why did it pick those up? Like it wasn't doing it. Like if it had to wiggle its its toes or wiggle its feet or sit up or it's not like every time it made an advance like that it was like, okay, this is what I need to do to figure out how to walk. It was doing them for other, just like the vacuum tube in the computer. It just, that's interesting in its own right. I'm gonna see if I can do this. Like I I'm not thinking about walking right now.

Vance Crowe [01:07:16] Yeah. And then there's this also like when you add in the layer of parenting, there's this weird thing and I I, you know, I think we used to live in communities much more so you'd be surrounded by children, you'd have been watching them be raised, you'd, you'd have a sense like, I don't know why we sing this song or, or play the itsy bitsy spider or do the, the this little piggy went to market. But that, those things ended up helping guide the child. And so now living in this world where I'm having to be like, okay, I know we used to do these songs, I think they're important, but, but then not really knowing and like them being remnants of our culture as a thing that has to be passed on in order to be able to help the child get to those things that were inevitable.

Kenneth Stanley Novelty Search [01:07:59] Yeah. Yeah. That's interesting too. Yeah. There are these like just cultural things we don't even remember. Like why, why is that the way we do things? It's just there.

Vance Crowe [01:08:10] So as a, as a parent when you think about the inevitability things, but you also think about, you know, by four years old they have got to be socialized. 'cause if they're not socialized, then they're likely not going to be. How do you think about your role in, in interacting with the inevitability?

Kenneth Stanley Novelty Search [01:08:29] Hmm. Well I guess it's the, the thing is, well that's a good point. Yeah. Like, I guess it, some things are more inevitable than others. I guess I would say. Like, like it's true that even walking is not truly inevitable. I mean, if you did something horrible, you don't let the kids sit up for it's the first two years of life. Like, it maybe it, it won't learn to walk. So like there has to be some reasonable conditions around like the expected conditions. Like, especially like if you don't talk, like it won't learn any language obviously no one ever talks. And so, so yeah, like when it gets to more complex, higher level things like socializing, I think yeah, things get a little less inevitable. Like there's still some things are still kind of inevitable, but, but you know, it's, maybe it's a continuum. It's not like it is inevitable or it isn't, there's just a degree of inevitability and so like complete social deprivation probably means that some things that normally are inevitable won't happen and you're, you're gonna have real problems. You try to give them like some of the like reasonable experiences I guess that that, that you think help it to, to climb those stepping stones and discover those things for, for itself.

Vance Crowe [01:09:48] And as you're imagining the world that your children will go into, what do you think you will guide them towards such that they'll they'll be resilient in a world that may be filled with artificial intelligence that can outcompete them Oh yeah.

Kenneth Stanley Novelty Search [01:10:02] On,

Vance Crowe [01:10:02] On every domain of intelligence.

Kenneth Stanley Novelty Search [01:10:05] Yeah. Well that's a really hard question 'cause we don't really understand that world at all. We're not, I I guess I don't think we're preparing them for that. I mean, I mean we, as in like anybody, I I don't think we're preparing our children for that world 'cause we don't know anything about it. We we're, we're preparing children for a world where other people are in control of the world. And, and so that's sort of how we understand the world that they're entering into. Like there's no training or there's no experiences to help them get used to dealing with artificial intelligence. Like, I don't know what they're, what to deal with them. We should build the artificial intelligence to accommodate us hopefully. So we don't have to rea accommodate it to it. But, but it's gonna be, it's just that world is so exotic and different that, that I just don't know what, what to do about it. I have no idea. I don't know what it's like at all.

Vance Crowe [01:11:03] Are you and I'll, and then I'll relent on the parenting questions, but it's something that fills my mind. I'm, so I'm very curious about how other people handle it. But how do you think about things like computer screens and, and the amount of time children spend in these spaces where so much of their learning is, is instituted through electronic means?

Kenneth Stanley Novelty Search [01:11:24] Yeah, I mean I think right now it's really hard with the pandemic to really be principled in addressing that issue. 'cause it's just like, it's like required for certain things. So right now I think you just have to compromise. Like there's gonna be some, there's gonna be some screen time no matter what your particular beliefs are. But I guess in general, yeah, ideally I think the ideal would be to not be completely saturated in screen time. Like I just, I guess my intuition is that like being around actual physical stuff is good for you mentally. So, so hopefully a, a significant part of your time can be in that. But, but I also don't think, I wouldn't, I'm not like a super purist, like I don't think it's like horrible to have some screen time either. I don't think that's terrible. That's my, my, I kind of think with that things work out. I mean, I'm not a child psychologist so I could be completely wrong, but I kinda think like as long as there's a generally reasonable amount of normal experiences, like things will probably work out okay. So I'm not super worried about it, but I do think that like you want, you want for mental health and you, you want some amount of like natural experience. Like that's how we evolved. Like we're, we're, we're part of the physical world and and that's probably good for us is my guess

Vance Crowe [01:12:50] One of my favorite questions to ask people is I call it the Peter Teel paradox, which is what is one thing that you believe that almost no one agrees with you on?

Kenneth Stanley Novelty Search [01:13:02] That's a really interesting question. Let me think about that. Okay, well I can, I can name one thing. I don't know if it's like the ultimate thing, but like that I think that we should eliminate standardized testing as a way of helping to improve the school system, I guess. Oh

Vance Crowe [01:13:38] Man, some people might agree. Say more about that. That's very interesting.

Kenneth Stanley Novelty Search [01:13:42] Well actually now I think some people probably do agree with that, so I'm probably not the only person that thinks that. But there's enough, there's a lot of people who disagree with it, at least the way the government certainly disagrees with that. So yeah, I I it's relate, I mean it's in the book it's related to the novelty search principle, but like the education system is like a really big ambitious problem. Like the way like, like the way these other problems are that I'm thinking about, about that. Like, we can't approach objectively. So it's something where like we want every child to, to get to some, some threshold level of educated ness. And that might be way past where most kids are getting today. Like a lot of people are behind where we wish they were. And so the, so this like overarching question of like how do we get everybody up to that threshold is sort of an objective question. Like we're saying that's our objective. All the kids can do this amount of stuff. And so it's very natural then that the next thing we get is all these standardized tests to measure against the objective. So here's our objective, let's measure it. And then like if they're not making progress, then we just, there's something wrong with that school or something like that. Let's like eliminate that or do something else. But the, but the problem is just like all super hard problems, all ambitious problems. We don't actually know what the stepping stones are that lead to like all the kids in the country getting above this threshold. We just don't know. I mean if we did, we would just follow those stepping stones.

Kenneth Stanley Novelty Search [01:15:13] Obviously we would just do it. And obviously we don't know because we never succeed. And so it's actually naive to think that that somehow an objective driven metric, like a test like this, a standardized test is going to be the solution to this problem. Because all problems like this, and this actually we didn't discuss this much, but this is a big point in the book and, and with novelty search is that all very hard problems are deceptive. And deception means that the stepping stones that lead to the solution to the problem don't resemble the problem itself. Which is again, like vacuum tubes and computers. Like if you're making progress in vacuum tubes, it doesn't look like you're making progress towards computers. Like you wouldn't think that you're like closer to actually, you know, like, like this, this zoom setup that we have right now. 'cause I have a new vacuum tube. But actually you are getting closer. You just don't realize it because it's deceptive. And on the other hand, there's another kind of deception which is that you can seem like you are making progress, but actually you're just heading towards a brick wall. And that's often the case. It's like you can make a few inches of progress, but actually you need to go miles and all you're really doing just heading towards a brick wall. And so of course like any really hard problem is deceptive. That's why it's hard. It's basically the definition of hard. If it wasn't hard then it wouldn't be deceptive because we would know how to get there, but we don't know how to get there. That's the whole problem. That's why it's considered a problem. And so if you have some simple objective metric, like a test, like a standardized test, then you're basically ignoring the fact that it's probably deceptive.

Kenneth Stanley Novelty Search [01:16:44] So it may seem that you're making progress for a while, but you will hit a brick wall, you will hit a dead end inevitably because it's going to be deceptive. And then what Well then like you have to back up, which no one will want to do ever. So they'll just create, create more tests, test, test, test. But sometimes you have to go down before you go up, you know, and that's, that's just not acceptable to us when we think about education and standardized testing. But really what you need, it's not going down and going up is really the point. What you really need is diversity. That's the real problem here is that like we don't know what works and we don't know where things that work work. 'cause it's probably not a, there's no universal solution to every single situation in every school system. And so we need a diversity of approaches, we need a diversity of innovative approaches, like things that we haven't yet thought of and diversity just does not flourish under conditions that are objective. Like if everybody has to be going up in score on some particular test, then there cannot be diversity because some degree of risk taking means that some of these test scores are gonna go down, but it's not allowed to happen. So you don't get any risk taking. So you don't get any diversity. And then the other thing that you need in order to make progress on a really deceptive problem is you need to have communication. So like if there's a lot of diversity and so people are trying different ideas, we need to be exposed to each other's ideas. So I need to find out your idea and find out how it worked in the context that you're in and then compare it to my context and then maybe I'll think of a new idea based on that.

Kenneth Stanley Novelty Search [01:18:18] That's a stepping stone. So basically I need to be exposed to the potential stepping stones. One blanket standardized test. I mean the word standardized tells you it's just not diverse. It's the exact opposite. There are no stepping stones in that world. Everybody is lockstep on the exact same road at the exact same time. And so all diversity has been flushed out of the system. All innovation is flushed outta the system. We will never solve the problem.

Vance Crowe [01:18:43] Oh man, I completely agree. I probably would've on my, on its face been like, yeah, I agree with you. Standardized destiny's not great. But I wouldn't have had that way of thinking about it. Ken Stanley, I have taken up so much of your evening, man, this, this was a great conversation. It did not disappoint. Thank you so much for joining me, man.

Kenneth Stanley Novelty Search [01:19:04] Thank you Vince. Yeah, this was great. Really enjoyed being here.

Vance Crowe [01:19:07] And if people wanted to learn more about your work or find out about your book, how would they go about doing that?

Kenneth Stanley Novelty Search [01:19:14] They can find my book at, well you can look up why greatness cannot be planned. You can find the book online and in general for my work, you could just google my name and you'll find, you'll find actually my old page at at the University of Central Florida, which has links to a lot of my publications and stuff like that. Or you could find me through Google Scholar, so. So you can find me online pretty easily.

Vance Crowe [01:19:42] Yeah, your Google Scholar page is a a wild ride of fun. I was just perusing it earlier today.

Kenneth Stanley Novelty Search [01:19:50] It's fun to hear. Yeah.

Vance Crowe [01:19:52] Ken Stanley, thank you so much for coming on.