DEMO: Why Not ANNs? Neurons Are TOO SLOW for Brain-Like AI
Publicado no IA Dots ·
AGIAI Signaldot-artificial_intelligencedot-llmidioma-enInglêsyt-N3O750WKT38
Vídeo em Inglês, publicado pelo canal Future AI Society.
About this content: The summary, key points and topics below are generated automatically from the video transcript. They reflect the content, opinions, claims and predictions presented in the video and do not represent a position, validation or fact-check by IA Dots.
Video summary
O vídeo apresenta uma discussão e sessão de feedback sobre um livro em desenvolvimento focado na arquitetura cerebral, redes neurais artificiais e o conceito de 'pensamentos atômicos', explorando limitações biológicas dos neurônios e a representação de conhecimento.
Key points
- O autor compartilha o rascunho de um prefácio que responde à pergunta sobre as motivações para escrever o livro, buscando alinhar objetivos como o avanço científico e a ajuda à humanidade.
- Apresenta-se uma demonstração utilizando o Brain Simulator 2 para ilustrar como os períodos de recuperação dos neurônios biológicos tornam os cálculos tradicionais de redes neurais lentos e impraticáveis.
- A discussão aborda a hipótese de que os disparos neurais individuais (spikes) e os 'pensamentos atômicos' possuem significado próprio, permitindo maior eficiência e processamento baseado em eventos.
- Participantes discutem o público-alvo do livro, ponderando se a obra deve se direcionar a especialistas em inteligência artificial, engenheiros de LLM ou ao público leigo.
- São debatidos temas como a importância do tempo no processamento neural, conexões recorrentes, bidirecionalidade de relações e a ambiguidade na linguagem e nos conceitos.
Topics
Redes neuraisPensamentos atômicosNeurobiologiaSimulação cerebralRepresentação de conhecimentoInteligência artificialProcessamento temporal
Video transcript
Transcript from available captions; it may contain errors.
[music] >> Okay. Well, people all let's get officially started and get to the to the material at hand. Um I want to cover lots of things because I'm going to be going on vacation starting tomorrow and hopefully I will be able to get the video out this afternoon so that uh it will be available to you. Uh one of the things that I'd like to do as we're getting moving forward, you know, if you have a chance to uh plug the video with somebody or put it on a social media site or go to the web to even just go to YouTube and give it a like or something like that, that will help because my objective is to get this message out to a lot of people. Uh we're going to be going through kind of an overview of the book and I want to give you some idea of what my thinking has been and I'm going to start by sharing a screen that we'll be using for a while and then we will get to the discussion stuff. Okay, there's a screen with a pretty picture on it. Um for those of you who want to read along, I have I think put Yes, if I press enter, this is the link that you go to to get the most recent copy of the um uh of the manuscript uh and you know it's the most recent manuscript if you look at it and it says September 10th. The the date on the title page is the date the the PDF was generated. So, you can always know if you're going to be following along with stuff that uh that uh uh is the current version. And one of the things that I went I started to prepare for this presentation and realized that there's a huge piece of of the book that I haven't I haven't discussed and that's why why did I write this book? And I'm going to start off with So, I wrote a preference preface that answers that question and I want to start by going to the preface. There. So, why this book? And I'm going to read through this because it's only two pages, but it kind of puts a lot of things together and asks and answers a number of questions about why we're why we're all here today. >> Ch- Charles Charles, when you're talking about URLs, I think you should have one URL that remains constant, that's always the latest, and then a variety of URLs for each version. And that way we, the readers who want the latest version, just keep a constant URL and we win. >> Um the URL that I gave is that URL that always shows you the most recent stuff. >> Ah ah, because I Okay. >> you do have to do a refresh. >> Yeah, I understood. Yeah. >> Okay, why this book? For decades I've pursued the question, can we build a system that thinks like a person? It began kind of as an intellectual curiosity. The human brain obviously does remarkable things that computers still struggle with, particularly the things we consider to be common sense. The brain applies what it knows flexibly in new situations, often in a fraction of a second, while using remarkably little energy. Now, how can a biological system do that? Artificial neural networks seemed like a natural place to start, and I've been pursuing artificial neural networks since the '80s, and so I got in early on that bandwagon. They were inspired by neurons, and they have proved enormously useful. Modern neural networks and transformers have taken that approach to levels which were once unimaginable. But, useful is not the same as biologically plausible. That distinction became increasingly obvious as I explored these ideas in the original spiking brain simulator in 1985, and much more extensively in brain simulator 2 in 2017. The more accurately I simulated the neurons, the more obvious the problems became. Neurons are too slow for that. A biological neuron cannot perform millions or billions of precise numerical operations per second. Neural spikes take milliseconds, and after firing, a neuron enters a recovery period during which it cannot fire again, leading to at most a few hundred spikes per second. The more faithfully I modeled these constraints, the harder they became to ignore. Even relatively simple artificial neural network computations ran into trouble being far too slow. Representing values through firing rates is fine for incoming sensory nerves, but within the brain, the greater the required precision, the slower the calculation must run. And accounting for recovery periods can slow things by another order of magnitude. Biological neurons could not be performing familiar AI calculations this way. Brains would be far too slow to be useful. So, the brain must be doing a fundamentally different kind of computation. And that led to a central question. What if inside the brain's computation, an individual neural spike has meaning? If a spike represents something, rather than merely contributing to a continuously varying number, that computational picture changes dramatically. A neuron's multiple spikes don't represent a value. Instead, they can represent increased confidence or importance of the information carried by the initial spike. Recognition changes, too. In a conventional artificial neural network, many values are calculated and the largest value is selected. A biological system doesn't have time for that, either. And neurons have no obvious mechanism for comparing all those different values. A much faster alternative is first to fire. Whichever candidate accumulates enough evidence becomes the most likely answer. That difference is profound. And instead of waiting for an entire network to finish calculating and then comparing its values, useful results can emerge as soon as evidence exists. So, I began working backwards. If neurons are this slow, what kind of representation could still allow knowledge to be stored and retrieved quickly, and how could a new fact be learned in a fraction of a second? How could objects be recognized from incomplete information? How could a brain represent the kinds of knowledge that seem obvious, including attribute inheritance exceptions sequences imaginary situations, and even learned mental procedures? All with large numbers of slow, similar neural elements. Atomic thoughts grew out of my attempt to answer those questions. They were not my starting assumption. They were the architecture I arrived at by repeatedly asking what kind of computational structure could fit both the limitations of neurons and the obvious capabilities of the human mind. An atomic thought represents something the system can know, and relationships among thoughts represent the meaning and structure. Even thoughts Each thought could plausibly implement be implemented by a small cluster of neurons, and the resulting architecture supports sparse event-driven processing in which useful results can emerge after only a small number of neural firing cycles. Understanding the brain is not only useful for building brain-like machines. The brain has capabilities that remain difficult for today's AI, and understanding how the brain achieves them may suggest better approaches for all forms of artificial intelligence. The same insights may also help us understand what happens when brain processes fail, and how learning, education, and behavioral development might work more effectively. Ultimately, exploring how minds work is not just about building better machines. It can help us understand and improve human lives. Atomic thoughts do not definitely explain how the brain works. We do not yet know enough about the brain to say that. But after decades of trying to reconcile computation with the physical limitations of neurons, atomic thoughts are the only architecture I have found that can explain a surprisingly wide range of cognitive capabilities while taking those limitations seriously. And this book is the result of following those constraints where they led. >> Reactions? >> Okay. So, I didn't like that answer because it itself was a whole thesis dissertation. It [laughter] had two ideas in it that I want you to pop up level and maybe that's it and then and then the rest follows. One One answer, why did you write the book? Is to advance general scientific knowledge about how the human mind works. And I don't know whether you want to use brain or mind there. So, that's one. Another one is to help humanity. Get period or you know, we don't I'm not even going to have to and so those are like the the net results of the whole thing and then there's this long-winded thing about, you know, I don't know, memory and all this stuff. >> Right, right. >> But those two key ideas, I think, frame I think frame what you're after. I want to see those be front and center um rather than getting lost in the weeds of this this >> And the it's like another chapter there. >> Yeah, that's another chapter, right? It's It's I'm not saying any of it is bad content. I'm saying it was a it it somebody if if somebody asked If somebody asked you at a cocktail party, why did you write the book? You wouldn't give that response, would you? >> No, I I need I need an elevator pitch. >> Well, okay, but I think you've got the two ideas in there and and you're and the idea is you're spanning the gulf between those two ideas of what or what we could ask the question, why should we advance general purpose science knowledge? >> Mhm. >> And you know, I would say to help humanity live better lives. Roughly. And then the rest is kind of details. Okay. Anybody else have something to add? >> Yeah. >> The one question I think people people might >> always jamming it? Okay. >> It's sorry. Need to be closer to it's the laptop. The one question that I've got for you is with this cuz I'm sort of with Freyja on where it sounds, it's really about your motivation. >> Mhm. >> around this. But also the other question because there's going to be people reading this who've come from AI back AI backgrounds, etc. etc. Or maybe indeed AI. Part of the thing you've got to put I'd like to I don't know if you can work it in, but it's to see why you think your way is right or a good way. Yeah? >> Mhm. >> You must have had an aha moment you you've sort of you sort of gone through it with Brain Sim 1, Brain Sim 2. But then you've got an you've got a it's that aha moment. It I think it's what Fry correct me if I'm wrong, but it's the similar thing. It's where's the aha moment? >> Okay. >> Yeah? >> Mhm. >> To get the people because there's going to be a lot of people who going to potentially read this. They're all coming from how we say classic AI backgrounds is easy. They've all got the blinkers on, you know, they've all been stuck in in this and you want to get to that. You've got to get through to that audience on a human level, I think saying here is here is my epiphany moment. In a very crude way. >> Mhm. >> Yeah. >> I think you kind of did that, Charles, but maybe it could be more succinct. But you just raised another interesting question. Who is this book written for? And I can think of three categories. One is the LLM person, you know, the other is the classic AI person, and the other is the person that doesn't know anything about either of them. >> What do you mean by that? >> And so how you phrase a lot of these things is going to depend on who is it written for? And and I I suggested this to you in an earlier email, Charles, but you might want to say, you know, on the back cover, this book is for these types of people. If you're not this type of people, put the book down and buy something else. >> [laughter] >> Yeah. But I love the put the book down idea. That >> But I like the group I like is the person that doesn't know about LLMs or classic AI. That's the group and that's the far larger group of I mean, that's 99% of people right there. You know, and then hitting that audience and then giving them more than they thought they could understand, but in a way that doesn't sound stupid or bad to the LLM or the classic AI person cuz they're actually they're real people, too, in addition to being it. And those people are going to be able to easily understand this stuff. The harder audience is the is the guy that says, "I don't even know what LLM stands for." Right? That's the That's the harder audience, but that's also the where the biggest value is cuz I think these ideas, knowledge representation here How could you even think without knowing what knowledge representation? I think knowledge representation ought to be taught at least in middle school, if not before. I'm going to be extremist on this, right? But how can you know how You know, do you teach about the process of learning in school? No. Do you teach what education is? No. Do you teach what language is? No. Those are the fundamental things, the highest leverage thing, and you have the opportunity to grab that. Now, maybe there's two books here. So, I don't want to [laughter] try to try to constrain it, but you know, it's it's the big people and we've got problem We've got big problems with billions of people not understanding things. And if they only understood things, oh boy, you know, we'd get rid of war and poverty and all of that stuff. Okay. I want to give you a little demonstration of of one of the things that really brought me up to brought me up short on this. And then we can discuss at the end of the quick demonstration if this is an appendix or part of the book or doesn't belong at all, but it's just interesting side of notes. And so, this is the Brain Simulator 2. And many of you may not have ever seen the Brain Simulator later 2, but it is a neural network simulator. And it is a spiking simulator, and we're going to focus on these three uh neurons, A, B, and X. And the little lines are uh are synapses. And these synapses have a weight of one, which we could tell by inspection or because they are white. And if I run this network Oh, the A, B, and X are the interesting ones, and all this other stuff is allowing me to set up a a series of signals that will uh allow me to show you what I've what we got here. And if I turn this on now we can see that this source fires every 10 milliseconds. And it generates a spike. And because there is a weight of one synapse between A and X X fires shortly after shortly after A, and we can zoom in and say, "Aha! Yes, X is firing shortly after A. In fact, you can see the spike, and then you can see the simulation of the recovery period here. And so the output happens, and then it has to recover. Now, let's assume for a moment that these these are are measured at 10 milliseconds apart. Let's call that value a half. And so I'm going to invent another signal and plug it into B and run the network. And run the network. If I hit the button, there we go. And now we can see the spikes coming in at B. And if we stop and take a closer look, we say the input is running at a half, this A is running at a half, and B is running at a half, and X is running at double that rate. And so you say to yourself, "Aha, as a neural network person, this is working exactly as I expected it to, and that proves that this is how your brain works." The problem is if I change this input to B to be slightly different timing, which I can do by connecting somewhere else. Now it is connected, I can't point with my finger. Now it's connected over here, and the timing will run this network some more, and now you can see that B fires shortly after A, and now instead of getting a half plus a half equals one, now a half plus a half equals a half. And the reason for that is once again, if we zoom in, you can see that B is firing during the recovery period of X, and so that input spike is ignored. And when you do a little observation, you find out that you get the wrong answer 70% of the time. You only get the right answer 30% of the time. And the only way to recover from this problem is to make the system run slower. And the slower you make it run, the less the recovery period is going to impact your result and the better things work. Let me add to that is that our output X can only represent three values in this model. The first is it can not fire at all, uh which we'll call zero. It can rep work at a level of one, uh or or a half like this, or it can have two spikes, uh which may or may not be evenly spaced, which we can do by doing this. Now we're getting the right answer of two spikes in 10 milliseconds, but they're not uh evenly spaced. Now it doesn't take us very long to realize that if we want to represent more different values, it's going to take us more time. And we're already at 10 milliseconds per value, which is what we would think of as ponderously slow. And if you want more values, if you want to do something like even a simple neural network that might be able to function with only 10 values, we're talking about a value represented in 50 milliseconds. 50 milliseconds is something that you your mind is almost capable of observing. That's a 20th of a second. And you can observe a 10th of a second quite reliably. So clearly this is not the way to go, and the aha moment says all of this stuff simply can't work, and it has to be that the individual spike has the meaning. Now what are we going to do? So any questions about that quick demo? >> When you were talking about the recovery period, if you can show the screen again, I can tell, but after a spike, it goes below the line for just a little bit. And the recovery period is the amount of time that it's below the line, right? There's this green There's this dip. And that's all it is. It's just a It's just like what a tenth or something of the entire width between the spikes. >> It's a relatively fixed um 3 milliseconds after the peak of the spike. And what happens during that time is that all of the ions that you used to create the spike and determine what the threshold was and all of that all of those molecules get shoved back to where they came from. And so a huge a huge >> Seeking equilibrium again. >> Yeah, a huge amount of of secret chemistry and molecular transport is going on in that time, and you absolutely can't live without it. And that's what's going on, and one has to presume that if you have larger neurons, that this process takes somewhat longer. But I haven't seen any paper about that. >> So, you're saying first past the post is the rule. >> I I didn't understand what you meant. >> Well, first past the post. In other words, the first spike is the one that triggers the signal. >> Right. Because because I set my weights to one, so whenever A fires, X is going to fire, and whenever B fires, X is going to fire. And as long as these are spaced out, and the of course in a neural network, the information about the relative spike positions of A and B is simply not even represented. The only thing that people might represent in a in a typical A&N is the spike rate, never the phase. >> Right. So, when you look at spike trains, what you care about is the leading edge and not necessarily the burst. >> Yes. Now, the problem is uh to extend this, when you want to do pattern recognition, your you're never you can't use a synapse weight of one. So, if you say I I want to recognize something I've got 10 attributes, I set the attributes I set the weight to a 10th. Then, if you get all the attributes, it fires. But, then if you don't get all 10 attributes, then it doesn't fire and you have to fire the input again. And so, everything from the atomic thought point of view, you've got to think in terms of when an atomic thought becomes active, it fires a burst and it keeps firing that burst until some output fires. Or, a burst >> A burst is a hedge. >> Yeah. So, so you're you can expect that that when you when you want to find out about Fido, that you're going to simply fire Fido until you get some useful output. And that might be three spikes and it might be 10 spikes, depending on the number of other attributes that are going on. >> So, you're saying that it's kind of lossy, that maybe if you give fire the first one may not make it. The first spike in a train may not have the desired effect and if there's a cluster, then that sort of emphasizes it as a higher probability of making it downstream. >> Yeah, let me slightly rephrase what what you're saying. The first spike almost never makes a difference because it will only make a difference in the event of a perfect match and a perfect match almost never happens. When you look at an object, you never see all of its attributes. You always see some subset subset of its attributes. So, it's going to take you a number of spikes to make things work. >> But, that does not define the Aren't you effectively defining the lower limit of thinking at that point or thought? >> Sort of. >> The each atomic entity you find out is a that you know you've got because you're limited to if you let's say on average firing three or four. >> Yes. >> That's It's 10 milliseconds, isn't it, basically, in certain cells? >> Well, let's look at it Yes, if firing at a maximum rate, let's say 4 milliseconds a spike. So, if you fire want to fire four, you're going to take the better part of 20 milliseconds. Let's say five to get to a 20 milliseconds. And you're right out there. If you can't If essentially, I'm looking at if you can't recognize Fido in in 20 milliseconds, you're probably not going to be able to run a brain. >> Yeah, I I'm sort of I'm I'll be honest, I'm sort of lost as where you're going with this one because I get the fact that you got the spiking, and I understand the fact of you potentially losing the signals. I get that, and I don't understand how that necessarily how you've jumped from that to atomic thought. >> Yes, the driving factor is that you can't do this kind of calculation. It's simply going to be too slow. >> Where do your recurring connections come into play, though? Yeah, I don't see you have you don't have any recurrent loops in there. Aren't there a lot of connections that loop back? And so, a signal goes out, and it comes back and reinforces itself. >> Let's defer that for a moment to get to finish the Chapman question. Um this drives me to the fact that that uh the individual spike must mean something. That's the key. That the individual So, now I have an individual spike that means something. I can bang on it with a burst and that helps it get through the pattern recognition process or it says this is important, pay attention to me. But [snorts] it really doesn't have any impact on the value. So, this spike within the brain might represent gray level three. But there is a different neuron that represents gray level four and the fact that you're representing that that's the spike train has nothing to do with the intensity of the color. That's what it led me to believe. So, that's why I had to go to the atomic thought. That the atomic thought itself means something. Now back at uh back at the John question, there one of the things that atomic thoughts do is it allows um uh a representation that to say this is this is fur and this is this is Fido. I can say say that fur is a a contributor to recognizing Fido. But it also means that I can say that Fido has fur and that means I have to follow this synapse in reverse and following synapses in reverse is something biology simply doesn't support. The chemistry is not there to follow it. Therefore, there has to be the recurrent connection almost everywhere. Because you want to be able to follow all of these connections in reverse and and the biology simply doesn't support it. >> Right, but isn't that how we hold state from moment to moment? Is there some kind of recursion going on that sort of repeats and says, "Okay, I'm thinking about this particular item." In other words, you have 7 plus or minus two pieces of short-term memory. And so, how [clears throat] do you hold those seven pieces if there isn't recursion? >> Yes. Um essentially, there are I mean, that's simply a way of saying, "I've activated Fido, but I want him to stay activated for 50 milliseconds." And how do I do that? And I can do that with a with a neuron that does something like that that keeps him that keeps him activated forever. But um uh uh I essentially, you build a loop, and that essentially makes a set of neurons ring so that they will continue to be activated over a longer period of time. And I see both that what you're saying both that long-term activation and the and the requirement to follow uh uh follow attributes in reverse. You have to be able to follow relationships in reverse, and that also leads me to there is zero possibility that an atomic thought can be represented by an individual neuron. You have to have a cluster of neurons in order to do what the thought can do. Including setting things up so it can learn in a hurry or uh any one of a number of other activities. Um one of the most important is when you see Fido and you've never seen Fido before, you have to somehow decide which cluster of neurons you're going to use to represent Fido. And your brain has to be able to do this very rapidly and say, "I'm representing Fido over here and not in all these other 100 possible places and that requires another few dozen neurons. >> So, time is very important in the system. >> Exactly. >> This is one thing that that just always is a head-scratcher for me is that with the artificial neural neural neural networks um they've washed time out. Everything is serialized. Even when you have parallel processing across multiple GPUs, it all serializes in the last layer and then you have a soft max that picks off one number. So, you don't have So, time is scrubbed out of the system entirely. >> It's the what that's Yeah, it's the one thing that always gets me is if you've got a neural network and you've got different layers you can have a neuron going from a higher level back down. >> Yes. >> And why does no one do that? Cuz it's logical. >> Because the math is too complicated. >> Yeah, but that's state. The high level the further down you've got more state so you can affect back. >> Right. So, so anyhow, this is this is the thing that that drove me away from neural simulation to individual neurons because you have to say So, now I'm going to say, "Well, I see that that there isn't enough there isn't enough DNA to define specifically what the connections are in your brain. How can you have a repeating mechanism that does the things I need to do like the uh recurrent connection and the ability to handle bidirectional relationships and stuff like that. >> Well, it's cellular [snorts] automata, right? >> Exactly. It's a sort of It is sort of What can What can I define that would do all of that stuff? And uh so, that's where where I went with it. >> Charles, are you sorry sorry quick one. Are you actually trying to now take Atomic Thought and are you thinking down the lines of taking Atomic Thought and seeing if how you can map that actually down to the neurological layer? Is that where you're going with is this your driving force you think? >> Um, no I'm going to think other people can do that. Other people [clears throat] are far more qualified at doing that, but I think it should happen. I think that Atomic Thought suggests a whole slew of experiments that could be done on brains that would drastically improve our knowledge of how brains work. >> Mhm. >> Yeah, yeah. I was just wondering if you were taking that tentative step of taking the concepts and then going down. That was that was all just a question I was just >> I think that should happen. I'm not the guy who can [laughter] do that. >> I mean if you were going to build this in a machine would it be an analog computer? >> No. Yeah, for a number of reasons by the way John, I'm so old that I have actually used an analog computer. And the reason we stopped using analog computers is with a modern processor we can uh uh execute analog calculations faster than the analog computer could calculate them given that the faster you run an analog circuit the more noise problems and other problems it runs into. >> I don't know. Let me restate this. Would would you describe the brain as a an analog computer? >> No. Um and that's partly my bias you know to every to man who only has a hammer every problem looks like a nail. And and my background is entirely um not entirely, it is largely digital circuit creation. And so that's my forte and and I can see everything in terms of yeah, I can make this work with a digital circuit. And what I and part of that is when we get into neurons the the spike of a neuron is essentially a digital operation. For for what we can perceive all spikes are interchangeable. The um even though there are small variation among them, they're essentially digital functions. Did that answer your question? >> Um sort of. I mean, I think I I I've looked at bees, you know, crawling over flowers and thinking, "Wow, that's a really amazingly compact processor that doesn't use a lot of power." >> That is correct. >> And [snorts] and so you step back and it's not there's there's no clock. There's no oscillator chip. >> Right. >> You know, it's just a bunch of chemicals. And so you know, to me that seems more analog than digital. >> Mhm. >> But chemical the the chemicals can form clocks. >> Well, another you John, you bring up a really interesting point and that is uh we as as computer designers are usually run with clock circuits. But uh when I built a computer out of telephone relays every function was so slow and took a varying amount of time that you needed to build an entire system that was based on the idea that you have a done signal for every function and you uh every everything you do waits. It's simply you start something and then you wait to get a done signal. And the done signal comes in uh wherever it comes, so you can have an add operation that runs at different speeds depending on how many ones are in it. For example. And so that's an entirely different mindset for creating a um uh to create for creating a digital circuit. And obviously, brains have to work that way. There is no underlying clock. >> Right. >> And that but that that's the see that's the bias. That this is where people shy away from this approach. Uh I'm not saying that it would be trivial to discover. I I think that people avoid it because it is difficult and it took nature a long time to figure it out and there's no guarantee that you'd be able to figure it out in your lifetime or many lifetimes. >> Right. And and and and certainly what you're saying is is is true of of physical uh physical things you can do. And I don't know how a bee walks, but I have a pretty good idea how a mammal walks with a very complicated sequence of neural spikes going out on motor nerves. Except that it all runs with feedback that you know what to take you never take one step before you've taken the previous step or once in a while you do and then you fall down. And and [clears throat] so [laughter] um uh the process has to be very I mean it's essentially always waiting for feedback because you can walk at any speed you want and still keep your balance because each step is waiting for the previous step to finish. And at a more micro level each muscle contraction is waiting until it gets feedback that not only did I send out the the signal to ask this muscle to contract, but I got a feedback signal saying, yep, it actually the the joint actually moved the way the way I wanted it to. >> Well, the brain has to maintain an equilibrium because if you get too many or too few spikes, it stops. If you get too many spikes, you get something like epilepsy. So, it has to maintain a continuous stream to keep otherwise in either extreme it just stops. Well, and so your your feedback observation is sort of baked in the cake, isn't it? >> Yes, and and also I mean essentially it is the input that's driving the whole system and some of that input is feedback and some of that input is sensory stuff and and the feedback drives the circuitry. And so, I mean looking at a more macro view, so given that I want to build an atomic thought, how do you make knowledge fit into the atomic thought? And that's where we go down the road of this is what a thought these are the characteristics of a thought has to have and and Zaid, I want to thank you for suggesting the that and I incorporated a section based on your suggestion that there was a recent uh uh test with chat GPT passing one of the ARC AGI tests reasonably well and it did so at a cost of $26,000. Now, we don't know what $26,000 means but it was certainly more than any human could pass a similar test. And so, I added a section that did that, in chapter 12, which is all about um energy. And even even at the best-case, the human brain, the any brain is so energy-efficient, partly because it's so slow, but partly because it's chemical. But uh uh it is so efficient that even at your worst-case, we have to imagine that the human brain solves this problem at a factor of 2,000 times less energy than ChatGPT can solve a similar set of questions. >> [laughter] [snorts] >> Now, they're a little closed-mouth about what their actual data is, but we can we can assume that. So, I wanted to thank you for making that suggestion, and whenever I get a suggestion, I give it, you know, a serious look and say, "Well, should we include that?" And the question, by the way, about this um this uh uh spiking presentation that I did on Brain Sim 2, uh if I were to include that, it would only even be interesting to AI people, and more particularly in in neural network people, most of whom either wouldn't believe it or wouldn't understand it, and I'm in the process of talking myself out of including it in the book at all, but [laughter] uh but it is one of the things that absolutely drove me to to go down this road, that that if you try to do more classical neural network things in in a neuron neuron simulator, it simply won't work. >> Now, have you had mutating nodes in your simulator, where they can you know, generate their own connections? >> Yes. Uh they can modify their own connections. You got to put the connection in there first. They don't have a way of adding a connection. But I excluded that specifically because uh I think the same is true with with biological neurons. That uh if you want to know if you want to learn something, um you can uh do that very quickly by strengthening an existing connection. You can take a a synapse from well below threshold to significantly above a threshold in maybe half a dozen spikes. And that seems very practical. Uh because half a dozen spikes, 20 ms to learn something, okay, I'm on board with that. But to grow a new connection, we know that new connections grow all the time, but they grow in periods of minutes to hours. And that's just not helpful in the time frames that we want to do things in AI. So, I would presume that your brain runs along doing the things that it you want it to do, uh learning the things you want it to learn. And if it detects that it is running low of potential connections in some area, then some longer-term process grows some more so that you don't run out of um learning capacity in a certain area. And I would I would speculate without any any evidence at all that one of the reasons that we learn the way we do at school is that you learn a couple of things and that overloads that area of your brain and then you go back to sleep and it grows some more connections and you come back the next day and you can learn some more in the same area. But that's that is entirely a speculation on my part. >> Right, but how long does it take for the the the you know the axons and the dendrites, you know, the the buckets of neural chemicals, they they get bigger and smaller. Which it makes makes it, you know, more efficient or less efficient. You have more neurotransmitter and the sacs or whatever they're called. And that the >> That's exactly what I was saying, John, that that uh um a a synapse actually has a whole set of of potential uh uh channels, ion channels, and it can migrate these channels around to the face of the synapse where they become effective and migrate them away again. And it can do this in in very short order. Uh but growing a new synapse takes a lot longer. So, the synapse itself can can change its weight in a very short period of time. Does that Andre? >> I I recently saw a small video that uh talked about how our brain works, how it learns. And it looked like the neuron was actually just dividing and just forming a new link. >> Yes. Uh yeah, uh and and it does, but it does this over a period of minutes and hours. It doesn't do that over a period of seconds or milliseconds, which is what we need to do that kind of learning. So uh exactly, I see these things all working together. Yep. So, I have a planned and the ability to go through the entire book in gory detail, but I think that what we want to do is to is to look for any wrap-up questions and and and we'll pick this up in what I want to do is I'll I'll get back from uh from my vacation and and take the time to actually produce some interesting new content before before I schedule another uh meeting. In the interim, I'll be able to handle email. I look forward to any suggestions. If you go through the manuscript and you say, "Yeah, but this whole section is is wrong or this whole section is missing." Let me know and I will add it. And uh uh I look forward to getting whatever input you've got. >> [snorts] >> Mr. Dave, what do you got? >> Yeah, I just had a quick question about I know you've been spending a lot of time focused on the book lately, but um I was wondering if the Brainstormer thought project on GitHub's code base was updated with some of those latest experiments at this time. >> Uh only within the branch I have been working on. The branch I have been working on is named UKS_CJS. And if you down if you get that branch, you get all of the recent stuff and some but you also get the experimental junk that doesn't work. >> [laughter] >> Um and usually before I before I post it to the master or main branch, I like to delete the junk that doesn't work. Um although some of it does still sneak in. Mr. Fry. >> First of all, I want to admit that I've been negligent, uh but I have now started from the beginning of the book and reading it through chapter 3, and that has been very helpful to me, Charles. So, I I apologize for skipping around before my earlier comments. Okay, I have my biggest question is, I think it was in chapter 2. I have a quote, "Ambiguity exists in language, not in the concepts themselves." So, if I'm to take the word concepts to mean a general English version of the concept, I have the following um scenario uh uh to to to to describe. Um let's say I have two pets. Uh they're both small black pets. One is Fido, a dog, and one is Fluffy, the cat. And I see from my peripheral vision some black thing running across the floor. Right? Now, I don't know whether it's Fido or Fluffy. That's ambiguity. But, I'm not concerned about the mapping between the words Fido and the words Fluffy. It's not a language ambiguity. It's concept ambiguity. >> Um let's invent another word. And the reason that that let me because I'm I'm using I'm explaining things at a slightly different level, and that is that you have a thought that represents Fluffy. And a thought that represents Fido. And you know which of those thoughts is which without any ambiguity in those thoughts. That when you see the black running object, you don't know at what what it is. It might you might equally activate Fido and Fluffy. >> Yes. >> But, and so you've got some uh what's a good you you haven't achieved a resolution to know exactly what it is. But, you know that if it resolves to it going to potentially resolve to one or the other and the thought that you have about fly fight or fluffy is is the unambiguous part. >> Sure but but yes but but what you are also representing this this event that something black crossed your path as a thought. >> Ah. >> Or a series of thoughts. >> Yeah. >> So those thoughts are ambiguous. >> that's an interesting way to look at it. And the >> Well thoughts are representing everything right so I don't I don't I don't think I'm inventing something here. >> No. But my You know let's say specially. Yes because it's going to be the fight or fluffy thought. >> Mhm. And that I can send I'll send you an email of my little little scenario here so. >> Sure. >> But anyway I'm really I'm really enjoying going from the beginning of the book and it's helping me understand the concepts a lot more so that that's good and I also want to thank both John and Chapman during this this particular session because your comments and hearing the response to your comments is really helped my understanding of this complicated stuff. >> Mhm. >> Okay Zaid you get the last word. >> Yeah I just [clears throat] wanted to add on to the whole ambiguity discussion. Like we also see within our like within our vision we also sort of like there are certain scenarios where it like visual input is ambiguous like if you've ever looked at a color wheel where all the colors sort of fade into each other. that there's no clear set boundary when you go from yellow to green to red it just sort of it all blurs together. >> Mhm. >> So like I guess is there a would that be represented as an atomic thought that represents the blur or like is that just an effect of post processing or something within the mind? >> What an interesting question. Um You know do you have an idea on that? >> Yeah cuz like what I've seen cuz like I wear glasses and when I take them off you things kind of look blurry but I've noticed if I focus a lot on it I it's actually just it's a it's the base image of whatever object I'm looking at but it's repeated a thousand times and when I put my attention on it I can see like all the uh well not individuals but like it's like a ghost image of the main thing just repeated a bunch of times but when I'm not focusing on it the brain sort of just abstracts that away into like a little blurry thing. So like in my from my experience it looks like it's represent it represents it like I guess it's kind of like in those image formats where they compress things and it's just you see a uh sort of a choppy gradient but then so I believe it's sort of a it's represented as a like as a discreetly but then it's post processed later to give that uh blur effect at least for our conscious view of it. >> Yeah and and you know when we talked about vision last time, um in the mental model, you do some recognition, and then you instantiate a thought of this is where I think Fido is. And now we've got two you a uh uh uh sample from Fry and a sample from you of now at this location I'm going to instantiate something, but I don't know quite what it is. And and uh >> Well, wouldn't that just pop up as a novelty? I mean, isn't that what we are tuned to is we we throw our attention towards novelty? >> Sure. So, that would get your attention. >> I mean, if you recognize it, then you can dismiss it. >> Yep. That's the best answer I've got. What do you think, Zaid? >> Yeah, it's it's certainly interesting how cuz like what we we perceive what we believe we're perceiving. And again I again it's sort of I notice it more when I take my glasses off, cuz I think I'm seeing one thing, and then as I get closer it changes. >> Mhm. >> But it's also [clears throat] kind of interesting how the brain is like when you when I put my glasses on I I almost never it's interesting how fine-tuned it is to um to 20/20 vision. >> Mhm. >> Yeah, so it's like I think I don't have a clear answer to it, but I'm leaning towards some level of post-processing. >> Well, that's a I mean, that's an interesting problem that that I'll I'll cogitate on the airplane. Okay, I want to thank you all for coming. I hope to have a video out. I hope when the video comes out, I'll put out a a piece of email. I hope everybody uh uh goes to uh YouTube and watches it or likes it or writes a comment or or recommends it to their friend or shares it on their on their extensive social network and I appreciate that. You know, as the more we get the idea out the more input we get back and the more accurate the answer can be. I appreciate all of your time and I look forward to talking to you in perhaps two months. Thanks a lot. >> What time is it in Japan? >> It's 12:45. >> Good night. >> [music]
Assistir ao vídeo original no YouTube
Assuntos: AGI. Publicado no YouTube em 2026-09-11.
Seleção automática por assunto e idioma. O vídeo pertence ao canal de origem.