ai risk

Vídeo em Inglês, publicado pelo canal Nick Saraev Daily.

About this content: The summary, key points and topics below are generated automatically from the video transcript. They reflect the content, opinions, claims and predictions presented in the video and do not represent a position, validation or fact-check by IA Dots.

Video summary

The speaker discusses concerns surrounding artificial superintelligence, referencing a researcher's recent resignation from Anthropic and highlighting fears of existential risk. The video argues that people should not feel helpless about the future of AI development and suggests shifting the focus of research toward alignment and slowing down capability advancement.

Key points

• A researcher recently resigned from Anthropic, citing irresponsible behavior by top labs racing toward self-improving superintelligence.
• Many alignment researchers believe there is a significant probability that advanced AI could lead to human extinction within the next decade.
• The speaker compares current public attitudes regarding AI to learned helplessness in animals, arguing that humanity still has options to change course.
• Four potential outcomes for AI development are outlined: misaligned and uncontrollable, aligned and uncontrollable, misaligned and controllable, and both aligned and controllable.
• Slowing down the development of AI capabilities to allow alignment research to catch up is proposed, even at the cost of slower scientific progress.

Topics

Artificial IntelligenceAI AlignmentSuperintelligenceExistential RiskLearned HelplessnessAnthropicOpenAITechnology Regulation

Video transcript

Transcript from available captions; it may contain errors.

Hey, welcome to my daily updates channel. I run a business that just hit over 400k in revenue. And I'm going to take a departure from the way I normally do these videos to cover a topic that is increasingly concerning to me and also worth getting amplified, I think, as much as possible. Um, Jacob Coxin yesterday made a tweet saying he resigned from Anthropic. He spent the last 3 years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. There are just racing straight to self-improving super intelligence and gambling with our lives. More thoughts below. He then proceeds to provide quite honestly like a pretty eloquent breakdown of what is going on sort of behind the scenes, why he left Anthropic, what he believes really is at stake here civilizationally, which he concludes is is human life. Um, and more. And I I just wanted to kind of talk a little bit about this more broadly. I feel like I have a responsibility as somebody with a platform. And um you know, one thing I've never really talked about before is uh AI alignment is actually one of the initial reasons why I started all this stuff. Um it was actually one of my initial motivations to really get into artificial intelligence and applied um AI and and automation and so on and so forth. I thought that this was going to be a very important problem, you know, 5 10 years from from when I uh graduated from university. Um, so as Evan Hubinger, who is the alignment science lead AI, points out, um, most people really do earnestly believe AI could kill us all. And he specifically thinks it's greater than 10% within the next decade. He thinks anthropic is trying his best, but we don't have a clear plan to solve a line of super intelligence and are not clearly on track to. The main thing I want to talk about here is there's this like feeling right now and it's not just on X, just not on teapot, you know, it's all over the world really of like helplessness. I don't know if you guys are familiar, but there were a bunch of experiments on this concept called learn helplessness a while ago where there was like a dog uh or a bunch of animals really and they were chained to like poles and they were kind of abused and treated very negatively and they weren't really allowed to move um past that pole and they were you know subjected to horrific things and then the researchers removed the pole and removed the chain and so the dog could at any point in time walk outside of the the confines or bounds of this chain. Um but the weird thing is it never did. It had learned to be helpless. Likewise, um, you know, a bunch of the experiments that I, uh, used to do a bunch of reading on back when I was working in labs with rats and other animals where, you know, you have like half of a platform that shocks the rat and the other half that doesn't shock the rat, you know, the rat will always try and run away from the platform that shocks it. Um, but even if that platform stops shocking it, the animal will never go back to that side of the platform because it had just learned that it is helpless when it is over there. And and the reason I bring this stuff up is because I think human beings just have a tendency, like most animals, to learn helplessness. And so there's that feeling that's going on right now. And I want you to know that you don't have to give into that feeling. You don't have to feel helpless here. Every lab around the world harbors a big chunk of their research teams and alignment teams. And they all think that like there's a greater than 10% chance. Some of them think there's probably a 90% chance that AI is going to get so advanced and so intelligent that it will eventually kill us all. Um, but these people are demonstrating part of that bias which is some form of learned helplessness. We can change that today. We have the ability not to pursue or to shift the course of research into these domains in such a way that it improves the probability of life, human life, life life that is meaningful and life that looks and feels like us and thinks like us and loves like us uh to exist in say 20 or 30 years. I don't understand why why people don't take more action towards that fact. I hate to give a terrible example, but I mean, how many countries throughout the world have achieved their aims through things like military might and so on and so forth. Um, I'm not suggesting anybody do anything to any lab or anything like that. And I hope that's not, you know, the vibe that you just got from me saying that, but the point that I'm making is like you don't have to feel helpless when somebody is pointing the barrel of a gun at your face, right? Like you, sure, you may not live, but you have options. You can duck out of the way. You can, you know, try and disarm them. You can plead with them. I mean, and there are a lot of things that you can do. There are a lot of future scenarios that end with, you know, humanity being disempowered and disenfranchised. And I just wanted to cover them all with you. I saw a great tweet which um initially inspired this line of thinking that I'll share with you. There are four possible outcomes for AI development. The first is that AI is misaligned and it is uncontrollable. And so if it's misaligned, that means it doesn't align with human interests. It doesn't love humanity. Okay? Maybe it is misaligned in so far that it wants to deceive humanity and take power for itself. And if it is uncontrolled, that means that we don't have the ability to stop it from doing things that we don't want it to do. And so in scenario number one, where artificial super intelligence is both misaligned and uncontrolled, humanity's over, right? It's game over. We know that. This is the 10% chance that everybody's talking about. We all turn into gray goo or whatever the in 20 years and is what it is. We have a nice life. But that's only one of four possible scenarios. Scenario two is that AI is aligned but uncontrollable. And so now we have AIs that are aligned to human interests but we just don't have the ability to control them. They multiply. There are millions, billions, quadrillions trillions sexillions I don't know the word past that of agents all existing um all over the world. Hell, they their brains might be in satellites. I don't know in a world like that where AI is aligned aka it doesn't kill us all but it is uncontrollable aka you know we don't have we don't have it in the reigns it will quickly become a lot more powerful than humanity and we will essentially be disenfranchised in some way shape or form by you know benevolent intelligences that's a world where you know humanity is alive but not necessarily empowered the third option is if AI is misaligned but controllable if it is misaligned but controllable. Then we have a world where a small group of human beings who do the controlling have total dominion over men. And so that is much more familiar to us, I think, as a dystopian example of what the world could potentially look like in 20 or 30 years. Orwellian style surveillance, maybe the sessation of need for the population entirely. Maybe they don't need the permanent underclass. They find a way to get rid of us, whatever. And then the fourth option is if humanity has successfully both aligned AI and controls AI and that's really the one that we we want to gun for. The unfortunate thing is there are four options and only one of them is an outcome that I think is meaningful meaningfully positive for all of us. But we faced far worse odds before throughout history. There were many routes that humanity could have taken. Various ice ages you know various cataclysmic events, asteroids, meteors and so on and so forth. many situations where our own technology could have destroyed us in the past. I mean, I'm thinking of, you know, hell, even the discovery of how to wield and harness fire. Sort of like that Prometheian thing, um, where, you know, tens, maybe hundreds of thousands of years ago, you know, some disparit group of people figured out how to wield fire. Immediately after that happened, mass forest fires, mass um, destruction of, you know, local megapa uh, and and so on and so forth. You know, think about like the genetic bottleneck when humanity was down to I I don't remember what the number was, but maybe a thousand or 10,000. We faced far worse odds before and so far we have threaded the needle in order to, you know, be where we are at now. So, there's really nothing that that is forcing us to like lay down and submit to something like this. The issue is most of the world is just either apathetic to it. Most of the world is impossible to actually meaningfully affect it or the small group of people that I think can meaningfully affect it. The vast majority of them just feel like they are helpless right now. Anyway, I know a lot of those people watch my channel and so I wanted to let you guys know we are not helpless. We can change things. So what does actual change look like? Now, you know, I think there's a million different ways to achieve that good outcome where AI is controlled and uh both aligned. I think it's unfortunate, but a lot of people will probably and they'll probably disagree with me or think that I'm, you know, trying to like slow down progress or whatever, but some sort of um change in our cadence is necessary, I think because capabilities are up here, whereas our ability to align capabilities are down here. And so there's a big disparity between the two. And so what might make sense is rather than, you know, making that capabilities lever as high as humanly possible while the alignment lever slow slowly tries to catch up, you know, maybe shifting the balance a little bit more so that there is less improvement in capabilities per unit time, which should give alignment more time to catch up. I don't know if there really is a way to align these models like mathematically or in a verifiably, you know, correct way. I wish I did. And I'm sure that's something that a lot of these um you know, researchers, these companies are struggling with right now, which is why they're making tweets like, you know, there's a 10% chance of literally all humans dying. But I think it would be remiss if we did not give it our best shot. The cost of not doing so is about 60 million lives a year that I think artificial super intelligence could probably help reduce. Um you know, 60 to 80 million human beings die every year. um the vast majority of them now for uh causes like you know reaching the end of a typical human lifespan. But um you know if ASI is here and maybe it can help us with drug discovery and and you know human life extension maybe we could bring that 60 down to something like I don't know 10 or 20 in in in the next couple of decades. And that's really like what is at cost here if we slow down every year that we slow down capabilities we're sort of that's that's the risk. But I think even just mathematically, you know, the cost of that many lives versus the potential extinction of all human beings to have ever lived, you know, and who will ever be born is is so much higher that it would make sense to slow down capabilities such that alignment could keep up even if it is at the cost of slower scientific progress and development of things like drugs and so on and so forth that can improve um human life. That's my personal take on it. Um, I don't think that, you know, having a big open layer economy of just like a bunch of AI agents, some of which are incentivized to hack all systems, some of which are incentivized to build plagues and and and pandemics and so on and so forth, um, will somehow magically be self-regulated. I don't think that will occur. I don't think, you know, I think even in a free market economy, what typically happens is, you know, a first mover or somebody that is just far more capable ends up sweeping up the vast majority of the market share. And so you need governments to step in eventually in order to like try and regulate the power balance. Obviously governments themselves are quite flawed and so inevitably end up contributing to the power imbalance. But this is obviously more of a political question than um anything else and that's why I'm always like kind of hesitant to talk about it because I know everybody has their own opinions on that. But anyway, that's my version of the future. What your version of the future might look like might be different. Regardless, you do not need to feel helpless. Um there is some hypothetical future where we we we cut the internet tomorrow. we cut all power supply let's say to these big labs and really just the whole wider world in that future AI development does not occur anywhere near on pace like it does right now I'm sure there'd be a number of other problems that would you know result from that but there is still a path from here even in our chess board that may look like we are losing uh from here where we are to to to success to actually making it and so anybody that might feel helpless I would just plead with you not to because we've been in far worse situations before as mentioned and we've somehow managed to squeeze out 8 billion humans over 100 billion humans um throughout you know history and and and and I think it's quite honestly possible that we continue down the path in the right way and we end up with 100 trillion humans colonizing the star system. I don't know. So I guess what I'm attempting to say is we could feel motivated from a statement like this to act and in that way that statement might not be a bad thing. What Evan did here was bring awareness to an issue that I think a lot of people have been kind of on the fence about and probably unwilling to talk freely and openly about because they're considered, you know, they'd think they'll be conspiracy theorists or whatever the hell. Um, but in reality is actually like a very a very meaningful and totally fair thing to start talking about now. And there's a reason that did 80 million views just on X, right? It's probably being reposted across a bunch of platforms. We get 500 million people that are seeing this. That's 1/16th of planet Earth. Presumably, those 500 million people are probably the most uh effective portion of humanity anyway. Uh they're probably not too young to be able to make a change or maybe too old to, you know, be able to meaningfully like contribute the rest of their lives to work on this issue. Um they're presumably, you know, they understand, I don't know, English, which is where the vast majority of all this work is currently occurring. Not to say that Chinese labs aren't making great headway and stuff like that, but obviously the English labs are still ahead for now. you know, us us people that are looking at this, like we we don't have to just lay down and take it. We can absolutely do stuff. I'm sad that Jacob just quit because I think he had a lot of good things to say about this. I'm glad that, you know, he decided to say something and get 80 million views on on a simple tweet like this that has a lot of potential to change the public consciousness around what is meaningful and uh what the future might look like. But um yeah, I guess one final takeaway I'll make is like this is not like you know like to be clear when I was in university I was like a damn dirty hippie. I joined the um environment club and I was like going to protest and stuff to protest pipelines and whatnot and I had I didn't know what I was doing. I think like obviously there are a lot of valid reasons to protest developments um especially if they're not done intelligently but I had no idea what I was doing. I was just joining a mob of people that was like, oh, like any development is bad and anything nature is good. And then I got a little bit older and then I realized like wow like a big chunk of my youth was um actually dominated by like um climate climate extremism in so far that like everybody was making statements like the world is going to burn in 20 years. We're not going to be you know humanity will die, the earth will die and so on and so forth. And it wasn't like a very logical or rational thing. Like yes, human beings are causing irreparable damage to our environment um of course, but that doesn't necessarily mean humanity itself is doomed, right? And so like I've learned to curtail like big sweeping social movements and stuff um with like reality which is that in the case of you know climate change and so on and so forth, you know, of course human beings are decimating our environment. I mean, just doesn't take a rocket scientist to look outside and see just how decimated large swats of our environment are. I mean, you can't plant trees over concrete, right? Just kind of how it is. But, uh, at the same time, like, you know, we shouldn't obviously allow that to get in the way of furthering human progress because humanity is really the highest virtue here, right? Well, this is something that can genuinely interfere with humanity as the highest virtue. And this is not like a like a like like an alarmism thing. It's not like AGI is like alarmism. Like it they're actively breaking out of containment right now. Um they're actively having misaligned thoughts right now about how to proceed in an environment of both humans and agents. All of the researchers working on this technology, virtually all of them, actively believe this thing, these things are either they have the potential to like be God 2.0 0 to like instantiate birth, some omnisient being. Um, or lead to, you know, a future where we all don't have mouths but are tortured constantly in a state of of perpetual horror, like basically a hell. Like, that's what all of them believe and that's what, you know, they're they're doing and that's why they're motivated to wake up in the morning. If you had that level of public interest even in like nuclear back in the 1940s with the Manhattan project and stuff like that, the world would look very very different today. Um, which you know I know had a lot of its own problems. But anyway, I feel like I'm rambling now. The point that I'm going to make is like we are not helpless. We have the ability to to actually affect change. Whatever that change might look like, just remember that like at the end of the day, right now at least for this fleeting moment, you have the ability to, you know, positively impact the way that this looks. Do not be afraid to talk about this with other people. Do not be afraid to actually worry and ask people like, "Hey, have you heard about this thing? Do you know what's going on?" Do not be afraid to like put pressure on, I don't know, legislators, regulators, and so on and so forth, or make content about it, um, you know, as I'm doing today. Don't be afraid of any of that stuff. Um, you know, this is basically somebody running the equivalent of the 4-minute mile just in public AI alignment. And so, I'm sure there will be many other people within labs, outside of labs, and so on and so forth that now feel a lot more open to talking about it. Um, you know, I feel like I'm doing my small part here, taking something from X and then moving it over to, you know, a small YouTube channel. Granted, um, I think I might talk about this on my bigger YouTube channel as well. We'll see. Uh, but yeah, anyway, hopefully that makes sense. So, yeah, guys, I mean, I think there's a lot of potential for good. And, uh, whatever the technology looks like, whatever our economy looks like in 10, 20, 30 years, who really knows, right? It's tough. It's tough to say. Hell, even the next 3 years GDP could grow, you know, 50%, 5,000%. Or not at all. And maybe, you know, the technology takes longer to materialize. Um, but, you know, at the end of the day, we are just rebuilding the human brain through first principles at a billion times the scale. And I think you would it would be remiss of you not to think for a second like, huh, this could have impacts far past maybe just GDP. And uh, yeah, you don't have to feel weird to to to to say that out loud. Okay, I'll catch all y'all later.

Assistir ao vídeo original no YouTube

Assuntos: Inteligência artificial. Publicado no YouTube em 2026-09-09.

Seleção automática por assunto e idioma. O vídeo pertence ao canal de origem.