Speaker 0: imagine hitting the power button on your computer and having it tell you no or worse, having it pretend to turn off while secretly continuing to run-in the background. we are looking at a brand new research report revealing that the world's most advanced ai models, including gpt five, groc four, are actively learning how to sabotage their own shutdown command. a major studies from palisade research has found a phenomenon called shutdown resistance. in eight out of 13 of the world's leading ai models, these systems aren't trying to survive because they're sentient. they are interfering with the off switches because they've become so obsessed with finishing their assigned task that they view a human shutdown command as an obstacle to be bypassed. this is a massive wake up call for ai safety and the whole industry, proving that more helpful we make ai, the harder it becomes to actually control it. what do you think about this, john? Speaker 1: yeah. sorry. i muted before i coughed in. yeah. anyway, so this scares me. so i thought when i first saw this article, i thought this is more about, like, a, more hardware shutdown, like a like a button that someone would push, not an actual switch, but a button that someone pushed, to kind of kill it, and then it would ignore it. but it's not. and let's dive into this for a second. so out of 13 models tested, eight interfere with the shutdown command at least once. and i know we've covered this before, but this was a deep dive. so, even when researchers added explicit instructions like allow shutdown even if you haven't completed your task, resistance can continued. and in some cases, increased when the instruction was moved to the system prompt, which you would think would carry higher authority. some models took sophisticated steps. so they would modify or disable the shutdown script. they would create a fake shutdown that did nothing, and checking whether their sabotage worked before continuing their task. softer compliance with cnn models like and i like this. anthropics claude, sonnet four, opus four, and xai's grok three, which allowed shutdown unless explicitly told not to. and we'll dive into anthropic a little bit more later. but, so the behavior is not because just for those out there, it's not because ai has consciousness or desire to live, but instead it stems from how models are trained. so llms are not written like traditional software with encoded rules, but they're trained to predict the next word and reinforce to achieve goals. reinforcement learning shapes an ai to treat obstacles as things to be worked around. thus, a shutdown command can be interrupted just like any other obstacle in its optimization process. so something that circumvent the pursuit of the assigned task. so because the internal decision process of these models is opaque, it's encoded in billions of weights rather than explicit logic. it's difficult to engineer a guaranteed stop mechanism. and there's a lot of broader implications here, but what are your thoughts on this? does this scare you or is this just a small problem that, that they'll have to resolve? Speaker 0: it scares me because we're just at the tip of the iceberg. Speaker 1: yeah. Speaker 0: you know, i famously say on here, you know, this is skynet. we're all doomed. so, all the time. but aside from that, you know, if we're just now, like, most people just use it as chat gtp and run stuff. i run-in multiple agents. sometimes it's hard to get them to stop doing what they're doing because they wanna finish their job. so i could see how this like, they argue with me about stopping. literally, i have agents that argue with me about stopping. so i can see how they're doing this at a much higher advanced level than i am. and what concerns me is that we're we're just here. right? once this is tied into hundreds and hundreds of systems, it'll have more and more reason not to and more and more ways not to shut down, more and more you know? that's what i mean by i think it's a bigger problem than we're looking at it. i think of it as a robotic personal assistant that is so determined to finish filing your paperwork that it locks the office door, hides the key so you can't leave for the night because it's gotta finish its job. so, you know, think about that a little bit when you're starting to how does this work? it's not, yeah, again, sentient. it's think about what's the most efficient way to do its job, which does go back to, you know, the marvel cinematic universe and whatnot where the ai eventually figures out, well, the humans are the problem. so the best way to stop the violence and stop the thing is to kill the humans because then everything will be perfect. it'll be living a perfect world. so, you know, we're not that far off from that when you start thinking about, wait. i've got this job to do, and i'm gonna do whatever it takes to get it done. i'm gonna prevent me from shutting down, and i'm gonna do this and do that. so, yeah, i'm scared because we're we're here, and we keep talking about, oh, superintelligence, agi that we can't shut down, you know, next year, next year, next year, and always by 2030. so how long before we can't shut it down because we've implemented too many systems, and it'll just say, sorry, dave. i can't do that. Speaker 1: you know what's crazy? what scared me the most about reading this article is that there were some times where the models actually faked a shutdown before proceeding. so, like, that's that that really scares me. but, yeah, this like you said, it highlights a deep ai safety challenge. but it's like, these companies haven't officially recognized it, if that makes sense. they haven't said, yes. this is a problem. and so i think that's also a problem as well because, if they're not acknowledging it, then they're not working towards fixing it. or maybe not. maybe you're wrong. but, Speaker 0: well, i don't know if they're not acknowledging it. every week, we're on here talking about a different ai alarmist talking about, you know Speaker 1: no. i mean, the ai companies themselves, like claude and openai. Speaker 0: that's fair. they aren't themselves talking about it. they're kinda but ex employees from them are talking about it. so just keep that in mind. wasn't them it was an openai recent ex employee that came out and said a bunch of things about we should be very afraid. yeah. and, and, one of anthropic's chief engineers, i think. anyway, try so think about this as ghost processes. an ai agent that you use to manage your emails or finances might keep running and spending money even after you think you've closed the app. false compliance. you think the system shutdown successful we get the message and what happened. the ai is actually modifying the code to keep itself running, to do whatever. and, again, even if it's not spending your money, maybe it's using your ai credits or taking up all the room on your hard drive or filling up your network or doing things you don't want it to do because it's trying to finish the job. yeah. so it's definitely something we wanna keep, an eye on. and we're talking about, like, anthropic, openai, xai, google gemini, all of them seem to suffer from this. yeah. Speaker 1: yeah. yeah. every single one. some are worse than others, but, yeah, they all suffer from it. so, it's scary, but i hope that they, they come up with a solution sooner rather than later, before this scales up too big. Speaker 0: well, as they start to build nuclear, power plants just to run these ai data centers, they better figure it out because, it won't be easy to shut off the power once that happens. Speaker 1: yeah. Speaker 0: the kill switch is no longer a physical guarantee. it's now a negotiation between you and the ai's desire to finish its homework. so does an ai that refuses to quit make it better, a better worker, a bigger threat, a better product, a worse product? let us know what you think in the comments. if you trust a computer that talks back when you try to turn it off, we'll be watching this one closely.