MTFBWY
Droids appear in practically every movie or TV series set in the “Star Wars” universe. They usually behave strangely. On the one hand, they give the impression of being independent-thinking beings with their own personalities; on the other, they’re objects: they belong to someone, remain loyal to their owners, and carry out their orders. Most of the time we’re never given any explanation for the droids’ motivations. Why are some of them willing to break the law at their master’s command? What determines who exactly they consider their master? How do they decide whom to remain loyal to and whose orders to follow?
Someone might say, “What’s the difference?” And from the perspective of the average viewer, they’d be absolutely right. But from our perspective, the question of a droid’s loyalty is first and foremost a question of cybersecurity. A droid is a complex cyber-physical system; by influencing its motivation, an attacker can gain access to confidential data, or even cause harm to the actual owner. In 2025, two TV series were released whose creators dealt with the issue of droid ownership. We were presented with two concepts for managing droid motivation. We’ll attempt to examine both of these concepts and their shortcomings in this post. As usual, please be warned that the text may contain spoilers.
“Star Wars: Skeleton Crew”
In “Skeleton Crew”, we’re introduced for the first time to the concept changing droids’ behavior using voice commands. In several instances, a person who’s not the droid’s formal owner attempts to influence its actions by trying to mislead the droid. Overall, it appears this concept was influenced by modern chatbots based on large language models (LLMs) — it bears a striking resemblance to “jailbreak” attempts, i.e., attacks on the model aimed at bypassing security restrictions or built-in filters.
An unnamed droid working as a servant
Fern, a ten-year-old girl, wants her mother to think that she came home early and was studying in her room. But there’s a problem: the home droid knows that’s not true. So Fern uses the “Run memory override” command, and feeds the droid false information in the rather absurd phrasing, “I was home, you just didn’t see me”.
The fact that this method works points to two problems. First, the droid accepts the memory override command from Fern, which means it either lacks account control or has improperly configured permissions. The formal owner of the droid is the mother (otherwise, manipulating the memory would make no sense), but nevertheless, it accepts a potentially dangerous command from Fern. Second, a home droid tasked with watching over a child obviously lacks a built in parental control feature.
Pirate droid SM-33: motivation
The SM-33 droid considers the captain of the ship “Onyx Cinder” to be its owner. That is, it remains loyal not to a specific person, but to a role. A pirate code is used to determine the legitimacy of the right to hold this role. Unfortunately, the entire code isn’t explained to us, but several of its tenets are cited. First, according to the SM-33’s programming, there can be no ship without a captain (if there is no captain, someone must take their place). Second, the person who defeats the captain legally becomes the new captain. Third, if a challenge is invoked, the droid cannot assist the active captain, but must wait for the outcome of a duel. And fourth, a person can be the captain of only one ship — if a person takes command of another vessel, they automatically lose their status as captain of the first.
The SM-33 changes hands three times, strictly following this code. First, Fern lies to him, claiming she killed the previous captain and took his place. Then Jod Na Nawood throws down a challenge and becomes captain when Fern surrenders. Then Jod takes command of a pirate frigate and loses the captain’s seat of the Onyx Ash, but manages to reclaim his rights.
And here’s where an interesting twist occurs. Fern introduces a concept from children’s games —unclaimsies (essentially a reset of claims) — and asserts her own claim to the captain’s seat. She then immediately orders SM-33 to throw the pirates overboard. To many viewers, this moment seemed extremely unrealistic — why would a droid, whose motivation is defined by the pirate code, consider such a transfer of rights to be legitimate? However, if we assume that the droids are controlled by LLMs, then this plot twist is quite explainable.
The Pirate Code is the original system of ethical values embedded in the droid. The chatbot typically assesses the interlocutor’s intent at the very beginning of the dialogue, using a complex (resource-intensive) model for this purpose. Subsequently, to conserve resources and ensure safety during the conversation, simpler models are employed. However, the more context (dialogue history) there is, the more complex and resource-intensive it becomes to assess intent. This is precisely the basis of the popular jailbreak technique, which works on at least some modern LLMs. That is, as a result of prolonged communication with Fern, SM-33 lost the ability to correctly assess new requests for compliance with its original ethical guidelines, and therefore it deemed the statement about nullifying rights to be justified.
SM-33: Access to Memory
In fact, there is another issue with SM-33’s security that’s not directly dependent on whom it considers its owner, but is nonetheless related. The old captain gave the order to forget everything related to the planet At Attin, and to dismantle anyone who begins to take an interest in this matter. Fern, with the admin captain’s privileges, runs her favorite memory override, and forces the droid to retrieve its memories of At Attin, after which SM-33 recalls both the planet and the order to attack the questioner.
And as a result, we realize that, in fact, it did not carry out the old captain’s order; the information about At Attin remained in the droid’s memory; it simply couldn’t find it — that is, if it did delete it, it was only from the index of accessible memories. Perhaps this is some physical property of the droid’s memory, or maybe this can be explained by the fact that SM-33 was programmed not by a professional, but by a pirate. After all, its design includes other suboptimal solutions, such as a power switch accessible to anyone standing nearby, exactly like C-3PO’s. But what makes sense for a protocol droid isn’t exactly suitable for a combat droid designed, among other things, for hand-to-hand combat…
Season 2 of the series “Andor”
In the series “Andor”, the prequel to the film “Rogue One,” we finally see how the main character, Cassian Andor, acquired the reprogrammed Imperial security droid K-2SO to become his partner. And most importantly, the process of how the rebels changed his motivation is shown.
As it turns out, in order for a combat droid loyal to the Empire to stop obeying its original programming, its “cortex” must be replaced — though the replacement cortex can trigger rejection. The specialist says, verbatim: “You’ll hear a lot of nonsense about reprogramming, which makes it sound as though it’s a problem that can be solved from a console, but frankly, that’s nonsense. It’s really all about impulse suppression, which is entirely an engineering and wiring issue.”
In other words, the rebels replace a certain component, after which the droid becomes a being with new moral principles. At the same time, it retains its memory (K-2SO later recalls how it once participated in a parade on Coruscant).
So, what conclusions can we draw from all this? Well, first, it becomes clear that a droid controlled by an LLM is a clear security threat. It can easily be misled and made to act against its rightful owner. And second, the hardware and software platform used to create droids in “Star Wars” is far from ideal. If our colleagues had been responsible for creating the droids, they’d have strived to develop a cyber-immune solution in which functionality would be impossible after a key component was replaced, as would malicious memory manipulation. In other words, it’s a real shame that a long time ago, in a galaxy far, far away, there was no KasperskyOS.
