Govern Persistence, Not Erasure: Why Deletion-Based Privacy Fails Against a Medium | Common S3nse

[music] Well, we got a nice small intimate group this time. So, if you guys want to come up closer if you'd like. I know everybody wants to, you know, be very cool and work on their laptops in the back, but if you guys want to come up, you should. And you can. All right.

It'll be just a minute to officially begin. Um, I'm sure some other folks will join, but we'll get started. All good. All right. Okay.

I want everybody here to think back to the social technology of the '9s. The internet, a revolution governed by extreme fear, extreme excitement. And the fear sounded obvious. You never upload a photo. You never give personal information, put a credit card into a web page.

[snorts]

And you would certainly never uh how could you trust the internet? How could you trust information written on web pages by people you don't know? I think we're in a very very similar moment when it comes to large language models and AI. When would you ever trust an AI with your personal information, with uh purchasing power, with your photograph, your face? How do you know when an AI has your information and when it doesn't?

I think the key to understanding these systems is this. A trained model is a new medium. It's the successor to the network of networks. So, for the next 20-ish minutes, I'm going to give you my theory and if it's true, what follows for security and privacy. My name is Joseph Bianke.

I'm a philosopher of technology, a consultant. I'm the uh as far as I checked the speaker list. I'm your only existing philosophy track for this whole conference. Uh the core of this talk comes from a paper of mine that's under peer review right now. I won't deliver the academic version um that exists.

If you want it, you can go to josephy.com. This afternoon, I'm just translating. So here's what you'll have when I walk off. First, a reclassification.

what kind of an object a trained model actually is because it's not a mind and it's not a database and most of what you've heard about AI is already downstream of those two mistakes. Second, you'll have a proof small enough to put on a napkin that erased from the model is a claim that no test can ever confirm. not is hard to verify cannot in principle. Third, what you'll have is what actually can be governed instead, which turns out to be the conclusion this community has been arguing for for 30 years. The program printed my conclusion next to my name.

Govern persistence, not eraser. The t-shirt version is a lot shorter. You cannot be deleted from a trained model. You can watch the handover happening in web search. For 25 years, a search engine was the address book of the old medium.

Crawl every page, index every page, and when somebody asks, you hand back a list of addresses. The thing you want lives here. Now you ask and AI answers. The links are still down there. Smaller, fewer, less, and less clicked.

The engines that perfected finding information at its location are teaching the world to stop asking for locations at all. So the web page, it's a document at an [snorts] address. The atom this whole information order was built upon is losing its place as the unit of knowledge. Its replacement is the trained model. And the new medium is powerful for the exact reason it's dangerous.

Nothing in it has an address and nothing can ever be shown to have had it. The public argument runs on two pictures and both are wrong. Picture one, the model is a mind, a someone. So I'll just be blunt and say no. Humans have found faces in clouds, spirits and rivers, and voices in thunder for as long as there have been humans.

that we now find a person in a pile of multiplication is a fact about us and the power of language, not about the technology itself. So the alignment debates, the consciousness debates, the doomers and the accelerationists are all downstream that same personification reflex. Picture two, the model is a database. It's a something with your information in it. It read my posts, so my posts are in there so I can demand to get them back out.

That picture gives you copyright suits, right to be forgotten requests, every deletion demand ever served on a research lab, but wrong, too. And the rest of the rest of the talk falls out of that. There's even a third picture on the books. European privacy regulator, the Hamburg data commissioner, took the position that a trained model contained no personal data at all. It's not directly in there, so there's nothing to regulate.

That of course doesn't survive measurement. Models do memorize. The capacity has been quantified about 3.6 bits per parameter. And extracting attacks run something of what you of what went in there is still in there.

So it's not a mind. It's not a database and it's not empty. It's a new kind of thing. The paper's name for it is a medium, a new way for information to exist. And what separates it from the medium it's replacing is one property.

On the internet, information has an address. In a model, it does not. So if anybody knows, training a model, what it actually does, model starts as billions of random numbers. training shows it. The corpus, the crawl, all the books, your blog, and after every pass, billions of those numbers shift by a hair.

Your data is never filed anywhere. It's metabolized. It becomes a tiny adjustment to everything tangled with the tiny adjustments of everything else. If you keep one sentence from this talk, keep this. Your data enters the model as a cause, not a component.

Not a cause the way a match starts a fire, but a cause the way a childhood causes an adult. You can't remember most of the meals that you've eaten, but they made you anyway. Now, you could be forming an objection in your head that the weights are just numbers. They're discreet. They're printable.

They're right there. Surely, my content is in there somewhere. But think of the egg in a baked cake. The egg is real. It went in.

The cake would certainly be different without it. Now point to the egg. There is no slice. That is the egg. Every crumb has a bit of egg.

Baking doesn't store ingredients. It spends them. Training is baking at a planetary scale. And your info has been baked. My paper argues that where in the weights is my info has no answer at all.

It's not that it's a hard question, it's that it's the wrong one. And in case that sounds hypothetical before this talk, I looked myself up in the common crawl. You should too. It's the great scrape that feeds these models. My publication on Substack entered the crawl in March and it's been captured in every single snapshot since.

Nobody asked me. The crawler just keeps coming back. If you've ever written in public mailing list, flame out, the git commits, the blog you deleted in 2019, you went in there with me almost certainly and as a cause, not at a locatable address. That's the reclassification of what a large language model is. So now the three claims I build on it and I'll tell you exactly how strong of them all are because I think I have one proved, one measured and one open.

The first claim is proved. No test can ever confirm a deletion. Period. Some of you may have uh experimented with deletion pipelines. Uh a right to be forgotten request comes in against a database.

The drill locate, delete, cascade, verify, close the ticket. Now the same request arrives to uh against a trained model. The genuine question so try to answer it. What test do you run on the weight to confirm the deletion happened by by a show of hands? Has anybody tried that?

Anybody know of a test they've tried that? There isn't one and there can't be. Whether you were deleted is not a fact about the model in front of you. It's a fact about the model's history. And the weights of a model do not record their history.

You've all handled an object like this. Think a a Euro dollar, a bank note. Uh consider it once stolen, and you'd never know. Stolen isn't printed on it. You might have one that's stolen and not know it.

It's a fact about where the note has been. The paper itself keeps no ledger. That's the weights. the proof. Imagine somebody sells you a deletion tool anyway with a certificate, a test fixed in advance run on the weights that the tool reliably passes.

One condition, the tool has to keep everything the rest of the corpus taught because a deletion that works by wrecking the model isn't deletion. That condition is the trap. The rest of the corpus that the model is trained on could have been anything. So the test has to accept so many different possible types of models and some of the models it accept it accepts still know about you. A cheating tool picks one of these and hands it over.

The test comes back green and your information is still inside. Any test the honest tool can pass. The cheat passes with you still in there. A green test never proves anything. So that's the theorem.

This covers models trained the way today's big models are trained. Everything in one pot optimized together. The run is over by the time the request arrives. And the test assumes your record and the rest of the corpus are independent. Neither secretly contain the other and the model learned from both.

So every deletion claim is a trust me from whoever ran the eraser. And this room has a word for trusted third parties. You can shrink the trust, attest the training log or the training run, log it, know what's in it, and that's good work and you should do it. But notice what your proof is about. What anybody's proof would be about.

It's about the process, never the object itself. The strongest addestation you could ever force ends at the same wall because the weights keep no receipts. So the second claim that's already been measured and by others, not myself, is that nobody has ever demonstrated removal from a standing model, only suppression. By removal, I mean pulling one record out of an already trained model in place. There's a research field triing that you may know of, machine unlearning, editing facts out of a model.

And at uh ICLR, one of machine learning's flagship venues, it was put under attack in 2024. Famously, the facts everyone had deleted were still sitting in the model's intermediate layers, readable. So against the strongest defense, blackbox attacks recovered the deleted material 19 to 28% of the time. A rephrase, a jailbreak, a later fine tune of a prompt that lifts the gag. The plain word for that is not cured.

It's remission. The thing is still in there. the model just stopped saying it when you asked it politely. Now, you may have heard that deletion sometimes works. It does, but only by unmaking the thing itself.

Retrieval systems keep your record in an index outside of the model, and the model reaches out and pulls from it at answer time. It's a compartment design. It pins your data inside one small submodel that can be retrained alone. So, look what the the engineers actually built there. They built the old internet in miniature.

You have a record, a node, an address. You've just created more models, not fewer, just one with your name on it. Your data sits out there at a location and never dissolves into the main model at all. Deletion works in this case because they're still aware. That isn't deletion from the new medium.

It's just creating another model. So is there any way to truly pull something out of a model if it's already entered one only exactly one and it's just that you destroy you destroy the model and you train a new one. You have to train it on scratch from everything except the one thing that you don't want to be in there. That's not surgery. It's not eraser.

It's not deletion. It's demolition and reconstruction. What you get back is not your model minus you. It's a completely different stranger. All of its billions of parameters have been changed.

The weights are different. So you can't run this backwards in time is the most important sociological conclusion. The models that exist today have already metabolized the crawl. If your data was ever in it, it's in them as of this moment and no future retraining ever reaches the copies that are already out in the world. So the third claim, lastly, [snorts] if you put the first two claims together, a deletion can never be verified and a deletion has never been demonstrated.

Claim three is just one step further. It's impossible. I believe it's impossible. I don't think any method ever will carve one record out of finished weights while leaving the rest of the model standing. Everything points that way.

What is not proven? It's technically an open question. I won't round it up to Ethereum just because I happen to have the microphone. So take it as a standing bounty. Show me one record carved out of a large model in place judged by the weights themselves, not by whether the model uh stays quiet under your strongest prompts.

[snorts] So the implication of whether it's completely impossible or slightly impossible, I've chosen the three that belong in this room that might be very practical to everyone here. Um, first, the law has already quietly reorganized itself around the impossibility. Here's what eraser compliance actually is. Once a request hits a model, Europe's top court in the Google Spain case gave dellisting. The web search index forgets where to point.

No information is actually deleted. The court orders you to break the route of deploying the information. So, that's one strategy. Or you have anonymize, transform the whole model until the law just stops seeing it, aka repress the information. We already covered that.

Or filter. The model still knows. It's just not allowed to say. It's just another command on top of the deployment. So it's not deletion.

It's a gag order. Of course, you have the very last resort, which is you retrain from scratch without the offending data. The demolition from claim two dressed up as compliance. On the old internet, removal was a repair. You delete the page, control F the server, scrub the infringing text.

Here, there's nothing to find and nothing to repair. Four options on the menu for the law and not one of them is the thing actually ordered. The most complete survey of deployed legal remedies, the scholar Daniel Wolf Townsen ran it. He found zero orders that reach into Weights in Colorado record. Zero in the entire record.

When the FTC went after models, Cambridge Analytica is among them, it ordered them destroyed whole on the grounds they were provenence, they were trained on unlawfully collected data. The New York Times suing OpenAI right now is asking the court to destroy uh the models trained on its archive. The models whole because there's nothing smaller to point at. So if the state wants a record gone from a model has two moves. You order the impossible or you burn down the library.

So second implication and this one is yours directly. Cippher punk movement spent 30 years engineering censorship resistance mixets to blockchain on purpose with cryptography. Then the most censorship resistant medium of all time arrived. On the old internet, keeping a document alive was a race. Copy it faster than they take it down.

Here, even the copy they catch leaves nothing smaller to burn than the entire AI. EU's chat control is the old reflex at full volume. If the state can see every record, it can govern every record. On this medium, there are no records to see. Once a corpus is metabolized, and the weights are loose, and open weight models are loose, torrented, forked everywhere, a voice absorbed into them has never been carved out and probably cannot.

You simply cannot subpoena what something became. So, so many old internet features were opt-in, or at least by design or principle. you chose what went onchain. This one doesn't. It doesn't ask.

By design, it keeps everything that was fed to it. It protects the corpus, the info from the reaches of the state, but also from you. Maximum censorship resistance and zero exit possible. So the third implication is that there's nobody to negotiate with. Suppose I'm wrong about all of this.

Suppose verifiable deletion existed and you had the lawyers and the leverage to force it out of a lab. Which lab? There are openweight models already loose deep quen llama mistrol thousand smaller that no request could ever reach, especially no request of yours. An EU court order could reach Meta tomorrow and it's already too late. And the trend of course runs only one way.

I'd bet that within 10 years, I'd half bet two years, that serious models get trained in a spare bedroom on a rack of secondhand GPUs on whatever corpus their owner chooses to train it on. So, the medium itself has no center. Deletion isn't just unverifiable in the one model in front of you, it's uninforceable across the entire species. Any governance worth building starts by admitting there is no global delete button. So stop asking what the model contains because it doesn't have any addresses.

Ask where you can stand. Relative to any party, a model is only ever in two states. Somebody holds the file or somebody actually runs it. Everything governable passes then through three gates. This is the stuff you can touch.

First you have the file. You can govern persistence aka custody, origin, history. You can know who trained what and on whose data. Those are all answerable questions with evidence that survives. The second gate is the interface.

You can govern access for everyone who doesn't hold the file. Whatever the model absorbed can be reached in exactly one place, the query and it's measurable there from outside. You can estimate how much of a record a model memorized. And lastly, the third gate deployment. You can govern use who may run it for what on what terms.

So those three things, persistence, access, use, you've already watched the state find two of the three. The FTC's orders were custody moves and the European filters were interface moves. Enforcement already lives at these gates. So the important thing for the builders, since this week is full of people who actually ship, anyone selling deletion from the weights is selling paperwork. It's just about the process.

instead provenence metering measuring what a model actually memorized that's real and nearly empty of good builders. So build there. In 1993, the manifesto this whole week descends from gave privacy the best definition anyone has managed. The power to selectively reveal oneself to the world. It's the only definition that the new medium can honor.

A model can never prove it forgot you. That much I conclude. Whether it can forget you or any information is potentially open. But what an AI model reveals to whom at which gate on what terms that is entirely buildable. And the same manifesto hands this room its three-word job description which is cippher punks write code.

So the meals you can't remember eating, the childhood you can't change, the Euro dollar that won't confess where it's been. You've always been made of causes. And now the internet successor is as well. The courts, the public will spend a decade learning what you know in these 20 minutes. So write code for the medium that exists, not the one the people thinks exists.

And when courts order the impossible, whether it's pro privacy or anti-privacy, and they will, be in the room that knows why, and hand them some info governance that actually helps. So, last thing, I know that this talk is being recorded. The recordings end up on the internet, the internet ends up in the crawl, and the crawl becomes the next generation of models. So whatever I get wrong up here this afternoon is probably going into the weights not as a cause but a or as a cause not a component and with my name attached beyond anybody's power to ever certify it gone or remove it. So most speakers walk off stage worrying they'll be forgotten.

I don't get that option. None of us do anymore. So if I have any errors, it's up to you guys to find it fast because the crawl won't wait. All right, [applause] I'll be here. Anybody?

So you are talking about uh deletion of records uh so that the models uh themselves do not uh have the information uh I mean once the data is uploaded to the model's memory using the training data then it cannot be removed that's true but uh also there was like one study which recently happened uh where they tried to create uh a models training data using images uh and in different batches of images uh they used the same samples in some of the batches and some of the batches they didn't use it and they trained multiple different classifiers uh using these uh different batches of training data and then at the end uh not the classifiers but uh the transformers and uh at the end what they try to do with these different versions of uh transformers was try to give it uh the same prompt and uh generate the output image uh output image image and text which should not come uh unless uh there's some part of uh the uh base image available uh to the training data. uh but at the end uh with a very high probability it was found that uh the uh even if one particular sample of image is not present in the uh training data uh given that uh multiple other samples uh are present in the uh training data which are related to uh that particular sample then the output doesn't change that much. uh so what I'm trying to say is like even if a particular news article for example is not present in the uh training data and that particular news in some form or the other is present in the training data the output might not actually change. So like even if uh someone has a claim that like OpenAI for example used their uh specific data that data might be present in some form or the other uh in some other data source which open AAI used and we cannot just say that like you have the IP right over this model uh because your data was used because even if that was not used the model might produce a slightly uh lower quality output but it would produce that output anyways. you uh that was found in the study at least.

Yeah. information. Not any particular information like even if you remove a particular source of information that information is present in some other sources. Most of the information in the world is present in multiple sources. Yeah.

Sure. Any question? Okay. Thank you. So on this same topic like it's about uh who controls like as you said uh you know what gets scrolled is it the one who creates the data or is it the the uh big labs because you know the decision is who says okay the big lab says I don't want these data like if New York Times for example I don't know how it plays out Maybe I would love to understand from you uh how do you see the what's the outcome of new uh the the the paper New York paper versus uh open AI because that will be interesting that will set a precedence in terms of how every other um data source would go against these big models um but in general who controls it mean I individual as an individual I control okay these are my data not I I say don't all or is like the the uh you know it's more of a consent driven by the the big labs they say okay these are the data for which we have consent to use this is like very tricky because yeah