Decentralized, Confidential Machine Learning – San Francisco 2025

[Music] hi guys I'm Alex uh let's talk about user owned AI user on AI is a is a term that I guess is not very standard but what we mean by that is uh how do we deploy how do we Design Systems and processes around artificial intelligence such that uh applications tools that people use uh powered by artificial intelligence are not benefiting you know centralized actors but benefiting users uh and you know how do how do we run them in a way that is uh respectful of the privacy of the user uh that it is censorship resistant you know that nobody can decide that a particular set of users you know don't get to use artificial intelligence uh today I want uh to talk about two particular aspects of uh the way we use artificial intelligence today one is more pressing more shortterm and other one is more long-term but more existential uh very much in continuation of what Ben was talking before me here today the first issue is how do we deploy artificial intelligence in such a way that users can use it without giving away their data today primarily the way we use artificial intelligence is through centralized World Gardens such as chat GPT clae deep seek app the one hosted by them and when you use application like that all your data goes and is stored on those centralized you know the centralized providers collect your data and generally that on itself is not a good idea though I guess over the the past decades we've been conditioned that this is okay you know most of us use Gmail giving away already all our data to Google but if you think about the alternative to that the alternative to that would be some sort of a decentralized system where you know decentralized nodes run by completely random participants run the model that is strictly worse because in this case you're giving away your data to someone who is completely random definitely not something you would want to do right so it's very interesting uh how how does one in one approach uh you know deploying systems that would be competing with centralized companies like open AI in a way that users will actually use them will trust the data uh and uh and the second problem uh that is sort of orthogonal but the solution to them will be surprisingly similar similar uh is how do we make the community to ensure that going forward we actually do have competing state-of-the-art systems that are not controlled by centralized entities right so until recently open Ai and entropic were Generations ahead of Open Source Alternatives right so you as a company if you build an AR you know a product that is using artificial intelligence you either take a dependency on open AI in which case you know you're effectively at their Mercy you know if they decide that you don't get to use it your company is dead or you use open source solution in which case you're at a disadvantage relative to your competition because your model is less powerful now we are at a the rare moment when it is not the case we have deep seek which is uh competing competitive with the centralized uh you know proprietary models but that will not be forever right so when open Ai and entropic increase the Gap again do we get again to get a model you know that you know randomly appears out of nowhere or do we need to do something to make sure that it does um happen right and the problem here is uh very um relevant to the uh to the to the name of the event is that you need to fund the common you need somehow people the community the society uh to you know to make it happen you know like you need money you need data you need compute uh all of that is not free models are notoriously expensive to train uh however once the model is trained the way you use it is you either deploy it behind the wall Garden the way chat GPT is deployed today in which case it is not in any way community owned or you open source it in which case it is free you know everybody can download it use it so you do not get to recoup your um you know the price of of actually training it and you know uh you know like stability for example so Ben used an example of Minal stability the same you know two of the companies that were uh pioneering open source artificial artificial intelligence both had to stop publishing uh tier one artificial intelligence model because you know the business model just doesn't work okay so now that the problems are presented I'm going to I'm going to I'm going to start U very far and then I will be building you know using usual building blocks until we get to a very nice solution uh first I will I will do a very quick rown of what are the options to solve those problems especially the problem of the inference right uh there's multiple one uh problems specifically for inference the most promising one the the most fundamental solution would be what is called fhe which stands for a fully homomorphic encryption uh where uh the model is built in such a way that cryptog it is encrypted and your input is encrypted and the model is being fully executed in the encrypted Manner and nobody can ever see your data right unfortunately for us the mathematics behind fully homomorphic encryption today is not there and it's extremely slow right it's uh the the overhead is uh 100x and more to run the model so it's completely impractical right so I guess the most viable solution today is local many people run models locally but locally you can only run models that are smaller uh right and moreover it's a it's a it's a very different experience right like if you run something through you know like if you if you use like open AI or if you use Gro or or the ones powered by fireworks or by uh cerebrus you know the output is very fast it's practically real time anything that trans locally is much slower much less enjoyable to use uh but the solution I want to talk about is called the transit execution environments uh it has practically no overhead and it's going to be literally no overhead very soon uh effectively next year uh and it's uh it's a concept uh that easiest to describe with an example of private inference right so imagine I'm a user I want to run inference on someone else's machine I don't trust open a I want to use someone you know I want to use decentralized service and there's a hardware provider which for all I know is probably malicious and all they want to do is get my data sell it uh or you know blackmail me I don't know right so if I send them my query get get my response they have all the data right however Intel processors AMD processors most of the processors today have this concept called confid um called trusted execution environments uh the idea of which is that on the hardware on the on the processor you have a vault uh within with which I as a user can establish a connection and I can run a computation in that vault in such a way that the owner of the hardware has no way of seeing what happens inside and moreover I have a proof signed by Intel uh that that is the case right so in this case I still have to trust someone I have to trust Intel but I would way rather trust Intel than uh that guy and starting with the most recent gpus h100s and going forward all the new gpus they also have trust at execution environments meaning that we can deploy machine learning models in a way that they're running on someone else's Hardware but I have full certainty that you know unless Intel is colluding with the some random person on the internet I I'm I'm I'm safe right and so inference now inference just works right we establish you know uh the hardware provider just trans VM or any other open source framework that does inference on their machine inside of the te I connect to it I know it's uh I know my connection is secure I know the code is running inside of the te so it's safe for me to you know send my query get my response I know that neither my neither my prompt nor responses nor the actual computation can be seen by uh by the owner of the hardware um now besides inference that interestingly enables a lot of very interesting use cases that are otherwise many people don't even think about it because they uh without G is are fundamentally impossible today one of them is if I go to a person and they say hey I'm hosting here you know deep Sig V3 you send me my prompt it's going to be inside of the te you get the response right well uh the you and you pay me you know the amount of money appropriate for deep Sig V3 what the malicious Hardware owner can do is run a significantly cheaper model you know like a two billion parameter model and pretend it's deep sig3 there is no way for for me to prove besides observing inferior results right but with T is what you can do uh and there's a uh I have a little bit of technical details here but I'm not going to go um too deep in there uh but you can you can design it in such a way that when I connect to the machine and do the inference I'm absolutely certain that the model running inside of the te is the one that the that the owner promised to me right and the the core idea happens to be that you you know you compute the hash of the model ahead of time and it's known I I can Rec comput it if I want to be 100% certain and the uh virtual machine that is running in there is a virtual machine that when it spins up the first thing it does is it makes sure that the model loaded has the same hash and if it's not it crushes right and one thing that te guarantee when I connect to it is that the code running there is in fact the code uh that the hardware provider promises me it is right so I can go to GitHub look at the virtual machine see that line that says I'm going to immediately Crush of the model is not deepseeker one you know and you know but by the fact that I was able to connect to the machine I know didn't crash I know that the model must be in fact deepseeker one you know and then I can use um you know a model being absolutely certain it's the model that was promised to me from there uh you can take one step forward and and you can say hey I have a model and that model is something I value right it could be maybe you know I trained it from scratch maybe I fine tuned it from uh you know I got a existing open source model but I have proprietary data I I fine-tuned it and it's a valuable model what I can do now is I can encrypt the model put the encrypted model you know like on3 you name it and then Design This virtual machine in such a way that it downloads the model it has internally um a decryption key but the model can only be decrypted inside of the te it can never be decrypted outside right and the way it works is that for the first machine the one I have you know I spin up my personal machine and there is this endpoint set SK right I I just I have the secret key so I set it and you know my virtual machine now has it and then you design the virtual machine in such a way that any two of them can connect to each other established through Intel that they can trust each other and then the one that has the secret key will give it to another right and so this case for as long as I know any person in the world who already has the secret key inside of their machine I can connect to them get it to mine and I can get to use the model uh but nobody ever will see the secret key besides myself besides the original Creator nobody will see the secret key outside of the te and they will never be able to decrypt the model which in itself is not very useful until uh you I I it can also be verifiable you know by by the same uh uh mechanic I described previously but what you can do now is you can say that that very virtual machine that now has the that can decrypt the model internally uh would only do inference for you if you establish a payment Channel with it and pay for it right so now I can distribute the model in a way that is completely decentralized right for as long as any person knows anybody else who already has the model they can get it right so nobody unless the entire world is censoring them they will be able to get the model they will be able to get a usable copy locally but they will not be able to use it without paying me the model creator for using it right so that's a it's a very important building block right because now you can if you have a model that is valuable right and you put resources into creating it and you know people will will be willing to pay for it you don't have to host it behind the W Garden you can actually just distributed in a decentralized fashion you know creating community-owned but not free uh machine learning models uh and that brings me to my final and most cool concept which is community owned model right so in the previous example the model was still there was still one person in the world who has the model right the creator of the model uh but that is not even that sometimes is not desirable right so so if we want a lot of people you know like we want the the the whole society to cheap in to create cre the you know state-ofthe-art model you know hundreds of people chip in any person now who has access to the model can run away with it you know monetize it uh take advantage of it so what you want to happen is that you you want the model to be created in such a way that nobody ever sees it everyone can use it and whenever someone uses it everybody who chipped in to create it gets their you know royalty uh and the way it works uh given everything I showed before is you have a smart contract right and the smart contract is rating everything related to the model and the first thing you do is crowdfunding that is obviously a completely solved problem in 2025 right so people fund uh then you have people running Hardware all of them have a specifically designed CVM uh running virtual machine running inside of the tees right they provide Hardware uh you know the or the smart contract is orchestrating training um that requires a little more details but I'm not going to go there right importantly they can provide Hardware get paid from that money that people uh crowdfunded and once it is trained it exists inside of those cvms encrypted but nobody the key the encryption key was created inside of the CVM and no person ever in the world has seen it right so the model only exists outside of the uh tees in an encrypted form right nobody can ever see uh no person or or agent or anything will ever see the model unencrypted it will only exist inside of the cvms but anyone can use it right anyone can get the secret key to the r CVM connect to it open this payment Channel pay for the inference you know send the prompt get the result uh and the money the smart contract will distribute money back to all the people who who contributed right so that gives you an ability for the society to crowdfund the model train it uh you know crowdfund the hardware for the model uh and then deploy it in a way that is completely decentralized sensorship resistant uh and more most importantly uh um privac uh privacy conscious right confidential so people who use it they use the CVM by by design by default meaning that no prompts no responses no data is ever seen by anyone besides the user themselves right so that's a effectively Holy Grail of uh decentralized AI um yeah so that's uh uh that's the nice thought experiment I wanted to share with you guys that concludes my talk uh thanks a lot [Applause] [Music]