Zero Knowledge Machine Learning and its use cases by DC Builder

models would give us useful sort of outcomes so your knowledge cryptography essentially gives us two main properties one is the verifiability of execution which stem from the cryptographic properties of completeness and soundness and the second part is essentially hiding parts of the computation that you're performing without sacrificing this sort of verifiability um when we're talking about zkml we're usually talking about proof of inference we're creating a proof that a model has been evaluated on some input and that it indeed created some output and we can verify this entire process cryptographically we can see sort of like a a simple schema over here where this is sort of like the inference um a step where you have some input you feed it into some model and out you have an output and you're able to sort of create a zero knowledge proof of this entire process why do we create only journalist proofs of let's say inference I'm not training um so training in and of itself is already extremely computationally intensive because you need to run it on co-located data centers and uh like essentially like let's say like AWS or Google cloud or like these huge distributed computers that like are working and like training for months on end it's millions of dollars that's suspend on hardware and electricity and maintaining all this infrastructure just to train a model and zero knowledge cryptography if you're proving a computation inside of your knowledge it creates a lot of computational overhead in the order of like let's say 100 to a thousand X right so it's about 100x as computationally more expensive to create a zero knowledge proof that perform the computation itself so if you want to use your knowledge training prepare to be training a model for let's say decades in order to actually create a proof for that training um so that's why we are sort of talking about proof of inference because when the model is already trained it's just running a few linear combinations and a few like Matrix multiplications and you'd be able to create a proof within that a lot more easily than with training so um here I'm going to talk a little bit about like different Primitives and how you can combine them sort of to sort of give zero knowledge machine learning so to sort of create a good uh intuition for what we're trying to do um so on the left side we see privacy and computational integrity uh if you combine these properties you get what you would call as your knowledge cryptography if you uh combine uh privacy and heuristic optimization so heuristic optimization essentially what you can think of it as is sort of like an algorithm that's able to give you a good enough approximation to a good problem that you're trying to solve um in the context of ml to give an example you can for example have a model that classifies an image as a dog or a cat and this model has some accuracy on some data that you test it with right it can have like 90 99.1 whatever right so like this is sort of like a heuristic and if you're trying to sort of optimize this problem that's what you'd call like heuristic optimization so this this heuristic optimization things that just you can think of it's just like ml pretty much and if you want to for example have like private ml you could use something like fully homomorphic encryption um fully homomorphic encryption is the type of cryptography that allows you to encrypt an input perform some operations on some encrypted input then you have an output that's encrypted and then when you decrypt the output it will have that operation performed on top of it so you're able to have data privacy uh if you combine uh ml with computational Integrity you get validity ml right so you only get this proof that some computation happened correctly but you have you're not hiding any of the computational steps uh and if you combine it all right like if you get privacy and heuristic optimization and computational integrity it's sort of like what we would calls your knowledge machine learning also one note that I'd like to point out is that the difference between privacy in terms of ZK and the privacy in terms of fully homomorphic encryption is different because fully homomorphic encryption allows you to have a private data whereas if you're doing uh zero knowledge ml the data is not private it will always be available to the approver but the approval will be able to perform some computations and prove that those computations happened without revealing the computation but it will always have access to that private information so that's like an important important step to note um yeah so this is just uh repeating a little bit of what I said before you have like more standardized definitions of what they mean and I'll also talk a little bit about like the difference between zero knowledge and validity um proofs in our in our space many people are sort of um conflating the two concepts right so for example when people talk about zero knowledge Roll-Ups they're not really talking about zero knowledge Roll-Ups um they're talking about validity Roll-Ups like creating proofs that some State transition happened correctly but you're not hiding parts of that state transition right like things darknet scroll uh or any other ZK roll up um right if you're actually hiding set computation think like Aztec right like where you're having some like private transfers where you don't see balances or things zcash that's actually zero knowledge cryptography not um not validity where like uh validity you're essentially doing as your knowledge proof without hiding any computation right so like when you're talking about validity ml is just not hiding any computation but actually proving that it happened um so I'm going to talk a little bit about the current state of the art uh of what we're able to do with zero knowledge machine learning in terms of like the technology and how far it is along um so um zero language machine learning is at its infancy um it's just like currently like zero knowledge Moving Systems have become mature enough that people have started exploring sort of creating proofs of more computationally intensive operations uh such as like machine learning models right so it's in a very early r d phase research is as like there's a bunch of papers that have come out in the past few years uh there's a few tools that I've tried that have come out like better more easier like ZK proving systems and better apis for these so you're able to create actually like groups of more more complex operations and so here I'll mention my friends for modulus Labs we're sitting in the front row um they recently released a paper called called the cost of intelligence where they tried to Benchmark different proving systems uh against a different size models and um so here I'll give you an example from their paper of what you can actually do nowadays uh you're able to prove a model of about 18 million parameters in size in about 50 seconds running on a powerful machine on nlbs by powerful I mean 64 cores and 192 gigs of RAM so it's a really huge machine and they're used for this specific one they use the plan K2 proving system built by the polygon zero team so now I'll show you sort of like a graph of comparing like all these different proving systems um for the ones that are interested in the in the list there's Gemini gross 16 Halo 2 blonky 2 Winterfell and approver that was custom built by a paper in within a paper called zkcnn and on the x-axis you can see the number of parameters and on the y-axis you can see how long it took to create a proof given a approving system for that specific model right so the the the one that's on the lowest that's like orange dark orange um is the one with splunky 2 that I just mentioned earlier the very last dot on the right on the bottom right so I'll talk a little bit about the tooling and like the state of the art like what you can actually use today um so this tool is the first one that I mentioned is Ezekiel um so by the way like these names are just the what you the slug that you have on GitHub just like if you go to github.com that's like the actual tool so you can use it um so Ezekiel is a tool that allows you to create proofs of on the next computational graphs on an X is this like standardized format that you can use to export machine learning models from different Frameworks so if you're an ml engineer let's say you use pytorch or Keras or tensorflow you're able to take on a Model that you have within your framework export it to nnx and then feed it into Ezekiel and create a proof that that actually happened correctly um Ezekiel is using a fork from Halo 2's proving system um that was created by the Privacy scaling Explorations Group by the ethereum foundation and they're working on an easy to use API for developers so that if you're an ml engineer you can easily prove um these computations uh another um tool is circumlet ml who is built by Dr Kathy so um so she she built out of this library that allows you to sort of create a model um with different layers using the circom domain specific language which was originally built by the Island 3 team right so if you want to build a simple model you can use circum for it you can use that one and if you have a Keras model you can use Keras the circum which is just a transpiler that allows you to convert any Keras model to a circum representation and then create a proof of it another good mention is tachikoma from the linear a team I just used the zkml handle because that's the one that they have on GitHub but the team is called linear a and essentially similar to Ezekiel where they're trying to create a proof of inference of some model in a standardized way of course it's early days more people are looking into ckml more people are trying to explore with it's primitive what they're able to do with it and so more more teams are building out different tools and I'm sure like in the next coming will improve severely and so now I'll talk about his use cases um the reason why I put the use cases in the end is um because sort of people um like usually work it backwards um I try to sort of work from The Primitives that you have and then trying to combine these Primitives to sort of think of useful use cases and also trying to give some useful context about its current constraints so that you can sort of think of like what you can actually do what you cannot do so like give you some limitations of what is possible today so here I'll mention a few of them um the the use cases that only use validity proofs where you do not hide any part of the computation is let's say machine learning service providers right like machine learning as a service if you want to use a model but you don't have it locally on your train like on your on your physical device you can just um let's say delegate it to open AI to run these models for you and um they're essentially you you as a user do not have any guarantee that they're actually running the correct model right let's say that the newest model is gpt4 and you want to use gpt4 but instead they give you gpt3 because it's easier to operate and you're not you're not able to tell a difference so in in a possible future when you're able to create proofs of these large language models for example uh you'd be able essentially to query an API and the API would have to give you a proof that they actually run the model that you want and they would not be able to lie to you about the model they're running on the API since it's a block box and you don't have like you're not able to see what they're actually running on their devices uh another one is verifiable on-chain classifiers or regressors right so this is um this is so that you're essentially able to verify um the result of a machine learning algorithm on chain without having to perform the computation on chain right like to perform a computation inside of a distributed system like let's say a theorem is extremely expensive uh and so essentially what you'd be able to do is run it locally create a proof locally and then you'd be able to put it on chain um a few use cases of this could be for example creating a proof that um let's say that you want to predict the cost of like housing somewhere right so you have some inputs which is like square meters uh where it is in the area how many bathrooms all these sorts of things and you'd be able to prove like to a new input um that like this this is like the actual result uh to a Smart contract and then the smart contract could execute some logic based on the result of this machine learning model um another one is anomaly detection or trial prevention so there's many models that are sort of uh built to sort of detect anomalies or things that are not supposed to happen or even like fraudulent transactions right so you can train a model that runs on some odd chain activity and if this model finds an anomaly or a fraud transaction and then labels it you can for example have a governance protocol that sort of determines that we can stop a protocol if we find some fraud right and this is established as like a zero knowledge proof of some fraud and like the the actual protocol will be able to stop if and only if you provided a proof that some model that people have agreed upon is indeed find has indeed found some vulnerability to be able to automatically find stop protocols if somebody finds an anomaly um another one in terms of like more like zero knowledge we're actually hiding computation it's uh for example like the centralized goggle so for context cargo is a platform where you can do sort of competitions for trying to Source the most um the most efficient model for a given task so let's say you want to build a car the dock classifier and you want to have the highest accuracy you can submit account competition and different people will compete to get sort of this this uh this price that a person uh set up for a competition and if you win then you get the price however this relies on a trusted party which in this case is kaggle so the person running the competition gives money to kaggle and then the person uh running or like applying for this competition submits the model to kaggle and once all of those parties agree then kaggle just like distributes the money to the person that won the competition and it gives the model to the person that created a competition so instead you would be able to make it decentralized in a way where a person would be able to create a zero knowledge proof of how accurate their model was without revealing what their model actually is and then they'll be able to automatically claim um the the reward based on submitting this proof um to to the party that created the the competition another one that's useful is for example running inference on private or sensitive data and you'd be able to essentially prove that let's say that a classifier that's trying to find whether a patient has cancer or not you'd be able to improve with with some accuracy that you run some model on the sort of sensitive data and you'll be able to hide after this execution trace and you'll be able to prove to either the patient or let's say like some insurance company or to like the actual medical expert that they they run some model and they actually classified the result in some way um so me personally I work at worldcoin we're building a privacy preserving proof of personhood protocol um I'll show you in a little bit later after the presentation if you want I have the the sort of like orb device that we have it's a it's an iris scanner essentially and the way that we create a unique identifier is running a machine learning model right however if we ever uh think of updating the machine learning model then the people would have to go again to a hardware device and essentially get onboarded again however if you use zero noise machine learning um you'd be able to create a proof um on their device on their on their device on their phone that they indeed created this sort of identifier correctly without having to sort of go to this physical device again um also we'd be able to make this sort of device trustless in a way where you'd be able to prove um sort of all of the all of the fraud sort of or like civil attacks that you're trying to perform whether it's like tampering with its firmware or tampering with its sensors you'd be able to essentially prove that all of it all the device is running like the correct algorithms on it and that is actually checked for for every single thing in order so you cannot attack the the system so this is like these are like one of the few use cases that are most prominent in my opinion uh there's of course many more many people are thinking about what they could use these Technologies for and as there is better Hardware there is more optimized proving systems and like as in general like the state of the art evolves I'm sure that many more use cases will will be available um so where to learn more where can you find more about zero knowledge machine learning so here I have three QR codes um one is the zero noise machine Learning Community essentially a telegram group where um I created like a group for me and my friends originally that were interested in zero launch machine learning and we've started like talking about the topic right whether it's about like the state of the art in terms of like papers and the scientific side of it or whether about implementation some cryptography things or just in general like hang out and talk about like cool interesting things that we're trying to build uh the middle one is called awesome zkml which is a resource aggregator that I put out that I put it together essentially that has like a list of all the common like resources on the paper the scientific papers that talk about zkml the code bases that that like mentions ekml are trying to do zkml related things companies that are building in the area and so on and so forth and the third one is sort of um um like a pitch to a zero knowledge hackathon that we're organizing in Lisbon it's called ZK hack Lisbon it's an in-person hackathon that starts on the 31st of March ends on the 2nd of April and essentially a hackathon for any builders that are interested in zero knowledge cryptography and there'll be many people building with zkml Primitives as well uh so whether you want to build with Ezekiel or any of the other tools that I mentioned or you have like a cool idea we'd love to have you there cool so that was my talk um I'm ready for any questions that you want that you want to answer ask me if you do not manage to ask me a question feel free to send me a DM on telegram at DC build 3r or on Twitter or just just pull me aside once I finish my talk um so thank you for listening [Applause] [Music]