20240322 ETHTaipei Workshop_ZKBankai @G Part 1

yes hi everyone hi hi hi hi hi can we sit a little closer starting with the sessions the first he needs the the uh cool uh so yeah good morning and yeah we contributors for ZK Bankai and I work at polyan apparently so that's an addition to me so yeah uh this morning uh we started with a few sessions starting with G who's a part of a session on NPCs so who do you awesome so uh you all you all know what NPC is right the idea behind NPC it doesn't even not know the idea okay great so uh with this session I hope to like give you the ination like how NPC actually works so uh yeah like so the problem of NC is that there are two par they want to comp function of a and are the inputs that both of these parties have and they want to Compu without trusting a third party without sharing the data to like the other person so now largely there are two techniques used to like do this one is called secret sharing and the other is like this so today I be focusing on like how it's done with secret sh so yeah this I'm planning on like taking a practical example of say if you want to compute this is the function that you want to compute and and uh a is the value that a has and B is the value that B has so yeah so how do you do this so yeah for this example be operating in a field of seven like the finite field of seven so in this case say C + 5 which is 8 in in a general like Decimal System it would be like uh 3 + 1 2 3 4 5 so one would be so like 8.7 sense uh like this can be split into like an addition and a multiplication problem right so uh first you need to like add a and together so how do you do that without like sharing a and without a having to share his data and we have to share data so yeah Step One is uh randomly like sampling so yeah individually what happens first go to like what each of these parties do like locally and then how get the data from the other yeah a what a does samples a value from like this set so for example if the value that a has uh and the sampled value does not be four So Sol it I mean oh yeah so uh yeah but the yeah once the random value has been sampled so like the random is here uh a calculates 3 - 4 subtracts like the value from the original value which turns out to be like 3 - 1 2 3 4 so 6 is the value that a has now the value that you got from subra his original value with value so notice that six doesn't anything about like what the original value is and this is the value that a will share with B uh with b and similarly B does the same with his value so say the value that we add is 5 and the random value is six now 5 - 6 like 1 2 3 4 5 6 again so now we have like two random values that are shared with the the the other person uh so right now so yeah like after this random has sample and this R1 and R2 cool now these values these randomly generated values are what we work with now so to calculate a + b which is the original uh Drop Like A plus b is what you need yeah yeah um this is a mess calulate back yeah okay you you have sh hands you have very all right cool hi hi everyone good morning I'm s and I'm going to be presenting my work on Fast performance and private Transformer inference using MPC I know those are like 10 of words and hopefully a lot of it makes sense in the next 10 minutes right so um just before I start like a quick show of hands who here is like familiar with the Transformer architecture in AI all right uh all right uh okay maybe another question who here has worked with a lot of these chat based applications like chat GPT get copilot here yeah almost everybody right and these are great of course and uh a lot of this AI revolution has happened based on this model called Transformer and this is essentially a neural network architecture that lets you train on massive amount of data in a paralyzed manner in a very very fast manner right so you can uh so this wasn't uh what people uh thought how yeah so this is the architecture that gets run behind a lot of these cloudbased uh AI inference Services right like get have copilot and chat GT but a lot of these uh create privacy concerns right so the way for example let's take GitHub copilot for example so the way it works say you uh you use your favorite ID and you send your code to the server the server then uh runs it's the inference on uh the prediction on the kind of code that needs to go next and it sends the code back to the user right so this is the general workflow but uh you kind of have a big trust assumption on the AI model provider so this is open air for this case but it can be any uh any sort of uh AI inference provider right so uh there are a lot of privacy just involved and this is not just for chat based models this is for AI any AI model uh in general so how can you solve this so we need a way to solve this privacy issue of uh of having these AI inferences done and we use a lot of these MPC techniques to achieve this right so what happens here is we essentially transform this into a two-party computation mode wherein the user prompt is encrypted and the model is encrypted as well and both the parties come together and eventually generate the next token prediction right so uh uh so I want to highlight the important Point here that it's not just the case that the user prop is encrypted it's also the case that the model weights are encrypted as well so if if a model provider wants to protect its source right if if there's a proprietary model that doesn't want to open source the data this has a way of uh the person uh releasing the model as well and both these parties come together you can generate a private inference using these LPC techniques uh right so U hopefully you kind of got a intuition around one of the techniqu that's super important with this uh from G's previous session and it's called secret sharing where uh both the parties have these encrypted inputs but they don't share those encrypt uh they don't share these uh inputs In The Raw format with each other but they rather communicate among each other using this technique of secret sharing and eventually they come together collaborate and generate the final result and and the problem with this approach is if you just try to throw this generic MPC tool at like a neural network model right it's it's extremely inefficient because uh uh all so if if you go down into the neural network architecture there are a lot of uh operations that happen uh I'm sure you all know that matrix multiplication is a big operation that happens over and over again right uh in this lab language models but then there are these other layers as well which are called activation functions like G and soft Max and uh unlike matrix multiplication these are nonlinear right so uh so in MPC you you I want only linear computation to be done you want to avoid as much nonlinearities as possible so you so you so this is although this does not get used much this you can see from the numbers most of the time is spent on just these layers right so you you need some way of bringing down the time here right and if yeah so for a normal model if you just throw a normal NPC tool at it it take over 60 seconds to run a inference which is which is terrible right with something like opening ey you expected to get the result back in under a second so we need to optimize this in some ways right so the way we do it is we don't we don't actually use the formulas for G and soft Max directly instead we use the approximated version of these layers so internally these these layers right they have this exponential function which is a nonlinear function that runs over and over again so rather than using this exponential function you approximate that using a quadratic quadratic form right and this is much more amable to MPC computation right so if if you take a look at some numbers if if you just use the raw G which is a nonlinear function it takes about 11 seconds but you apply this approximation technique and it brings it down to like 2.5 like a 20x Improvement right so similarly the same technique has been app to the soft Max there as well so you bring it down from 40 seconds all the way down to 3 seconds right so so this sounds great but uh does it also brings some drawbacks with that obviously uh the accuracy of your model goes down a little bit so uh let's say your model was performing with 82% accuracy you just apply this transformation the accuracy dips to about 3% for this a data set called U right so you so this is not the solution this is a part of the solution you need to apply another technique to make the model more performant so with this you've got you kind of achieve High uh high speed but you kind of also want high accuracy right so there is another technique that we adopt and it's called knowledge distillation now right this is the second technique which which kind of makes the whole thing come together and makes the whole thing good perform it right so uh this is like the internals of the Transformer and I wouldn't go into it right now but uh just to give you an intution behind this technique right what this basically tries to do is uh imagine this is the raw model without any kind of approximation the vanilla model and this is your and let's call this the teacher model okay this is the V1 model without any approximations and this is the model after the approximation let's call this one the student model okay so what we are basically trying to do is we trying to uh distill the weights from every intermediate layer onto the student model so we trying to kind of match these two as close as possible right so so uh we run a normal training Loop like it happens in machine learning uh training to make to decrease the L to decrease the difference between every individual here right and eventually with this this technique of knowledge distillation you can actually [Music] get you can actually achieve the same level of accuracy that you had with the initial model right so just to put everything into perspective right so this is like the whole architecture so you you start off with the Bena pre model you apply some approximations to make it faster but then you lose out on the accuracy and then you apply the second technique of knowledge distillation to get back the accuracy as well and then you run the MPC engine over it to achieve uh these privacy guaranteed that you want and yeah at the end you can use it like any other tool like chat GP right at the end yeah and these are some Benchmark numbers so the TDS be with all of these optimizations we can achieve up to 5 to 6X Improvement compared to the vula MPC for the transform model and uh in my opinion there are a lot of other things that are underere explored this number can be brought up even more so yeah that I I'll be looking into a lot of other optimizations and the code is public and it's still work B but yeah you can take a look into this and yeah that's it thank you so much uh any questions so that second round where you're coming back and uh the SL before that you're essentially like doing another round of training that's but yeah uh just like just to like finish this so like I have settle down now so uh yeah like the goal is to perform this Edition here so a plus b is what tech to perform let call the first value R1 and the original value minus R1 is the second value so each party has two a has R1 and R2 and B has R3 and R4 now to calculate what a does is shares take the first hand party with the second party so B is left we has like four from here and like similarly also shares like one random value to a so yeah so finally a has R1 and R4 from R1 from itself and R4 from B and B has R3 from itself and R R2 from G now they locally compute like R1 R4 these twoes and R3 R and uh once this value is like uh they share it with the other party and finally the it comes out to be like it preserve like additive homomorphism so yeah the is the same as what the original computation of we would so if we have to that so 4 + 6 is 10 and uh 6 + 6 is 12 so 10 + 12 that's 22 and 7 of that that's you have any question Sor in this Paradigm do we have like operations what limits do we have on the mathematical operations so addition and multiplication is typically what we can do with this Techni so with multiplication it's a bit hard you need to use something called VI triplet right you need to generate three more random values uh to like perform uh like multiplication is like much more expensive than addition but addition and multiplication is typically what you can use does this work way too loud good for the recording recing haveing so uh thanks for the session uh the next session is from ji and we'll be talking about proofs and aggregation layer how is the aggregation being done yeah I invite J the stage [Applause] and well as you know the the dation layer is very much the technique for when you have many chains many rollups that is to communicate you own transfer value between uh one chain uh one chain and the other um this is based so the the basic thing is that what we call the bridge but at the end is a bridge when you deposit are working for example poly first thing you do is you deposit the the you deposit the funds okay and and these funds they are well they are assigned to um to a network to to to a rollup okay and then you can move collaps from one to the other okay and and how the let me explain a little bit how how how it how it works okay so if we start with layer work so we have what we call we have a three that's a pen only three is like kind of datase so every time we do a deposit uh sure we create a a leaf here and here in this Leaf what we are saying here okay we are saying the maybe the token don't know address of the token maybe Matic usdc ether whatever token token address if you want we have the have also the token origin in this case is going to be layer one it's going to be zero we have uh the destination mean the destination layer uh this may this one destination layer stadi roll up let's say we want to send it to roll up a okay uh of course here M sending 10 matics then whatever and uh there is also let's say destination address or if you want some other messages that I can pass with this transfer something that's specific to so when you are depositing there so what you want to do uh with these Tok in there okay so there destination I put destination address maybe something else or something else okay so this is um every time that I do a deposit layer one actually what I'm doing is uh uh I'm creating one of these so it's just an only just creating the tree and this tree that by smart contract the only thing that can do is just increase so every time I and putting a new le every time the get deposit okay so um if this is a this is a state and this a state um the idea is that this state in the in the rollups this state is available um so in the uh so in the rollup this state is um embedded or is included in the in the the layer to so what what we do is we put that State uh so we put this this this this root okay uh we make them available in the in the in the rollup okay so in the roll up main what you what you do is you have a smart contract where you have the root of this of this three so you can prove maybe one of the leaves so then what I do is okay I see that there is a leaf that says that uh then matics goes to this address I create a rid uh a rid uh token that represents the this case the the Matic or whatever whatever token is in there uh and I send it to the so you you claim it in the user and of course I Mark I mean I say okay this Le I already use it so you cannot unload you cannot claim these Fones uh these phones again okay this is and okay so this is and uh for withdrawing from going from from L2 to L1 you do exactly the same in L2 you have a a Dre okay we have the RO and every time that you send deposits to layer one you create the Lea in this case the token address is going to be the same token address the origin is going to be the L1 because this is the token that you are you are sending the station roll up in this case is going to be the not roll up a a is going to be the L one that's another Network or if you're sending to another rollup it's going to be a different rollup the amount destination address what you want to do in this other network and maybe some messages so this is exactly the same even if it's Layer Two or layer one actually this is what we call a local local local execute this the local execute of the layer one this can be the loal