Marlin 🛠 Web3's off chain compute marketplace

foreign [Applause] [Music] in this presentation I'll be walking you through the different features and use cases of modeling and how you can offload intensive compute onto it double secure and scalable decentralized applications the flow representation is as follows we'll start with a brief introduction to Marlin really quickly skim over the different services that the Mali Network provides and this will be followed up by some example use cases which might even Inspire your project once you have an understanding of the capabilities you have access to when using Marvel I will walk you through some tutorials for using some of these features specifically using valid for privacy preserving or confidential Computing offloading the operation of backing Services onto a decentralized network as well as Outsource growth generation for GK applications finally we pointed to some appropriate resources and channels to further your exploration modern and short is a network of nodes that make their Hardware resources available to users under a ps0 model these resources can include anything from a vanilla service to trusted execution environments like Intel sjx or AWS Nitro enclaves gpus or even fpgs for Hardware acceleration of say AC proofs or storage and bandwidth for caching a trusted execution environment of De in short is a special piece of Hardware in computers where record and data is predicted from interference or leakage to external processes any code or data sent to the TV can be encrypted and decrypted only inside the deal with special key is ensuring that when you send private data like Identity or like Financial or medical records to someone else's device the owner of the device or any program running in the device is index able to access the data since the data might include code itself programs can be run securely in these environments with the assurance that no one can manipulate the execution of the code and that its output is interpreted sdes can be used to serve HTTP or PC requests like entire backends can be offloaded to a network of tees to decentralized services that you might otherwise run on a centralized server building a zika based applications involves a proving step which is computation intensive but can be optimized using gpus and fpgas some applications require that such operations we run client-side to ensure privacy of like secret inputs involved resulting in minutes of wait times for end users however such computations can be outsourced to the Mali Network where even the secret keys can be secured by running the operations inside secure enclaves so under normal circumstances validators control The Ordering of transactions within a block however interesting applications can be built if like ordering preferences of transactions can be communicated to validators so users who are familiar with like using flashbots on ethereum can now use it with polygon as well using the Malian relay so as mentioned earlier these can be used for building decentralized applications where the confidentiality of data and integrity of execution matters for instance identity for credit scoring and insurance protocols applications that involve running machine learning algorithms or private data sets in fact even private keys can be shared in certain Clips so one can deploy a VB Bots or applications that use such Bots for example there are Decks that shares Mev profits with users entire backends like trading engines for a closed order book decks gateways for ipfs RFQ based access Etc can be deployed inside Netflix can be used to build a games wallets voting apps ml applications and a lot more and if you want to have the provers Run efficiently without without utilizing all of your or your users resources just proving operations can be outsourced to Marlin you can also build Integrations at like existing zika ecosystems by Outsourcing some of their Proverbs to our Network using our flashbot today you can build Rescue Bots that help users rescue funds from hacked wallets Bots that help users get priority doing and actually means I may be sharing taxes that are shared captured me with users and a lot more so without further Ado let's jump into some walkthroughs for some of these features in this tutorial we'll take a look at how to create a simple node.js server and deploy it inside not click the tutorial that we'll be following for this is the one from our docs you can find it under user guides on Place tutorials deploy.js server I also have with me uh fresh Ubuntu 20.0 for instance where I'll be running all the commands so before you do anything uh we need to install dependencies that we are going to need later so we need Docker to actually build The Enclave image so let's install that server let's go ahead and install that as well and we also need to install the mask but I already have it installed so I'm just going to skip that step great let's move on to the next step now so let's actually create our node.js server so let's create our working directory for it first and let's just create a basic node.

js project great so we'll create our server using Express so let's install that and let's actually create our server file great so you can actually deploy node.j servers directly inside and place which install node and use it to use that to run it but if you were seeing our steps from before installing node it takes a lot of time and then installs a lot of stuff so that basically loads the image size significantly so what we are going to do is we're gonna use PKG to package up the server file and node.js into a compressed binary so that you can just run it directly without installing any dependencies so let's go ahead and do that I would like to install the pk2 first and let's actually create the binary great so if you see I have a an app binary now and you should just be able to run it and yeah it just opens up a server that's listening on Port 4000. let's now move on to creating our Enclave image first we will set up a development environment and for that we'll use our Nitro CLI image which sets up a developer environment for you out of the box so you don't have to do manual installation steps so let's do that great it's running now I will just open a new terminal and leave the other one running and we'll basically get a shell inside a r container using this so this container basically comes pre-installed like Docker and Nitro CLI the most important parts that you need to actually build down click so let's navigate to uh our server here and we'll see that a server from outside has basically been mounted inside the container so any changes that you make to files uh directly on the instance itself will reflect over here the next step is to actually prepare our image specification and for this let's go to this link what we'll be using is we'll be using the default image specification that we provide and we'll basically just be customizing all this stuff so let's first create a Docker file I'll just open a YouTube for this let's go inside yeah let's create a Docker file first the next one is setup.sh 3 back and and finally we have the supervisor conflict let's also set that off so something that you'll notice is that uh all of these files basically have customization points uh built into them where you can basically put our custom setup code and that's exactly what we'll be doing in this tutorial so now that I've copied the files let's go back um yeah so the docker file basically sets up a lot of like default networking and black supervisor related stuff for you and what we are going to be doing is we want to run our server uh when The Enclave boots up so we'll just copy our server Into The Enclave first in the docker file yeah and the setup.

sh file doesn't really need any modifications and what we'll do is we'll edit supervisory to basically start up our Enclave program set up our server program when The Enclave starts up so let's copy this so this will make your server startup but one thing that you need when you're running inside enclaves is uh you need a proxy to basically expose it to the outside world and not just inside your account clip so let's set that up as well great yeah that should do it so let's go back to the terminal that we had before this one and let's first build a Docker image based on the docker file that we just created okay and you should be able to see the image that you just made over here concrete latest now let's use this Docker image to create our Enclave image great and it basically prints some measurements over here these are used to ensure that the Integrity of The Enclave is maintained let's move on to the final step which is uh deploying the image on the model Network so here one thing you need is you need to host the image somewhere where it can be like publicly downloaded by nodes in the Mala Network so for this purpose I have chosen to go with the service called web3 storage it you can upload files to the service and it stores them on ipfs and like pins and I'm using filecon essentially so I have already uploaded my image over here that you can see and it gives you a URL uh if you just right click and click copy link it gives you a URL that you can use to basically copy the own clear image so once I've done that uh let's actually just verify uh whether the image hosting and stuff is working properly so let me just copy the link address let's go here and let's try to actually download it one thing about W3 storage is you need to manually add the file name at the end but yeah seems like it's able to download it in which case it's hosted properly so that shouldn't be an issue now let's create a job for this for that we go to the UI that's hosted over here using the URL link and what we do is we basically fill in Fields as given over here so for example this is the provider that we run for eth India so we put that over here and we put 250 as the deposit amount that will make your own place around for like roughly two to three days and we also give the link that we just saw to the victory storage yeah we need two steps one is we need to approve uh the contract to actually get your tokens let's do that yeah it's done and let's now create our job that should be done as well yeah that's confirmed so once you've done this you can basically come to us and we can give you the IP address of the Enfield that you just deployed so in my case it happens to be this so what I'll do is to ensure that The Enclave is running uh I'll just try to call 4000. yeah there we go there's the response from a server that we just deployed and so that concludes uh this particular tutorial and thank you let's try something a little bit more complex now in this tutorial you'll learn how to perform privacy preserving compute inside enclaves this tutorial is also from our docs you'll find it under Oscar tutorials more specifically we are going to learn how to load numbers inside an enclave also how to perform compute on them in in this case it's going to be addition so we just expect The Enclave to return the sum of the numbers that we are going to be loading inside it I have an Ubuntu instance with me where I am going to be running most of the commands let's start by installing some dependencies so I need a Docker to build the entire image so let's install that most of our programs are also written in Rust so let's install that as well uh let's install a few more uh programs that we need to actually build downloads you also need to install a task wallet but I already have it in web browser so I'm just going to be skipping this step let's go on with creating our project now let's create a working directory for it first and let's just create a basic cross project let's Now set up our dependencies and the biodies that we're going to be building so here I just have a bunch of dependencies that are going to be used by the programs that you're gonna build and this list of the programs that let's first create the app server so the app server is the main program that's gonna do the addition inside the enclave and it accepts two types of messages one message is used by the loader to load numbers into the enclave and the other message is used by the requester to essentially fetch the results of the addition operations the Privacy is maintained here because the program for us does a different element key exchange to generate a shared secret key and it expects the data that is coming in to be encrypted by this key so nobody can eavesdrop on the data and the data can't leave the envelope either let's create our loader now foreign is the one that's going to be sending data to The Enclave uh in our case we've chosen the numbers to be 12 and 43 so our expectation is that The Enclave is going to add these numbers up and return as a result of 55. the loader also does the same diffie-hellman uh key exchange to generate the shared key and it encodes the data using this key to ensure that nobody is eavesdropping on it let's write our request or not requested pretty much just does a query of the result that's inside the enclave and it uh prints that on the console that's pretty much it let's create a key generator program that we're gonna need to generate keys for the development key exchange and let's create the verification program now if I start by creating an AWS dot set file and this certificate is the root certificate of the AWS Nitro infrastructure so but this gives you an assurance that uh you are actually communicating to an enclave or not any other random server let's create our verify program now the verifier uh basically downloads the attestation from The Enclave it does a verification of uh is that station signed properly or the certificate uh change signed properly are they expired or not and does the chain end up at the AWS root certificate things like that foreign yeah that's pretty much all our components let's now build everything that's done let's generate our keys that we're going to use for the key exchange we should have our loader and request a keys over here let's now build our image let's first set up a development environment by using the Nitro CLI Docker image that we have this already comes built in with a lot of the dependencies that you need mainly like Docker Nitro C line on that let's get a shell into this and you see that it already has a the main dependencies that you would need to build and killing edges let's navigate to our directory and then we have the directory that we had before mounted inside this Enclave now let's now start preparing the specification for our image so the images that we're going to be using are going to be based on our default image specification so you'll find it on the link here which basically consists of three main files I'll just open a new terminal for this let's create a Docker file first you'll see that the docker file has a customization point where you can write your custom setup code let's also create the setup file and same here the setup also has the customization points where you can put stuff and let's finally create our supervisor file this has all the programs that are going to be running inside the enclave and you can add your own over here let's customize the image uh let's go back here and once we've built the image you should you should see uh that we have an app binary over here that's our actual server so we need to copy it inside the enclave and start running it so let's do the copying first we also need to copy the public keys in order to support the different key exchange let's do that yeah the setup file doesn't really need any changes so we'll leave that alone and let's now copy our program specifications and start running them our server goes here and we also did a proxy to expose the server Outside The Enclave so let's add that as well that goes here so yeah that's pretty much it and let's now build that image let's go back to the term that we had and let's start building our image we first build our Docker image using the docker file that we had and now we use Nitro CLI to convert the docker image into an enclave image so this command prints some measurements uh make a note of these values because we'll be using them during verification editor let's now deploy the image so for this we first need to upload the image onto somewhere where it can be downloaded by the nodes in the network so uh for this I've chosen we have three story uh it stores files on the RPS ipfs Network and pins them using file coin uh already uploaded my image over here that's the first one here and you can just right click to get a URL link to the CID archive that's on filecon one thing that's good to do is check whether the URL is publicly accessible using just W get a call you can just download it and just verify this step let's move on with creating a job so for that I first go to the oyster portal and I already preferred in some data that's over here uh this is the address of the infra provider that you've deployed for each India and uh you can just put a deposit amount and the link that we just saw from every storage goes here one thing to remember is uh you need to add the entry aaf file name at the end manually over here just make a note of that let's go ahead and approve the tokens for transfer yeah that's done let's go ahead and create a job that is now confirmed as well so once you've done it you can come to us and we can share the IP address of The Enclave that we just deployed let's now move on to verification so for this part uh I'm just going to use an existing Enclave that's already deployed just so I don't have to uh wait for the current one to get deployed and that's over here you can see it's pretty much has almost all of the same files that you had previously and The Enclave image that was previously deployed had the following measurements and it was deployed on the following IP so I'm going to be just using this data so let's first verify whether we are able to communicate with The Enclave whether the attestation matches and all that so let's do that you'll see a lot of blank spaces here that you'll have to fill in you can fill the IP first and the PCR values once it prints the puppy like this you know that the verification was successful and it also creates an app.pop file that contains the public key of The Enclave that you can use to basically do that if your element give Exchange let's now load data into the enclave and there we go it says get succeeded and recall that we the data that we loaded with the numbers 12 and 43 so our final step should give us the result of 55 which is the sum and there we go so that's how you do privacy preserving compute inside The Enclave of course I just did simple addition you can do any sort of computation that you want inside that and that concludes this tutorial thank you in this tutorial we'll create a small circum program and take a look at how to Outsource compilation and proving onto our program Network the tutorial that will be following for this is again from our docs so you can find it under user guides Provost tutorials let's start with installing some dependencies that we need in later steps so first let's install scenarios that we're going to use to verify truths foreign network is gonna send us yeah so let's create our program now uh let's first create a working directory for it and let's create our program the program that we're going to use today is a simple multiplication program so you have two secret inputs A and B and the goal is to uh check whether you know a particular combination of A and B that when multiplied together gives rise to a public output now let's Outsource compilation of this program onto the program Network we do that by making a simple API call and when you're running it yourself you would probably want to change the entropy value to like some other random combination of letters that stuff and make a note of this project ID we'll need it while generating the proof before generating the proof uh let's we need to First specify what the values of A and B are going to be so for this tutorial I've picked C to be 1024 so I've picked a particular combination of A and B as like 16 and 64 that will multiplied together gives you 1024. so let's create our inputs file and let's now generate the proof this has the project ID blank over here let me just copy and fill in from before great now we have an output.

zip file which contains the proof let's extract that now foreign Network match with what we expect so I recall that we wanted RC to be 1024 which matches over here so that's all good and let's do the final verification step now yep seems like all good and yeah that that's essentially uh like compilation and proving Outsource to our program Network and you can easily verify that locally using Snapchats and that concludes this tutorial thank you can reach us on these platforms uh and we'll also be in person at the India event itself so if you guys have any feedback any questions or want help with the hackathon feel free to hit us up thank you and the best of luck