EthCC 2: Shawn Tabrizi

I guess I should plug right now that if you are interested in big building on substrate and you can have no idea where to start tomorrow at 10:00 a.m. we have a workshop set up so you can definitely show up there look at the calendar I think had the most recent room location all right okay so I say my name is Sean Tabrizi I'm a developer at peri technologies and today I'll be talking to you how substrate enables for cleese runtime upgrades or forecast blockchain upgrades a little back on myself I spent four years working at Microsoft working on cloud identity and cloud infrastructure then during my time Microsoft I got interested in theory em in the theorem gap development and now I'm focusing on substrate and subject development mostly working on building runtimes and something that you heard about and something I'll talk about more in my talk so to give you an overview first we talked about boxing upgrades today we've talked about substrate rust and wasum as building blocks for substrate for CLIs upgrades and of course how you can build a substrate at the end of this talk so I guess to jump in blockchain upgrades really what we're talking about here is software upgrades and they saw FERC written notes so as you upgrade all the notes to network then the actual network will change and you know act accordingly to the upgrades that you get on the nodes there's really kind of two ways you can do an upgrade we have this idea of a soft fork which is kind of a kind of a nicer way to upgrade I kind of limits what you can do but in general the nice thing about soft Forks is that old nodes will still maintain compatibility with the network so you can have a bunch of new nodes a new software out there the own nodes will still accept and agree to the blocks they're being produced by that network but what you still have to do to upgrade the network is to have a majority of nodes actually upgrade that software so even though all denotes kind of stay on the old software and cause you to work the majority of nodes have to upgrade such that more block and produced with the new logic than the old logic but slop upgrades or something like we use all time and it's actually a really kind of good upgrade mechanism the kind of thing that we talk about right now is hard forks and this is kind of a more I guess the way you can do a more disrupt it's more disruptive the network but allows you do like larger upgrades pasting hard quirks are changes to the protocol or to the code which is incompatible with older software so as you create this new software the old notes which don't upgrade will not be able to be talked to or gain consensus with the new node software and really you know in order to maintain consensus the network all nodes and all miners have to upgrade to the software in order to maintain partner or they will fork off this is the hard fork part of it and it's got two ways this has happened in the past you have these like contentious or disruptive or disagreement upgrades things like you know Bitcoin to Bitcoin cash theorem classic where there might be some disagreement how we should move forward and two kind of networks spin out but also you have things like you know that are planned other upgrades things like you know the constant upgrade that just happened reom is a plan upgrade and you know usually it's not contentious and everyone kind of goes in upgrades but still you you have the possibility of forking your network and I'm gonna talk about that so let's look a hardwork upgrade so you can see we have a block up top we'll bring some old logic and we decide sometime you know some block number we're gonna have all these nodes you know upgrade their software and at this block number change to a different you know consensus algorithm are changed a different protocol which is something that's not compatible to old one unfortunately you know actually getting all of the nodes to upgrade their software is a big task and not everyone will open at the same time so you end up happening for a period of time maybe even forever is that having old nodes producing blocks from incompatible with the new network and you actually have this fork where you have a block chain of new logic i blushing of old logic and this is kind of not a great thing for you know block upgrades it makes it kind of the whole process really Kanaskie and what we've seen the past is there is a need for upgrades you know block chains are meant to become these persistent persist for long periods of time and you know there's no perfect watching today that's what we're all here kind of developing blockchain technologies and if there was they probably won't be for long you know as time and technology develops you know you want to also be able to iterate your blush in and make it usable for the modern day so you know think you might want to upgrade still you know in the future like things like important security vulnerabilities you know changes to rules or the protocol you can add new functionality or you can even repair change state and you know in general is mentioning like you know blush and upgrades or hardware upgrades we've heard a lot of coordination you need to have all these people who are running notes or running the infrastructure to know when the upgrades gonna happen to download a new software run it you know maybe take another node at the time that they actually do the upgrade and you know this is such a kind of a hard problem even the information right now is asking for people for a person as a hard work coordinator for such a job so this is really you know a big task and you know comes even a bigger problem when you start talking about governance and signalling towards like how do we make an agreement on what the right upgrades are and what the new software should be and if we should run no or not right this whole kind of process is not really handled I guess in a in a fixed way so it's kind of hard to understand exactly what should happen with upgrades so when building substrate we decide to make upgrades a first-class citizen of the entire process so before I jump into that I want a table about substrate so I'm sure all of you who have been kind of hearing about all day but to give you a quick overview substrate is an open source modular and extensible framework for building block chains it's really you get all of the core components needed to build a block chain things like the database layer the networking layer consensus engine transaction queue library content modules and the best thing about this is that all of these if different components are customized are customizable and extendable so that you can go and you know take this framework and take your idea of what you want to do take the pieces that you don't need to rebuild you know know that you have some like production level ready level software but then also modify the things you do want to modify and get your own custom logic whether it be in your networking layer maybe be a new runtime module or anything like that so let's talk about the library runtime modules so what is a runtime the runtime really is the block execution logic of the blockchain or instruction it's like all of the actual things that the blockchain does from block to block is handled in this runtime and in substrate it's composed of these things all runtime modules so you have these were modules and you should all have their own kind of like special functionality and you can pick and choose from them like we've done here and create a entire runtime which actually just finds like what your blockchain does we have this again I mentioned a runtime module library which comes to substrate and again all of these things you can choose and pick from and actually build your own custom runtime and of course you can create your own - so let's look at the runtime and how it fits into the overall substrate architecture so you can see it's actually the runtime is wrapped with a webassembly consensus in networking layer these are all written in rust and built into a native binary but then you also the runtime itself which lives inside of the web assembly which is also written in rust but it's built into all the native and wasum binary and this is really powerful we're going to talk about these technologies and kind of how they work to enable this for quest runtime upgrade so one of these might notice that the entire stack of substrate is written in rust and it's a new system programming language which is uh sort of on Mozilla and no preparing Firefox and it's actually a tool that we have parody used for nearly all of our clients so uh it's just something I wrote about so like why do we choose rust well as I mentioned parity is a rust team as much as a blockchain team but you get a lot of really amazing things when you use rust to build systems you get inherent safety so rust ensures that programs are free from undefined behavior any data races any memory safety issues that never gets all free when you compile your code if it kabah successfully you get these things for free with rust rust is crazy fast and most importantly it's very lightweight so it has almost zero runtime overhead and just really really fast lightweight code then I've mention also we have a website believe so we use we use web assembly for the runtime as kind of a general platform independent execution format what and wasman is great for this runtime environment because it's sandbox Abul uh you know the prek user from buggy malicious code it's deterministic when you remove floating-point arithmetic which is obviously very important for blocking systems it was built to be very close to the native it's really close to your native execution engine so it runs really quickly and can go in your native speed and then also wasn't very compact so it can be easily transfer of the web and there's built as a web format and so as I mentioned you both rust and wasum also work my veil together you can see that rust builds directly to blossom which allows for the same code base to generate both the Watson binary and the native later binders that you can run in your own computers so without having to like maintain action from force of a code or having to maintain different pieces of Technology you can have a single codebase generate both kinds of binaries and we have built a wasum interpreter for in rust that allows you to run was in binary and it's something that we've integrated into substrate and as I mentioned because rust is very compact itself and wasn't very compact you actually end up with really small Watson files which is kind of how the runtime actually lives when you finally when you finally build it so if we think about okay well I have this really small wasn't file what we can do actually now which we couldn't do before is actually store the Oise and file on chain and this is really the magic that allows us to do these for festrunk i'm upgrades that this small wasn't probably actually put it on the blockchain and have people get consensus and agree or what is the correct runtime of the blockchain at a certain point in time and so we take a look the logic that actually enables what a portal start time upgrade does you see here we have the Oise and on chain and we also have the Native Client running if the version of the runtime on the chain is different than the Native Client that you're running we actually fall back and use the Oise in runtime and awasum interpreter that we built and actually execute blocks using that version of the code that's on the chain and that has consensus if the version of your runtime is the same as it is on chain then you actually just run the native the native binary on your computer and it's much faster but it ensures that all points in time that anyone running your chain will always be consensus because you can always know that everyone is running that at least the same runtime logic but their same version and it's really this kind of fork in the code that enables us to maintain consensus through upgrades without having to cause any quirks so one of things talked about you know when you turn over to phrases you know it's obviously a very powerful process and needs to be treated with such so we talked about you know how can you just update your chain we all sniffing about like okay we're the governing rules around upgrading your chain so the runtime code is stored in our substrate database which is they see a key value store and is stored under a special key called code and is accessible through our consensus modules one of the modules that you get when you build on substrate and you can actually call this set code function and waise when one of the modules you provided something a pseudo routed module which is great for if you're doing like you know testing or doing like you know private networks where you want to have a single key be able to control the runtime and that key can basically call into the pseudo module and directly change and upgrade the the code that of your blockchain for more like public or in distributed systems you we have an entire conching governing system built across the number of modules which allow you to propose and have people vote with the democracy and via councils what upgrade should happen and make sure that you know if an upgrade happens Network it's something that was decided by by the network and you know it's built in the distributed the kind of decentralized system and then finally you know you of course if you're building on substrate you can control run some upgrades with your own moderator and logic so you don't have to use our Unchained governance system if you have an idea for how you want to do you have some other alternative way you want to touch the runtime logical upgrades you can of course build your own module in logic for that and I guess the most important thing talk about is right upgrades are optional so you know you could also build a block channel substrate which doesn't have workers at all so if you want to make sure your um your block chain make sure has a fixed state and never changes that you can go and just remove the runtime upgrade option and you just never have to worry about this kind of this pathway so the one thing I want talk about is creating gap chains and substrate so substrate is of course built to be a really powerful platform for building you know any kind of block chain things like utx no chains like Bitcoin smart contract chains but I think one of the things that are most accessible right now especially to you know you all here at a theorem conference it may be developing adapts and build adapt chains so I want to talk about some of the I guess you know the problems with modern apps and their piece of scaling issues so right now if you're building adapt your gonna you know run to problems where you're gonna have to compete with other gaps on the same network that are trying to compete for you know basic transaction priority resources you're gonna have to take the security of whatever a network you're on so even if your DAP is not maybe the most like you know important like you know multi-billion dollar service is to be something fun like a game you still have to maintain the security of whatever that you're on and that can be expensive not really performant for your DAP and then of course you know upgradable contracts are a thing but usually it involves having that contract be controlled by central authority so it's really you know there's kind of some problems here with taps on the current platform but I'm proposing that gap chains on substrate can actually solve some of these problems so as you mentioned things like transaction resources transaction fees when you control your own blockchain you control what the fees you impose on your users who are using your blockchain you can control how the resources are managed and how they're um you know basically how your blockchain runs entirely you you can choose the right security model for your blockchain you know if there's a consensus protocol or a different you know system that you that you think is beneficial that has the right ways the right options between security and scalability and speed you can of course choose that because you control the entire blockchain stack and of course as I mentioned before runtime upgrades you can control how the upgrade ability happens whether it be something you want centralized or something you want to committed in a more on chain governance system and that really makes it really powerful so uh you know one of the things I wanted to ask or you know want to show you here is like what would happen if we have the crypto kiddies like that but it was put on substrate and that's something I want to demo to you right now by doing an upgrade so you have to give me a second I doing a little bit of hack here cuz I wasn't able to project from my screen I'm gonna be sharing my screen over a chat service and see if this works so give me one second and then me one second over here let's connect on Zoom let's see if this works across your fingers here this is true bootstrapping okay let me share my screen over here and it might get a little bit of lag but it looks like it's working yep okay so I'll show you everything close this window it's blocking the view I hope this is continued okay good I did not break anything okay so uh let me show you what I got here so I have a project open um this is a subject project there's two folders here there's something called subject kitty the securities you I basically substrate kiddies is a kind of book called the substrate no template it's basically something you could download out of the box and you can basically aren't hacking stuff you right away in this no template I've only really changed one file which is my new module which is kind of controlling this new substrate kiddies I guess functionality of my blockchain and then the second folder here is the subject um such a key DS UI which is a react app which you can also just download out of the box and it will run it will work out of the box with the no template and then you can of course modify it to add new UI elements or new functionalities that your blocks represents so if we go into the substrate kiddies project part of substrate you can see here I've introduced this one file and actually breaks down policing please I've minimized bunch of the code but if you could look at the overview here we have a structure which is they pick kitty struct you know it has things you might expect like DNA prices generation we have some an event type we have a macro for declaring all the events in our in our module so things like you know Kitty being created a pricing set things being transferred this is like again this looks a big a minimal version of what cryptokeys might look like on substrate we have a declare storage macro which declares all the different storage items that our module manages and course we have a declare module macro which goes and actually creates all the public functions that you can use to interact with this module on things like you know create a kitty set a price transfer by key these are all the things that are there and so what I'm gonna do real quick is I'm going to run this once again now also make sure that there is a yep my node.js project is running so if i refresh this page you can see I have a blockchain running so you can see here that as a height one and if I go to the Explorer and I make this a little bit smaller let's see we should see a new block being produced no yeah so we see new blocks in produce so I have a local blockchain running here what we can do you know some basic blushing stuff here to begin like send fun so I'm gonna send funds from Alice account and Alice is pre funded with a bunch of money and sent to myself so let's do that real quick so myself ten thousand units and it works okay so we have here and we see some events that's great and then we also see here we have action by module here so I really I've already written some UI that allows me to create a new Kitty so I can write my name create a kitty and when block gets produced we should see a new cat up here so what I'm going to show you now is basically we're gonna add new functionality to my runtime dual runtime upgrade and show that my UI can reflect that those new changes so let's actually create two cats because what we're gonna do is the new functionality were gonna build is breeding a one second real quick so over here in our runtime I have written some code and we can actually walk the real quick real quick it's a new function called breed kitty basically what we're gonna do is run take two cats use their DNA mixing together and produce a new cat using those DNA and hopefully this cat will have elements from both of the the kitties that were used to generate it so if we copy this code and we add it to our declare module from macro and I save this what we need to do now is rebuild the wasum so again I've mentioned the runtime is stored as a wasn't file and we can do is run this script which will actually generate a new wasn't file based off of the new function that we are then entire new runtime that we've built here which includes this new function you see here okay it's created now want to go back to my UI here and select the runtime to be upgraded so you can see here this file is created a at 5:30 to it is 5:30 to will click this you'll press upgrade and the last thing I want to show off here is that my UI has um basically an if statement where if the Breede functionality doesn't exist it will hide some UI elements but if it does exist it'll show you some new UI so before you saw we can only create a kitty but um if the brief functionality exists you'll see some new buttons a new UI then which allows you to breed a kitty and this is kind of one of the powerful things you can see how upgrade happen I'm live so here you can see that there's a event that was released called sooted which is again from the sudo module saying that we have upgraded our chain and if i refresh this page and scroll down to our kitty app you actually see there's new UI elements here which represent hey we have a brief functionality and let's actually try to use it so if you have these two cats this one has a lot of calls grey with a spot this one has a scarf we take their DNA and we try to mix them together so I'm doing this one-handed right now and we have to give it to someone so let's give it to Alice okay that's pretty kitty and hopefully the rng gods are good on me today we'll get a new cat here I guess a this was a hell of a scarf it has the flask that this one has you know it has this smile has the same color and you can see here we've basically taken a chain adapt chain here added a new functionality to it upgraded it without any basics any problems any Forks or if you have to restart your node and we were able to basically introduce and upgrade our chain and fork is better thanks okay let me exit here I just have a one or two more slides at the end let's go back to here we did an upgrade and it worked good okay yes so I don't talk about this workshop so everything I did and I showed you right here about building a cryptokeys chain on substrate something that we have available as a workshop so we can teach you how to run a local substrate node run development teach all the best practices building a working chain with UI and even if you have minimal rest experience we are there taking that into account making sure that you're able to get through the entire process and kind of learn how to build your first chain and so there's something that we offer and I think anyone today here I can use this URL and pick up and build the chain that I showed you today including a runtime upgrade and as I mentioned tomorrow we have a workshop at 10:00 a.m.

where we will myself and another developer a parody will be working through this this workshop and basically talk to you through the process showing you how it's done answering any questions and making sure that you are able to leave today or even work on this hackathon this weekend a building on substrate so this is a really great first up for anyone who's interested but on substrate we make it as easy as possible with the tech with a hopefully an idea that you're all familiar with and translating those ideas to substrate and substrate worlds yeah so let's go overview I talked about substrate about rust and wasum and how they enable on these very small wasn't files which can sort on chain which allows you do these four plus upgrades and final items about the workshop which you can all access today and learn how to build your first substrate chain we're hiring if you want to if you're interested in rust interesting blush development you can go to $30 slash jobs we have an events and updates subscription that you can go to on the newsletter and finally you can ask me any questions if I still have time Thanks [Applause] questions for Shawn we're running low on time so one or two great talk Thanks so the if you're running your own blockchain it's the sort of liveness of it depends on the there being at least sort of one node online at any time what happens if it's a very small group of nodes and they're all offline is the blockchain sort of pause and continue or is it relying on liveness from from another source yeah I think these are the substrate nodes aren't much different than any other blockchain nodes so if all the Bitcoin or the three nodes went offline blocks are being produced and you know the network would halt so in this demo I only have one node and running locally if I stopped it yeah blocks are stopping produced and if I start it again I'd pick up right where I left off you know I'd be able to you know continue building new blocks off of the last head blockhead the substrate is built such that you know you can produce these node binaries release them and they'll automatically fine each other using network layer reusing the p2p and as long as there are nodes out there there you can oh you can basically this network reproducing blocks that other other people using your same blockchain can find and yeah I mean yeah that's cool okay great thank you any other questions all right okay thank you [Applause]