The Complete Guide Building Blockchains With the Polkadot SDK | Bader Youssef - Web3 Foundation

[Music] in this specific uh presentation I want to go through building blockchains with the polka dot SDK now this is mainly an intro so polka dot and the PO SDK is really a really vast thing there's a lot of technology in there and while i' I've refrained from putting in code in this presentation so you should consider yourselves lucky um I'll be glossing over the introductions to some of these key Concepts but also in general when it comes to actually building blockchains and building layer ones so it's not just something specific to polka dot but also Concepts which carry over in really any other ecosystem so just s introduction about myself my name is B YF I'm a technical educator at the web 3 Foundation uh you can reach me at any of those uh handles there so I'm going to start really basic right and I'm going to gloss over this quickly because everyone probably already knows this but you know what does it mean to build on web 3 right and web 3 is kind of this uh you you know this this kind of magical word um where we associate a lot of this in terms with it but it doesn't really mean too much but essentially what we're doing is building a more intentional internet that's fundamentally what we're doing it's something which is intentional and um actually has a backbone compared to the traditional current Internet and blockchains give us that backbone for the most part it's not the primary way to implement a web3 solution but it is one of the fundamental ways and a few Concepts I want to get into your brain because as an engineer if you're going to be engineering blockchains you need to actually be learning what these concepts mean especially if you're getting down in the protocol layer you're going to run into some of these terms so for example blockchains are essentially deterministic State machines they move from one state to the next right and this state is dictated if I give X input I will always get y input that's the meaning of something being deterministic and blockchains additionally are also of course backed by Game Theory and economics uh to where we can actually you know somewhat predict the behavior of people even though that's quite difficult in most cases um but blockchains fundamentally are made up of these three components of course and also with cryptographic guarantees we cannot forget that and like I said I want to stress that they're supposed to be deterministic and you'll see why this is important later but they're not just you know any state machine it has to be a deterministic one and a good example I like to always use is that of a vending machine because once you get um you know a vending machine has certain rules and if you abide by those rules and you give it a certain amount of coins or input then you'll always get uh that same output theoretically and block should work in roughly the same manner and the primary use of even using these right uh a common mistake that I've seen is people they especially when they're getting introduced to uh using blockchains is they want to use it as some sort of like distributed database or distributed storage and that's not quite the right um use for it or it's not using it to its maximum capability really the use of these distributed ledgers is for data Integrity we want to ensure that data is retrieved in the same state was recorded and we want to be able to do that across the board and generalize that which obviously we have challenges doing that um but that's what we want to get to eventually and now before we dive into the layer one and protocol development aspects let's look at the current state of web 3 for more context so you can kind of classify very very broadly and maybe it's not fair that I put rollups on the same level as smart contracts but you can kind of count uh or look at the web 3 stack as this kind of um uh as this kind of layers so you can have the layer zero which is like a meta protocol like Cosmos or polka dot which connects blockchains together connects layer ones together then you have the layer ones themselves which are usually application specific sometimes they're generalized like evm is generalized uh then you based off of the layer one you can build rollups and layer twos you have Smart contracts which acts as your kind of middleware and then of course your adaps at the bottom so polka dot you can kind of think as as a composer of protocols so polka dot will essentially allow you to grab a bunch of different Services compose them together into a single product because the trust assumptions already there because they're secured and backed by polka dot itself since polka dot connects different blockchains together you can assume that all the blockchains connected by polka dot and backed by its Economic Security uh will be just fine to use and on top of that there's also cross change communication between all those change automatically and a bunch of other things which normally would take you uh quite a bit to consider and now let's actually get into the me ofit so the polka SDK now this is an overview of some of the things in polka and we'll be focusing on the top one called I'm calling protocol land um but essentially you can also look at it as kind of three different Stacks right you have user land which are just daps you have uh the second stack which are smart contracts right kind of the middleware of web 3 and then you have the protocol kernel land and you can think of this as the backend of web 3 so layer twos layer ones um uh layer zeros metor protocols Etc so the po. SDK is now uh put into a single mono where you have all the tools you need to actually build all these things that I just talked about so you can use um see oh yeah okay so you can see here we have frame we have substrate cumulus and then the actual polka dot note implementation itself so substrate is kind of the bread and butter of polka dot it's the a set of libraries which take care of all the Primitives and all the things you need to build a blockchain you can think of it as kind of like a react how react gives you all these different building blocks for building a website substrate kind of gives you all those building blocks things like networking um defining a blockchain's runtime blockchain storage uh upgrading the blockchain um into One Singular uh code base and that's what substrate you know substrate is a foundation for something so naturally substrate is a foundational layer for you to build blockchains the result you get distributed State machines which can come to consensus so you build layer ones and then in other words you can also say that this is kind of a set of generic implementations for common components all across the board and then after that we have cumulus which is how parir chains or how blockchains connect to polka dot so they have to abide by the parin protocol in order to be part of the polka dot Network and cumulus also built with substrate so you can see now substrate truly is truly the foundation of polka dot and cumulus allows you to connect to Polkka Dot and it's basically a developer kit used to build out these par chains so now let's dive into what a substrate node actually looks like now you can see it's made up of three main component on you have the client which handles the lip P top kind of networking side gossiping with other nodes then you have the runtime in the middle and this is the part you'll be interacting with the most then you have the storage so storing our blockchain's state and each node uh is always usually divided into these three categories now most of the time we don't have to mess with the client or storage we only have to use the runtime and essentially the runtime is a web assembly blob there's a there's a wasm time uh runtime inside of polka dot which takes these run times and ensures that the state transitions uploaded by a pair chain are valid through this runtime and the runtime is where you put all your business logic right you can customize your chain build it however you like and now to actually divide this runtime now substrate runtime you can build it just using substrate however if you noticed there's something else called frame which you will encounter now frame actually and it stands for framework for runtime aggregation of modulized entities it's a very long name uh we just call it frame because it's easier but essentially there are modules to build your blockchain with so different types of modules ranging from consensus to nfts to even running a full evm inside of pokot if you wanted to are all implemented as pallets and modules and then you can roughly see now how a runtime can come about so you can actually either write your own pallets or include a set of already preconfigured pallets configure them and then all those compiled to web assembly as your runtime each Pet's configuration can be change you can mess with this configuration adjust the parameters um let's say you want to set certain pseudo privileges to certain accounts you can do that if you wish or you can also just completely remove uh pallets so let's say I want to remove the pseudo pallet and put in some sort of governance system instead now the whole goal of creating these run times right is to create infallible ones so blockchain runtimes on their own uh they should not be failing you should not have a runtime which stops over some unexpected user input so you as a developer should be in charge of creating defensive as well as infallible runtimes and there's some you know key things to keep in mind such as using your blockchain storage wisely do not bloat the state of your blockchain only use very specific um uh pointers to items rather than the items themselves and you want to use defensive programming as much as possible you don't want to roll your own crypto of course and all of that to get to the goal of creating infallible run times now as a use case to kind of put all this stuff together I want to talk about Frontier now Frontier is a native evm compatibility layer inside of substrate and what it allows you to do is essentially run an evm inside of a substrate based blockchain so let's say if you wanted to upload solidity based smart contracts then you could easily upload that if you have the frontier pallets already included and again Frontiers implemented as a set of pallets so you can see how extensive you can go if you wanted to you could Implement um other protocols as pallets inside of substrate thereby making your sub train compatible with those protocols you can see here that Frontier is made up of two pallets so pallet evm which lets you actually upload solidity smart contracts and then pallet ethereum which exposes an RPC implementation so you can connect metamask to a substrate or polka dot based chain and there's actually quite a few polka dot chains which you can connect metamask to if you wish and uh use an ethereum account that way so what do you need to learn to do all this stuff to create blockchain runtimes to create your own blockchains essentially right all of this is getting towards the goal of essentially uh either creating your own solo chains or par chains sub can be used for both uh you're not limited in that regard it's a very generic set of tooling that allows you to take care of networking consensus storage Etc all the hard parts of blockchain are taken care of for you so and I put this with four exclamation marks because the entire polka dot SDK and ecosystem is all rust based so you're going to have to learn how to use rust uh and use it well and some people have called some of the syntax and po the SDK like rust V2 because it's a little out there but the learning curve is definitely worth it and of course the willingness to learn this kind of generic way of programming you're given this generic uh implementation of consensus generic implementation of networking and you have to kind of put these things together and but as a result um you get an entire blockchain which normally could take you months and an entire team maybe even years now you can probably spin up a blockchain in about an hour if you want of course the mindset is to innovate create new things don't just copy code and then just you know um uh use that as your basis you can actually you have the the ability to create something entirely new and also I encourage you to utilize existing Solutions so or Bridge them together so like I said polka dot I like to view it as a composer of different things Al together the idea is that you can create a product by composing all these things together in some cases you may not even have to build a blockchain yourself you just can use utilize the existing ones and compose them in a smart way so what has been built so far using all these tools well polka dot itself is built using substrate and polka dot is also built using substrate cumulus frame all these things that I mentioned polka dot itself has been built and you can see if you dive into the polka dot code base it's very extensive and very particular in the way that it actually you know it's a layer zero a meta protocol which validates the states of other protocols so you can imagine how extensive substrate is when you consider the complexity of that and substrate gives you those buildings blocks to create these decentralized systems it doesn't even have to be a blockchain uh as we'll soon see and of course what can be built you can build a Fess blockchain if you want to again you have full control over the your runtime there are specific macros specific ways to configure the fees in your chain maybe you don't even want fees at all and instead you just want to use Economic Security from polka dot and use dot as your as your uh method of paying fees maybe you want to implement a completely custom consensus mechanism maybe something that has very fast finality you you can also do that using substrate and then put that as part of your substrate train you can replace the accounts model with the utxo transaction model instead like Bitcoin and if you want to get you know a little more specific you can use like an iot type supply chain traceability solution like an app chain so you can completely tailor just purely the business logic portion of a chain and just some examples so these are two system pair of chains and respectively um should be switched around but it's Bridge Hub and asset Hub Bridge Hub is a par chain on polka do that manages incoming connections from other Bridges other ecosystems and then from there on it uses cross uh cross consensus me messaging to talk to other par chains using Bridge Hub and again that's implemented as a par chain using substrate and frame and pallets same with asset Hub which is an asset registry and the point of these system par chains is to kind of uh divulge the work from the relay chain which is poka do poka do Po's job is to validate the state of other protocols uh whereas these are specific a really good example of a solo chain built using substrate that has a completely custom consensus mechanism uh with dag based consensus I think they have like subsec finality or something crazy uh is Alf zero and they they are a solo chain so they're not even connected to polka dot but they're using polka Dot's tooling to accomplish um their consensus and of course polka dot itself um utilizes something called open gov which is how decisions are made on how chains are to be upgraded different modes of funding and such open gov is also built as a series of pallets which work on chain and work intimately with the core protocol and then lastly just to get a little bit into this more of a fun use case um iut and sub XT so I've actually done this before it's possible that you can take something like an esp32 and if you know Hardware it's a very tiny chip and you can compile substrate libraries onto that chip in order to create transactions encode them sign them and then send them them to a substrate based chain so you're can to have like direct transactions being created from a tiny little chip sending up to a par chain and that par chain then gets finalized by polka dot so you can see how you can really finalize and guarantee the state of many different things because the tooling is just very diverse so how do you actually start well there's quite a few initiatives so there's the polka dot blockchain Academy which is essentially five week like super intensive program five weeks of nothing but blockchain and polka dot um you're sitting there with like 70 other people and you're just learning blockchain the entire time uh it's a pretty rigorous entrance exam but it's well well worth it because you get to build connections you get taught by industry-leading experts um and you just spend all of your time learning the fundamentals like these things I gloss over in the beginning to do with uh blockchains estate machines you're going to be learning all of that at a really really basic level and it gives you that extra bit of understanding uh we have the polat developer heos program just a community of developers um that use polka dot and it's a cool Network you can go and talk to people about you have and then there's some upcoming edex courses so we have two right now and then there's two more what we're working on which is an intro to rust for substrate and then also an intro to substrate development and of course there's substrate polka do hackathons we have a I believe we have a um a bounty here for that there's also treasury funded polka dot education initiative so if you go to open goov if you're brave enough to put up a a proposal to the community and propose something to build either at to PO po do or build on polka dot you can do that as well um yeah and on the pokka dot Wii there's like a starters guide I put together where you can just kind of go through all of these different resources and there's a QR code for that right here if you want to get started on that and then there's one more I had to added in last second it's a really good guide for it's more development oriented it's like a hacker guide to polka dot all the different templates templates you can use you can actually like right now on your computer start a polka dot node or a substrate based node modify the runtime compile it again view it running view it State transitions everything like that uh yeah that's all I have for today thank you guys