EthCC 2: Emiel van der Hoek

so I'm going to talk about poker scam which is a blog Explorer for substrate and I'm going to go into some specifics there are particular to poker scanner so first of all I'm going to start a bit about what pokers can is and what it does I'm going to go into some specifics of substrate the substrate run time and I guess this I'm kind of aiming for an audience of future substrate implementers I guess I have some best practices for substrate implementers to ensure optimal compatibility with a polka scan in the future and I'll go a bit into our roadmap so basically we want to make the multi-chain data accessible and understandable yes our mission well we predict the future that will have many chains built with only a few frameworks and obviously we think that substrate is one of these frameworks that allows for that but you get into well increasing specialization you need quite a lot of skills to actually start working with that and I think what you see a lot in new industries is that you get some sort of outsourcing with that so specialized organizations will emerge to do service management or of blog explorers for example so poker scan is our multi chain block Explorer and web 3 scan is the organization I work for and we intend to be one of these service providers doing full service stuff for block explorers so what has always interested me is the data it's I've been building a block explorers since 2015 especially a theorem got me interested in that and there there's some data you can easily get out of notes and lots of the interesting stuff that's actually really hard to get out of notes and especially if you want to search for stuff that's happening on a blockchain or if you want to go into some sort of aggregation or statistics or correlations or get the full history then not only the configuration of your note is important but you get into what blog explorers came to be well with focus can we try to do all that stuff that's on the right-hand side and we try to do that not only for a single chain but for multiple chains as well basically the model of our block Explorer is you have a data source such as a node which has an API we have an harvester application which does nothing more than get as much data out in a systematic way it puts it in a general storage which is a large database these databases can grow up to 10 terabytes for example we put an API on top of that and we put a user interface on top of that the idea with a multi-chain Explorer is that you have all these harvesters and block chains in parallel but we put a single user interface on top so you can get access to all these different different block chains through a single UI and that's where we get into web 3 scan the service provider I can imagine a future where you do all sorts of data integration service level management quality assurance on that data layer for organizations interest in that so the idea for a poker scan is if we can build one generalized block Explorer stack for subscribe we can have block Explorers for all the chains that will emerge in the polka dot ecosystem well these are a number of projects that are currently in that polka dots eco system already and I bet that that number will grow these are a few of these this is an example of an image of how a single relay chain composes an ecosystem you have many specialized chains there are multiple nesting levels and even if one level only has a hundred chains for example a second level could have another 100 change you could get to an ecosystem of say 10,000 chains won't go that fast and we won't get there very soon but this is the general direction this is one of the images from Jack's recent article about the polka dot ecosystem I call this a polka dot universe basically one relay chain interconnecting all these different block chains in one such universe and all these lines between these block chains that's the interoperability or should represent the interoperability I guess what we're trying to build is one single block Explorer for such polka dot universe so I want to talk a bit about this substrate runtime which makes everything we're doing possible so we've heard a couple of times before already today that substrate is a development framework for block chains and with substrate you can build different classes of blockchains from bitcoin like UT EXO classes of blockchains to full smart contract blockchains to single or dual function block chains of which the pocket of relay chain is an example or the etherium to Jesper specification is being built as a such single function block chain on substrate and we've heard in a Rob Rob Emmaus talk about how light clients can be built of different chains within other chains so that these are all these different functions you can build with substrate and basically substrate consists of a number of layers and the only difference between the chains built with substrate is the runtime the runtime is what defines what differentiates one chain from another so the thing is that substrate is really abstract it needs to be abstract if you want to be able to do anything with it but if you want to build block Explorers then you really want to talk about meaningful data stuff you know transactions tokens balancers accounts all these things don't exist in substrate in vanilla substrate you only have blocks extrinsic and storage and extrinsic is a general term for a transaction but it doesn't have any meaning it just looks like a blob or a string of bytes and there is storage that's and all the stuff that is stored within the blockchain the extrinsic s-- is the stuff that triggers the state changes within the block chains and the blocks themselves they are not really more than an ordering mechanism so to tell what comes first and what comes next so substrate doesn't even have accounts it doesn't have time steps it has nothing and all the data that is stored in the runtime has no there's no decoding context for that so and that is a problem if you want to build a block Explorer for that type of technology but there's a good thing a runtime that part that defines what the blockchain does has a an API or the metadata so that's one of these things that substrate substrate provides and it gives you the decoding context so I'll give an outline of the DES five components of death metadata and it gives you an idea of what your runtime does so we've heard this a couple of times before as well SRM L substrate runtime module libraries these are basically logical groups within the substrate runtime so we can see that there is a runtime module for accounts for balances for Treasury even for smart contracts so if your substrate runtime is built with the contracts module then your substrate chain you build has support it has smart contract support well and these modules have a number of special functions in them one is core functions these are basically they define what your blockchain can do so you can call this from the outside and you could look at them as your transaction types so you could input a transaction you could send a transaction of a certain type to the blockchain which is a transfer type transaction and you can give an address which is the recipient of that transaction and a balance and that type of transaction would allow you to transfer something but again this is not a native function of substrate because substrate has no native function this is one function which is part of this one particular runtime module which is built by parity in this case or I guess this function exists on the pocket of relay chain as well because you need to be able to transfer your dots from one account to another well and you can have tens or hundreds of different transaction types and these are the core functions and the metadata the ABI of the runtime will allow you to identify and decode these type of transactions so what this means for a block Explorer is that if you harvest the data from your block Explorer you can classify the transactions by mapping them to the metadata specification you get from substrate and you can list for example all the people on strands for transactions of this particular blockchain which makes your block exploring generic another chain would perhaps not have such a function and your block explorer would not show that part another component of the runtime or storage functions basically it allows you to query the state of the blockchain using the metadata if you don't have that metadata you don't know what's a query for it's a bit like having a smart contract without the API you wouldn't know where to start looking [Music] yes there's one special storage container it's called events I guess you could compare that to the locks you have in ëthere ium and you can use these as a cheap or effective notification of a state transition if you built the triggers for these events into your runtime one example you could you use a storage function to query for changes but you would really need to monitor the things you are looking for events these are fired when stuff happens or stuff changes so you don't have to pull for stuff every block but you know when to check the storage what important components of the substrate runtime metadata definition is that there is a type definition and type composition if I go back a number of pages you see all the pink words these are basically data types and address a balance a referendum index or a votes a balance an account ID and so on these types have a meaning in a composition and there's some improvement I guess here there could be some improvement still here but all the data types what you get from a substrate note is nothing more than a byte representation each data type definitions really help you how to interpret the bytes so if you know that your thing maps to an unsigned 32 integer you know that you have 32 bits to scrape from that string and to go to the interpretation and the meaning of that since all types are nicely defined in the substrate runtime that allows a very nice low-level mechanism to get meaningful data of higher-level objects in your block Explorer yes so the runtime the run ABI promotes substrate second-class citizens to first class citizens in pokus again so as I said the substrate only has a handful of first-class citizens blocks extrinsic sand well storage you come to anything with that in a block Explorer all the interesting stuff is within the runtime so it's the accounts the balances defaults the proposals that are voted on these are all second-class citizens and certain substrate but the ones they have these meaningful data entities in your block Explorer that's the stuff you would want to know that's why you interact with a block chain to get that meaning out of it so we have been prototyping poke escape and the harvester and we've been compiling the substrate notes for many of the projects that were in image before and there are some things that help compatibility for with polka scan so one of these things for example this is perhaps a 101 on that on how to deal with github but that you I'm talking to substrate implement implementers now that's change and release management procedures are followed or whatever this is even a parity sometimes it goes wrong because we're looking at a version Oh 3:15 but the cargo files are still on a different version number so there should be some sort of process for that it helps if these things are sorted then there is the Virgin version change release management for the runtimes themselves substrate has this great feature that the runtimes that what defines what the blockchain is and thus that it can be upgraded through Unchained governance but there again it is important to have good release management practices for that well I think we've seen many examples already today of these runtime also that are being developed by a different teams some runtime modules I guess will become standard modules or go to modules if you want accounts token module I think we will see that there will be some convergence on some standards with that I think it makes sense to reuse runtime modules where possible and not go you know in vented all again well you should be careful when you introduce new struts or type definitions one of the things we learned for example in building our first proof of concepts is that what the balanced type actually means can change with runtime upgrades and this is problematic it really makes a difference if you scrape 64 bits or 128 bits because you just don't know where you're the meaning of your byte string or bit string stops and where the next section starts so I guess it would be good for the ecosystem to come up with some sort of registry of reserved words or some kind and perhaps focus can can be that resource for a well at least until we address these type of problems I've taught you about these events events which are triggers can be triggered when a an important state change happens I guess in our tea sets and our prototypes of focus can we came to rely on these events heavily and we will be indexing the events that are fired in any substrate we support that allows for indexes and lookups on date that is provided in these events and basically it means if you fire an event for every new accounts that is created you get a search function in your block Explorer for all the accounts if you know if you create an event in your runtime for all the balance transfers which have a recipient sender and a balance and a transaction fee perhaps then because we are indexing these components you can get lists of what the largest transactions are in an ascending or descending order or you can look up all the different transfers from a specific account or to a specific account because we've built these indexes based on these data structures oh and perhaps a non-technical best practice is for all these organizations and companies and individuals in this new emerging polka-dot ecosystem to get to know each other to be aware of each other and want to work on relationships there's lots of synergies possible and for starters we would like to get in touch with all substrate implementers so we can maximize our compatibility with all these new changes that are emerging so roadmap so I spend most of I think 10 months now researching how to build a multi-chain Explorer and substrate that research is done we have applied for an ecosystem grant if that works out we will be providing the community with an open-source solo chain Explorer which is the entire stack for one single substrate chain and our business activities the web 3 scan activities will continue in building the multi chain Explorer and service just around death thank you that's it for now um one more thing this is the Pokes can dot IO there has been some hostility and our website was ddosed so not everything is up and running right now but hang tight in a few days that will be solved so if you want to talk to me reach out come find me yeah so are we with time yeah who has the second mic the question oh hey thank you for the talk one one question so for substrate chains that have included the contract's module so support smart contracts are you also I guess that's kind like an extra there so that indirection or depth in terms of having a block Explorer that can understand smart contracts deployed onto a substrate chain it's that something you're also looking at all planning on dealing with yes well the first test nets have been deployed with that support but I don't really know what that looks like yet so I bet for starters that if a smart contract is deployed like on etherium a good runtime design would trigger the firing of a contract created event of some kinds and it would have some meaningful data for example at least to allow the block Explorer to identify and list the smart contracts that have been deployed but when you talk about interaction with the smart contract itself I guess that's just another layer of being such a codec which we have for the runtime but then for the contracts themselves yeah that may say so I guess usually the ABI yeah I guess yeah the ABI for this for a smart contract is not I've provided by the chain itself I would have to be provided somewhat independently by the I guess bigger strategies in an open system anyone could deploy the bytecode of a smart contract yeah and I don't know where of the details of these processes yet it's just really early in this ecosystem and I I bet that the edgeware project which are while relying heavily on that runtime module for their smart contracts and governance a platform that as soon as we start interacting with with them more that more details will follow thank you what else any other questions for a meal here we go hello sir do you think that could be interesting in some point to start thinking how to and beat some visualizers in like in substrate like in in a way that like polka scan or web 3's can can show some transaction data so it is in building some kind of this very basic basic busy visualization of transactions so in terms that it can be rendered in the user interface when you access to this transaction so it's using beaming zombies organization I have difficulty sorry now understanding yeah right it is for instance for instance in the serial now the problem is international internal transactions sometimes it's very hard to to trace yeah yes because maybe you don't know exactly what's that but even some point for instance my contact week if some way to you could visualize this transaction in this way in a thorough scan for instance did be easy easier for users to understand what's happening with there are no sections I do think that these kind of things could be useful also if your question is I've built an EVM block Explorer so I'm well aware of what makes it difficult and usable and in in ether scan they introduced the term internal transactions yes which is all the stuff that is triggered by the contracts themselves so or a triggered bank transactions hmm when I am using using an application I need to trust in this application and these you talk to me afterwards I can't understand your question as difficulty understanding okay I see you later okay sorry all right so thank you so much Emile [Applause]