Umbrella | Introduction to Umbrella

hello my name is miguel morales i'm one of the product leads here at umbrella and today i'll be reviewing the umbrella product um the the the purpose of this video is to show all the developers all the different hackathon projects that we have available it's only a couple of them and give developers an introduction to the umbrella architecture to the umbrella apis and sdks so that um you know developers are able to build applications and to um and to be able to participate in these hackathons and complete projects that are um going to be judged well against their other competitors so let's get started um first i just want to sort of give a very high level overview of what is umbrella sunbrella is a is a oracle network and we deliver um data from a side chain onto the different uh blockchains uh primarily binance smart chain and ethereum and uh that that data um includes like cryptographic uh paired data so for example the price of each usd and btc usd and so on and so forth um and and we also provide other types of data such as such an options data and uh you know the we are also adding a lot more data so for example you can find um different types of more obscure pairs and you can see the whole um the whole uh key list in our block explorer so here you can see you can we can go and see our binance block explorer and all you have to do is click on layer 2 data and that's going to show you all the different uh data points that umbrella provides so um as you may have seen me mention we have two types of data layer layer one data which we call first class data and then we have a layer two um first class data is essentially uh just sort of data that you can use directly from your smart contracts and you can also use layer 2 data directly from your smart contracts it just requires um retrieving some data from an off chain api to submit to your smart contract to be able to use that that the layer two data in your smart contracts so first but let me take a step back and um let me see let me present the the architecture of the product for a little bit um so the way that the way that oracle systems currently work is um right you have different feeds uh in which you know validators essentially uh submit the price of a crypto pair let's say for example you have the eqsd feed um in um in oracle system oracle validators essentially just submit a transaction to the to the smart contract which is a feed contract right and um this can be very expensive especially for the validators as they have to essentially pay uh once they have to pay a transaction fee for each data pair so imagine a validator has to pay for uh submitting the price of ethiosd for btc uh you know abdc usc and so on and so forth and that's just very um very ineffective and very expensive for for validators and alongside that for developers is actually pretty pretty easy to use so these feeds are able to just directly call from the smart contract and um and they're fully they're fully um they're funded and essentially you can just uh you can just query uh let's say the price of you know ethiosd i think so they'll get get a little bit trickier um when you're using um sort of like data that's a little bit rarer uh in in which case you know traditional uh wellies like uh market leading um oracle systems have the concept of networks and so you have like different networks to um that that holds all these different types of data with umbrella it's a lot different so there is no need for different networks um and that is because we have two two types of data as you mentioned there's first class data which has exact which acts exactly like um the native feats that oracles traditionally have and then you have um you have you know what we call layer two data and so instead of creating all these different dispersed networks uh we have one sort of one network one global network that includes all all these different types of data so for example you know we if we're to add like um sports scores or the weather data for example um those providers would simply provide an api and then that gets added to our layer two data um which can ultimately be promoted to layer one data but um but the way that this works is through the use of merkle trees uh so here you can see a diagram of how how our um our side chain works so essentially we have a side chain of different validators decentralized validators and they come to a consensus right and the notes basically they independently validate the data so um so whenever there's consensus they they're double checking that no one is trying to manipulate the price right they're independently verifying and loading the data from the from the apis and double checking it against other data and then when i do arrive at a consensus that consensus is this memorialized um on a smart contract and what's what's in that consensus um so essentially there's some key pieces of data so one piece of data is called the merkle root and so the merkle rule just simply represents um thousands of crypto pairs into one singular value and so um and along sideways we also include what we call again the first class data and that has um a list of the crypto pairs uh natively so before for cases when it's like you know a little bit more rarer data it will be in the layer 2 data set which is stored as a merkle tree uh here i'll give you a little bit sort of uh more insights into how our our validator network works uh you know you feel free to skip all this if you know if you want to learn how how the network is architected i'll start talking about um developing apps in a little bit so essentially what we have is a validator registry and in the validator registry the validators is where they uh you know they are registered and you can find information such as their name their their their blockchain address and the location of of their node of their validator node validators are requested to set tokens at stake so um so they do that by depositing umb tokens to set at stake in the staking bank and then we have what's called the chain contract and we're also adding a delegation and essentially the way that that works is um the chain contract make sure that make sure that the the data being submitted is has been signed by re like valid registered uh validators and such that the list of validators are signed in their stake must be more than uh more than 50 of the total stake and that's how you achieve a majority once essentially so the the chain smart contract um validates these signatures is so what essentially is happening is all this data is being batched into one transaction so as opposed to you know another in other oracle systems where it's like the validator submitting uh independent transactions uh in this case uh we just submit one transaction the whoever's the leader in that round uh submits one transaction that transaction includes uh the dozens a dozen or so first class data which represents all the you know all the popular feeds and then alongside that with the merkle root of the consensus data and the signatures that say yes we agreed that for this round this is the merkle root this is the the values of the first class data and then all that gets stuffed into a single transaction and that's what gets submitted into the into the blockchain and um yeah a little bit just to wrap up how consensus works uh there's a leader that's deterministically uh elected and and um they are responsible for submitting the block for that round if they fail to respond then you know rotate to the next leader um and so the leader basically proposes a block to other validators and other validators double check data and if they agree on the value of that block they sign off on it and so and so that's when the leaders uh gra gathers the signatures off chain and then submits it uh as a batch uh on and so yes so essentially um now and this is important because uh on chain you have this key pieces of data which again is the it's like the merkle root um and and the first class data and the and the values and all of that um and so here's sort of a a brief overview of how an application may interact with the different with this um you know with umbrella uh smart contracts okay now we'll go over first class data so essentially as i mentioned before first class data is very much just like standard um data that you can access just like you could from the standard feeds that other oracles provide so the best way to to to work with first class data is to first go and see the list of first class data available essentially one of these you can think of it as a feed of data that's um being continuously updated uh on the blockchain so here i'm looking at the our block at the umbrella block explorer on the binance chain uh so binance is essentially our core anchor chain uh and then we have bridges to other chains uh but in this um for this hackathon and uh for the time being we're only going to focus on the finance um smart contracts on the binance first class data and not not worry about bridges just yet so uh so you can go here into the to the binance block explorer and click on first class data and when you click on first class data you're going to see a list of all the different um uh first time data keys and and their latest value here now you can just simply query a query a smart contract and i'll go over that in a second and get this value directly from your from your smart contracts so that's that's um that's the the very basic concept is extremely easy we have documentation onto a more comprehensive introduction to first class data and it tells you everything that you can that you can do and how to access this specific set of keys from an api if if you need to if you need that uh we also have uh an sdk so this allows you to use the first class data from your smart contracts so essentially it's just a set of files that you can import from your from your smart contract to be able to use first class data so here i'll just kind of go through the example and um and just kind of help you along understand some of these concepts so the first thing that you want to do is is um is to import the chain contract this is an an interface uh file but it's it's uh it's essentially just a way to communicate with the change contract um you'll also need to pull in the registry contract um and you also need to pull in the decoder uh library this this allows you to decode the values essentially so um so essentially the the registry it's it's simply a pointer to the real uh chain contract because periodically there may be updates to the chain contract you don't want to talk to the chain contract directly you want to find out what is the latest chain contract first and then talk to that contract so that's what essentially this registry is this is called the contract registry and so you're going to query the registry in order to to find out what the latest chain is so when you initialize your your smart contract here you need to make sure that you specify the correct contract the contract registry we also have a page here that that lists all the different contract registries for the different networks so you're gonna you know in this example we're using binance mainnet so this would be the the registry address that we pass it so as the registry address that we pass in and then we we store that address here and we're going to be working with that in a second now i'm going to just focus on the the most common use case the the the core functionality and that's just being able to get the price of a crypto pair for example so here's the way that would work so from you uh so from your smart contract um you would essentially first query let's see the chain so here in this example we have a helper method that essentially returns the chain from uh by by resolving it through the through the contract registry so here we're using the we're first querying the registry and we're asking hey contract registry give me the address the latest address of the chain contract so that's this contract here and uh and that essentially here resolves um the chain contract okay once you have the chain contract resolved you can start uh querying that contract so the the most common scenario is getting the current value for a given key so again for example we want to uh find out what is the price of umb and this is the key and so you you there's a little nice little helper here in the block explorer that allows you to just get the get the bytes directly that represent this key so we just copy that and that's what you would pass in uh to the chain contract to get the value and what you get in return is the value in the timestamp of when that value was last updated so this is continuously updated and so when you do this you're gonna get the latest the latest value here you're just checking whether there is a value maybe somebody passing the wrong key or something like that um and um and that's how you get the value for first class data so it's yeah it's very straightforward to uh to use um the the block the you can see all these um you can find out more what what exactly this uh this chain contract uh interface looks like so you have all the different methods and um that's available in our sdk okay so next we're gonna go over uh go over layer two data uh layer two data it's a little bit more complex to work with uh however it's the mechanism that allows the umbrella network to support uh so much data and and be able to store it compactly and be able to use it compactly within the blockchain right um so first i'm going to just quickly review the concept at a high level so the way that it works is that the umbrella network the umbrella oracle system uses a merkle tree to store data compactly now the way the the merkle tree is essentially allows allows somebody to prove uh the existence of a specific um key within a data set and the beauty of this is that you can prove the existence of a key within the data set um without having to to have the entire data set and so uh you know merkle trees are pretty fantastic uh they're used within ethereum and and you know in other systems and essentially the way that it works is um let's say that we you know we want to store the price of btc usd uh it would be stored within one of the sleeves in the merkle tree and then if you want to be able to prove uh to prove this this value uh let's say that the price of btc usd is you know 50 000 at this point in time you want to prove to a smart contract specifically your smart contract the smart contract that you're developing um that this value is essentially uh correct and it's the value that was um that was derived from the consensus mechanism right um so typically uh what developers will do is they will write an off-chain service that periodically updates a value that they're interested in uh and and sends it to their smart contract right so essentially um running like their own um oracle node but the beauty of it is that uh that they don't need to run consensus they don't need to do uh any of that they essentially just copy write the layer two data um include the the latest value and the merkle proof and send that to their smart contract and the smart contract is going to validate the merkle proof to make sure that that value was derived from consensus and this is where you have a trust a trustless mechanism um of natively uh promoting right layer two data to layer one data um you don't you don't necessarily need to uh you know to do um to to do all this um but it's uh you know you for example could every time you make a transaction in your smart contract you have the user right past you the value and the merkle proof but the more typical scenario is the developer is going to write an off-chain service that again sends that data includes the marco the merkle proof and the smart contract essentially validates it and updates its own internal value so um so the first thing you're going to want to do when you want when you're working without with layer 2 data is go to the block explorer and there's two views that are important so first the layer 2 uh data view here shows you all the different data data points are currently in the layer two data set so right now we're close to a thousand a thousand data points and uh as us umbrella gets this developed um and and um keeps improving there's gonna be a lot more data pairs to be available here and within this data this data set is a lot of more exotic uh data points right so uh so very various uh sort of more uh smaller cap coins that may not be available through traditional uh oracle feeds and this also uh includes like you know there's like v walk b wap uh data there's gonna be options data and essentially it's going to be like you know sports scores and just anything you know a merkle tree can encompass hundreds of thousands of entries um and still be compacted to a 32 byte hash at the end and the beauty of it is and why why why is umbrella better than other other um other oracles is because for example in the leading the leading competitor right they have was called the marketplace and the marketplace allows right uh different uh there's like different oracle networks and you as a developer you can essentially you know you can select your your notes and you can select the oracle and the type of data that you wanna you wanna that you wanna use but um you know that creates like this dispersed networks right and who's who's um you know the stability may not be the you know may not be the best ability um and it's just creates all like different dispersed dispersed separated segregated networks um and it may sound cool right someone's running a note right like oh we have you know this data provided running but you don't need to do that right uh with with umbrella we have this one global network and that global network includes all the data all the different data points from all different categories right so a data provider doesn't have to run a node they essentially just provide an api the validators talk to that api and then the validators come to a consensus and it's all stored compactly without incurring any additional cost so that's kind of the the beauty of the way that layer 2 data is designed okay so next we're going to show you how to how to get information necessary to use layer 2 data and essentially we're going to go over one sort of scenario right so so one of the things that we have as a hackathon project is the ability to [Music] bring layer two data into layer one so this is a typical uh a typical workflow so let's say you're a developer and you want to have uh this layer two data delivered into your smart contract uh such that uh your smart contract can still be trustless so your users can know uh that you know even though you are updating let's say the value of some certain um some certain key um it's still still trustless because you're providing a cryptographic proof so let's say that you're building an application that needs that needs um you know the latest uh sports right or um you know time weighted data um or just something specific uh the first thing that you want to do is first uh look at the entire data set of layer 2 data available so you can you can see here there's a lot more sort of you know less popular data pairs with lower market caps and you're also going to see right uh you know like time again time weighted uh time weighted averages and and just other types of data so find the data that you're interested in and you can you know you can see that here currently we have a little less than a thousand uh data pairs available and the way that it works is you need to basically get certain piece of information uh to be able to let's say submit a transaction to update a value um that uh that you know so that your smart contract can validate that value so here we can look at uh we can look at a specific block so if you remember periodically there's a new block minted on when there's consensus and this block includes the mercury and um and so let's say for example we want to update um the price of ave btc based on the latest consensus round to our smart contract essentially we would write an off chain service that periodically checks if there's a new block minted and um in and will you know periodically send data for certain blocks we'll send the key and the value and the markup proof and then that on this on the developer smart contract then they would you know process whatever logic they need to process and then update the the internal value uh within their system at the same time it triggers you know you will be triggering um any so any sort of process within your smart contract so so here you see an example let's say we want to update the price of ave btc uh you will need some key pieces of information uh and so you as a developer you're going to be communicating with the with the chain smart contract just like you did uh just like i went over in the first class data part and um so you're gonna need the you you're gonna need to communicate with the chain smart contract and you're gonna use uh you're gonna pass certain uh pieces of information to be able to validate uh a specific value so uh so there's a method within um within the the chain contract called verify proof for block and you're going to pass in the the block the block id that you want to use this data from you're going to pass the miracle proof and here is the key and the value that you're proving so for example here i would just copy the key copy the value this is what i'm proving we're proving that the the price of ave btc is you know this fraction of a penny here uh of of a bitcoin and so um and so you're just going to be able to pass that key in value and then more the most important most importantly you're going to pass the merkle proof so this merkle proof proves that this is was indeed the value um and that it was contained and asked this value within this this merkle root that we showed here earlier so once you pass that data to the smart contract you can just execute the function and it'll validate uh if that proof was correct okay so next we're going to go through the developer api and how do we retrieve um this off chain proofs such that you can use them within your um on chain smart contract um so you can you know so there's different uh api endpoints and for example so we have an endpoint to get the first class data set uh and to get the layer two data set you can get information about the latest blocks so if you're scanning right you're scanning for new blocks and you want to you know update a value this is what this is where you'll be in continuously polling is to get the blocks you can get information about a certain block id if you want to um uh get all the leaves for a specific block and then uh this is probably the the more the most useful one uh so what this does is this gets the proofs for us a specific set of keys that you include and this is going to return the latest value from the latest block id so let's say that you want to update the value in your smart contract you know every hour uh you would you know add the hour retrieve the latest proof and and this is going to include all the information that i showed earlier that you need to pass to the chainsmart contract to validate it there's also some very important things um that you can find on our github so for the purposes of the hackathon i think you're going to want to focus on two things number one is our example application and that's called babel and so this is kind of has a couple has one a smart contract that you can use a sort of a building base um to see like the you know to see the system in action um so so we have two examples here one how to use the first class data and then another one on how to use the layer two data uh so here you know we're gonna focus on the layer 2 data so remember that um you know you're going to want to import uh the sdk and that is called canvas so you're going to want to import that and there's you know of course information in our in our developer docs on how to how to import your sdk you wanna you're gonna wanna import uh the chain uh contract um interface and the contract register interface and uh the the value decoder um because you know the later two data is encoded so in order to um to be able to extract the like the actual decoder value you're going to have to like you know do some decoding um and i'll show you i'll show you some examples here so um so imagine that you know you this is your developer your contract and um and you want to update right the the price of uh you know avebtc or whatever it is that your contract is working on so you know you will be developing this off-chain service that is going to submit a transaction let's say you're doing it every hour all right and what you're going to do is you're going to um get that block id the latest block id along with the latest value and submit it and your method here uh you can have your method right validate the freshness of that block uh in in other things so you're gonna first resolve the the address of the chain through the contract registry and this is the same for us as first class data um after that you do that you can then validate a proof and so you know in our example we kept it simple but uh you can also query the chain contract for information about about a certain block so let's say you know you pass in a block id and theoretically it can be anybody right like this this this is a public a public method so um so that because it's secure and you use cryptographic proofs to prove things um and so but you know in this case the developer is the one developing the service they will pat get the latest block id uh get the key and the value that the contract cares about and the proof of that you know that of that value um and so you can use this this method to to validate the the marker proof being passed in and this is the exact same method with the exact same data uh that i um that i went over that we did on ether scan so this is just uh this is just how you would do it um you know on your on-chain smart contract and again there's other other methods here say where you can get information about a block right and you can get um you can get like how old a given block is and so let's let's take this example right let's look at the latest block or this block here and all right let's get uh let's say what when was this block minted and it will give you the time stamp and so you can you know implement various things let's say that you only you want you only want you would only accept the value um based on freshness right so like you can query the block id see if it's too old and not accepted or maybe you store you know keep track of the block id that was last submitted you want to make sure that the the new information being being uh being updated is the latest in which um or rather is like at least fresher than the last time you would check both the timestamp and then you would check to make sure that the last submitted block id is lower than the new block id being submitted so you can have all these different uh conditions in your smart contract uh to make sure that you know your users trust trust your you know um and that do you do you're using even though you're submitting this data yourself that the data itself was derived from consensus from the umbrella validator network and so once your once your smart contract validates the merkle proof and you know you call the api to to to validate it then you can just go ahead and update that value and um and use that value within your smart contract uh as you normally would right and so you know in this case uh the developers you're you know you're you're paying the cost of being on your own oracle but uh you know through the transaction fees but uh in essence right like you you don't have to worry about integrating with apis you don't have to worry about like oh it's just me like um there's no you don't have to have consensus like you're proving that this was there was already consensus on this and so this allows basically anybody to a use the the vast layer two data set and b uh integrated into their smart contracts and essentially promoting write that layer two data into native layer one data within your developer smart contract so um you know as i mentioned there's definitely um the more uh complicated scenario of instead of just using the first class data directly but that's why you know when we are judging this this hackathon's uh projects um you know we're looking for the complexity and how well you understood this more complex um you know just like this complex methods and this more complicated way of using you write the more exotic data set so um so we definitely uh prefer you know to see uh projects using layer two data directly um and it's you know and it's very easy to use this later two data natively slayer one data on chain so with that um good luck