The Graph 🛠 Querying and Indexing Smart Contract Data

[Music] the presentation that I like un it's it's a half an hour or 20 minutes um but I have prepared a special one for you that is the less technical um thing like how to actually query data and how to find the data how how to work with this um Depending on time we can also go into the subgraph development but um yeah I think it's uh it's cool to have this also that was never recorded subcraft development stuff is recorded from time to time uh anyways let's get it started so um clearing and indexing smart contract data on ethereum with the graph um I start here really with the basics um that is what is actually the smart contract anyways um and I have like an interesting uh like an example you can think about of a smart contract that it has two things basically it has storage like data and you can think about the storage in smart contract as a simple Google Sheets or Excel sheet where you have um just rows and columns with uh with data in it um when we look at the nft contract it's my uh main example because it's it's such simple in in terms of Technology um you you would have like in a in a very simple way you have a ID that the token ID of the um of the nft and then you have an owner which is an ethereum address and then you might or might not have a metadata U and other stuff but but in nutshell nft the e721 contract def findes token ID and an owner so that's that's the data view on it right um if you want to yeah the problem is with with the data is that the smart contract and the blockchain it it usually holds the the state as of now right the it is a good tool to to see which who has which nft who owns Which nft but it is not very good um in in knowing uh who owns the nft before and that's exactly what's going on here um that's a problem that we don't we don't see who owned it before in the smart contract state by itself um this is where the graph jumps in uh that it actually helps to also find a way into the history of a smart contract and on top of that making the current state actually also easier to access so you can think about without the graph it is just this and with the graph we have a history but um we will go a little bit deeper the second thing that a smart contract does um besides uh holding holding state or data is um it also has rules how to change that data and that's very important because it don't want like have everybody just changing who owns Which nft it it it follows certain rules um which can be encod in a smart contract and that's also why it's called smart contract um because it the the data is um yeah can only be changed according to the rules so let's define quickly the concept of a blockchain again um so they basically a blockchain is a distributed data storage um secured by cryptography uh that that means like all the transactions they are in they are in blocks and then hash is coded on all all over that those transactions that makes a block and then the Block it's all always um linked to its pre previous block that makes the whole thing temper proof so you can never change one thing um distribute means that it is the blockchain not runs not on one server it is basically copied over thousand or 100 10,000 100 thousand times all across the world that makes it so uh censorship resistant that you would basically need to shut down every single node that runs the ethereum blockchain on it or whatever blockchain in order to stop it and that that's basically impossible um because we have not that Global jurisdiction where you you could enforce such a thing um that makes it very very interesting but going forward we think of the blockchain as one thing we do not care about that it is distribut put it across all all these machines so we we think of it as a logical uh one unit so that's just the blockchain right and then smart contract as I said before is the data and the logic how to change the data that is stored on blockchain and how does it work if you want to change the data you send a transaction to the blockchain which is then distributed across all these um copies but um you you sign it graphically if your with your secret key or your wallet so that um everybody knows that this this is a electric transaction and um then the transactions included on the blockchain we will look at that later on um but there is a problem with the data on the blockchain as I said before um but the the user interfaces as we initially saw um they want to read how they want to write to the blockchain and as you saw before it is incentivized so if you send a transaction to the blockchain you pay some gas or you have to put some gas on top of it and the gas is consumed by by the miners or now no now now it's the Pro St by the validators and then that's why you pay and by paying it for it the whole thing works um but when you want to read from the blockchain there's no incentivization for anybody to provide you that data per se not in the protocol so that's why when you go on ether scan you see a lot of advertisements um which sometimes are also a little bit Shady in order so that ether scan can uh fund its own business or the same is is also with um Alchemy if you if you do a lot of request to Alchemy or in fura you need to pay for them um so they're jumping into it but the protocol by itself incentivize reading right that's a big problem going back to the um analogy from before each of these things here is one transaction and also a bigger problem with with ethereum blockchain especially is that we have all these the smart contracts or deps um that have stuff on the blockchain and every time someone sends a transaction it just laid on top of each other so in the end if you want to if you want to see okay what was actually going on let's say in um Foundation then you would need to oops then you would need to go through all these transactions and and pick the ones out that are according to foundation and that's quite tedious work and it takes a lot of time um that's not something that you want to have in the front end and this is also uh what we see currently or before the graph that frontends do this is really code that from a project that I um that that I was advising um how they were try to get the data out um so for those that understand um JavaScript or typescript that reads as follows um do you get you try to get a contract but all these aweight methods especially those that we see here and and these here they all take 200 to 300 milliseconds to resolve because you always call back to some some node um which you usually need to pay somehow that they give you that that data and then you wait and you go through so this code if you are if you own let's say 10 or 20 nfts easily gets five to 10 or 20 seconds to load just to show you uh a list of the tokens that you hold and uh this is this is like super inefficient and um yeah users just move away after a while because they say like whatever I don't have time to wait 20 seconds for my stuff to resolve um yes and that's actually now we had like already two um reasons why the graph exists so for for one to get the historical data but for two also to have a very good access to the data um yeah on the website you probably know it's index and Protocols of cre networks like ipfs there's also support for RB um near Cosmos and and others so in the end uh the graph will index all blockchains more come soon TM um on the Hoster you see currently one point or one billion qu per per day or even a little bit more depends um so it is really powering um the a lot of the Decks that we see currently Unis swap is the most famous example that uses this but we also kind of others like enzyme or Odus orm stable um yeah there are a bunch of them um yeah and it's not only the host service it's also a e centralized indexing Network protocol so um there are more than 180 indexes I think currently worldwide owned by individuals oops um and that that means these are independent indexes that that that one the things so it is similar to what we talked about before but ethereum or blockchain general is that we have a distributed Network um there is this redundancy so if a subc is indexed by uh one indexer all across the world if one goes down and other is still around um that also gives you fast and cheap reliable access to that data and um yeah the whole mission is that we we we go towards a global open API of of the data that we need um yeah that's very cool then in the end we can now build truly decentralized apps right it's very that's what we try to to do here so quick re uh into the sub so I told you before about subgraphs subgraphs in general are just a database and a database is the same as you saw before in these Google Slides just multiple of them so database consists of M tables and you can Define the tables as As You Wish um then you have the mappings which is this gear here but that's a logic of how to get data out of the blockchain into that database and in the end you you make it available so and the subgraph by itself is only um subgraph by itself is only uh the code that defines how this is done but it doesn't do it necessarily by itself the in indexes they need to index the subgraph which by indexing that means they create the database and they run the code how to fill that database up so um yeah that's the quick concept of the subgraph so what happens is that we saw this mess of transactions that are just there on top of each other that they are um created like very nice need buckets for for each of the protocol in the database per each protocol so we can we can easily quer the data and and make much more sense about what's going on um that in the end we end up with uh to to go back to the screenshot from before that we have a very nice and neat graph qu query so one query that has a has a round trip of 100 to 300 milliseconds usually uh it's much quicker that's what we expect um yeah to to get the data out all right um I was just reading the chat so I will also share with you the presentation afterwards I all my presentations are open uh you can you can use them even if you want to give the presentation to to someone maybe you want to remove the Ed and not logo then but yeah it's public goods um so we end up with this uh modern dep our architecture where we have the user interface on top and then uh the blockchain on one that we knew already in between is this indexing protocol that is incentivized and and and built for data read and that sits in between and reads data from the blockchain and and stores in in this indexing layer all right so how does it look like then or how can you think about it this is the um illustration that we that's the the official one how the graph note works so again on top is the da a da can be be like in this purest sense adap is H it's just HTML CSS and JavaScript um that defines the app it could be also other programming language but that's what we currently see um and as we talked before when they send transactions to Smart contracts they are change you can change St so um that is buying an nft on a nft Marketplace or swapping something on unit swap or open LP position whatever cool thing you can do in the transactions transactions they emit events um and me with events smart track basically tells the outside world what what what happened we can um look into this later what an event is but that's basic basically what what smart contracts emit so you can watch the blockchain or watch smart contracts for those events these events are then in chested into the graph note which will um then with according to those events run MP which is uh in best web assembly module and put it put then the stuff into the store so here is is the database um where where the data is stored for later retrieval then the other way uh that we had also in in this image from above is that the dep the dep also needs queries so if you go to Unis swap um at the analytics page it it needs to load data somewhere so it sends queries to the graphical API which sits on top of the graph node and that graph API will get data from the store and then um send it back to the dep it's very um very neat right um also a very funny uh GI that we used to display this one like um how how sub crops work we have this massive uh transactions again and then magically somehow they are sorted into uh this this pocket so that you know what's going on have an overv all right so now when you think okay it's very cool but now how can I actually find data tool query the best thing in my opinion is this subgraphs that XY set from Missori they are writing very sophisticated subgraphs um okay um and I also send you this here the chat that's Pro currently for me the best entry point I mean you can also go to the graph oops gra.com and go here to the graph Explorer and then you see also STP graphs here um but the one from M like these are the ones that dep thre for themselves or also some of mesari see they also in the tops here um but sub graphs here are very good so we can for example say cool um let's have a look at I know uh so so they have some on the decentralized network and some on the hosted service um so you can look through them and and find out they like hundreds of is it okay 10 production 47 no 74 47 100 development and 100 um six deployment because why is that because they deploy to different chains that see curve they are usually the same on all cool um so so when you want to look into one of those another good example actually is uh let's switch to this one is the Crypt Punk subg grth um yeah here you need to be careful to take the one with the highest coration because somehow all people just deployed sub grows that's a downside of a decentralized network um now if you want to query that subcraft then you can just go here and go into the playground and start to play around um with with that subc but um what I like to do and that's where the where I think it makes sense to see this Workshop is actually open up a a very um yeah a little bit more sophisticated playground although this a revamp of this one is in the works but I can and show you how this work works so what we do if if you want to query a subgraph on De centralized Network what you need is kind of this is this query URL you can copy this query go here then you can and go to uh um explore like this and and paste it now the problem with subgraphs on the decentralized network is that you need to have an AP key and that's inside of that URL is is this placeholder for the API key but we first need to find one right so let's do this together so what you do is you go to um to the subc studio actually connect the wallet uh and I will just create a new account and then um sign in and yeah I use this already to Tolo some desktop CS but what's important for you is that you go to API keys and not to subr and then create a new API key you can click here say like uh demo cool and then I can receive free queries that what is by giving an email it's op Shields okay so I can caim the queries nice and now I have two th free queries on this API key that's very good and I already have the API key here so I can just copy here go back over here and paste it oh um I mean yeah feel free to just abuse that one I mean like everybody can have thousand free queries cool um now what we have here is on the top we have the subgraph and on the bottom we have um the Explorer and the documentation so the cool thing about graph is because it is um very um strongly typed language that see um we can have this Explorer on the right we can also have the docks a graph Quil query has so called root types and here we have the root type query and root type subscription although subscriptions do not work currently so we go with query if you go on query then you see a a detailed documentation about what you can query um so in this example when you look about look at um the Crypt Punk sub graph uh it has accounts which are uh holders of punks then we have the actual punks and we have metadata of the punk and all of these you see there is kind this repeating pattern of account and then accounts with s so this is the first one is to get just one and then you need to provide an ID or you can have many and then you can have a filter we will look into this and then the result is an account or a uh just one account or a list of account same if punks and see account punks metadata trades asks bits and contract and so on so forth like there's lots going on so now what do we want to do a funny thing to do with the crypto PS actually is to say I want to know which was actually the most expensive sale ever happened on a PK and uh you could go to um crypto punks here okay cool and I think you can try now lot sale so cool they say the largest sale ever happened is this one 5822 uh for um 8,000 e always always funny for me to see that the people actually pay that much money for a pixel but yeah that's how it is um now we could tried to reproduce this one um and that would be cool we go here to the sales and I like this Explorer we will have soon something like this in also on the on the the gra.com um it always looks the same we can say okay what we want to do we can actually order by the amount of the sale and the order direction we make it descending so that says um it will sort the list of results by amount and the the highest amount to be on the top and then we can say okay in which hash it was it happened we can say which uh ID it is and we want to see also the amount actually want to see the transaction and then we maybe want to have the from and to let's see and see this is for me very explorative that's why it's also called Explorer I can just um click around here and and and read stuff and and trying to figure out what act what beta is here and then I can send it cool now we see um so the highest according to the chakra uh is actually this one and funny enough when we go back here here we see this is 8k and this but that's the second one um maybe I should also get the bu out all right let's send this again we can remove this one actually have a little bit better view so 5822 yes that's what we saw here for 8K but a funny thing is that actually the highest sale ever is this one inter and start again yeah Ro call and why is why is that what happened here so when when we go to e to way by the way the usually it's um in way so what happened here this is interesting know um be out 8K is the highest sale that's what they say here but actually um we also see when we look on the blockchain um that it is uh 124k e for one Punk um and now you wonder like why is it displayed here and we can start to do investigations and that's what I think is is is the graph perfect for and say ah there is this transaction hash we go to either scan have a new compute so like aut complete not there yet and um and and con start to look into the thing and what you see here is that um someone uh uh did made a flash loan I think here we see um uh someone got oh what is this five 100 million die in a Flash loan out from I think from compound Chang in rupt e that's why we end up with this a weird number and then buy Punk and then gives it gives it all back and sells p g this person tried to uh fake this transaction and we can and interestingly uh on the official side they censor this um but they talk about it uh in a an interesting way so we can try to see the punk with the number and go here and see they say I okay this uh blah blah blah they are flash loan and they sendor it on their website yeah cool um yeah I think this explains how to query the graph