The Graph 🛠 Building Subgraphs for the decentralized Graph Network

[Music] [Applause] [Music] seems like i need a new laptop yeah my name is simon i'm a solutions engineer at your node it's one of the core devs behind the graph and because we lost a little bit of time i'd just go very quick and maybe have some time for uh questions so like for adap you need to be fully decentralized this is what the graph can do so you can have no servers you can break free from authorities so this is roughly how it looks so you have like adapts they communicate with smart contracts that's the problem there is that the blockchain is not really made for data reading it's made for data writing so um you pay gas to the miners to to get your block in but you do not pay anybody to read the data so like nobody is actually incentivized to do this and if you look at the protocols it's kind of a mess it's everything mixed together and then if you want to retrieve data in the front end it looks like this kind of very messy uh code so the graph is kind of this api like it provides apis for this decentralized future that we all work towards together we have currently one billion queries on the hosted service per day and um it's a decentralized indexing protocol actually it's not only the hosted service so when you look back to those um those all transactions that are kind of mixed on the blockchain what the graph actually does is kind of it it it it creates separate databases for each of the of the protocols so then you can like uh just query the one data that you actually needing so kind of the new architecture is like you have an user interface that directly writes to the blockchain but reads from an indexing protocol like the graph and you have the data sources this um ethereum but also near its integrated arbitrary optimism and rv is being worked on so like there are even more data sources that come that can be indexed by the graph so in the end we end up with this very neat graphql query that um yeah that was the same like before but it's just much much nicer so um since one year you can really deploy your subgraph to a decentralized network there are 150 indexes all over the world so and then it's it's going up so it's really possible to have like a decentralized data storage for your for your dap um yeah so you can really have fully truly decentralized apps yeah this workshop usually it's one hour but at 215 minutes so we look at nfts so nfts basically are these simple contracts or interfaces basically we just look at the transfer event so every time you move an nft around then a transfer event is emitted and in the subgraph you just store that as i said before you can yeah you can watch for these events and then store it into the database that is how rough it looks on the top view of the depth it writes to the smart contracts the graph node indexes the data from the smart contract puts it into the store and on the left side tab queries through a graphical api this data i have some tips and tricks here but i skipped that um also there are some patterns but we will see them later but um actually when you scan that qr code here then you arrive on a on a github page and on github page this presentation is linked so you can kind of at your own time go through because what i really want to show you is how it's actually made because that's why we are here kind of it's a hackathon not a i know a pump and dump conference so um yeah and here is my twitter but yeah let's start with this so i should have it open already here it is all right um so basically we just take a nft contract oh it's the wrong one yeah it's old so we take nft contract so in this uh in this example we take the hash marks contract so it's kind of this uh this address on um on ethercan and then we can actually you need to install the graph cli and then you can run this simple uh or maybe even a little bit complex uh command what it does it says like i want to create a new subgraph i want to have it in using this upgrade studio so i'm ready to deploy the decentralized network um i did specify the contract that i want to index um i want to index all the events automatically that's cool then i give a name to the for the contract to say which network this is ethereum net but you can also do binance polygon arbitrary optimism cello and whatever on any other kind of evm compatible blockchain but also near but then the contracts look a bit different but i mean it's if amsterdam here so we do ethereum um yes then you can provide an api actually on ethereum you don't need to provide an api and then you give a name to the for the folder and lastly a name for the for the sub graph that you create later in studio all right let's do this um i go quickly in my uh folder so let's just copy paste this in i think it should work without api so let's kick that let's see so what it does yeah all right um this was kind of expected so yeah i tested it before so this directory already existed but it fetches the api from either scan then um it creates this subgraph scaffold which we just look into now and then it initialized the whole thing it installs a dependency with yarn and um yeah and then bootstraps everything it's actually quite convenient especially this index event like if you get started with a subgraph development then you have kind of a contract you want to index i would advise you to use that dash just index events because that creates you boilerplate code that makes more or less sense so we can now go into this um let's just say hashmastercraft tool yeah here it is um right so this is how it looks like so when i when i look at the subgraph um then usually what's interesting is actually it's a package json because it's javascript um project so here in the package json it's actually not not that much going on so you have these two dependencies graph cli and graph yes and then you have these code chain and build commands and we will execute them so we'll you will just see also the deploy command is already here so it's quite convenient then the next thing that i look at is the subgraph yaml here is kind of the manifest of the subgraph here it's written what's going on so you have stuff like again like which which which network to be indexed what's the name of the contract uh what's the contract address the api name and here the event handlers like which events we all start to to to to watch for and yes and what's important like unfortunately not not comes out of the box is that i would really advise to if you have the source that you provide the start block because then you tell the graph like just start at this block and not like from block 0 which makes indexing much faster so we can actually go in this creation transaction and then we have here the start block so we can take the start block put it here and yeah i would say that's it and then it also generates a schema it's kind of a database schema um you have uh the entities entities you can think about it's kind of tables so you have uh now we just we just store the events it's not super super uh sophisticated so we just have this approval stuff the name change that is a little bit special for the hashmasks then ownership transferred but then the transfer event which is actually the one that i'm most interested in and then we have a mapping and the mappings is kind of this glue code so here you say like which events do we want to watch for so we have the event handler so we want to say for example every transfer i want to do something then you say like okay this is the handler then it jumps into uh it jumps into this mappings and checks for this handle transfer and then executes this also this is actually quite simple so we just create a new entity we give it some id um it makes sense to use the transaction hash and the login deck so it's kind of identical or like yeah identical per per per actual event then we have the from two token id and we just save it it's not it's not so so crazy and now if you have this we can already do this um yarn code jen this in the background creates the boilerplate code so it goes into this generated folder and writes um uh according to the ap to the api and the database schema it writes these helper functions which in the end and in development help so you can for example say like oh i want to know what's inside of the of the transfer event because assembly script is similar to typescript it just works so i can click click here and then i see oh it's an ethereum event you can go into an ethereum event i test these these parameters and stuff so you can start to explore the code through that feature then the next command is kind of yarn build so i want to see if we can actually build this uh sub graph now here locally before we try to deploy okay that works and yeah and then we can actually just go to the subgraph studio connect the wallet mm-hmm if your mainnet connect you can also log in with the nozis multisig so you have a little bit more control maybe of your team if you have bigger subgraphs that's what we do with those sub graphs that we currently migrate for a lot of projects this week we had gelato for example migrate to this interest network and um yeah kind of this is really going on like the theorem maintenance upgraphs they from the hosted service they are now slowly but surely coming over to the decent glass network um yeah sometimes you need to do refreshes because we're still early yes here it is sign message cool so you see i already did a lot of those test stuff i hope i don't have any hash mask yet let me check what i named it here hashmas okay so like all right this exists you click of create a new subgraph you say you index is your main net only mainnet if your maintenance is supported at the moment on the subgraph studio all right then you take the name here the this one this lock it's important that you have that this correspondence here in in this it's kind of the slog that you need to combine and then you need to authenticate so you run this graph off command and then you can do yarn deploy i forgot to save then it will then you will be asked for um version yeah whatever makes sense in your project kind of keep the version name and then yeah it's deployed sometimes this fails but um we are lucky today and then you if it fails you just retry i mean like it's the same if the browser force refresh retry eventually it will work yeah so we have it now deployed so we see it already here it starts syncing this is just in the in the subgraph studio so it's still on a centralized server if you really want to publish it to decentralized network then you could you click on this one you say ethereum can also be the first one to signal on it although i don't have any grts on that account but this is actually how it would work if you really deploy to decentralized network um yeah but we have it already here so you have a so a temporary query url that you can use for your development and you can go into playground i can for example say like yeah let's see what we have we should see already some transfers yes see all right so we see we see already the transfer so it's so it happened so i mean like it took me 10 minutes to to kind of bootstrap this this up graph uh from the contract that is already there but since um i i just now just continue working on that sub graph until they throw us out so the next step would be we say like okay i mean this is nice but with me down just the events index it's a good start already i mean it helps you have graphql for your events but it's not really i mean like it's it's not really cool so what i want but what what i said before is like what i'm actually interested in is just to transfer so we can go here and just start to clean out clean up this thing so we can just remove all that stuff also the entities the schema where is it and mappings then i can do yarn code gen and the other cool thing about this is that it will now tell me like hey you're trying to import stuff that doesn't exist because i regenerated the code so like the the editor now helps me i can just be like late night coding unprepared at the hackathon and the code the editor helps me to do everything and i can do again yarn build which then kind of confirms to me that the subgraph like it's everything it's correct and it looks not bad now the next step would be i mean you can follow that github repo here um to actually make a little bit more more more sense out of it so we can say okay we can for example introduce this pattern with the immutable events so we can say like okay and we can store the block number the timestamp so it makes a little bit more sense and we see like from who to who that the sub graph goes so we first update the schema so now we have from 2 token id timestamp block number that makes it a bit more sense and then also to um yeah and we also need to a little bit extend the mappings actually but i mean it's not that crazy difference between before we just we just get out a little bit more data out of this transfer event and see now it says me oh i don't have the entity the block number on the entity and the reason behind is that i changed the schema but i didn't run yarn code gen so the the editor doesn't know about like my new intentions and as soon as i do this magically after some while this disappears yeah here it is again okay now we have it extended we don't need to deploy now the next step is actually we are not just interested in transfers we are actually interested in tokens and owners and and the contract and we even maybe want to have kind of some aggregations on the code of those um tokens and also some aggregators on the owner so we want to know which owner does own how many tokens so the first step to actually go in this direction is that to write the schema this schema is more or less clear i would say it's similar now what we now have is this derived from field so you see that transfers um are linked to this transfers and and it goes through this token field that we have here so so we can have this reverse lookup up through the tables but you can also read that in the docs or come to our discord and ask me and i'm happy to explain it a little bit more verbose so this makes a little bit more sense so this kind of actually starts to really give us insights into what's going on um oh nice i didn't write the actual mappings code i think it's here yes so let's take this whole function and discuss it in vs code we see a lot of interesting patterns here all right again oh maybe i need to import everything and this is copy paste error let me check all right this is i can go go to this uh the editor also helps with um this autocomplete stuff all right so i can say like okay um where do you find the contract and then it says ah it's in this generated streamer then i can say it wants to have the event address to hex string all right the token it's probably also in the schema i can load it bigint will be in graph ts and here we have the two hex string quickly fix this token number then we have a token number id yeah just skip it right the hash mask they don't have a token your eyes i really skip that um okay this will work the address all right this looks already good not sure what is this and let's skip that for now okay so what we do here now we make it a little bit more complex so we have a transfer event again this is similar we store some stuff then we store token id you have this helper function get or create owner which tries to load the owner from the database and if it doesn't exist it will just create it and save it and then return it so that's quite handy um it's a good pattern like for for this kind of returning you don't know if the object already exists then here we um we load the contract from the database um as a kind of a singleton where we store these aggregations on top but also here we we've bind we bind now to the contract instance so we can do eve calls on the instance so like the name and the symbol these are eve calls that go back be careful with ease calls in general because they take a lot of time they are usually the the main reason that the subgraph does not index fast that's why i only do it like kind of once i try to say okay if we don't see the contract if uh initially then we don't we didn't have we didn't save it then i go through and initialize it and then i do the eve calls but then like in the next transfer we have the contract already in the database and then i don't need to um do these if calls again and so we kind of here is this kind of this initialized thing then the same with the token like the first time we see a transfer we see a token and then we can take the token yeah with other contracts i get the token uri but unfortunately hashmaster doesn't adhere to this metadata pattern um so i have to comment this out uh yeah and then i i say okay i i i am i go just up if the as soon as i see a new token then i know i can can increase the total supply of of the contract and then and then also i give the the token i assign an owner to the token i do add more to the transfer and then i save everything and that's it now let's see if i can build this it might run into weird problems because i kind of just comment the code out and the schema expects some values but now let's just deploy again now we increase the version so it doesn't collide yeah in the meantime um [Music] yeah if you i know it was a little bit quick and a little bit fast usually i have an hour for these workshops and my laptop doesn't have any problems but yeah we tried our best um but if you still have any questions like we have a graph booth i try to be there as much as possible so if you have any ideas what you want to do with the graph if you want to build a subgraph or you want to query an existing subgraph so and so forth feel free to come by there should be always kind of good people around to give you a nice answer feel also free to follow me on twitter and write me dms i try to answer to them although i mean like the hackathon's a bit uh hectic but um maybe you can try and i try my best to answer all those and and if that doesn't work you can also still go into the graph discord we have dedicated support to tech support people that are around here for the weekend especially for your questions so go to the graph discord or also to the um the eve global discord both should should have kind of some coverage um so it should you should really be able to find someone who helps you and uh yeah and last but not least um in the beginning of june there is the graph day in san francisco so for those who are in the us or like to travel to the us feel free to uh to come by there are also um scholarships for for hackers if you want to yes so let's go quickly back and see our results it should um yeah it has deployed but it might result in an indexing error not bad all right now we can start to query and now it's getting more interesting because we can actually see the tokens right so we see tokens owners we see some transfers on the tokens yeah so we have much better actually data now that makes much more sense contracts let's check this we can also see like how many talks we've already seen oh this maybe it's a book i never checked one last thing if you do this and you go into this query url yeah feel free and directly kind of open this into your browser then here you have kind of this very neat graphic ul interface it's better than the playground here so you can go here on the explorer and start to explore uh the subgraph actually the same applies to every subgraph that is on the decentralized network so when you for example go to i know it's a cool subgraph ens so you can do the same although it's not linked so this kind of a trick that i show you so you can click right click open address a new tab you have this very neat graphical interface and you can start to explore this one is a bit bigger than the one before but i mean like uh you can go there and see like accounts first 10 the id one with domains the name whatever send the query and then um yeah you see these are the burned ens addresses interesting yeah i learned something i think that's it thank you so much for your attention sorry for the stress and uh yeah fun hacking [Applause] [Music] [Applause] [Music] you