The Graph: Edge & Node 🛠 Building a custom NFT subgraph

[Applause] [Music] i think we start the thing is recorded but i'm also around at the booth for questions later so i'm hi my name is simon i'm a solutions engineer at the company edgernode that's a core dev working on the graph protocol um the graph protocol is also a sponsor of this hackathon we will look at the prices later but uh in this workshop i quickly show you how to build and publish a customized and decentralized nft sub graph so yeah eva bailin the director of the graph foundation once tweeted and said like yeah if you build the app that for the community but you using centralized components just for your vcs or whatever then you're not really building a dab you're just building an app using web3 stuff right um so the but the problem is uh really having a decentralized stack is quite is quite uh challenging so um but i i walked you through through these steps so the data storage of an nft contract can be roughly be seen as just you know a google spreadsheet where you have like owners and then ids and metadata and that's kind of the state of the blockchain so you can always just create a blockchain say who who owns which uh let's say the first punk but um it's very hard to see who owned that punk before you can do that by by digging through the blockchain but it's not easy so like that's what the nft people call the provenance so like who owned it before um yeah and uh the blockchain in basically works like this so you send the transaction to the blockchain and then with that transaction you change the state of the blockchain and then eventually and then immediately you can read the current state but you cannot really read what's what is going on so that's what you would like to have is kind of this time travel feature that we that we know from mac os so that you actually have like the same data but you can actually go back in time and see like how was it before and um yeah this is actually what the graph the graph enables um i i said like that this is the problem with with the with the depths in general so like initially the depths were kind of thinking about yeah we can have like a user interface that is just html css and javascript um it's deployed to ipfs so it's um it's already a permissionless storage and then it directly interacts with the blockchain so everybody runs their own blockchain node on their computer in a fully decentralized manner um time time was telling like this this is not the case and um like that the biggest problem is actually like blockchain is right optimized and incentivized so by sending transactions to the blockchain you pay gas and that gas the miners take and so they can kind of are incentivized to run that stuff but um if you want to just read data from the blockchain then you then you start to kind of talk to the miners and say or to the to the nodes and say like hey can i can i know this data and there is no incentivization mechanism to do so there were like more than one billion queries per day on that hosted service so um that's that's quite a lot and and we are currently in the process of migrating all these queries to the decentralized network so whoever has a sub graph on the hosted service and uses it currently um feel free to come to me i'm i'm helping you to migrate those subgraphs over to the decentralized network i also have t-shirts and pull-ups for those who do so i'm yeah grab me um as i said it's a decentralized indexing protocol um yeah animated um there are like more than 160 actually indexes world it's permissionless so like these indexes they are kind of independent everybody can join um and through that you have like this this redundancy across the globe that your data is always accessible if one indexer goes down another one is there to serve your queries um also because there are across the globe wherever your users are when they query the graph then they will receive a response from index next to them or like form index that can give a quick quick result um queries are kind of they are fast cheap and reliable um but what changes might be a little bit new is like now you pay for the queries so that's why it's now the queries are incentive so the you pay a small amount for the queries it's not that much but it kind of shifts this power structure um towards the users who you're only in control of stuff when you somehow pay for it right otherwise you have kind of data miners and the ads and stuff that try to make money out of it um yeah and so that's kind of the vision we're going through this global open decentralized api and i would invite you to join and um build on it so this enables us to build truly decentralized apps yeah so this is uh the rough overview kind of already are there any questions about the graph in on this level before we dive into the tech no otherwise we are around we have a booth come and uh find us so what is a sub graph so subgroup is kind of this this piece that um is at the core of the graph you can think of it like as we saw before the the blockchain is kind of actually a mess you have like all these transactions for all these different protocols more or less on top of each other now if you want to make sense out of it and see like what actually happened with let's say sora then you need to go through all these transactions and take the ones from sora out and pull them together to make sense out of it right so that's what happens and that's why it's so slow um but what would be nice is actually if we have like separated databases per protocol which store that data for that protocols in a way that you can easily query it and that's what actually a sub graph is so a subgraph is a definition of like how do i want to have like my data database schema and how do i want to put data that we see from the blockchain into that schema and then it generates you automatically interface to query it so in the end the lifecycle of a query or the whole thing works a little bit like this on the top we have the dap and we know that kind of just the front end html css it sends transactions to the smart contracts or to the blockchain node which the the the the smart contracts usually they emit events so the graph node in the middle that it listens to those events on the blockchain and when these events happen uh it they go through this mapping which is a web assembly module and the mapping makes sense out of these events and stores it into the database that we see on the bottom right and then on the other hand side like if the dap wants to send a query on the left that it goes to this graphql api that we saw before a very nice expressive query language to get that data out of the graph node so this is roughly the system architecture so because there was this nft nyc and because it's a cool example anyways let's see how a nft subgraph would look like and we can also kind of build one together quickly so like at the nutshell the erc 721 standard is just you know you have to transfer events that you see like which token moved from who to whom um you have like this balance off function where you can check the balance of a certain address you can check the owner and um yeah you have this transfer function that you can actually uh trigger transfers so it's it's basically super simple um so we can look at this when you get into sub graph development then here are some tips and tricks so i um yeah you should emit when you write today when you write the smart contracts you should be a little bit verbal with the events don't try to save gas there because it's you shoot you later in the food in kind of making sense again out of your contract and then with the subgraph development there are these eve calls so that's let's try to not call back let's try to have all the data that you need in the subgraph in the in in the events so that can make the subgraph indexing much much faster um yeah you can create entities for important objects and concepts we'll look that later more in practice you can store aggregation data on those entities like counts and stuff and link those entities together so you can have more meaningful queries and also more explorational queries and then there is kind of the standards nft standards of vt wtf it's a good resource um but yeah you can also uh ask me so then you can start to think about like okay what kind of questions do i wanna ask uh like what questions do i want to have answers through with my subgroups so like for the nfts kind of how many nfts are there in total how many nfgs have this trade x who owns which nft um what's the provenance who owned it before is it available when we have on-chain markets um what is the historic price uh what's the current asking price and so on so forth um so you you start to think about that and then you can come up with a schema so this is a rough schema how i would like model at nft so you have a contract that's the collection it has accounts about everything and it links to the owners and you have the accounts which are the owners you see the holdings which will other will also link to the nfts uh where you have the token id the metadata uri um then you have like the events you would just store the events also like transfer the means bid asks and sales if you have them block number block cache you want to have per event metadata and trades if available so let's do this so first you install the dependencies i created actually a small github repo that's here i will also tweet that later so that's here on my github that you actually can follow the steps um it's also linked to this slide so basically you install this graph cli globally to get started and then you can be going to look at the hash masks contract so you can go to this um to either scan you see that contract um so this that's the hash most so we know the contract address but also what's interesting is kind of this start block right because we do not want to start to index the blockchain from from genesis if you're just interested in a contract that was deployed at a certain amount of time a certain point in time and now this one this is actually the biggest time saver especially for the hackathon projects if you want to quickly start a subgraph from an existing smart contract on ether scan or on ethereum or any other chain that is supported not all of them are supported with this quick start you can also do this with if you can provide the api and it's a little bit more complex but we want to move fast here so what i do is like i just run this so what it does like it says okay graph initialize me a new subgraph i want to deploy to the subgraph studio which helps me to deploy it to the decentralized graph network then i kind of say like this is the contract address and this is the important one index event so i want to have like a scaffold which automatically indexes the events and then i want to have like that's the contract name is a steve hashmask which network i'm indexing and um yeah and then also the directory so let's fire this i did it before now you need just me to trust me that this works because before it took very long to yarn install everything so i was like maybe uh we can skip that so that that will be the outcome um so you have this thing here let's make it a bit bigger is it big enough yeah so by looking at um uh it is basically it's a javascript-ish environment so usually you can start to look at with package.json so we see here um you have some we have some dependencies and some some scripts pre-defined so that's already good then the next thing that i look at is kind of the subgraph yaml here is defined like how the subgraph um yeah kind of it keeps it all together it says like okay here are the data sources you can have one or many we see again like it's ethereum mainnet it's this is the address this is the api uh these are the entities we go later to the entities and then we also have the event handler so this um est 721 they emit these events like approval approval for all name change this is special for the hashmast because you can name them ownership transfer this also because ownable contract and also the transfer event um so this is here and then you say like okay and i want to put i want to run this in this mapping.ts but first we look quickly at the schema this is auto-generated so like for every event um it automatically creates the entity so despite that this is kind of named schema.graphql what this actually is is kind of a database schema it's not the exact graphql schema that will come out of it but we'll see that later so like everything has kind of id and more or less just uh whatever it was whatever is in this event kind of stored and then also we have ins in source mappings we also have the boilerplate stuff as we have these all these these handlers um actually the most interesting one is the handle transfer so every time a transfer happens on the ethereum blockchain of a of a hash mask this handler is triggered and what it does is kind of it creates a new transfer entity it creates this id for it which is just the hat the transaction hash and the log index so you so you have kind of canonical identification of that um event and then you add those things to the entity and then you just save it i mean it's it's super simple um right so uh you can check if everything works fine you you're on code gen which generates the boilerplate code here so it creates these helper functions um that help that help you to interact with the entities and um and also the the abi and these are just abi helper functions all right so this this works i can check with yarn build if i actually can build it on my local machine yeah okay cool that works right so um here it it has created the web assembly files that i can now upload i can go here um to the graph.com studio [Music] and yeah i'm already connected so you connect with with the metamask i can go here and say like and indexing ethereum net and uh yeah you're already here and then i can only do this i only need to do this graph auth and then [Music] graph deploy you need to give a version and uh then it creates this again this web assembly files uploads it to ipfs and then and and deploys this to the studio studio the graph studio is kind of a centralized service provided by edunote for you for easily developed subgraphs so you see it's it's kind of deployed already so i can quickly refresh here and it should index yeah so it already starts to scan to scan see we can see here the logs and we can also see here that it didn't sink yet all right it will it will take a while but that's okay because um oh i forgot to add the start block so start block it actually it indexes quicker cool i changed this and then i can just redeploy and i'll probably need to give a new version and that should be quicker um going back to the presentation so this was just like the super quick start of getting started with the graph now we have kind of this just all the events stored in entities but you probably want to do more complex stuff um so i i show you some patterns um so you want to store these immutable events that's actually what we did like events are typically immutable events because uh immutable entities because they are just happening once and you do not want to change an event after that you just store it and i also think about a little bit about events as the low level entities on top of that you start to build more interesting stuff like the account actual accounts actual nfts which you can derive from that events so you have an account so you see an event so you basically see an event it's a transfer and by seeing the transfer you can see okay from who to who was the transfer so you if so you have already two new um entities namely two new owners and then you can just start to count up and down so these are kind of a little bit the highest but for for the events i would go with immutable so this is the pattern for it um you can get have those get or create owner um helpers because you often want to get the owner or want to get the token or whatever and it's always a little bit the same if you if you see it for the first time you want to create it and if you have it already you just want to load it and maybe add something to it and uh in the in the uh schema you want to link the entities so here the contract and the transfers are linked and also you have the aggregation entities where you can um uh where you store counts so for example uni swap has aggregation entities so so they store one line for every day and then just just start to do the discounts so that in the end they can display their charts yeah before we go back to the code there are two categories of prices from from the graph the first is like best use of existing sub graphs this is actu this is in my opinion a little bit easier because you can just go um [Music] for example to uh to the graph explorer and then you look at an existing subgraph you have very cool subgraphs here so maybe this one is cool the 721 marketplace is one that all that indexes all the [Music] uh the the nft sales actually so you can go that go here and start to send your queries you can start to um explore how it how it is set up which entities you have which things you can just start to query it and think you can can use the end point to start to query so this is a little bit easier you don't need to learn sub graph development um and you already kind of have the the same price as for the new subgraphs so kind of a trick if you're a good front-end developer this is a a cool track to get to get some prize money the other track is the best new sub graph so here we expect you to write a new sub graph and and and we look at complexity innovation and um and that kind of stuff so again we have like three three like a first second third prize if uh you want to get some bonus points there there are two stuff that uh that you can do so use the decentralized graph network that's what i showed you here if you go to this explorer graph.

com there is also the hosted service which you can still use but if you use the decentralized network you will get bonus points and also by using substream so substreams is a very new technology that we recently announced at the graph day in san francisco and it's actually kind of a new paradigm to write subgraphs uh it's it there will be written in rust the technology is very young but it's very powerful so like if you're into rust and into the indexing blockchain data or kind of trying to stream blockchain data um yeah think about to make your hand dirty with um substreams and that would be also kind of a big bonus point so like if you're writing in a substream you will be very high up for this best new subgraph um yeah so this is my this is my twitter um you can always uh idms are open you can write me on twitter or you come to the graph booth or there's also the discord channel that you can go in and now we can try to see if um yes if that had started to index yeah already that's cool so see we see for example like we have we can we can check the approvals and we already have approvals here or we can check for the name changes can see like which are the the recent name changes that we saw from the hash masks so this is this is just given by just indexing those those events um in my github repo that i linked here i was actually going further of then removing unused entities extend entities and write more stuff to create these logical entities on top of it then you can all kind of start to look into this so there's the schema how it looks like later on and there is kind of a mapping how it would look like if you really go through those steps but you saw like my goal for this presentation is just to say to you look with the graph init command you can very quickly spin up the soft subgraph it already kind of gives you very good insights and from that you can start to um explore explore the stuff and if you have a very cool sub graph on the decentralized network uh on on ethereum and you want to publish it to the decentralized network then you can click here on publish you say where you want to publish and then [Music] i can wouldn't drink a bee or may ring b if you want to just test it out or mainnet if you wanted to do it and then um yeah you can you can do this and really be part of uh the decentralization movement um yeah that's the finish the end of my talk are there any questions yes um no pop publishing um so by publishing you you should signal 10k trt on it um and then just the on chain action is probably around 0.05 eth or something like that it's it's not super crazy um and for the for the grts for the initial signal you can actually reach out to me or to the graph foundation so we help you bootstrap this all the questions cool thank you so much for your attention i bring my thing here again and happy hacking you