Building an NFT API with the Graph - Nader Dabit

we're good all right thank you for helping me get set up hello everyone i'm here to talk about building nft apis on the graph protocol and this is a workshop i kind of intended to have a little more time but we're going to actually walk through how to do this and i've also created a like open source github repo for anyone that wants to actually build this themselves and maybe take a little more time but the idea here is that we want to find a smart contract that has been deployed to a network we want to index data out of that contract and we want to kind of make it queryable through some api endpoint and we're going to be using the graph protocol so the graph is a indexing and querying protocol for getting data from blockchain applications or i would say from blockchain networks there is a couple of different ways that you can build sub graphs we have a hosted service and a decentralized network and today we're going to be kind of walking through how all this stuff works so the reason that the graph exists well when you build out applications in the traditional um i would say like quote unquote like web 2 space or you're maybe dealing with traditional databases databases like uh sql and nosql databases are optimized for both read and write operations whereas blockchains are mainly optimized and all of the technology and all the innovation is typically happening around the transactions and transaction times transaction costs optimizing there but reading this data isn't quite so simple i would say so if you want to read data from a application or some type of smart contract that's been deployed to a network you might have a couple of functions that have some type of selection set that you can retrieve that data but if you want anything beyond something basic if you need something like filtering searching full text search querying things like relational data all that stuff a lot of times developers will end up building out an indexing server where you kind of take all of this data you store it in a database and you serve it up via some api endpoint the graph offers kind of a decentralized way to do that but also we have a protocol that enables people to kind of participate in the network and we can verify that the data that is being returned is actually correct so we have about 15 or so minutes and i want to kind of show how this all works if you would like to follow along with this now or later you can go to my github and it's github.com dabit3 the first workshop or the first code base that you'll see there is this denver workshop and i'm choosing the crypto coven nft project to kind of build on but this actual code base could work for almost anything so if there's any nft project that's been deployed to any of the 30 or so networks that we support you can kind of take this code base and build out your own apis on top of any of those smart contracts and obviously this doesn't only apply to nfts a lot of d5 projects like unit swap or using the graph gaming pretty much anything in the web3 space so what i'm going to do is start off just by like building out this entire thing in the end and deploying it so what we want to do is we want to first install the graph cli and i already have that installed so this is a tool that allows you to create boilerplate projects and also um just have tooling and stuff for your project so once you've deployed once you've built your api you can kind of deploy it from the command line using the graph cli so once we have the graph cli installed we can initialize a new boilerplate project so what we kind of want to do is kind of like have all the different pieces that we need to get started building and we can kind of scaffold out some example code for us to kind of then update and we're going to be using the graph init command so graph init just initializes a new subgraph and this subgraph boilerplate will be what we can use to build upon and we can pass in a few different flags so we want to pass in the contract address for the crypto coven nft uh that's been deployed to ethereum so i have a link to that here and all you would need to do for any of these apis to be to build any of these apis you just need to kind of have that contract address so we're going to say graph a net and we're going to pass in the contract address along with a couple of other flags and this will initialize kind of like a boilerplate project for us now as we're doing this we we first also need to go into the graph dashboard and create an example project for us to deploy to so i'm going to go to my dashboard here in the graph.com and i'm going to click add subgraph and i'm going to call this like denver api and this will be a crypto coven api and this will go ahead and create this subgraph for us and once this is created we can go ahead and initialize our local project and my internet's a little slow because i'm actually connected to my hotspot because for some reason the wi-fi isn't working my computer so i'll give this a couple of seconds oh yeah sure there we go yeah and while that's saving i can go ahead actually back to my command line and just assume that's that's worked and it's going to walk us through a couple of options i'm going to choose ethereum for the protocol we support a couple of different networks evm compatible near and in the near future cosmos sowana i'm going to choose the hosted service for the subgraph name i'm going to put in my username slash the name of the api i just created and this will go ahead and create it in a directory i can choose a theory of mainnet the contract address and the token name now this is going to go ahead and create a boilerplate code base for us and this might take a few seconds because i have a slow internet connection so it looks like we have a few files here that we can go ahead and get started with though so when you scaffold out a new graphql api with the graph you're going to be starting with a couple of different uh files and folders to kind of get you started but there's really three main things that you need to know when you're building subgraphs you need to have um your graphql schema defined and this is kind of your data model so if you've ever modeled data in pretty much any language any type language it's very similar to to that and if you've ever used graphql then you're probably familiar with graphql so you model your data in your graphql schema you have a main configuration file which is the subgraph.yml this defines the things like the contract addresses um the different events that you would like to index as well as the entities that you've defined in your graphql schema as well as a start block so you don't have to start indexing the network from the genesis block and then finally the src folder folder has a file called mappings and these are the function handlers that are going to be invoked whenever one of these events gets fired and we're going to kind of walk through how this works so with those three things in mind let me try this one more time here it looks like that was actually created so let me go to my subgraphs okay there we go all right so well with that in mind kind of like walking through those three pieces let's go ahead and fill these pieces out so the first thing i'm going to do probably is go ahead and define my data model and the data model is going to be something like this where we have a token and the token is going to be an nft you know non-fungible token and by default from the contract we can get the token id and we can get the owner of that token but other than that we don't really have any other information but from within the subgraph uh mapping we can actually call out to both ipfs as well as the graph node and also um the network itself so we can call the contract get information we can call back to the graph node and we can also talk to ipfs so using those three different um http calls we can actually take the id and actually get all the other metadata that we would like off of ipfs so with that being said i'm going to go ahead and copy this graphql schema go ahead and do the the token oops this is the wrong there we go so the token is going to have an id it's going to have a token uri it's going to have a few other pieces of metadata like the image the name and the description and we're going to also have a user and then we're going to be able to actually map a relationship between the token and the user and vice versa so this way we can actually query for tokens and their users and we can also query for users and their tokens and then finally we want to add full text search so here with this definition that i have here we can define which actual entity that we want to make searchable so we have that token entity and then we can define the fields that we want to make searchable so here we have all these different pieces of metadata i'm essentially just kind of like grabbing a handful of those and putting them there so that's all we need for our graphql schema the next thing that we'll go and update is our subgraph.yml so for our entities we can just update update this to be the token and the user and um for the event handlers the only event that we're going to be dealing with in this example is the transfer event and this is going to fire when a token is created or minted but it's also going to fire when someone transfers that so this kind of keeps up to date all the tokens that are available and then finally we want to go ahead and define the start block and this is when the contract uh had its first token created and this is essentially going to be where we start indexing so i can say start block put that there all right so it looks like our sub graph configuration is done and i can go to the terminal and get some boilerplate code that we can now use in our mappings based on our graphql schema and our subgraph yemo by running graph cogen and this is going to introspect our graphql schema it's going to give us some nice types as well as some apis we're interacting both with the graph node as well as the contract itself and it's going to generate those for us in a folder called generated and we can now use those in our mappings so i'm going to delete those and the last thing we need to do is write the mappings and this is kind of the actual business logic that you're going to have to write as a developer like everything else is just configuration stuff but here you're actually saying okay this is what i want to do with the code i want to actually take this token id i want to like you know call this contract i want to get this information i want to take this token object and i'm going to append these different properties to it and then we want to save it to the graph node and then from there it would be made available so i'm going to go ahead and start by just pasting in these imports these are the apis as well as the types for interacting with uh the contract itself and then these are apis for interacting with the graph node which is kind of like you could think of as the database where all this is stored and then we also have the ipfs hash and you can get the base uri for almost any erc 721 by going to the contract and they have a function called git base uri and this is kind of where you would get that and kind of using this ipfs hash we can say like this ipfs slash tokenid.

json get the metadata append it to our object and save it to the graph node and that's what's going to be happening in this very very long function and again like this is probably not the right time 20 minutes for a workshop like this for me to actually walk through all this code but what i can tell you what's happening is in this function right here this handle transfer every time a function is transferred from one user to another or one is minted this function is fired and then this is where we're going to be actually storing that information in the graph node so we first check to see if a token exists if it does exist all we do is update the owner because all the metadata should be the same if it doesn't exist we kind of go through and we um construct this object with all the different metadata by talking to ipfs by calling this ipfs. this ipfs api that we have from the graph typescript library so if i save that file where we're actually done with everything and i can go here and i can go ahead and run a build so i should be able to run graph build to kind of make sure everything is configured properly and if this is configured properly we can now deploy this and test it out the build's completed i can now go to the cli and i need to deploy this to that project that i created in the graph dashboard and to do that i need to first authenticate so no one can kind of deploy um to my you know i guess app and all we need to do is just like copy this access token i know this is public but i'm okay with that this is a test account i'm going to go ahead and say graph auth and it's going to go ahead and authenticate and then now i can actually deploy from this session so i can say yarn deploy and this is going to go ahead and deploy the api and we should be able to test it out now this uh is running on my phone's wi-fi so i'm not sure how long this is going to take but it seems to be working all right all right that was actually fast so if i go back to this and i refresh we should see we have a nice uh graphql text editor graphical editor for us to start sending queries and by default we have kind of an example query we see that we're getting our our information back and we can say okay i want to get the image i want to get the name and i want to get the description and we're getting all the information back and if i kind of like take this image and i paste it here whoops then we should get a crypto coven so it looks like our api is working the next thing we might want to do is like oh okay i want to um now add full text search to off to my app so i can now invoke that query that we created earlier called i think it was like covent search and i can pass in the text that i would like to search so here i'm going to say something like nyx i guess and we're going to only get the um the crypto covens with the name nix and i'll just define the data that i want returned and there we go we have that one that came back a couple of other queries that you might want to do would be stuff like um pagination so here i'm just like saying i'm going to skip the first 100 and i'm going to kind of like get uh that second page also you can do stuff like ordering so here i might want to say i'm going to order by the updated timestamp so we can kind of have like a nice ui that shows the most recently created and updated i mean the most recently created and transferred uh nfts so that query is coming back and then maybe there's one more query that we haven't checked out yeah so there's also filtering so in addition to the full text search you might want to say something like okay i only want to get the nft crypto covens that have this sign which is capricorn and then we're getting all that back and the beauty of graphql is that you can define the data that you won't return so here we can go into this graphical edit i'm sorry the graphql schema introspection on the right and we can see all these different properties that are available and we can define the ones that we want to bring back and therefore you're kind of like saving on the bandwidth and therefore you kind of have like a more optimized api that is um more performant in areas where you might have limited internet bandwidth and stuff like that so that's it we kind of built an entire graphql api from scratch in about 20 minutes and if you've ever tried to build your own indexer you realize the pain and suffering that something like this might help you with so um i would encourage you to check it out and check out this workshop check out the graph docs uh we're on twitter at the graph protocol and uh yeah there's a bunch of tutorials and stuff out there so check all that stuff out and if anyone has any questions i'll go ahead and start answering those now yes the only networks oh yeah the question was like we called ipfs from within the the graph node uh mapping the only networks that we allow calls to be made to are ipfs the graph note itself and also the network uh for which your uh api is deployed to so in our case it would be ethereum mainnet so you can like call the contract and uh you know get information from there but other than that no you can't just make an arbitrary like http call yeah yes yes yes so the graph network basically has a network of indexers and if you want to become an indexer you stake grt graph tokens and you're now operating a node on the network and um if you are serving like the wrong improper data you get slash so therefore that's kind of like where that data verifiability comes because if you call a blockchain network directly through an rpc that you own you know that data is is proper but if you're um running your own index or you're kind of like calling a centralized indexer you don't know if that uh data is verified so that's kind of the whole idea around how the network works do you support event triggers or web hooks based on specific so graphql has this idea of like graphql subscriptions which are really nice and we actually support subscriptions but not i would say in a way that it's currently production ready some people are using them but uh but right now we don't support subscriptions on the client the graph node itself like once you deploy it will have listeners set up and it will continue to index new um new data that's coming through but on your client-side application you might have to set up some type of your own event listener and kind of like either refetch or maybe just deal with that data as it's coming through yourself i don't know if i missed this in the beginning but do you need grt to deploy a subnet and to query it it's all no there's there's there's a couple of ways just following the tutorial here that you can deploy and run it in production you should be fine so you never need grt to do any so there's like a the the service that we use is the hosted service so this is kind of i would say not the decentralized network so if you wanted that data fair viability then you would need grt to deploy and then use your api on the on the network okay so thank you sure hey you had mentioned that uh these indexers are running could you go a little bit into like the incentive models on how they are rewarded for running these indexers um is the can you repeat the the main part of the question i did hear you're kind of asking about indexers yeah so someone is staking their grt and they're running an index or how are they rewarded for for doing so okay yeah so you get rewarded as an indexer uh with the combination of the query fees that the the developer that's kind of like using that api is paying along with network wide inflation so there's like a token inflation that gets distributed to all the network participants that are running indexers and you can also be a non-technical person and delegate tokens to an indexer so you don't have to kind of be a developer to to participate in the network anybody else all right thank you so much so a couple things real quick uh first of all