Indexing Smart Contract Data with Subgraphs - ETHGlobal Bangkok 2024

[Music] all right thank you everybody for coming uh my name is Kevin Jones I'm a developer relation my name is Kevin Jones I am a developer relations engineer working for the graph protocol I am also a core member of the build Guild I'll be doing a workshop later tonight uh at around 9:30 on building with scaffold eth uh but today uh right now we're going to be talking about the graph and how you can index your just kind of go and get started so again going Kevin uh please come find me after if you have questions uh we're going to talk about what and why the graph is important we're going from a architectural standpoint the blockchain is uh kind of like this decentralized database that keeps track of a state of variables blockchain right events and so uh what we want to do is actually capture those events and index those events right uh but there are some problems with getting access to that data um on the blockchain first of all uh there's lots going on there's lots of protocols that are uh you know competing for Block space there's lots of transactions there's balance transfers uh there's just a lot in a given block right uh on top of that the original uh uh data into your application directly from the blockchain but that doesn't really scale uh the RPC provider for a blockchain is really not designed actually from Reading perspective there is no incentive for the blockchain to really give you that uh data in a reliable way and quick way so what we actually want to do is kind of replace this layer uh and still use the uh incentivization of the blockchain's um consensus algorithm and the security of gas and you know writing transactions but but but replace this read uh configuration we also can't get historical information really easily on the blockchain right so we need a way to be able to go back in time and look at see exactly what happens at a particular time on the blockchain so we actually want to do something like this where we have this concept of like time machine which uh I I was really surprised to find out that time machine is still a thing it's actually on my laptop I did know that I never use it but it is a really cool concept where you can go back in time right and see what happened to a particular file well the same thing goes with the blockchain you need to know over time what has happened right on the blockchain uh and this is just not easy to do unless you know exactly what block number an nft was transferred right so you end up having to do some kind of indexing additionally like front ends are not really uh IDE ideal for quering an RPC directly um this is an example of someone who wanted to get a list of all the nfts in one wallet and so this is great if you only have like one or two nfts but if you have hundreds or like me I think I have like almost a thousand nfts in my wallet this would just completely make the application take forever to load uh upwards of like 10 20 30 seconds um so we want to actually use the graph right the graph is a decentralized data indexing protocol and it is uh built with with values that are uh you know pushing for decentralization and the access of that data there's over 100,000 delegators which are providing stake to the network for security there's over uh 10,000 subgraphs on the network which are like two those um subgraphs to uh you know popularize them and whatnot so we take this mess of a blockchain and what we want to do is create what are called subgraphs right so subgraphs are these unique developer focused apis that people can accessible way so we can replace that kind of earlier method where we're kind of looping and going across the Json RPC with a single graph ql query so graphql is one of the core components of the graph and this is a optimized language especially for like frontend uh applications where you can specifically say what you want to get access to and get all of the results in a single query as opposed to having to Loop over like uh you were doing in this previous example we also have uh this problem again we're it's not incentivized for rights so what we want to do is by paying e you know your gas basically your way uh to put your uh stuff on the blockchain but for reading the data you have this middleware lels we're going to look at them in detail but it's a manifest which is a high Lev config file it's an entities which is keeping track of how the data should be stored on the database and how it should be exposed with graphql and then there's the third configuration which is how we process the data so you can imagine uh we we need to be able to to listen to these events as they come off of the blockchain and we need to be able to put those into the database and we use we do that with what's called a wasen module called assembly script it's a subset of typescript so it doesn't quite have all the full functionality of typescript but it's really powerful it's got type safety and it's very performant and very secure so as events come off the blockchain we can process those in wasm right so we run these things called handlers which take the data and process it and store it into a database and this is exactly what the indexers are running and then we expose that data for the consumer over graphql so imagine all the indexers have a database running that's storing the data but a consumer does not need to understand how to do SQL queries they can exit point for all the data we'll go back figure out how to get the data how to query it and how to serve it to the consumer in a fast and reliable way so this is what we end up with we end up with the DAP we we still have our dap that is written in HTML CSS JavaScript right in contract or from the smart contract they get picked up and processed using this WM module right down here and again this is all written in assembly script it's very similar to typescripts and all that data is going to be stored in a database and then the graph node here exposes that data kind of a RPC method we have a direct integration with chain link they're here so if you're hacking you can get the graph Bounty and you can get the chain lick bounty in one shot which is really awesome uh so check this out uh this is available on the smart contract kit GitHub uh under functions the graph Unis Swap and there's an example of how you can do chain link functions with the graph so listening to the graph and running some processing some some functions and uh the chain link guy is going to talk about this a little bit more you might want to go over there and see his uh workshop and then we also have a direct integration with scaffold eth I'm going to talk about scaffold eth tonight at 9:30 in more detail I'm having a full 1hour workshop on how to do that but today we're going to actually show this off if we have time on how you can uh spin up a subgraph with scaffold eth as well and this is all done using this one single command this will spin up a front end nextjs a backend with either hard hat or Foundry and it will also spin up the graph running in this middleware layer for you it's all Docker based and we're going to set it up right now cool there's also a telegram chat if you guys want to get some help uh you can join here obviously we're going to be at the booth but um and we're going to be in the Discord but if you have question you can also uh scan this and get access okay cool let's do a live demo um we're going to just kind of go right away so uh we're going to uh spin up a new instance so we're going to do npx create d e at project- e is saying I want to use an extension and I want to use the subgraph extension so I'm going to use that this is going to spin up my project it's going to ask me what I want to name it I'm going to call it uh Bangkok and it's it's going to ask me what chain I want to use so I can use hard hat or Foundry I'm going to go ahead and use hard hat it's going to scaffold this out for me okay um you don't need to know this but basically um we need to splin up a couple screens I'll go into this more detail at my next workshop at 9:30 but we're going to spin up our chain on the left screen up here and we're going to spin up our front end over here okay and then we're going to use this other window down here to do our orchestration okay so I'm going to come down here and I'm going to split this screen up as well because we're going to need one more screen to set up the graph inside of Docker okay so this is a yarn package so it's super will spin up your front end yarn deploy will deploy your contract and then yarn run will spin up your uh subgraph environment so let's let that run the internet's a little slow here so let's see if it's going to finish soon hopefully it does uh while while it's doing that let's go ahead and go into this uh folder on all these and let's look at the code okay so go to there project and what's cool about it is it's got a monor repo style of thing so it's got nextjs here it's got hard hat and it's got the subgraph folder so inside the Hard Hat folder it's just like any hard hat instance it's got the contracts so it comes with a contract it's kind of like a very basic hello world contract and it comes with a deploy script already written for you to deploy your contract so you're ready to go now this contract is very simple I just I'm only going to talk about it for a minute it but it's got a string called greeting it's got an event that emits when the greeting changes and then it's got a function that allows you to update down below here we also have a nextjs package which has a subgraph tab and it has an example of how you can query the graph as well so we use what's called graph client which is a special SDK that's available from the graph framework so we have this kind of example of how you can like query the blockchain sorry query the subgraph and then this last part is where the subg here we got our uh blockchain if I can type blockchain's on the left our front end is here on the right yarn start and then we're going to deploy our contract over here yarn deploy [Applause] so where is it running at ah here it is sorry I had a previous instance running let me get rid of that so yarn Shan and yarn start there we go cool so now I have the block because I deployed it to the old instance so let's go do yarn deploy reset there we go uh so that's going to push my contract then I can come over here and spin up okay so that way I can take my subgraph and deploy it locally without having to use uh you know a test net or something like that so it's really good for local development okay so we know this is going to be ready when at some point it's going to say uh waiting for ethereum blocks or or like and while that's doing that let's go to the scaffold eth so scaffold eth runs on Port Local Host 3000 and we going to have to compile and then now we have this um uh kind of scaffolded environment you can see there's this subgraph tab which will tell you how to get your subgraph up and running and we also have this concept of a burner wallet where we can use this as a wallet inside of our application we don't have to connect to metamask Super handy because we can just grab some funds from the faucet and start signing transactions so let's just send a greeting real quick let's say set greeting so the thing about scaffold eth is it has this debug contracts tab which will let you um automatically start working types for your frontend application so we can instantly come in here and start testing I'm going to go into this much more DET detailed at 9:30 but we can come in here and change the greeting using this kind of built-in UI so let's say uh hello Bangkok like this hit send there we go so do yarn um well we can come here and look we can go subgraph we can do yarn uh run node is to spin up the docker containers yarn local create is to create the subgraph yarn local ship is to deploy the subgraph to your Docker instance and then this builds the assets if you need to like if you've made changes so let's run these commands Okay yarn local create this is going to basically create a very basic scaffolded uh subgraph for us then local ship is going to actually ship our um subgraph it's going to basically compile it so um it comes with a subgraph already written so it's here in this kind of uh structure so we have this subgraph configuration file that I mentioned we have the source uh which is the has the schema the graph ql configuration and it has the mapping which is the how Network that you care about tells you what address you want to index it tells you what entities that you want to save how you want to store those entities and it tells you what data types that you want to index uh and then we process those again with the mappings file so what this kind of logic okay and we will basically save that data in the database after we set all the values for the entity okay so we import those entities up here as well so these are all the types for the entities so it's really easy to work with if you know typescript it's like super easy to write a subgraph and then we have the uh data types for the that that data as well and then we do some interesting stuff here where we link the data together with dve from but let's try to deploy it so we're going to put version one here we go now it deployed it so now we can grab this URL that it gives us right here and we can load it up and we should be able to see that data that we looked for so if you go to explore you'll be able to see the data so we have all the greetings we want to grab the first five and you can kind of build out your graph ql query what the greeting is what the ID is uh who sent the greeting address was and then we hit play and there we go we get that pure Docker instance of the graph and then you deploy your changes and then you can start seeing them right away so really cool and then if we go back to where scaffold eth is here we can refresh and we should be able to see the data there at the bottom there we go so this is the example that we see on that subgraph tab gra.com we have two we have two prize categories we have best use of the graph which is first second and third place and fourth place as well I think so first second third fourth and it's split up with different uh amounts then we have best substreams which I'm not going to talk about today but you can own subgraph you can use the studio so you go to subgraph Studio you can create a subgraph here and this is going to give you all the uh examp so if I put like example like this create a subgraph got one more minute one more minute and I'm just going to answer questions outside so that way we don't have to worry about taking up too much time uh but here it gives you all the commands to spin up a subgraph if you run these it will bootstrap a subgraph for you so you don't have to write it by on YouTube and cut the speed by like a quarter and then you can watch it again [Applause] [Music]