Rapid dApp Development with GraphQL: Let's Get RADD – Brendan Asselstine

so we are here with Brennan deceased from from pool together everybody my name is Brennan ass this time from pool together I'm really excited to be here because last year we debuted pool together at eath Denver that's where the hackathon projects started so it's really cool to come back it's kind of like our homecoming our anniversary and and so I'm happy to present how we actually built the front-end because prior to pool together we had actually been running a dev shop for about a year so in total we've been running building DAPs for about two years now and so I want to share some of the insights that we've gleaned from basically doing the data binding with the front end because that we found was the hairiest piece because dealing with an etherium node and talking to it directly can be very complicated so this is what going to cover we're gonna cover an introduction to graph QL we're gonna cover how pool together leverages graph QL and we're gonna build adapt very quickly its I actually have a lot to cover so we're probably gonna just zip right through that but there is a tutorial online and then we're gonna extend the DAP using the graph protocol we really like the graph protocol it gives us totally flexible data so we can actually go back and like as we're gonna do we're gonna get a term report as well as the existing sub graph so you can build as many sub graphs you want so you get as much data as you want and then finally we're gonna finish with discussing AIP 1767 which is really important because that's the etherium client graph QL endpoints standard which is slowly permeating through all the different client implementations so it's not quite there yet though because we're really waiting on you in Ferrara because people don't necessarily want to run their own nose all the time because it's complicated when we did it with AWS we got shut down because they thought I was a spam bot because with all the peer connections it was just crazy and then I want to close with questions so this could be stuff about pulled together smart contracts it can be about being the graphic you all work or anything so let's jump in out of you guys who has experience with graph QL you could play to the poor and play with it before so so I'll kind of fly through this then because it's you know it's pretty straightforward so graph QL is a language to define query and mutate data so and that's at its core what it is it's just a way for you to communicate with a server it's just a language the actual implementation could be anything you can use Postgres you could use Redis you can basically bind it to any data source you just need graft well resolvers to be able to return the data as you need the other half of that is the graph QL query language so once you do define your type definitions you can then query it in a nested way using the query language so let's have a look at that now so this is a super simple example it demonstrates that you can set in ohms so enums are could be basically treated like primitives once you define them did you define your entities in this case we define a starship I actually pulled this off a Apollo client example sheet and and the idea is very important there you can see that it's ID exclamation mark and this really helps clients cache data because knowing the ID and the type of an object it's very easy to store it client-side and refresh it only when you need to and then you can see we've got the name diff definition and we have length as well and it's interesting to note here that length is a function so we're actually passing it a unit which it then will affect the size of the float that it returns so if it's meter or feet that'll be what the length returns and then on the right side this is actually a very special type type query because these are the top-level queries where you start in your graph 12 query so everything will have to start with starships or starship and then within starship you can select one of its attributes you can also have associations so you can select other entities within that and here you can see so starships on the right it returns an array as defined by the square brackets of starship and then as the singular one where you pass an ID returns a singular star so this is what a query looks like you can see that you can pack as much data in here as you want to and this is one of the series benefits of graphic QL and that you don't need to do a bunch of different queries to pull in a bunch of different data you get exactly what you need and you can see that here in the hero : starship so we've called that top-level starship query and we tell it we want the ID in the name but we don't need length so we don't even tell it that we need that so it's not gonna return us that data and then we rename it as hero so that in the front-end we can just go data hero and access it that way and then finally you can see that we're accessing all the starships as well so why graph QL why is it so useful why is it important because JSON RPC API is like existing HTTP restful api s mean you're either over fetching or under fetching your data and you're doing a lot of queries you're just drilling your server with all these different queries pulling in your different model data points so using a single query you can grab all the data that you need no more no less and this is one reason why Facebook champion this technology is because they develop so many different apps and all these apps wanted to pull in facebook data but they're pulling tons of data they're like okay we need we need to solve this and this was a great way to solve that problem one thing I do want to note though about the queries is that graph QL doesn't have any concept of skip or offset or any kind of like count or aggregate functions or anything like that it's purely just a language which you can represent that so in your actual implementation however you build that in the server you could be like okay well it will implement paging this way like graph protocol is implemented paging in a very certain way so it's it's a very very loosely defined I think that's one reason why graph koala's how people have a hard time understanding it the other major benefit of graph QL is because of these type definitions you can see what your server is offering it allows you to introspect the API without needing any kind of documentation because the servers will serve up this type def so that clients know exactly what they can query so using a graph Grille server you can just point an off-the-shelf tools such as graphical at it and you'll be able to explore your API we could actually take a quick dip out to the graft protocol because you can see that here once you have a subgraph it's a little grainy there and let's let's boost it a little bit but you can see your schema over here so if you go to our top-level query we see it's this entity pool prize and you can click on it and then see all its different values so it's super generic it's really really practical really handy that way and so generally what does a graft rollout look like given our abstract as languages how do we actually integrate it so like I said that you can use Postgres reddit Redis whatever your back-end database is you can just build a graft well layer on top of that typically an HTTP endpoint that you just send queries and then your client can create the endpoint just using raw strings just queries disassembled as strings super simple but typically clients will want to use a framework such as Apollo client or relay because they have really elaborate caching mechanisms so they only request the data when it's fresh when it's new it needs a little bit of help to push it along that way so that's why I versioning and other things are built on top but out of the box they're really powerful and they really can help speed your application up it's important to note as well that a lot of these queries they're being fired off from the client to the server but you can also fire off queries to client side resolvers client side query resolvers and this is a really cool feature that we've used and abused and cool together because it means that we can interface with a theorem node really cleanly so this is a powerful powerful feature it's not standardized though it's it's different in an Apollo client and relay but we'll dig into that so how does pool together use graphic UL well our architecture is very simple and it kind of originates out of some of our first clients when we're back when we read F sharp in that they were very ideologically opposed to having a server or any kind of centralized to anything even having a static front end on natla files like okay we're not gonna use IP FS let's just let's do this so really all we have is static web app we've got our smart contracts and we've got our craft protocol subgraphs and finally in furor because running a node is challenging and they've nailed it um and so like I mentioned before with these client-side resolvers we use Apollo client in two ways we have our client-side resolvers talking directly to the etherium nodes and then we also have like normal graph to all queries that get fired off to the sub graph so the client-side resolvers are kind of special as well because they're customed to apollo client we decided to extend Apollo client with our own framework interesting to note that last year at Eve Denver there's a project called eath palo i like to say 'yes poyo because there's a big chicken so it's clearly Mexican in nature but they effectively implemented something similar to what we did except we took a little bit farther because we've now integrated multi call a smart contract by maker it's really cool because you can basically compile an array of all these different calls smart contract calls and send it to it in a single eath underscore call to the JSON RPC API and the smart contract will go and execute all of those and then return the bundle of data so you can batch these calls together so you can within your queries you can pile in say a dozen calls and they come back instantly in one shot so it's been a huge performance improvement for us and we're really excited to be able to launch with that and then of course we have other things because we're interfacing with ethers j/s you want things like the current network their account blocks the current block pass blocks past events all that kind of stuff we also support from transaction mutations but that API is is is mutating itself so we don't want to go to public with it so let's have a look at a tight beam query so this is our local client-side resolvers that are communicating directly with the etherium client we can see that as mentioned before with these sort of top-level queries previously with starships and starship here we've added our typing top-level query call which corresponds to eath call and we've renamed these with graph QL aliasing you can see the green balance total supply and die balance any any resolved field can be renamed client-side and it makes it really handy when you need to switch things up but keep the same name so you have to change your front-end code and then what defines that as a client side resolver is little add client on the very end there but you can see that this is really readable we are wanting this contract pooled I the functions balance of and then we pass it the user address so that's another important thing to note is that queries can take variables as well parameters rather so let's build a DAP I can seem there's not a lot of laptops up so we're gonna rip through this really fast I'll just go through the readme because there's a tutorial that is basically verbatim cut and paste code it's stupid simple so you can basically recreate the exact app or building just by cutting and pasting so let's hop over to github I know a lot of people don't like the style of github but I've gotten used to it so I don't mind looking at it so bear with me here so first things first we want to set up the app and so this is this we use it next because next j/s is dead simple to use comes a lot of configuration out of the box we don't want to mess with webpack configs we've done enough of that so this has a lot of great out-of-the-box features including server-side rendering so you can actually render part of your application server side which then gets loaded and then the rendering is completed to client client side so that's all pretty straightforward we include ethers Apollo client everything we need now this is the the most complicated piece I'd say so this is where we initialize Apollo client and Apollo find a big part of it like I mentioned before is the caching mechanism so you want to create your cache right here and then we need to set the default cache data so this is stuff like in our case this is setting transactions there's a certain shape we expect which is just an empty array cuz that means your front-end won't choke on it and you don't have to do null null checks everywhere it's nice and simple and then next we attach the typing resolvers so these are the client-side resolvers that communicate directly with aetherium and then finally we link so this is this is a really crazy abstraction that Apollo has but it's actually very cool it works kind of like HTTP event sort of handling server-side if you guys are familiar with that it's a chain that gets passed on to the next and next and next so in our case you want it to hit our type be multi call link which will batch all the calls and then it'll get passed onto the state link which is the client resolvers so it goes one at a time and the multi call link is able to batch everything and finally we return the Apollo client here so and this is what we love about Apollo is now the front end once we create it we just pass it to an Apollo provider which sets a react context so that you could pull it in anywhere you need nice and easy so next up let's integrate the api's we do have we'vewe've now use NPM and versioning so you can pull in our api's or api's first using NPM j/s but in this case we're just gonna pull it in from ether scan nice and simple so you can just create the JSON files locally and live api's and then this is the other part of tightbeam that makes things really easy to use is you actually instantiate it with an API mapping so this Maps contract names and networks two addresses and api's so that means when we do that previous call as you saw over here when we say pool died it knows exactly what we mean it's like okay I'll just figure out what network we're on and then because you use this name it'll be this address so it makes things really convenient when you have say a front-end app deployed to Kovan and main net and you just want to switch back and forth and it just pulls in the addresses stupid simple you don't have to do any work and it'll reach rygart retrigger all the queries nice and easy so so here we you can see we add a contract there's the address there's a JSON we're going to add it to type beam as a part of the options you pass the constructor and then let's display some things so this is where things get fun because this is what we love about this framework because it separates your retrieval of data from aetherium which is a nightmare from your actual rendering of the components and in this case you can see that all we do is we define at the top of query that pulls in the different supplies you want from both pool and our pool tokens our so excuse me died in our pool two tokens and then down here we render it so we're using react hooks here so we're going use query with our query and then we once we have the data populated we just render it down here so this is step four so let's quickly jump to that and just demonstrate so I believe I'm at the right yes let's go so as part of the app repos there's branches for each step so if you want to jump ahead if you don't a mess around editing you can just check out the branch and easy peasy it's right there so we'll go yarn make sure everything is ready to go yarn dev so this will spin up the next server and then we'll spin over to localhost and give it a second so there we go nice and easy loads pretty quickly oh hey that's cool these you know these values are static but why don't we make them dynamic if we switch over to here let's just move that into the background and open up our component here so now we're looking at the actual source code so here's use query let's add a polling interval here so if we go pull interval 2000 this will spin up again and should rerender might have to save this again do we trigger it and now every two seconds that query will refresh and update these numbers so with very very little code were able to make a very responsive UI and then you could hook up events you could hook up you know graph protocol subscriptions there's all sorts of different ways that you can have it trigger and refresh these queries so jumping back to get up we set up her token supplies component that's all done and then we integrated that to our index so you can see took uh supplies being rented here nice and simple and now this is the next step so smart contracts are great in that it's always available super robust highly distributed yadda yadda yadda but they have no reporting mechanisms built in so if you want to do interesting data visualizations or have some kind of interesting time series info like I want to go back to block 600 and see what our deposits were technically that's possible with etherium but in fewer shuts that off so that's a very highly paid feature so if you want to introspect that kind of stuff using graph protocol makes it super simple because you literally can just pass the block number as part of the queries so and and I'm actually been really impressed with how much graph protocols improved over the last year previously you have to do this whole docker setup that was very challenging but now it's incredibly streamlines and we don't even need to run it locally because it works so fast on their hosted servers so and not only that but initializing a project you can just pass it an address it will pull in the API from you Thirsk and set everything up for you so all you need to do is fill in the blanks so here we initialize the project graphing it very simple let's have a quick look at that code and what it sets up is a basic project and it creates a mapping file so based on the API that you provide it it sets up all these handlers for every single event that's fired by that smart contract so it's very easy for you to just have this template to work within it also has the subgraph Yama file so this defines the address that it's actually pulling from as well as the start block and I highly recommend if you create sub wraps to add the start block because that'll speed things up tremendously in the front end it won't have to process like years of blocks from way back when so jumping back so here we set up the correct address because I actually up here this is the implementation down here is the proxy so it's pulled together smart contractor upgradeable so there's a separation there because if you pointed it the initialization at the proxy it would have created basically the Zeppelin upgrade API code which is not what we want so now we define our schema and it's the fun part what questions do we have for our application what do we want to find out so for pulled together let's find out what the churn is you want to know how many people on board and how many people drop off per prize pretty simple pretty simple analytics to look at so first thing we do if you recall the type definitions from earlier is we're gonna define a pool prize type so of course it has an ID they all need IDs we're gonna sell it the dry ID there's a bit of a confusing nomenclature here because we say draw but it also means prize but just don't worry about it and so for each prize we want to track the number of deposits and the total deposits likewise for withdrawals so very simple stats that'll tell us the inputs and outputs per prize easy peasy so we'll call yarn cogent now and that will the graph protocol project that we set up we'll already know how to populate that so let's actually check out that step 6 I don't think that's right I need to refresh this page step 2 there we go let's go back to here was checkout step two we'll go yarn coach N and the graph will do its thing so here it's generating your actual like the entities the typescript representation of the entities as well as I guess here building it into web assemblies so it's kind of it's it's doing a lot I believe me that's the yarn build step that I can remember so now that we have our schema defined that's that's our end result that's what we want now we need to process the events and this is really cool so this is a process I don't see it mentioned anywhere but it's called event sourcing where you actually have a static immutable history of events that you want to replay into whatever data shape that you want and the cool thing about it is that you can just scrap that shape of data and then just replay it and regenerate a new what they call a projection like what that the shape of that data looks like and so I wanted to play with this for a long time so grab protocols they make it's stupid simple to do and so what we want to do here is we don't want to start processing the events because the events in our theorem history are these these immutable little logs that we want to basically replay and kind of rehydrate some new some data structure in this case all the cool prize they'll be defined so I do a lot of description here about how pool together works so that's if you read the tutorial we can skip over that because people still don't understand it how we have this open and committed phase to prevent people from gaming the system but anyway that's that's further reading if you like so let's handle the open event so when a new prize is opened for deposits the open event is fired so what we're going to do is we're going to create a brand new entity or pool prize entity it's ID is going to be our draw ID that is included with the event and we're just going to default these different parameters nice and simple and then we'll save it and behind the scenes graph protocols doing all this fancy business with Postgres so we don't have to worry about that exactly how that works and now let's handle the deposit event so just to eliminate this a little bit so this is the source mapping file that was generated along with the project you can see here all the empty handlers we're just filling some of these in that's all we're doing you can leave a bunch of them empty you can fill only one of them in you can do whatever the heck you want easy peasy so now handle deposit is done and we have our deposited information the count and the amount so we'll do the withdrawal as well this is basically cut paste I think this is very simple very straightforward the same thing and we're done so let's check out step three so we're all caught up and we'll just use your own coaching again just in case and your own bill just in case make sure everything's tickety-boo yep we're good to go so now we deploy and this is so great it's so simple so I'm actually not going to navigate to the Explorer and this is for unit you need because it displays our API keys so I can't I can't actually show that I mean it's a bit of an operational security problem there so maybe if that could be hidden by default then I could display it so something for you guys but yeah so I've also added this little helper actually this might be another nice thing to add just to off very quickly against the graph API so what you do is you go yarn off and then your access token nice and easy so I've already done that though so none of you can see my access token not that anyone really taken I'm sure and then deploy we go yarn deploy so I believe this is already set up so it's going aren't deploying let's check out the latest state [Music] let's just go stupid Chuck yo that's last one so what this is gonna do it's going to build the web assembly and then upload all of the assets to the graph so this is their hosted version of the system and if we jump back to the Explorer let's see we can see now that there's a pending version and it was really fast because I guess we're only processing to event so it's just like very quick it's important to note as well that I actually skipped this step you need to create this through their website you can't create it through the command line you've got to bootstrap your subgraph through their interface and then you can deploy to it so that's something important to know so let's jump back to our app here so great so it's all loaded up we've done it now we can actually look at it through our Explorer here so they have a really great little Auto generation system here so we actually have a default query and now if we hit play it's kind of challenging to read it's very small let's hide the schema first of all maybe that helps a bit anyway let's hit play and we could see here all the data that's pulling in and we can request exactly what we want cuz it's graph QL is to skip that we'll just get the draw ID you know easy peasy so and this again is the benefits of graphic you all you pull in you're just sipping data only exactly what you need after we kind of went public with the dye pool our in Fuhrer usage exploded we went up to 6 million a day it was crazy because of the way the front-end was built so the multi call really helped with that it batched all the calls down into a single call which is really wonderful you've also started polling less because there's still a lot of polling we use ethers they don't yet have a WebSockets implementations we're kind of waiting on that but for the time being this the graph has really improved so we're kind of transitioning more and more data to the graph although we still want to have a few things straight at from a theorem node because then your reliability is much higher your service uptime is much better we have seen a few drops here and there so now let's integrate this in the front-end and this is where Apollo client shines I'll just jump back to our code here so integration now because we define our client-side resolver switches are effectively are abusing the Apollo client plants add resolver system now we can just see use Apollo client as it was meant to be used which is communicating with a regular graph QL server so back to our create Apollo client library there sorry a file all we do now like I mentioned before how the queries get passed from link to link to link to get processed and how they started with our multi call then they went to the state length of the client-side resolvers but now if a query didn't have the @ client attached to it it would need to be fulfilled by something else and so now it's gonna be passed on to the HTTP link which is now communicating with the graph protocol subgraph there and this is a dirty hack right here because next jace renders in both the server and the client and so it can be or it can really bite you in the ass sometimes so this that little Clause helps help prevent it from choking so now that's set up going back to our components let's just add a new component and we'll just pull in the prizes query here so now we're gonna pull in all approve pool prizes there's the ID deposit count deposit amount super simple going back down here we've just got another react hook and here we render it in a table I know I still use tables not supposed to so let's let's see what that looks like so that's step 5 so if you go back to our front-end we'll just kill that do you see a step 5 and then we'll go dev kick it back up and [Music] shellfish why that doesn't make any sense okay well let's let's see here I might have configured something wrong [Music] always the demos oh there we go I don't know what happened there so now we can see it loaded super fast we have all the information here that we want I actually had some surprising numbers here I realized that we had significant churn in some cases but it's probably just the whales entering and leaving they create a big splash so you gotta watch out for that in addition this doesn't include the sigh migrations to die so we're missing a few things here so as you can see it's really simple to make this data live using a polling interval we could even do the same thing for this but grab ql support subscriptions so you can get the WebSockets pushed for new data so it's very convenient that way I haven't yet played with that we said we have a lot of legacy code as well so that's the beauty of our client-side resolvers as well as graft protocol in tandem creating a really awesome front-end experience without having any server-side code so we're really stoked on this the structure and we think it's gonna last for a long time so the last thing I want to talk about because this is really important to talk about is VIP 1767 so back in I think it was January 2019 Nick Johnson completed the graphics well endpoint for death which is a big achievement it's super complicated and it allows you to query all this nested transaction data which if you're using the traditional JSON RPC API you would have to do so many calls to pull in all the little different bits and pieces that you needed so now with a single graphic you all call you can pull in all the transaction the logs and everything associated with it and it's very efficient just sipping data over the wire very nice at the same time Raul Kripalani and Christian from in Fira we're developing eath QL their own graph QL implementation and so they fantastically combined their efforts to create a IP 1767 which apparently was mostly east cue ELLs I guess they aligned it to that but it looks like the adoption hasn't proceeded quite as quickly as we might as hope as we might have hoped because we have foals client support and get than Pantheon in Fuhrer has stated that they have the sort of sort of standard eql then they have the extended eql which offers all of their additional bits and pieces on top but there's no public endpoint that you can communicate with so it's it might it might be just dormant right now the project I'm not sure what's that parity has partial implementation and so we're seeing some functionality across the different clients that being said we interface with ganache so much that having it in ganache should be really essential for local testing so we're kind of and that's why we have our client side resolvers because we're getting the batching of all the calls we're getting that sort of sort of advantage but we are able to work across ganache in any other client that we need to do so it it's a little crude and it's a little abusive to graph QL but it works really well and it's super fast that being said this will eventually happen I'm sure for us though as a DAP it's a little less relevant it's more for say you know blockchain introspection like if you wanted to build an efficient ether scan you would definitely want to use the graphical endpoint for all the you know block data transactional data all that stuff but we're communicating with smart contracts so 90% of our calls are just eighth call very simple so I want to open up the floor now to anybody that has questions about pull together or smart contracts or the our history or graph QL or anything really actually is you need from the COO of the graph here yeah I mean got unique as well well we're will be here all weekend right oh yeah oh yeah so yeah it's a great team questions because yes okay maybe save your crap protocol questions for you need then but does anyone have any pull together questions anything like that yeah all right and that concludes the talk you're actually out of time so we gotta get okay perfect well thanks everybody appreciate it [Music] you