E. Galano, Kris Shinn ~ Infura & EthQL: Improving the dApp Building Experience

[Music] all right hi everybody I'm eg I work on inferior a at consensus and today we want to talk to you a bit about graph QL ap is for aetherium so inferior runs nodes and then aetherium API as a service for developers to quickly get people started building applications and we've been around since about 2016 we were one of the first teams working at consensus trying to build tools to help bootstrap the developer ecosystem and so we really want to make sure that developers are getting the right experience right we want to make sure that the tools that they have are the best to build applications and we really felt like focusing on the infrastructure so that developers couldn't focus on whatever need protocol they were trying to create was valuable so talking a little bit about the evolution of etherium tooling we started out with one aetherium client and now we have a choice we can we can we started with the C++ client then we had death and parody now there's Pantheon etherium there's there's options now we also started with just EVM assembly and then we went from that to an improvement which was solidity and then now we're looking at sorry serpent and then we went to solidity and Viper and these other next-generation languages and smart contract languages for development we started with no frameworks and then we went and had truffle and then there many other versions that people can use now there's embark there's a parasol and then when we talked about the JSON RPC this is how people are building the front-end for these applications and and we haven't really iterated on it if you look at the JSON RPC right now versus the JSON RPC we had a year or two years and three years ago it's the same pretty much with only a couple of additional methods and that that's what we want to talk about is we we need to continue to improve on the tooling and let's start by taking a close look at what this query layer is for aetherium data and if you've been looking at you know outside of the crypto space in what is really interesting for people building applications graph QL has been a pretty popular API standard the last couple of years and is continuing to grow in terms of adoption so let's look at trying to build a graph QL API for aetherium and that standard that we've been developing over the last year in partnership with folks from the Pegasus team which works on the Pantheon client and other people in the ecosystem have been calling this effort fql be etherium standard for a graph QL API so when we try to pull data off of the blockchain it's difficult it's a when you think about when you're building an application right now you're querying raw information you're saying give me this block of information and then from that employing transaction information and then from those transactions I'm pulling events from a smart contract and that's that's a pretty low-level way of trying to build an application it'd be great if you can deal with it in more friendly terms like I want to get all balances back for er C 20 tokens for this address or I want to get all birthing events for you know crypto kitties within this time range right and if you want to build that type of functionality right now you don't you don't really have that abstraction layer and having a better API such as a graph QL pattern power the API can start to get you towards a better better developer experience which will help you build a more efficient application so that's what we're gonna talk about is let's get towards this next generation API so this is Christian also from our team he's been taking over a lot of the work of leading our ETL development after it and he's gonna walk you through the rest of the workshop thanks thanks Yugi so walking towards a next-gen API as Yugi was saying we base it on graph QL right so for those of you that aren't as familiar with graph QL it's what we're calling a schema fold data selection API and it basically views all data that you're querying as a graph it's about the data entities and the relationship between those entities encoded as a query language it's been used since it's been in use inside Facebook's since 2012 open source in 2015 and it had an MIT license applied into 2017 what's nice about this is it flips the model right with every API implementation it's the server that dictates what you're getting the server says this is the data that's available and it gives you all of the data back well with graph QL it it gives it now on the clients control or the application control right the application says I'm only interested in such-and-such a data I only want these three pieces of data back and the server will only transfer back those those pieces of data so it's a single self-describing endpoint unlike rest where you have to navigate through a series of resources it also contains a self description that rest you have to layer on you know swagger or json-rpc or another layer to make to give it that schema structure the ocean but with with graph QL it's it's built right in right so you can build a lot of tools on top of it because you can query if what the what is available for you to query and it solves a lot of challenges that you find with the rest having to navigate different resources relationships between resources hey do Spurgeon engaging api's paging extensibility like all of these things are built within the graph QL specification so it provides a really nice developer experience on both in designing the API and in consuming the API and by the way it provides valid query validation out-of-the-box so if you're building a server from it you don't have to worry about checking all of the arguments the specification itself since it's strongly typed will take care of all of that validation for you a little bit of what we mean by schema fool it basically means that all of the data relationships are encapsulated within the data types themselves so if you see on the right hand side we have a log type right an aetherium log and within that you have an account reference and a transaction reference coming back from the API itself it's usually just a hash or a reference in graph QL we actually unroll those for you so you can just select that nest it inside your query and I'll show you what I mean in a few minutes so presenting what we're calling each QL write graph QL plus aetherium it is east cue Oh what do we want to use each QL it presents a 30 M data as a navigable graph it integrates with powerful front-end libraries like Apollo clients so you get a lot of sort of batteries included functionality that you can just pull into your applications more importantly it fetches only the data you need no less or no more right the problem with JSON RPC and rest is that it suffers from data over fetching or under fetching you're either fetching way too much data that you only going to use a portion of or you have to make multiple calls in order to get the data that you need each queue all kind of puts solves all of that this means it reduces round trips to your server it gives you less data you have to parse and decode which means it's improving your user experience it's decreasing the time you have to wait for the network to return the data and it provides a better user experience for your users and it's a modern it supports a modern ecosystem that's built on top of this so like I said you get a lot of batteries including a lot of open source libraries that you could just plug into your application and get going essentially it eliminates the need for JSON RPC and a lot of the multiple colors that you need in order to resolve your data I just wanted to touch upon this right now this is a very rapidly developing space for us so there's there's two sort of flavors that we have there's a standard flavor and an extended flavor the standard version of youth QL is what we're working through to get ratified as an EIP we're trying to get this standardized and implemented directly in the nodes and there's initial implementation in Geth and I believe it's going to be on the next point release and the standard version supports mutations like sending raw transactions the extended version which is what I'm going to demonstrate today is an overlay on top of the SQL standard and it supports a lot more convenience functions things like log decoding a plug-in system for extending it with your own contracts a flexible deployment model there's a lot of convenience features that we've built into the extended version so the kind of features that we were support that I started to go into block selection on a single block multiple blocks lists of blocks navigating through a block into a transaction into accounts we have transaction filters so you can say only show me transactions that have input or that have logs has ERC 20 support so you can start to decode and unroll ecrt RC 20 events drill into storage and and it has a JSON RPC transport selection so until we get the standard ratified across all of them all of the nodes we basically do smart batching on top of the JSON RPC so it's faster less round trips and it compiles all of this data for you so getting started is pretty easy if you're running your own node like if you're developing against truffle or you have a light client on your local you can just clone the repo see the end of the repo yarn dev and so and supply your own endpoint that you want to query against if you have an inferior account or you want to sign up for URIs just as easy clone the repo CD into the directory specify your inferior ID and start the server the easiest is that we set up a playground for you guys to experiment with so you could just point your clients directly to this URL here and start playing with it and get going right away so this is kind of an interesting slide with the efficiency gains that we've seen in our testing in this in this set up we queried five blocks and we're fetching various data from the blocks right at the hash transactions the sending address and the value of the transaction over JSON RPC the sum of the entire data set to get this information was a little bit over a Meg and almost two thousand requests to the client to compile all of this information with eath QL the the bytes over the wire was a little over 200k 80% 81% reduction in the data across the wire and we did it in one call all right demo time so I'm gonna show you a little bit of how to play with each QL I'm gonna be doing this live typing so stick with me if it if I crash and burn all right so this is the SQL playground if you go to the URL that that was in the previous slide this is what you're gonna land on and now we can just start writing queries right so let's say we want to select the latest block and I just want to see what what the number is the latest block number okay great short and sweet for the rest of this demo I'm just gonna peg it to this to this particular block number great nothing changed let's say I want to select the transactions so there I just queried for all of the transactions in this block included the indexes and the hashes just for brevity as you can see all of them come back 0 1 2 3 4 all the transactions are there but say I only want the ones that have logs and it filters out all of the ones that don't have lugs right so now I'm just seeing index 1 index 5 6 7 12 all of the transactions that have lugs so now for something a little bit more functional let's say for each of these transactions I want to scan for GRC 20 tokens and just like that we see all of the transactions that happen in this block and it shows the symbol and the value traded this is a very short selection of the data that's available I can also query on account so I can see who it was sent to who was sent from but the point is is that trying to do this over JSON RPC would be much more difficult right you'd have to loop over a number of transactions and then you'd have to go and decode all of the logs so with SQL this is basically all at your fingertips alright so the roadmap this is a very active project for us so we plan to add a lot of functionality to this and we support standards we love standards and we want to support a lot of standards in the ecosystem ERC 7:21 GRC 165 ENS those are all functionalities that we want to add to decode its this is built on a modular plug-in system right so it can include your contract it conclude whatever whatever flavor that you want to support it's a flight we have flexible deployment models which I'm not going to go into right now but basically you can deploy each QL and in any number of ways we want to support graph QL subscriptions so it eliminate the need for polling we want to support the variant schema grants so various paging methods a a simpler plate paging and then the relay relay based schema paging we want to integrate with indexing backends to level cache and new filtering strategies and so this is basically driven by you guys like what do you want to see how do you want to filter your data those are the strategies that we want a place inside inside each qo so each key Willa is made for you the DAP developers so use it break it contribute to us let us know what you want let us know what's missing we're on getter this is the URL to the to the open source project for each key well say hi with a star open issues contribute and then for more information that we have we set up some documentation for the for this hackathon there's the playground URL that that I was just showing you and then finally that the repo so if your is sponsoring some prizes so there's three thousand dollars in prizes for the projects that use SQL so you know build with it and let us know thank you so much I think we have a little bit of time for questions maybe about five minutes yeah yes sir yes oh I was describing the two two flavors right standard and extended the the standard implementation that's being built in together and that we're trying to get implemented into notes will support mutations right now it's limited to send raw transaction but as we get further along and start developing on this you know you tell us what you need and and with that stuff that we would like to bake into the to this game API the mutations it is something that you can use today it will be it's not quite released in a point version of death yet the next point version will support it the one that we have set up is the extended that's right if you could build master on the etherium repo and play with it if you wanted to do that yes sir that is correct that's one of the deployment models so if you clone the repo you can basically point it at an inferior if you're running in fear adjust with the ID or you can specify your own URL like if you're running your own node or if you're developing local with truffle or something like that great thank you so much [Applause]