"State Channels at Scale" - Andrew Stewart & Jannis Pohlmann

and just follow up with the audience there will appreciate that um and with that we are ready to uh move on to our next talk so uh the next talk we're gonna be doing is going to be by andrew and janice uh andrews for consensus and jonas is from the graph and so they'll be talking about how do we actually manage state channels at scale uh this is something i'm really excited about um and i want to hear their thoughts on how they actually pull this off for the graph and just how do we think about this um as a large infrastructure problem so i see both andrew and the answer here and i'll let them kick off with their slideshare thank you so much and welcome thank you i think giannis is going to be sharing his slides all right um yeah thanks everybody for uh attending my first webinar um i'm gonna be talking about the recent goal of the state channel's team to build a highly scalable state channel's wallet um as uh kartik said um i'm andrew stewart i work on the magmo team at consensus and we're part of the state channels community the sd channel's organization which is a community of researchers and developers implementing open source state channel solutions um i actually have the pleasure of uh giving my first joint presentation um with uh giannis who's the the tech lead and the co-founder of the graph um we actually um started a collaboration with the graph a couple of months ago to introdu integrate nitro state channels into their query payment model and so giannis is here with me as well yeah welcome i'm glad to have you all here um so the structure of our talk will be as follows we'll talk about the graph and its use case for state channels especially going forward looking forward to the graph network then we'll talk about state channels by introducing the general concept but also how it applies to the graph specifically and then we're going to talk about not really jet packs but we're going to talk about the improvements um we made to push boundaries of state channels forward in the past couple of months and yeah so let's start with the graph um the graph for those of you who don't know what it is it's an indexing protocol for organizing and efficiently accessing data from blockchains such as ethereum and storage networks such as ipfs um we back in january 2019 we had graph day and a one day conference where we launched the hosted service that's a currently centralized solution to basically do everything that we want the network to do um just in a simpler way so that everybody could start using it right out of the gate and didn't have to wait another two years or so for the network to develop um that's been in uh yeah running since january 202019 and has since been picked up by a lot of uh well-known projects um you can see some of them here uni swap synthetics ave dios diagnosis aragon ens and many many more um are using uh the graph in production today and you can also see that when you look at the the daily query volume that we're processing so right now we're at about 300 million queries served a day um for all of the the projects are using us and that comes down to about three thousand to four thousand queries per second served um we recently launched our test net or incentivized test net called mission control um as we're moving closer towards the graph network launch now the graph network will replace this centralized hosted service where everything is run by a single company that you have to trust to you know stay around and keep everything running and also that you have to trust with your data replace that with a decentralized service and a decentralized network and a query market around that um so the way this this looks then is that instead of hitting a centralized servers the consumer will pick from a variety of indexes that are running around the world based on certain criteria and will send these indexes queries for the data and their incentives baked into the protocol to incentivize incentivize indexes to um first of all index these data sets that we call subgraphs keep them available and serve queries for them and get paid for these queries and then um there are a bunch of other incentives around that to keep the network healthy such as um curating uh sub graphs or uh delegating to indexes uh doing good work um and so on and so on um if we use the the hosted service as oh yeah one thing i should say so initially you want to start not with the consumers directly interacting with the indexers and paying them directly um rather we'll start with um so-called gateways that's it in the middle and that the app send queries to like um unlike the hosted service today or like any other um api service where they don't have to query uh pay for the queries and then these gateways will select indexers based on certain criteria that can be customized and will handle the payments and also the validation of query responses on behalf of these dapps and end users um so when you see consumer here you can also for now think of them as gateways um if we use the hosted service as a reference then we'll see a decentralized network that processes about 4 000 queries and payments per second um looking at for reference visa and they process about 16 1700 payments per second they can peak at 65 000 but the average is about that so we're looking at double the that average um at the ho for the ocean service right now and if we can translate that to the network and that would be a decentralized system processing that um now you just heard 4 000 paid queries per second on ethereum uh with its you know block times of 15 seconds and 10 transactions per second throughput um how does that work and for that i'll pass it over to andrew yeah the state channel's team would naturally ask at this case can we use state channels to pay for queries um and to start answering that question i'll first ask uh what are state channels well state channels are a layer two scaling technique for ethereum that enables trustless instant and zero-free transactions amongst a fixed step fixed group of users they're not exactly instant because a pair and a recipient are rarely in the same location and in addition to be trustless they do incur a small computational overhead to verify payments as we'll soon see but in any case for graph queries um state channels are a perfect application because uh once a gateway selects an appropriate indexer you now have a fixed set of peers who want to exchange some asset such as ether or some token for some information such as the query response um to for those who aren't uh familiar with nitro state channels i'm going to walk through the life cycle of a toy two-party state channel that's funded directly with two on-chain deposit transactions so for starters if we had um say alice and bob here with five tokens each and they wish to trade tokens between them they can use an on-chain adjudicator to mediate the off-chain token transfer using a nitro state channel they do so by exchanging signed states which the adjudicator will use to release funds based on based on states supported by all the parties in the channel so to start off they both sign what's called the pre-fund state here in this case the initial outcome of the channel allocates five tokens to alice and five tokens to bob once alice has the prefund state signed by bob it's safe for her to deposit into the adjudicator contract even if bob refuses to deposit the nitro protocol ensures that alice can recover her tokens in bounded time even in the presence of front-running attacks um after bob sees alice's deposit it's safe for him to deposit and the channel can there from there on be used to make payments by simply updating the outcome in the channel for instance if alice wants to pay one token to bob she can sign a state update where the tokens are allocated slightly differently with 4 to alice and 6 to bob in this case she's given one token to bob now nitro state channels are turn based meaning that if bob doesn't want to make a payment he needs to pass the buck back to alice in order for alice to be able to make the next payment i'll note that there are applications of nitro state channels that can avoid this but this assumption this allows wallets to make simplifying assumptions that enhances the security model now suppose after making a few payments there they decide to finish with the channel and settle the bill what they do is off chain they cosign a final state indicated here by the asterisk with the final outcome and they give it to the adjudicator in the on-chain transaction when the adjudicator is given a final state signed by all the parties the channel is instantly concluded on chain in other words if the peers in a channel collaborate then they can recover the funds without without having to wait a timeout period once the channel is concluded the adjudicator distributes the tokens according to the the last outcome that it held in this case alice would get eight and bob would get two um now this is just a toy uh state channel with a simple outcome um in general you don't want uh participants to be able to update the outcome arbitrarily and for the graph in particular you might want to make payments uh conditional on certain certain features yeah so this example translates pretty well to the graph network as well um one thing that our payments um where our payments are a little bit different from just accepting a new balance is that they are conditional on the indexes serving a query response that matches the query and that's correct for this query so what in our case consumers or gateways will do is they will send a query along with the conditional micropayment that can be unlocked with an attestation for the query response um so that is for the indexer to say i've successfully executed this query against the the correct data and i'm serving the correct data back um yeah and based on that that's basically the rules of the the state transitions and how payments can be made off chain between the consumer or gateway and exam um yeah this there's one um thing to note here these adaptations if they are incorrect for instance they can also be disputed later on chain so these transactions don't have to be final if it turns out that the data served was not correct [Music] um the graph network will most likely see us entering new dimensions in terms of scale um as more adapts adopt the graph as the the technology for querying um the underlying blockchain data um we will likely see thousands of them um we already see or have um i think between one and two maybe three thousand uh subgraphs that were created over time on the hosted service um so there's plenty of of interest there and those thousands of apps will likely power millions of users um and to serve that traffic we'll have dozens of gateways will have hundreds if not thousands of indexes that either index specific data sets or a variety of data sets and so there's there's a lot of um a lot of scale that's um that requires or that yeah requires the whole system to be very responsive and to have for both queries and payments to have um high throughput low latency to be reliable um that queries don't fail and payments don't fail to be trustless as well because you don't want to trust on the gateway that you don't know uh and uh or an indexer that you don't know that you don't even interact with if you interact with with it through a gateway um these gateways if we um think of them as deployed around the globe to to reduce latency they are largely independent when it comes to the state channels um so they will all have their own set of state channels that they manage with indexes they will have potentially their own balances and they may be run by um you know by completely different people with different keys um so for like looking at the scale of state channels um this like overall network is not extremely interesting what's more interesting is to look at a specific pair of a consumer or gateway and an indexer and to see how many paid queries we can execute between these two uh what boil next we hit and how we can remove those so to discuss the issue of the throughput through a single gateway index or pair i'm going to first look at the critical path required to make a payment um what what does a gateway indexer actually have to do to process a payment so the first thing is [Music] the gateway needs to construct a payment state s1 it then needs to sign it store it in its database send it to the indexer and then wait for a receipt state s2 from the indexer meanwhile the indexer is waiting for the payment state s1 from the gateway once it gets it it's going to push the payment state into the into its database it's going to then construct a receipt state s2 sign it store it in its database and send it back to the gateway once the gateway receives s2 it then pushes it into the database and the payment is is then processed so starting out we could initially hit on the order of one payment per second given these steps it was a bit more than one but on it was that order of magnitude and um why was it so slow well um each of these steps from uh one to six on the left and on the right they're blocking steps if you only have a single channel the gate we can't do anything to process another another query until it receives a receipt from the indexer um likewise if you're interacting with your database you can't your application can't do anything until it gets a response back from your database so to increase throughput we definitely need to uh use more channels um so using creating multiple uh creating parallel payment channels is where the nitro protocol really shines um where we can fund a single ledger channel with a single deposit on chain and use it to fund many payment channels via ledger state updates off gene um by using many of these parallel payment channels we could initially hit about 10 payments per second which is still slow so why is it slow well no js first of all node.js is a multi-threaded language runtime but it only provides a single dedicated thread for running both the event loop and your application code so you can't use more than 100 of a single core for this thread if you ever hit a hundred percent you're gonna have a bottleneck so for instance signing state updates need to be as quick as possible and initially they were slow um janis came in and helped us with that so one thing that's or the first step of constructing a state for a new payment that you can then send over to the indexer is that you have to encode the application data and that is very similar to how you encode a lot of data in ethereum you use an api encoder and you pass that data in and if you write a javascript or typescript application or library you're likely to use ethers or web3.js in this case we used initially used the ethers api encoder and it turned out that that was fairly slow because i think it's implemented in javascript luckily the the application data that kind of represents our um our state transitions um so like the the conditional micropayment or the the attestation they are fairly simple i have a few fields like the payment amount um a request identifier response identifier a signature by the indexer so in this case was very easy to just write like an almost by hand custom encoding routine that does this in a much faster way than a generic encoder would um and i think that boosted the throughput about 25 percent um after you've encoded your application state you will put it into the new state or the application data you put it into the new state and before you can send that over you have to encode that again um and hash it and sign it and these three different steps we're all using slow javascript based code so again the ether api encoder to encode the state um a javascript category 256 hash and a javascript method to sign the the outcome of that um and one thing that's really nice about javascript is that you can very easily extend it with um well initially you know if you run node.js you can initially extend it with maybe native code but today we also have a webassembly which we can run equally in the browser as well as node.js so one thing we did here was that we replaced the encoding of the state the hashing of the state and the signing of the state in which were initially three different steps in a single call to a webassembly module that was written in rust and compiled to webassembly using wise and bind-gen and that ended up being a javascript package that was that's you know as convenient as any other javascript library to import and to use um and what you can see here is yeah you don't have to like encode the state up front um also not doing several round trips to webassembly doing a single round trip to web assembly to perform all these operations um there uh what we use for this code is uh in rust is ethi a library written by um initially parity now open ethereum team um which we're also using pretty heavily at the graph and it's kind of like a partial equivalent to ethos.

js in the ros world this gives us another boost of about 20 in throughput so when it comes to storing states safely we're using a postgres backend but we chose to use objection a javascript query builder library to interact with the database but the result is that um if you write javascript code like this you're constructing the same query at runtime over and over again in this snippet you're running you want to construct a join statement between the user table and the emails table and it actually takes a non-trivial amount of time to construct and dispatch these fairly simple sql queries which is just wasting our precious main thread cpu cycles and it's not actually that bad to just write the sql query that you want by hand as a prepared sql statement stored in your source code that means that all you need to do is fill in your parameters and send it off to sql and if we switch to a library like a pg promise that embraces this technique we can actually seemingly boost the the throughput by uh something like 400 um in a single threaded application so with all these optimizations to get to hit um with uh with a few like a a couple a hundred payment channels running in parallel for instance we can hit um on the order of 100 payments per second in a single threaded javascript application um that's clearly still not enough these days you can get a powerful multi-core server and we're only using one of those cores so if we want to increase throughput more the natural thing is to hire more workers we can just spawn worker threads in a node.js application and offload the as much of the critical code as we can into a worker thread and then as long as we have enough cpu cycles to schedule more work and as long as our database can handle more database connections we can scale throughput linearly by just paying for more cpu cores um with this strategy we can then hit the next order of magnitude around a thousand payments per second but the main thread bottleneck will always remain um if you only have one main thread that's running on one core um that has to do a certain amount of work you just can't scale any more than that um and the solution here is again obvious we should hire more managers and by managers in this case andrew means more gateways or more not threads but processes that can manage more threads um so one thing you can do is horizontally scale the gateways and have them all use their own database connections and their own worker threads to do basically the same thing put them behind a load balancer and you're good right there are a few complications here when it comes to state channels um due to their sequential nature um you can't use the same channel in two different gateways at the same time so um these need to be synchronized a little bit otherwise um you know one gateway or two gateways might try to use the same channel at about the same time real life one real light one one does it the other realizes that channel is now busy um decides to retry and um pick a different um channel but maybe one of the other gateways has already picked that and so there's kind of a race to find a suitable channel um which needs to be optimized a little um so a very naive way to do this would be to just have them use completely disjoint channels um like a gateway would do that is or like two gateways would do that uh deploy to completely different regions uh on the globe um you know both managing their own channels having their own keys and yeah managing the channels with the indexers completely separately um that has the the downside of having to create a lot of channels in the first place to have to fund these channels um it's a lot of overhead uh just to get this extra scale and since they're all able to connect to the same database all that's really needed is kind of a lookup table that allows all of the gateways to efficiently and atomically pick a channel in the database that's not used yet and at the same time mark it as used so that nobody else will use it and then there's very little overhead in picking a channel um yeah it's it makes sense to kind of move the channel management out of the critical path as well so that doesn't that that doesn't um block any any cycles in the main thread of any of the gateways um so uh one approach that we've taken is to move the channel creation logic and all of that into a separate um service that runs alongside these gateways and um yeah manages just the channels and then the gateways just execute against the channels that they find in the database and so if we run more applications going from starting at a thousand requests payments per second where does it end well we'll see that's still um one of the open open points here we'll we've yet to see where that takes us but we're very optimistic that we can um take it to the numbers that we need to sustain the usage that the hosted service has seen already and carried over to the network so um to try and wrap things up i'll quickly summarize that we're making significant uh progress um at increasing the throughput of our state channel wallet um by optimizing the critical path making a multi-threader wallet and building tools to synchronize the sharing of a pool of payment channels um i'll i'll just add that the most of the work outlined above so far is open source um there is some part that channel management software that janus was talking about it's still loosely coupled to graph specific code but we plan on open sourcing that in the near future um and in conclusion the the um the graph has provided the graph is providing a highly scalable and robust network for indexing and querying decentralized data and our collaboration with the graph has been a really welcome boost to prove that state channels can work well at scale with the upcom with the upcoming graph network mainland launch expected uh later this year state channels are going to be one of the few highly scalable decentralized payment solutions using production um so we're really excited to continue with this collaboration um yeah to to just wrap it up um i'd like to thank the uh the ethereum foundation for their past and their ongoing funding of the state channels project um as well as eve global for organizing this session and uh everybody from the graph for welcome the state channels team welcoming the state channels team into their project um yeah thanks everybody for your attention and uh enjoy the remainder of your talks andrew janis thank you so much for uh this amazing talk and also a very beautiful presentation we have a few questions coming in from our audience and uh i have a list of envelope questions that uh that will ask you we have a couple minutes here and uh luckily uh mike uh kersner from the state channel's team has been answering a lot of them already so there may be some repetition here in the answers but i'll kind of ask the smaller ones for you i guess my first question here is uh just as a just for the audience to better understand what's possible here how programmable are state channels and kind of uh are they only supporting just transfers in any conditional logic or do you actually get more complexity on programmability here if you use a channels um the so the nitro protocol focused on simplicity um with the goal of um targeting the majority of applications that we could imagine using without adding too much complexity into the protocol so therefore we're currently limited to state channels where the state transition logic is a pure function of the existing state so you can write arbitrary code that determines what transitions are valid in solidity but the code must be a pure function of the current state and that's the only limitation of the nitro protocol right now there is some research into escaping that constraint but that's without a target use case um it's not uh considered right now so maybe just to follow up on on your answer then does that like i guess maybe it'd be great if you can also answer what um because what there's some live witness assumptions you have to make for nitro um and kind of how does that change yeah so um we actually have a pretty good blog post or a blog series about this um so there we were able to formally verify at least a portion of not formally verify and we have a tla plus specification of a part of the nature protocol itself um that can prove that as long as you have uh the ability to submit a transaction um then you will not you lose your funds um like if you can submit a transaction within a block time um like maybe within a one minute time or something like that um you're guaranteed not to lose your funds uh in spite of an uh an attacker who's willing to spend an arbitrary amount of money front running your transactions awesome um a question for uh for janice so as as a developer and cto craft um how has the developer experience been for you to sort of integrate their channels into the wallets api um and it's just mostly about like kind of how do you think about these libraries just being immediately ready to be imported into uh your project or just kind of where you're working on as a developer and just kind of how easy they are to use um so would love to kind of get some insights into how using savetown's production has been for you yeah um i mean they're not used in production just yet um but they will be very soon and exactly um so the collaboration started out um i think with uh the sections team adding a server wallet into uh their repertoire um because our gateways that we're developing our indexers are both running node.js processes so um that was badly needed before that i think the wallet was primarily um uh targeting targeting the browser um so that was that was necessary first after that the integration was um pretty smooth um the the interfaces that we now use um to the libraries that i'm maintained by the state channels team to kind of manage the the capacity of channels that we need uh to you know handle the throughput in terms of queries um are now automatically managed on by by the state channels team libraries and that's the kind of the code base that andrew mentioned is loosely coupled to the graph so we have these identifiers for which we need to generate a certain number of or ensure a certain number of state channels um and at some point these identifiers go away and then the state channels need to be closed um and so this life cycle management i think is pretty generic um you can imagine that in a lot of other situations as well so um that should be fairly easy to you know decouple from the graph and and reuse in other places as well um yeah the the cooperation has been been really fruitful and um the the end result is really good well i wouldn't say end result yet because we're still working on those horizontal scaling improvements um but yeah overall um it's been a really good experience and um yeah from the first moments that we talked to the stage films team about like their design and like their approach like how they wanted to move everything that's not that's potentially slow out of the critical path and and so on everything um yeah made sense pretty much from the start and that was that was really nice well it's uh that's a wonderful testimonial for uh all the work the state channel team has put in the last two years and uh maybe i'll just kind of close off with another question which is a lot of hackers here for the hackathon this month that are planning to use scalability and integrate that in any other solutions in their projects so kind of having used state channels for for the graph do you think uh the sdk and the state channels project is at a place where almost everybody can just import that and then get a very high thousands of pps built into their projects even if it's for a hackathon i think that's more a question for andrew to answer i think what you will need for that kind of scale would be to um to have these libraries more generalized i mean you can manage your your channels manually you know you can scale them up uh you yourself that's that's not too hard um but um yeah uh those those libraries that manage the channels on both both ends um that will be pretty crucial for everyone to to reach that kind of scale i think that's right yeah um yeah unfortunately it it um hasn't been able to be a priority to get this stuff out to be usable to the community um we're sort of running at a pretty bright pace trying to meet the scale that uh that the graph network is going to run at and it's our uh priority once we do that to um release these uh tools to the developer community so i i regret saying it but i'm not sure that it would be ready for um a hackathon that's um ongoing this month no worries i think this is a just a milestone for all of us who can achieve and uh especially as it gets integrated into the graph at the scale that they're operating at i think this is just a good recipe for others to take from so um thanks again andrew and yanis for the music presentation and answering the questions mr hand