FunFair Technologies' Fate Channels: Lessons learned Implementing State Channels (Devcon4)

hi i'm jeremy and co-founder of funfair and we've spent the last 18 months building a platform that uses state channels which is now live on mainland and has been for about a month so realistically I'm so I'm going to talk to you today about our experiences that we've learned over the last sat eighteen months doing this so what is fun first of fun fair is a casino platform which allows a wide variety of games to be run in real time but the gameplay is backed on on the chain using using state channels as we will be talking about today so I'm gonna blast through this we have a fairly specific sort of form of state channels that we've developed for our application rather than developing something massively generalized so we have a two-party bi-directional channel and we support a fully generic state machine so we will support state machines that pretty much can do anything as long as it can be expressed in the contract call that can be executed on chang'an I'm going to talk about that in some detail we also provide an integrated orangy into this into this which is a form of a complex commit reveal scheme using hashes it's deterministic but not predictable and we feed essentially one random number into every state transition that's that's available for the state machines to use it's optimized for use for casino games unlike a lot of the examples that people talk about when they have like chess people taking them one move and then another move and in another move we advance the action the valence of state with an action from both players at the same time so it's more like Rock Paper Scissors where both people are doing this at the same time and we combine that together too with the random number to produce the next state in the in the sequence um this actually makes the rights in the state machines themselves a lot simpler as there's only really one function to advance the state but it does make dispute resolution quite a bit harder so all of our disputes which can come from various reasons timeouts network network network problems incorrect state transitions just general software problems we reduce to a single statement which is that the other party is not following the protocol of the state channel and state channels it's really important to understand our protocol they are series of by which the state is advanced those were their rules at this day it needs to be followed so if that actually happens then the the the person with the claimant makes her claim on chain using the previous sign state co-signed state that was a post parties a degree 2 and what they think the next one should be and the contract validate so that's a valid state transition and if that is correct then we raise a dispute and then the counterparty can do a number of things they can go okay that was a problem right I better do something about this and they can provide the next state transition and if that's done and validated on chain then we have a great side effect which means that we can actually carry on from that point off chain and complete the contract this is a really really powerful thing the counterparty can say no dispute as in well you know yeah probably that's my fault and can we just I can do nothing if blown up or if they've gone on holiday and the channel times out and the claim out can close too close the channel subsequently to that with a penalty to the party that timed out or they can challenge with a later cosign state now in various implementations of that there's penalties or not whether or not you use someone's maliciously tried to close a channel with an earlier state we think that is that is malicious especially because we've tweeting with casino games here so we treat that as a penalty to the person that put the earliest date on the chain because of the variance so building a customer-facing zap is hard I think most people know that using state channels does not make this any easier if you want to build an automated system that interacts directly with the blockchain it's even harder still so this is really talk about what we face while we've done that with experiences from the team quite a few of which are here today so let's start with the topic du jour so customers non boarding and so fun fair we really aimed out our initial product at least people that had some crypto somewhere right or vaguely knew what was going on most most people he says hand-waving they hope to in exchange it's relatively few have actual war it's an interchange Eric Lee but we need to take people through this so we just did the usual stuff that you have to explain to people gas price gas limit what happens if you get them wrong if you get transactions stuck that can't be mined we have to take people through this process to be able to get them in front of the stage channel we didn't really want to roll out our own what we wanted to be wallet agnostic with this anyone looks got a wallet of any safe way shape or form you can use that you know lots of people are working on ball it's so right why do why do other people are doing but relatively few of these are out of beta always properly support I would free apps and even then we are at the mercy of someone else's code for an entire platform to function at all in the month we've been live metameric have broken our app four times which is really not particularly helpful this particular the first error is the day the day we launched when you want it to make a transaction to chain it made a transaction to the chain and then reported an error saying that transactions have not been made to the chain so this does not help writing automated code also it's very unclear what customers are actually signing when they open a channel I mean this is just like some data what does that actually mean that's the raw data for part of an open channel you might get to see something like this why wouldn't I press that button I don't really know we've got some all right was that so um but we've got to encourage our users to do that so we've got work to do here also there's another entertaining thing and we managed to get into this situation trying to open a state channel using a ledger with 16 button presses going through random streams of hex digits not particularly custom friendly so I know that people are working on this a lot but where we are today is not good enough to get in front of us in front of customers so realistically we need to wait for it to get better that's the lazy approach roll our own solution or collaborate with people that are doing it and I know lots of other people are doing it we're here on a mission to find a partner to help us with this so if you've if you've got anything interesting to say please do get in touch but it's certainly our biggest short-term problem as a customer facing live application at the moment finally it's hard to explains customers what's going on and why it's fair it's hard enough with blockchain in general but to try and talk to them about state channels you don't get very far without having to explain water hash function is and why it's important and to really prove that channel is fair you need to be able to understand and read contract code need to be able to validate the signatures of each state transition and you need to be able to validate that the state transitions were correctly executed by both parties which basically no one's going to do so we need to work on that a little bit and I think there's an interesting stuff along the lines of witnesses are watchtowers and Patricks going to talk about that later today and I think that might be able to provide some sort of a help here but it's just in terms of getting customers to understand what they're doing so we put up a nice screen underneath you can see that saying yes everything was fine the signatures were valid that's not valid but what does that mean it doesn't mean a huge amount it means that we said that that's the case if you want to do it yourself you've got to go and look at this lovely piece of JSON and try and submit it to the shame which also isn't work so we need to try and bring those two things together so that we can really get our customers to understand and believe that state channels are actually secure and work which they do right quick one liveness so state channels require participants to be live as in you have to always be there to respond to the messages to advance the state if you go away the channel can't advance and it's fairly fairly obvious in itself but if you go away how long do you go away for before you do something about it if you just turn your computer off and walk away you you have violated the protocol of a state channel so the longer the channel is expected to laugh the longer your timeouts need to be I think it's probably self-explanatory but clients need to be short or long there's a kind of period in the middle where it makes no sense for the timeouts to be so what situations are you trying to cover if the users just closed his laptop and walked away well you know then you can probably close the channel fairly quickly but if it's important enough and there's enough value in that channel that if they broken the laptop or they've gone on holiday and they've forgotten about it the risk to them of losing all of their funds you have to really understand and think when you're building a channel what what what values you want to choose for your time at going automated blockchain transactions I need a crack on we have written some hitless service processes that interact with the blockchain to provide the casino end of this casino game they don't really have any decisions to make themselves they just need a full of the protocol but it turns out this is quite hard there are hundreds of different situations that and failing to do any of them is a protocol violation and penalty anytime an automated process interacts with the blockchain you have to really watch this if you don't notice a transaction has been mined or an event issued that's a protocol violation if you more in turns out if you try and mine the transaction and it doesn't get mind that's a protocol violation so you need to make very careful assumptions about gas price and interwebs who reacts if it looks like a transaction that you put on the into the transaction port is now no longer going to get mind because the gas price has changed we have issues with event feeds but so we just triple check everything that every different way we possibly can and there's a quick one on uncles and reals I mean how long do you want to wait I mean we definitely there are people in this room that would say that we didn't wait long enough I'm fully aware about that too to open a channel but regardless of that not noticing a training organization is a protocol violation so you can have interesting situations where both parties in the channel want to raise a dispute separately they both do it in the same block one of them gets mind before the other one so this one throws because you can only have one dispute on the channel at a time then you have a training organization and they happen in the other order and we've got to have automated code that can spot that and react to that and continue and not crash or do the wrong thing and that's just one example of many of the things that need to happen and the client code needs to do that as well because you can't have your users try and work out and manually what's going on with the top turn when the chains been reorganized I will say that if anyone's actually developing this stuff we moved on to the main net from that various test nets way too late in the process and this it's very very different meaning that from any of the other test networks so if you if you're trying to do anything automated on the blockchain get online as soon as you can right state machines the kind of meat of it sorry I'm fronting not do everyone's implementation the definition of a state machine but a state machine is code that implements this function which is a function that takes a state and an action on that state and returns a new state and it has a function that has no side effects so it's entirely self-contained all state machines which represent the individual games the slot machines the Roulettes black around blackjack all of that stuff are written in written individually in solidity and we deploy these to the chain in advance there were people here that would say that there are other ways of doing that but we do it primarily so that it's there in advance and you know what you're getting into when you start a contract its leads to some interesting challenges though each state transition has to run in less than the max of gas for a block otherwise you can't advance the state easy but they still need to be cheap regardless is in a dispute both parties have to execute this code it's causing some unexpected design decisions for when it comes to writing generic games you know as we've done you know over the years you end up doing things like using complex look-up tables rather than just a very simple mathematic calculation figure that to get that stuff cheap you confront this cost to the deployment of the contract some of our original contracts had several mini multi-million dollar gap multi-million gas transactions to to be deployed on chain because of the size of the look-up tables but we've we've made significant progress with that the EVM is interesting to traditional developers when you're writing non kind of crypto code which this isn't really ok state machines limited stack size is a pain in the ass but we can get around it but it turns out it's also it's cheaper to store static data such as pay tables in actually in code rather than in storage but then you end up like with some stuff like this this code should never be written by anyone ever but it is the best way of doing it currently in solidity if you want to get this stuff done cheaply right also new problem machines a channel say machines can't really throw I mean they're not supposed to have any side effects so there's nothing to undo but they have to always work if you can reach a position where this in a state machine is in a given state and the next state transition will cause that state machine to have a overflow or even throw or some sort of exception or something that causes the transaction to fail you cannot resolve that dispute and this is actually exploitable so we have various validate calls that we do to make sure that you can't bet on 37 on roulette or minus one or something which could cause probably but this code has a whole new set of vulnerabilities different to other solidity vulnerabilities experience writing generic blockchain code and you need to think about it very very carefully debugging on this probably skip through this ste buggin is hard on the blockchain and that's really simple when you run into be running private nodes with a black box if they crash they go down it takes ages to restart the test that's depending on what their the week is sometimes work but they're not particularly representative main there remix debugging when it works is actually quite fun I'd love to stepping through individual transactions on main our customers have done but it's all pretty hard just be aware so packing and unpacking we are ultimately preventing the state channel is a interface that lets the state at the state advance but it calls out into the state machine to actually advance the state so this thing has to deal with lots of data that it doesn't understand you don't want the state channel to be able to know what in meaning of the actual state is beyond the actual balances of each individual participant so we've end up passing around you know blocks of binary data which the state machine can use freely but that then means that you end up having to pack and unpack this data fairly regularly so we we ended up writing a full code generator for packing and unpacking data and people say it's like Google's froze up outside wouldn't necessarily know the the the generate code for solidity typescript and c-sharp so that all three languages can actually unpack and unpack in the consistent way and so the code needs to understand this can and the code that can't doesn't need to I think I'm very much looking forward to the day when the experimental ABI encode of version 2 is no longer experimental and for ABI decode in solidity 0.50 I will say if you're doing any sort of say channel you should be looking forward to that too it's going to make life significantly easier for packing and unpacking arbitrary data inside solidity contracts upgrading contracts so yet again I mean that's obviously hard anyway you have to build in upgrade ability to most of the things that you're doing on the chain right now but you might have a channel that's open so your client software needs to be able to make sure that you can run both versions of channels at the same time but the most important part of this is that state channel's involves signing stuff off chain and putting it on chain so you don't get any of the natural replay protection stuff that a regular transition transaction would make so you have to be very careful about replay attacks it's very very easy to get that wrong I mean a simple way there are many way to get started is to ensure that the contract address of the code that's verifying the signature of the data is actually stored in the data and it's not the only way of doing it but it's a simple way of not making too many mistakes but if you have data that's validated over here and then pass on to other contracts that's not necessarily the case trust some blockchains trust this as a result of decentralization in consensus mechanisms state channels are fundamentally trust this through the protocol so the protocol defines a trusted system but how do you create a trust this implementation of a state channel and this is a problem for adapts in general it's all very well saying this stuff is secure but who wrote the type script code that's running in my browser this is a problem that will face again there are people looking at solutions to this stuff but we've got plenty of work to do here to prevent various attack vectors right okay good so i just threw a couple of pages of some actual real stats for you guys based on what we've discovered over the last month so all channels are designed to be short-lived all right so either d a few hours really the longest we would expect although we've had longer than that we're seeing an average of 200 state transitions per channel but we've had one that did nearly 5,000 back of the envelope calculation would suggest that this channel would have cost 1/2 billion gas or nearly three thousand dollars to run on chain it took 60 cents so this stuff does work stuff really does work and it does achieve a scalability and the cost saving reductions so that people say it was approximately 5% of those channels end in a user time and out 50% of those a ragequit switch is people just closing the laptop because they have limited value in the channel and this more effort for them just to turn the laptop off them to actually close the channel but actually you know I'm two and a half percent I'm taking that as a good number actually that's a lot better than we thought it was going to be and a significant number of the others are basically people getting bored trying to wait for the channel to open they put the wrong gas value earring or whatever we need to work on that as well but once you're in the channel things seems to work pretty smoothly ish so so these twos these two things I am actually very proud of even the first one we've had one failure of our automate you could see no system to correctly advance to state now that's a bug that's fine though these things happen but the protocol worked all right player raise its please raise the dispute and and and claim the dispute and they got paid and a casino did get penalized and actually this is good for the players this is the system actually doing what it's supposed to do we need to write less bugs it's fine you know but it's good and it really works and I'm really actually proud that it played out and then we've had two channels that are actually had a dispute resolve it on chain and come off train and continued and you know that's the thing that very very proud of as what go thank you very much I didn't think I'd get through that in time so I've got two minutes so I can probably take one question if anyone's got anything otherwise yeah so just a quick question regarding the main net and test net your observation what was the main difference what's the what's your what do you mean by that main it okay the key point on that one although my guys they can tell you more about it is that most of the test nets don't real I mean on a regular basis you don't see you don't see on coops we see a lot of uncle blocks on the main that don't make it into you know we've seen that transaction we've started to do things or tell the user that that's happened but then it's gone away in its code and you had to reel we've had to do you know do something else and if you don't catch that if you don't spot it that's the main that's the main one that's out of there obviously it's a bit slower sometimes and the different consensus mechanisms for the test nets behave somewhat differently but that's the main one it'll help you see uncle and then find out what you need to do about them to wait for confirmation we do wait for something right but we actually vary it based on I don't know if this works but some someone had the crazy idea of waiting for using the first derivative of the rate of change of the rate of change of a gas price and if a gas price is changing you are more likely to get uncle blocks I don't know that that's actually true or not but we use that as a metric to work out how much longer we should wait okay the user experience on this I think deserves channel well when we built a reasonably complex your in front of this all right so we've spent a lot of time on the UI to try and talk people through exactly what's going on so this stuff is all live by the way you can see a demo version of it on fun Federal IO and we have our first casinos life for those of you in the limited number of jurisdiction so that it's available for but you can you can go and see what we do with that but we talk people through the process we create the transactions we show them nice spinny things waiting for the lot to be mined and and then when it's all done the game the game launches and stuff so we a lot of effort into that at least anyway cool I think that's it thank you very much [Applause] you