Wendy: Adding Order Policies to the Consensus Level - Klaus Kursawe

so yeah thanks for having me in here and i think um this talk is now the proof that i should really sit together more with the cornell guys because we are pretty much going in the same direction um here so i'm talking about wendy which is another fair ordering approach it came out of a slightly different use case so in vega we are building a derivative trading system on a dedicated chain which is tendermint and our main goal is we need to convince traders from centralized exchange that uh decentralized is an alternative and that especially means um if any flash traders steal more money or get more money out of them than they do in a centralized exchange they're not coming so we need to pretty much minimize uh mev as much as possible and in this use case democratizing isn't really an option because if we democratize the rewards um the traders still stay at a centralized exchange because they make money more and more money there and the problem we are having here is that the price of the decentralization is that we have much larger attack surface so currently if we don't put in protection then flash traders actually have actually an easier game on a defy system than they have against the centralized exchange and we have spent 30 years plus work in doing consistency but much much less work on what's a good order in in the last time and only now has the problem big enough that people really care about that um other thing we want to do is we want to have minimal latency impact so we need a practical protocol and if we waste a lot of time then we are too slow compared to centralized exchange and again the traders will go away um there's another issue um which is sort of turning around icon uh economic arguments so for other reasons uh we want uh economic control over our validators so we want to set economic encouragement for things like diversity and performance if they make most of the money by mev and not by validator fees that the network can control then trying to do this is a little bit like trying to control smugglers by tax breaks it just doesn't work because this is not where they make the money so that's another reason why mev minimization is important for us because we want network control on what validators get and we want them to get the fair share so there's no doubt there but we want to link uh policies to economy our goal is a little bit different than from what are we described you want to integrate the protocol into the consensus layer as a modular add-on so we don't want to mess with the consensus code but we also don't want to run an oracle we want this to be part of the consensus code a little bit like kasper is for ethereum to add finality we want an add-on that adds fairness and our original chain is a bft style protocols we built on tendermint um i have one slide on how this would uh hold for ethereum in the end and our goal is a tool tool set for kind of everyone so we heard this argument before there's no point in being selfish and trying to have one chain that handles front running because we all suffer if this is a real or perceived problem so as many solutions as possible as many tools as possible is helpful um state of the defense um so there is a little bit of protocol archaeology there um already mentioned causality um there also are leaderless protocols which solve a lot of the problems by just not having one validator or minor setting the whole order but having a group already baked into the protocol um that's also the work i did back then in my phd thesis honey badger is doing that now but they never really took off so the bft style got more into fashion so now most implementations are stuck with leaders and we need to handle that um yeah and then we have the block order fairnesses and what i'm talking about now is um the evolution of wendy which is the fairness framework which also includes um causality in there and have a couple of links to most of the papers i mentioned so for further reading um you can see that on that slide um so we already saw some fairness definitions so um i have the luxury of being able to go over this very fast uh the one thing to add here is um we have a bunch of impossibility results so the um standard thing if everybody saw or if all honest validators saw a before b then it has to come before b isn't always decidable because there's uh loops um so that led to block order fairness so if you have a loop put the whole loop into one block and let's application sorted out but that tool runs into a problem that we need potentially unlimited block size since these loops can be arbitrary size even in a synchronous system and then we have a backup definition that is timing based based on local clocks so it doesn't add as much fairness but it doesn't lead to a paradox and our first approach in wendy was we go for the block order fairness when we can and if we see we are now growing a two big block we are running into trouble we switch back to the weaker definition of fairness resolves a deadlock and then move back to the original protocol once we had that um the next idea was ekta um was to expand this to an actual framework um but first setup and model so we need a known set of validators so i wouldn't call this a permission network but we're not registrationless validators need to say here i am and they need to know of each other we assume an existing consensus layer and we're trying to be as flexible as possible where we fit in and we assume a multi-use chain so we assume that we have a blockchain that does different things um in our case different markets if it's ethereum different smart contracts and not all transactions need fairness and some transactions may need a different kind of fairness so if you have different definitions of fairness different markets different smart contracts may actually have a different perception of what fairness they want um so we have a framework protocol that then can encompass different definitions of fairness both between different applications and switch for one application like i said for the block order if things go wrong we can switch the definition first phase is a dissemination phase that's a one round protocol that just makes sure that every potential block creator gets all the information they need to make an order um that can be time stems if you have trusted time you would throw times uh trusted time in there um for block order everybody just throws in the order in which they source things and send this to the potential block creators um then we compute blocking sets um so this is the transactions that need to be delayed because it cannot be fairly transfer now have a reveal thing so the nice thing if we have an ordering protocol if we link the ordering with the commit and reveal we can actually reveal at an earlier point than we could for normal commit and reveal protocol so normally i can only reveal once the order is set so once the transaction cannot be reordered anymore now i can reveal as soon as i know by the fairness rules i cannot be front run anymore and one of the problems of commit and reveal especially in non-finalizing chains is that it can take some time until i'm allowed to reveal this combination makes it actually very nice it can start revealing earlier so the two techniques are not competing they're actually working very very nicely together and then the more practical things blockchain interaction and post-processing that's getting the whole thing actually to run on an actual blockchain which is in the less exciting but very important and very tedious work um so the blocking and revealing rules and this is where the whole fairness definition comes in so everybody can every market or every smart contract can have their own fairness definitions through these rules and there's essentially three and a half important rules so a transaction is blocked if given what i know now there can be a transaction that i haven't seen yet that may have been scheduled earlier if this happens i cannot put this transaction into the next block it needs to wait then we have transaction dependencies um so given what i currently know um a transaction i know about but haven't executed yet has to come before my transaction i can then either put them in the same block or if one of them is blocked um both of them need to wait and then we have rev reveal ability given the current launch i have a transaction now has precedence over all newly generated transactions so this is where i can open the commit and reveal and weakly revealable is not as strong and i haven't found a very nice definition definition of it it just means now you need an insane amount of network control to still front run me and for all practical purposes this is just not gonna happen um now police's examples uh fair block order was mentioned already so if all honest parties see t1 before t2 then g1 must be in the same on earlier block and then the post processing will sort this out and this is then the rules for when it's blocked uh what the dependencies is and i'm not reading the rules for you and you can look at the slide maybe later so this implements fair block ordering uh by just defining those four rules on the back up okay slide i forgot um policy requirements so there's a couple of things i need to think about if i build a policy um what i want to do has to be measurable so we heard that also before if i need a trusted signed clock or if i want to have sender time then i need a trusted sign clock if i cannot measure sender time i cannot put this into a policy requirement ideally loop free ideally we should be efficiently terminating so that no transaction blocks forever and ideally once a transaction is unblocked it should stay unblocked and not go back now as in all requirements in the byzantine world is usually workarounds so the fair block order already violates two of them um so we have a couple of things measurability is sort of unavoidable um loop freeness is solved by putting stuff in the same block and then that's the applications settle things efficient terminating can be solved by detecting if you're not terminating and then switching the policy and one not to me we can tolerate a non-monolithic protocol if we don't want to do commit and reveal if we want to do commit and reveal then this is getting difficult um timed fairness the other one i mentioned it's just a different set of rules and just to show that we can get completely different um capitalistic plus social security that's essentially the ethereum model with an add-on that if you wait long enough um you didn't need to pay that much so the transactions your block by transaction that either paid more or is already waiting for a very long time so we can actually be relatively creative on what examples we want as long as we can formulate them as yeast in these three properties and then the protocol will support them now performance measurements we did two implementations one to be the real implementation which is still being integrated since memphian integration here is a bit complicated and a simulator to see how how the performance are so in terms of latency add which is our biggest worry the protocol runs completely parallel to the host blockchain so it doesn't delay anything in that sense but some transactions are blocked and then end up in a block that would be later than there otherwise would be in our experiments the number of this transaction depends between uh on this ratio of block time versus message delivery time so if i have very fast protocols if i use tendermint that's sort of 10 of all transactions if i use a slower protocol like ethereum you won't even notice that this is happening um transactions that don't need fairness are passed through right away and suffer no delay um and the weighting i mentioned above is actually the dominant factor transactions blocking each other is was in our experience pretty much negligible um the assumptions we have and that's where integration gets interesting is so we assume that the blockchain does not change the order of events that we propose we can work against the blockchain here but having a mempool where we can give the mempool an order or priority make things much more efficient if the blockchain needs ability to drop messages due to overload we need to coordinate this and if we have an other ordering mechanism that's of course a conflict so since most people here care more mostly about ethereum and this is where the whole mev problem was biggest so we can implement or use wendy with ethereum so it would be an ad on like custer also having its own sort of validator set like cusper that know of each other and can run a protocol where everybody needs to talk to each other efficiently so the good things you can make commit and reveal much more efficient uh as we solve the non-finability issue every smart contract can choose their own fairness definition and just use it and there's a lower performance impact what we need is to what you would need is um a defined set of fairness validators we need a hook into block and transaction validity verification so when you can say this transactions is now invalid and the biggest issue but there's certainly solutions to this that can be discussed is we need a mechanism to resolve a conflict between the native ordering mechanism which is gas fees and the wendy policy so we want to avoid a setting where i have a transaction that by when defenders will need to be executed next but it didn't pay any gas so everybody else is blocked by transactions that just refuse to pay gas and needs to pay for them or find something else and with this i'm at my end um so i guess we don't have time for questions anymore right yes unfortunately um however uh i think we should continue the discussion offline and we'll continue to collect these questions next up we have