'Vouch: Client Diversity without the Downsides' - Jim McDonald

so we're gonna do something different right now we're uh we're gonna have a special guest uh co-hosting this thing with me and uh i'd like to welcome uh frenzy um from the ethereum foundation who's working on the solidity team uh to uh co-host the next few talks uh with us and uh she'll be the one taking over uh for me and uh i i hope that you uh enjoy the rest of the today so frenzy welcome i'm super excited to have you back and uh hi yeah thank you for having me and uh it's awesome to uh to see you here join us from a different part of the world it's always awesome to get really really nice um members from our community and kind of make this event possible and we're seeing so much excitement we have a few hundred people joining us on youtube and in our tv chat so uh it just gives me even more joy to uh have a lot more people excited about the merge absolutely yeah thanks so much first of all for having me as a co-host for uh the second time now uh i love those events i think they are really the best way in the current state of the world to to stay connected with everybody and to keep uh on exchanging on the current state of ethereum and what's uh in store for the future of ethereum and yeah i've been following the the talks so far i really liked them um they were incredibly interesting and insightful and i think uh without further ado i will do my job as a co-host and introduce the first speaker that is up next that's jim mcdonald jim is cto of attestant that's a company that specializes in building tools that improve the stacking experience and today jim is going to talk about about client diversity without the downsides large is an open source validator client designed to operate with multiple beacon nodes and jim today will talk about why client diversity is a good thing and how vouch can provide client diversity without the downsides i am really excited to have you jim and the stage is yours thank you okay so i'm gonna mainly run to a set of slides um that makes life easy for everyone so hopefully you can see the the slide deck and what we're going to talk today as franzi said is really looking at some of the information around client diversity what it is why it matters how we can achieve it most easily with the current configuration and where we'd expect to go with it you know after the merge so obviously the first point is is to understand what client diversity actually is we hear a lot about it but the question is what does it really mean so from our point of view client diversity means securing the ethereum to beacon contained with multiple implementations of the specification so ethereum two is primarily a specification of which each beacon node carries out an implementation why do we want to do this you know why why don't we just have a single implementation and everyone's happy well first off is it it hardens the specification uh if there is an assumption made in someone building a beacon node and there's just one beacon node out there then that's the end of the story however if there are two or three beacon nodes out there then there can be questions about well what does this really mean i made an assumption here i i thought this corner case wasn't relevant whatever it might be as a result of that you end up with a specification with a single implementation being relatively loose so this hardens specification up everything in a specification has to be very clearly defined it also hardens up implementations so again if we have you know just one or two implementations out there then they can make assumptions they can read a certain piece of text in in a certain way uh unfortunately uh english is a language of most specifications is written in is not as precise as it could be and is indeed very ambiguous in places as a result of that you can take a single read of the specification and create an implementation you can create a second read of the specification and create a different implementation if those implementations have to talk to each other as beacon notes do that however then means that they have to finally agree on something so you cannot leave things open to your own interpretation once there's consensus involved because of that having client diversity hardens each individual implementation finally it separates specification and implementation now this is a really important point the ideal world you can take a specification and create a full beacon node implementation from it equally you could take a single beacon node implementation and re work backwards to create the full specification from that the reason why you want that is because it means that each one is well defined if your specification requires some kind of hidden knowledge inside the implementation then that means the specification isn't well formed it requires information from elsewhere so this means that client diversity helps everyone and everything in the infrastructure and ultimately it gives us a much more stable and stronger base in which we build the protocol so great if we've decided that client diversity is a good thing let's go pick a client uh there are lots of clients out there though so here's five of them these aren't the only clients out there that carry out the ethereum too spank but a good example there are lots of them out there so if we want to pick a client how do we do it what do we choose so let's have a look at some strategies one of them is let's go and pick the least popular well if we want to have client diversity we don't want to pick the most popular or the second most popular we want the least popular however maybe it's least popular for a reason maybe it's not quite there yet maybe some of the features aren't fully working maybe it has security issues who knows the problem is is that without any additional information just picking the least popular to increase client diversity could put you and your validating a risk so maybe you go for a primary that's a little bit more popular or well known and then you pick a backup trouble with that of course is that you don't use the backup very often so the backup might be sitting around waiting for the primary to fail and the primary never fails or fails three years from now because of that it's a very tricky situation just to say hey you know we'll we'll have a primary backup and we know the backup will be there unless you're continually testing it that's a lot of work a lot of effort makes life quite tricky maybe what you can do is you can measure the beacon beacon nodes you can have a look at each of them you can measure them according to certain metrics and find out which one you're happiest with problem with that of course is using the right metrics you might pick something like cpu footprint but then again a beacon node that doesn't carry out any proper work we'll have a much lower cpu footprint even though it doesn't succeed necessarily in creating the best blocks or the best attestations you might have one as the lowest disk usage or network throughput you might look for some metric that allows you to compare on-chain performance the problem is is that you don't know which metric to use and the metrics will change as well over the last three to four months since the beacon chain went live we have seen multiple revisions of all of the major clients as they go through their own testing internally their own performance metrics they find bugs they find optimizations they will change so even if you do find a perfect measurement and you measure it exactly and you pick a winner the chances are your information is out of date two weeks later so although there are lots of strategies for helping you to achieve client diversity none of them really come without downsides there are always issues to think about there are always concerns and as we've said it's very very much a moving target so what if what if you could run multiple clients at the same time you could take the best information from each of them and use them to help secure the network while also gaining superior performance and great reliability that's the type of thing that would be great because that way you take the best of every client you work with them well the the rougher edges are smoothed out by other clients you also get great performance and you get great reliability so that would be a really really nice thing to have and that is basically why we attestant built vouch so vouch is an open source validator client we have designed it from the ground up to work against multiple beacon nodes we don't twin it with any single beacon node in fact we most of the time actually have no knowledge of which beacon node we're talking to we have a heavily beacon agnostic view what we wanted to do is give us the information we need so we can do our work equally having just spent the last few minutes talking about client diversity the last thing you want to do is make vouch a new single point of failure make it a a system that everything else funnels through because if we do that then we're back to where we started where we have a ultimate client diversity of one so because we want to avoid that what we do is we offload as much work as we possibly can to the beacon nodes we let the beacon nodes individually do all of the things that beacon nodes do they gather state they run state transition they store historical data they transfer data between beacon nodes they do all of the things that you should do and what we do and in vouch is we look at the information that they give us and we act as a decision maker so we will look at the information provided to us and we will decide which information is good or which information is best and then finally we want to be resilient in the face of nodes that are unresponsive maybe unsynchronized use network errors or just internal problems or ones that are just slow the reality is is that beacon nodes all have different performance profiles and one big node might be fantastic generating proposals but doesn't do very well generating aggregate attestations one might have a fantastic network stack but uses an awful lot of cpu when we ask it to do work there's all sorts of different pros and cons with each of them as mentioned they're all getting way way better and they're all stepping up but the simple reality is they cannot all be best at everything it's just not the way the world works so we want to be able to talk to all of them and then be resilient if one of them suddenly stops responding if we hit the bug or one of them he has a problem with its network stack or we have a system where suddenly there's been some kind of packet flood attack and as a result one of the nodes goes offline we need to continue to be resilient in all of these situations so to give a more concrete example of of what we're talking about we'll we'll think about the idea of block proposals so in ethereum two when it becomes time to propose a block the each individual beacon node has its own view of state it has its own view of all of the attestations that have yet to be included it does its own work in creating aggregate attestations that will go into the blocks so this is this is great for us what we do is we say okay that's brilliant what we want is we want a block proposal from each of you so for every beacon node we're talking to and again it can be any number of beacon nodes we can run with any beacon node that works against the standard ethereum two api we can talk to the beacon nodes in parallel we can have multiple instances of the same beacon node so we could have you know two prism nodes or three prism nodes scattered around the planet so maybe they're getting different views that way uh you know we could have a detector in a lighthouse node wherever we want we can sort of mix and match as we choose and again as mentioned you know vouch doesn't really know or care what client software each speaker node is running physically where it's located or anything like that when it comes to propose a block vouch we'll just ask for a proposal from every client of which it knows it will do it in parallel it won't wait on one for the next one to come along and it will do this with a timeout as mentioned we need to be resilient in the face of a beacon node being unresponsive perhaps vouchers sitting in europe and a beacon node over in asia we can't contact because of some kind of network glitch we're not going to hang around and wait as a result so we contact all of the nodes we can we get it in parallel we bring the data back to us as each beacon node generates a block and returns it to us we have to decide which one we want to sign the way we do that is by carrying out a scoring mechanism so we'll start to dig into the contents of the block and we'll basically run the internal ethereum to scoring mechanism or similar based on the information that we have but also very importantly the information from from the wider network so for example if one beacon node is running uh for some reason say it has a clock issue we might end up getting a block that's earlier than actually the one that we asked for or one that doesn't have information about the correct parent block because it's had a network issue there are lots of reasons and as we run this in a live environment we see lots of these reasons where for some issue or another we end up with a block that is is non-optimal maybe it doesn't have that many attestations and it may be the attestations it has in it haven't been heavily aggregated whatever it might be the idea is that we will boil all of this down into a simple score so we will look at all of the attributes of the block we have received we will score each of them and we will assign that score to that block as more and more blocks come in we can compare one against the other and we will end up with after all the beacon nodes have returned their data or after we've had timeouts from the ones that aren't going to respond to us we will have a block proposal with the highest score as far as we have ranked it we've got that that's great we're now happy that we've we've done pretty well we've got a block we're happy with we will go through the standard signing process as you would expect then we submit the block now again we can just send the block out back to a single instance but that doesn't necessarily guarantee that it will get the the widest reach if you send it to a beacon node that potentially again has network issues so has very few peers that block may not reach much of the network so what you want to do is again in parallel and again with a timeout obviously if a beacon node is unresponsive or similar we give up in the end but we want to submit that signed beacon block to every client we can reach by getting it to as many beacon modes as possible what that means is we end up in a situation where we have the greatest chance of getting that information out there making it part of the chain and obviously ultimately what we're trying to do here is secure the chain in the best way possible and by doing this we are giving our block and the attestations in it the best chance possible to get out there be available and as a result of that we can ensure that we have the uh you know the best chances we say of that block being included in the network okay so one of the questions that has been flying around is well what kind of impact do we think the merge is going to have on on what we're doing at the moment well first off the merge will give us more data uh we're going to have more data on the beacon chain we're going to have more data flowing between nodes that is going to start showing up potential weaknesses in some network environments it's going to show potential weaknesses in network stacks so again we won't expect all beacon nodes to handle the new load as well as each other there are going to be areas where where one node might start to get even overloaded or similar and it's not just down to the software this involves potentially the hardware that our peers are running on uh you know it doesn't really matter how powerful our hardware is if all of our peers are running on low powered hardware and they're getting overwhelmed that's going to slow down movement of data around the network so having more data will make the world that little bit harder in terms of ensuring that we can communicate individually with all the rest of the nodes in the network and because of that obviously again the more nodes we talk to the better that we have in terms of a chance of getting that information out there secondly subjective data one of the interesting points about the beacon node is that at any point in time if we take state and then we say what should the next block look like in terms of who's its parent how many attestations should it have how should they be aggregated there is an objective ideal block that could be created because of that it makes life frankly relatively easily in terms of of block creation and and attestation selection um there are a few corners in there but ultimately it's a it's a pretty simple thing to do when we get to including application layer blocks that changes because now we're including transactions and the world of transactions is large and wild and there are any number of ways you can slice and dice the information out there this means that then we have a situation coming down the line where suddenly the subjective selection of one piece of data over another into the application block will start to matter a lot more this is a whole new area for ethereum two clients now potentially we had a lot of this off back to the ethereum one client to build but equally there are other alternatives out there there are people like flashbots who are building their own systems we we need to start spending more time looking at this but the fact that there is going to be subjective data makes things like scoring blocks makes providing optimal blocks and similar a lot more difficult so it's something that as it comes down the line it's really really important to understand that the merge actually will make this kind of thing a lot more important in terms of being able to look at multiple feeds multiple options for blocks and being able to score and select the best one and then of course more operational diversity um we're going to have eth1 clients and these two clients working with each other because they're now talking to each other in more than just the trivial fashion that they did with uh the current beacon chain it's going to become a a lot more complicated so we're going to see a big step change in terms of the complexity of the individual beacon nodes and their interactions so sort of just let's sort of summarize and and look at what this means well first off and i you know very very important understand client diversity is important it's not just important to the network it is important to you it is important to you as someone who wants to send transactions on the ethereum network it is important to you as a validator it matters because without client diversity will end up with non-permanent specification we'll have specifications that have uh ill-defined areas that are never caught because there's only a small number of implementations and they never find them as we've said though client diversity comes at a cost and the cost is your own ability to validate if you want to run a minority client fantastic but there are less resources out on the net to help you with them potentially it's not as secure in terms of some of the features and and systems that are running it simple example is some of the clients weren't actually active at the time that madasha had its rough time incident as a result of that there are situations that the less used clients have yet to be in so there's always a concern that if you start to run these you're going to end up in a situation where you are somehow disadvantaged and client diversity is great and network diversity is fantastic but very few people want to do it if it involves a direct cost to them compared to everyone else around them so vouch is one way that we've provided that will allow you to support client diversity it doesn't give you any sacrifice and performance or reliability and in fact can increase both you there might be a minority client out there that isn't brilliant on the networking it struggles a bit on cpu but you know what it's brilliant to creating aggregate attestations if that's what it does if that's the best piece of it that's fantastic vouch will basically use that when you bring a block and it will score it very highly it will use that so not every client will be best at everything not every client can be best at everything but you can still run them and get the benefits of each of them without experiencing the downsides that is it for the talk we'll do some q a in a second but just wanted to say as we mentioned this is an open source project there is a url there where you can download vouch uh it's used already in a number of areas obviously internally inside our own infrastructure but also elsewhere and there are other areas where obviously if you go on there you can file issues you can contact us on our discord there's a lot of areas where you can get in the information about vouch and obviously uh take a look to see if it's something that you would be interested in running to help provide that client diversity for the network amazing uh thank you so much jim that was uh truly fascinating so far i've only received a few comments here uh two attestations that you are indeed a rockstar and people were asking if vouch is open source you already responded to that yes it is open source and here in the github link you can um yeah have a look and probably also contribute um are there any other questions for jim let me check other than that i also believe he will be available in the chat right absolutely and it's worth saying obviously yes it is an open source project and there's there's a lot of stuff that uh can be done and added to it so uh you know we we have our own focus on what we think needs to be in there uh but we definitely have other people's ideas that are coming in and it's something that and again i you know i strongly believe that with projects like this the idea is to give people as much flexibility so they can create their own systems for example some people actually use vouch not to validate but actually to pull a bunch of additional metrics out so that you can provide comparative information across different beacon nodes in terms of how well they they create blocks or attestations or aggregates or similar so you can start to do that type of thing too so yeah absolutely you know it's it's it's primarily a validated client but we have had other people actually manage to uh to play with a few different variants on a theme and yes as as always as always pull requests are gratefully received yeah amazing interesting so now the questions are flowing in um the first question is with regards to your roadmap um what's next for vouch so the next thing well there's so there are two streams for vouch one obviously is keeping track of the new changes so uh we're starting to work on altair altair i'm sure you're all aware is the first hard fork for ethereum two and that's meant to be going we're not sure yet june july august whenever it may be so we're starting to build out some additional work there to support that uh the biggest piece for us will be uh sync committees so sync committees are a new feature that uh validators will have to support so that's going in there so obviously we're keeping track of that and a little bit further out we're also looking at the changes the merge provide uh separately we're always looking at enhancing some of the strategies that the value chooses we mentioned about scoring of blocks and attestations and similar there are always ways in which they can be improved where the efficiency can be increased or equally people can have different inputs on what they find valuable and what they want to score highly so absolutely that they're kind of the two rough streams that we're working with okay cool then the next question would be do you see vouch implementing other social ranking mechanisms like they are used in evolutionary computing so one of the things that we we played around with in in vouch was actually what we called synthetics the idea that we don't take any of the information from any of the existing beacon nodes as gospel and we don't take them whole but we use them all as advisory and then we break them apart and use that information to create our own synthetic proposals or attestations now part of that would obviously be trusting the information that's coming to us and there's a lot more of that coming down the line so um yes up to a point equally at this moment in time we let the beacon nodes guide us very heavily in terms of what comes through so again we want to be a little bit careful about taking doing too much ourselves as mentioned we want to be a relatively small piece in the puzzle that aids client diversity increases reliability and performance without attempting to kind of take over the world so we have to be a little bit careful about anything like social ranking and similar into the system okay and then uh last but not least do you see vouch also for eth1 nodes asked by a guest developer um vouch will integrate in some way with each one node so obviously when we get to the point of the merge if you think about the merge at some stage and an e2 validator will have to propose a block and the question then is where that block comes from um if ethereum two nodes themselves talk to ethereum one nodes and generate that that's fine alternatively and quite possibly vouch would talk directly to ethereum one nodes separately to again gain the best application block it can find um that could also incidentally apply as we said you know other areas like flashbots and similar so um yes i i certainly you know we certainly want to and i think we we will add that as as we said as part of the merge but yeah totally because the the benefits again for client diversity in ethereum one the application layer are as strong if not stronger than the benefits in the beacon node fascinating all right thank you so much jim for sharing all these insights with us