'Future of Eth1: Permissionless development' - Alexey Akhunov

the radio uh i'm here to introduce our next speaker alexei akhenov i think i did that right please correct me alexei um he's the founder and lead of the turbo guth client he leads research critical for the future of ethereum and today he's going to talk to us a little bit about how modular development will enable more people to be involved in the eth1 core development so without further ado please take it away hello emily thank you for introduction um yes so let's uh get straight into this um so i prepared a little presentation it will be shorter than the last time uh but i need to just just click couple of buttons [Music] yeah that's here okay okay so the so what i'm going to be talking about here is that um so i've started let's see just one second okay so i've started watching um ethereum core development uh somewhere uh maybe in the beginning of 2018 and it took me a while to really understand what's going on and i remember um we had some kind of uh gathering or summit in stanford and in the beginning of 2019 and kind of walking around the campus campus was a great place and somebody just asked me the question without any kind of uh without any sort of irony or sarcasm like so how does um a particular change get uh into ethereum like how does it work and then i stopped and thought about it and of course the prepared answer would be like oh there's this a process there's eips and you need to do this and that you need to go through the stages but actually i try to describe what what happens re what what happens in reality so not just as a process tells us but what actually happens and it sort of started led me onto this path of thinking about it a bit more and i kept coming back to the same um realization that we are seriously bottlenecked on the um the core developers and it was obvious to pretty much everybody that we have a very small number of people that are involved into the most critical part of this process and there doesn't matter how many people will be throwing in the eips and and kind of coming up with the interesting kind of ideas but eventually all this thing net all this deluge of stuff needs to get through a very small number of people um and the reason for that is because these people are what you would call the owners of the code and that's it i think it's understandable that if your own this code it's not like you're um it's not like you're not in a sense that you know it belongs to you but in the sense that you are irresponsible and accountable well you feel that you're accountable to responsible for the quality so if uh somebody gives you the change and you didn't verify it yourself you were basically feeling that well that might break i still i need at least to test it myself so i can tell you for example that in our project i do quite a lot of testing you know you would not you might not expect that but i actually spend a lot of my time testing what other people have written because i i feel that i'm i have an ownership and i need to make sure that i the any critical any serious problems don't just pass pass through so and i also understand that some people might other people might not completely get all the nuances and i need to basically do all the testing and so that applies to other core developers as well so who to who take their job seriously and they do need to look at everything and they need to try to understand everything and then sometimes or often times that causes some frustration uh from the people who propose things and so and i was trying to propose like figure out how we're gonna solve this you know there must be a way to to to widen this bottleneck to get more people in but without sacrificing the quality and back in 2019 and so there was this idea about the working groups and things like this and yeah wrote the blog post about it but then after a while i realized it didn't quite work and there the learnings from there was that the working groups that we created back then they didn't own any code so so they could basically do research and they could prepare some changes this and that but because they didn't own the code all these changes still had to go through the developers who did own the code and they had to apply their own rigorous you know checking testing process and things before changes go in um and then another realization is that most of the escort ethereum core developers is not implementing eips is something very different it's again it's a lot of testing it's a lot of uh some a lot of people do fuzzing it's a lot of um like figuring out the performance improvements which most of them don't have nothing to do with the ips so that's another thing i realized so that's the that view that the core developers are the people who implement the eips are it's actually not adequate and there's a lot of other nuances so and so then uh we zoom in so scroll into uh 2020 and some of you might might have seen we had about three or core dev calls uh which i i put on on slide like when they happen you can go and go back and and watch them or to listen to them and so we spend these three calls where we decided we're not going to talk about eips but we're going to be talking about other issues and three main issues that i put highlight here is the burnout of the court developers which is i think is the consequence of these uh sort of responsibilities that i talked uh about in the first slide and you know the pressure that you know they feel on themselves and the the pressure also combined with the pressure uh becoming from other people who want something to be changed but that you know that kind of double pressure from one from within and one from from from the outside and not many people not not everybody can deal with that so then um we were talked about diversity of different implementations and we also talked about the barrier to entry so this is the secondary issues where there like is we recognize that it's very difficult for anybody new to come in and just let's say produce a new implementation um so you know i kept thinking about those things i mean we did uh discover quite an interesting thoughts um and then i went on uh i went ahead and i tried to find the solution so my solution that i already proposed on these calls which some people were skeptical about but some people were supportive is essentially uh introducing modularity and it's actually something that i and my team can do can action rather than simply propose and you know try to push for so we can actually do that before we talk about modularity let's just quickly go into um the so on these calls one of the theme was that we do need a diversity implementations uh for this reason that if we have only um so if we have a basically the dominant implementation that there might be some um some bug happening and that bug is uh is essentially becomes the part because of the rules of the protocol which from my point of view sometimes it's okay it's permissible um if you can you know because that's already actually been done a couple of times but if we look at the uh the different implementations disagreeing about uh the the state what what people come call consensus failures it's actually sort of this is uh presented as the sort of the the scariest thing one of the scariest thing that could happen on a serial network consensus failure so if we look i mean that is my own observation because i do look at the to the past but i didn't like take the statistical analysis of it it does happen quite rarely but it does happen both on main net on on a test nets and so my observation is that most consensus failures actually happen in one specific place of the ethereum implementation and that's what i put here in diagram so what i call the intrablock state so it's not actually evm itself most of the time evm isn't generating consensus failure it's a sort of a layer around it which deals with the things like um caching the the the things that retrieved from the database it's the refund logic it's the uh self-destruction uh logic and things like this so they are not strictly i mean depending what you think vm is but there for example if you look at evmc which is the very popular well really popular interface for uh written nc for example evmc is if you if you look at my diagram you would see that evmc is actually interfaced between that evm block and that uh like a pink block of the inter block say so this is the evmc which is uh which is this boundary and then within this boundary for example uh we're going to be talking about evm1 which is one of the implementation it implements this but it does not implement that ping thing which needs to be wrapped around it so and i posted that this is where most of the consensus failures happen not in here not in the peer-to-peer things not in americalization not in the state with or anything it's actually mostly there and that would be important for my next uh cup slide so what we already did uh so as i said that i we started to take action on this i mean we've been doing this uh for for quite a long time so what we actually did um since may we have started uh so we have to get of course and that is the the derivative of uh go ethereum and we've replaced quite a lot of things in go ethereum but i think we still have a virtual machines pretty much unchanged which is inherited from uh go ethereum we have changed the interlock state pretty quite a lot so it's very different so that's why i'm saying tg here um but we also already produced the alternative implementation of these uh piece that where the most consensus problems happen and this is completely clean room implementation so there is no code lifting from anywhere we did not re-implement the evm which just took the evm one which has been written by other people and currently it works actually it works better than uh this one so so currently the the the recent benchmark was that we could run this uh this bit through the all the ethereum blocks in about 36 hours and we can run this through the same time blocks in 21 and a half hours which is pretty good improvement but what i'm going to say in this diagram is that you know if you look at those things that they have um completely different implementations of the most of the part which generates most consensus failures so for the in for this kind of purpose these could be deemed as is a different implementations and they would provide the diversity if you run uh let's say if you run turbo get in these two different configurations so by the way we did not finish the integration of the silkworm thing into tribogeth but i think we will uh finish this quite soon so next thing we did is that um we we started on the path of the modularization and taking out components and that was one of the first things we did we did it for the purpose of performance because we noticed that um the rpc request they do um they do tend to be quite cpu intensive um i think it's because mostly because of the transformations of data between formats and stuff like this so what we did we separated them into different uh processes and we have created the sort of very simple interface uh very low level interface between the rpc daemon and and the node um and that also has an interesting um consequence that they might they could be multiple rpc daemons uh for example implementing different variations of the rpc json rpc standards they could reside on the same machine or different machines um so currently we only there's um it works with the tcp here uh but actually we are we realized we can also make it work with a shared memory so if you need the if you can put them on the same machine and the your you need a really high performance you can do that it doesn't quite work yet we just need to fix it but it it could work in in principle but this is the way it all kind of started so this is by the way um for the um communication between the the the protocols our current standard we just basically do it for all the components if we want to separate them we use grpc which is basically http 2 based uh pro or sort of framework which is based on protobuf and it you can generate bindings for pretty much any language in existence so we for example this is like a protocol definition for our rpc demon as you can see that what it does it's basically it's like a it's like a database you can open a course or to the you can open a transaction and then you can you know inside the transaction you can open the course or to a table and then you just do um basically extract go first you go to the first record you can seek something in the database and you can retrieve it and it doesn't have all the semantics of prefetch and everything like this but the the point is that the interface is very universal and simple so you can implement pretty much any rpc request using that because what you're given is the remote database access and so that basically was the first thing and then that gave us ideas so that we can actually uh then next thing um whenever we had this uh issues of um where we wanted to do some variations uh we okay we said we can split these things as a component so the work currently ongoing on the consensus engine split and this is actually quite an important one so currently uh although people say that the consensus engine is pluggable but actually it's not quite pluggable because it still uh lives inside the the um it basically lives inside the code of the interpretation and anything that lives inside the code of implementation has to be verified as i mentioned before by the code owner it has to be like checked and everything but if imagine instead you had the consensus engines implemented as a component with a well-defined interface then the people who developed the consensus engines could own that code and so that could be already a start of separation of the code ownership and so the if let's say if there was a working group that worked on a consensus engines that working group not only just proposed something but they could actually own the code they can do their own releases as long as they you know there is a compatibility through the interface and then the sort of it's one of the examples where the core developers essentially give up the um the responsibility and the power they need to come at the same time so that they don't own this code anymore but they own the their implementation of the interface so the switch to different consensus engine can happen uh already with the participation of a different group and so that's currently we're still figuring out the first version of the interface because in this interface we're trying to support um three things already the already existing ones that we know about we trying to support each hash which is quite simple so initial hash you would have let's say these things that are marked green like um you know the core wants to verify a certain header it just sends it to the consensus and there's a bit of chatter going on between them but in the end it gets the result whether it's verified or not um so the consensus can ask for additional information for example for some parent blocks and for parent headers and stuff like this and but then another use case is when the core wants to do forked choice so and again the fork choice rule is the is the the job of consensus engine so it will ask out of these set of headers which one is the best and the consensus engine we say this we will say this one and it can again they can ask for some additional information for example if there is if these things are actually on a different forks it will ask for the predecessors of these headers to arrive at the common ancestor and then it will be able to figure out for example what's the highest difficulty or whatever consensus algorithm says um and then for the ceiling is like it's for the miners or for validators there's another request but we also so we're looking at the et hash we're looking at the click click could be implemented with this protocol as well and we also look in an aura but aura is the most complicated one but we actually are probably going to try to implement anyways so next practical example is the p2p sentry this is also work in progress and it's a bit further ahead than than than consensus engine we actually have uh initial implementations running and uh one thing to notice here is that we already have two implementations of century one and which is we basically separated the code the peer-to-peer pure code that uh existed in tubercult which was inherited from go ethereum and so literally like today i was testing it as the as a sentry and then we have another century written in rust which is basically completely clean room implementation and they are implement the the very similar protocol and so i'm not going to go into details but this is the protocol we're currently working around so what we're trying to do is that we're trying to sort of make these two sentries are essentially pluggable or even another interesting thing you can do is that you can actually have multiple sentries connected to the same node and with all the other um yeah so this is actually opens up some more interesting uh flexible um options because you can have like a yeah you can have a two different centuries of different implementations connected just in case one of them has a bug then you still um actually connect it through another one so so here basically the idea is that the diversity could come in many forms it doesn't have to be the entire implementation of the of the client but it could be a re-implementation of a specific bit or you you could actually have multiple implementation in one installation like with an example of sentries and the last bit that i'm going to show is the transaction pool split which we haven't started yet but we are going to start it very soon with some help from other people and you probably some of you might have watched the uh today's talk about account abstraction so for example the work on account abstraction requires a lot of um changes inside the transaction pool logic and how we're going to you know how i propose we're going to do it so first of all we're going to lift it out of the core component and as you can see here it could be done uh with the help of a peer-to-peer send p2p sentry so essentially a transaction pool becomes this kind of component suspended on the two interfaces uh so from on one hand it's interfacing with the peer-to-peer sentry for the transaction traffic and on the other hand it does require some access to the state but it can use the same interface that rpc daemon is using for the state because it can basically query the database and what is interesting here is that um you know i'm i predict that there will be multiple different implementations of transaction pool uh some for the purposes of the account abstraction experiments some of the purposes of some sort of muv experiments um and so forth we also have a project to try to implement it in a different language i think we're going to try to do it in python for example like you know how cool is that you can implement transaction pool in python connected to our century which is written in rustin to the turbo guest node which is written in go and it's all going to work um so yeah as a conclusion uh to unders score the things i just said that you know we need to think about implementation diversity in a very you know i would say more diverse way so it could actually come in different forms it could be uh not just basically just re reproduce the the whole thing in a different way but maybe reproduce the parts in a in a different way and also kind of recombine them another thing is that working group can and should own the code and that would make them because they are the owners they will be responsible for code quality and this is how we're going to widen bottleneck and another thought is that for everybody innovation always happens elsewhere we are not the smartest people in on on the planet right there will always be people who are smarter than us that will come and you know help us to uh you know to do things better yeah that's uh that's it awesome thank you so much alexa for sharing that with us um if you've got time to hang out for a little bit i do have a couple questions that came in from the chat um the first question is how big is the performance overhead for decoupling via json rpc um we actually haven't measured that uh to be honest we did have a project to measure this uh some time ago but it was long time ago and we we only saw the overhead on the things that are they were really really intense uh kind of chatter with the database uh i think we were testing something like get storage range at and it was some um considerable overhead however if we look at the current usage of uh rpc requests in inferior for example they had a blog post recently that the one of the most popular requests is is call which is essentially executing transactions uh you know onto the state which is in which is current current state or something and so this is where i expect a lot of benefits from taking that out of the node because you will run evm in the rpc daemon not in a node and it will just once in a while go into node to ask for a state so yeah the answer is i don't know but as i said as well we also introduce in the the the mode where you can do shared memory with the with the node and actually and think in this case the performance might not be even um different awesome um got a couple more questions okay uh next one is it i'm gonna mispronounce this i know it is it possible to use c plus plus silkworm evmone already this particular user would love to add it to their fuzzing efforts yes so the silkworm is open source and it's uh it's got apache 2 license so you can search for it um i mean or i think there are also links from the from the turbo get repository definitely i mean open source apache license i mean it's just out of the oven uh so you know it's not like we're we haven't uh did that a lot of testing with it but if they want to experiment yeah you know be my guest cool all right let me check on the chat i've got another one okay will this module will this modularization stay in turboget because there is more freedom to design or have other clients express an interest in adopting this approach um i sort of like yes and no because i think uh at the moment this is very new um a kind of new direction and i some people are still skeptical about this um and my sort of way to go about it is that i we will essentially just do it we will not try to sort of uh to to try to like get the to to try to basically do consensus on this before we actually just gonna prototype and we will see if this brings some benefits and if it does i'm sure other people will join and people will recognize that if somebody tries will try to come in and re-implement trooper gear for example or any other client it might it will be much easier for them to start with the one component and then get to another component and another one so they can you know the job that i was doing for the last almost three years would be much easier if i just had some small part to start with rather than just looking at this whole thing i'm sure there will be a lot of cooperation got it got it all right i'm gonna hang on just for like one half a minute to see if we have any more questions because the chat does seem to be pretty active uh let me see okay i think that's it um thank you alexei for joining us giving us an awesome talk um i'm sure i'll see you around okay bye