'State of the Merge' - Lakshman Sankar

so the next talk is titled state of the merge and uh this is going to be talked by lakshman who is a researcher at the ethereum foundation and he's going to be presenting the current state of uh what the merge is and where it is uh larson's been working on this thing internally for quite some time and i know we've all talked about the merge as a as a big event but exactly what's happening there and what um is planned is uh is kind of the goal for us to uncover today so with that please welcome lakshman and uh whenever you're ready you can share your screen and uh turn off video turn on video hey uh how's my is my video everything is great working great let me share my screen how's that how are we up perfect excellent all right hi everybody um my name is lakshman i work on a lot of things at the ethereum foundation primarily focused on kind of moving the protocol forward um i think what that's kind of fallen under you know recently is we're calling this like eth2 and the merge basically this big shift to proof-of-stake consensus and you know i i kind of wear many hats i'm involved in some r d stuff i'm involved in some coordination communication stuff education stuff um i'm always down to talk about this stuff so uh you know i'll on the last slide my my handles will be shared as well but i'm always available to chat if you're if you're interested in learning more just getting involved anything like that all right so uh this talk is going to be very specifically about the merge um i'm not going to go into too many technical details actually i'm more going to kind of like motivate what is this thing like why why are we trying to do this um what do we need to do to get there and where are we now what's remaining and finally talk a little bit about where where you can get involved where you can help where you can kind of like learn more um this talk there's a lot going on in ethereum protocol development um a lot of really cool ideas uh shall we shall we gave some excellent kind of historical context and some vision about the future and there's you know there's they're starting there's a whole bunch of cool stuff coming for for state size management you know like vocal trees um epic epic bay state experience you know 1559 is on the way uh so we're gonna have some cool new gas uh fee market kind of structure you know mev extraction you know like moving starks into layer one like there's all kinds of cool stuff out there this is not going to be about any of that this is going to be very focused on this kind of singular task in front of us which is upgrading from what we know as a theorem today which is theorem with uh with the proofs of consensus and um you know later in the day vitalik's actually gonna talk about what comes after merge so if you're if you're interested in all this stuff which is super cool as well definitely stick around for that so uh where are we going what what is what is the goal here sometimes it's valuable to just uh step back and remember what remember what this is about because there's a lot of like jargon there's a lot of like you know changing kind of like language around um you know what what we're trying to do and it's valuable to just kind of think about what is the end goal so this is this is a this is a near-term image um you know like we you know ethereum 20 years from now may look different but this is sort of what we're looking at um and what we're hoping to get to you know within you know don't quote me on this but within the next year uh you know uh i think i think last week there were a bunch of interesting talks about layer two side chains various mechanisms for scaling that don't exist in layer one um and it's really cool that all that's happening in parallel with these these things that are happening um in layer one uh i'll uh prove the stake um and so what we're working towards hopefully is something where these two things work in concert we have this very fast data layer uh and that that operates on proof of state you know environmentally friendly uh has economic finality has these has these has these great properties um that proof will work doesn't have and that data layer serves you know this this interesting execution context ordering context for transactions in a bunch of different you know who knows how many different l2s and and maybe they interact with each other but this is sort of like the rough mental model to that we're working towards and and this is why so this is uh this is from uh from a tweet vitalik made um at some point last year that i think you know while while maybe the uh the language is different um the concepts hold up um and and also don't don't worry too much about the actual numbers here but like you know this is this is what we're working towards we're going to get uh you know if if when roll-ups get there if roll-ups get there uh we're gonna we're gonna have like pretty excellent uh you know gas savings in tps um but if we can combine both of these worlds we go even further so that's that's that's what we're working towards so how do we get there so what what what is what is this merge thing like what is on on the theorem one layer or sorry and the layer one side how do we how do we actually what what needs to be done to actually get there um i sort of think of three different pieces to this um one is sort of like the uh the actual operational hot swap of one sub component of the system out for something else which you know we'll talk about consensus hot swap uh one is sort of the the actual client uh you know like if it's it's not enough to kind of theoretically just have uh a proof of stake and proof you know like in and and ethereum we need like a client we need like an actual client implementation that does all the things we wanted to do um that does all the things we want from what we call eighth one today and all the things all the stuff we want from the from the beacon chain and finally we want we want to kind of really trust that the new home for consensus is uh is robust is secure um and so uh some of this is just sort of like a lot of testing but some of this is also you know kind of more visibility around what's going on over there um so first of all this is uh you know there there there's con i realize there's a lot of like uh shifting kind of like um perspective on on what this looks like but i started thinking about uh an ethereum as sort of analogous to uh you know a like a traditional operating system and that there's different kind of layers in in terms of you know what it is like just as in a in an operating system you know you have kernel space which is kind of like the vital functions of the operating system and then you have like user space which is things like higher level stuff um you might think of ethereum as having kind of this consensus layer which is the core sort of like how do we uh how do we even grow a blockchain and then and then there's a lot of stuff that exists above that which maybe you know we can call the execution lectures like how do transactions get into the blockchain what determines that a transaction is legitimate i.e the evm so if if we use this model kind of like as our as our base this is kind of one useful way to think about what the merge is um we are keeping the execution layer the same the ethereum we all know and love today and of course you know like layer two is above this somewhere uh and we are upgrading the consensus layer to this new you know like super fancy fast uh consensus layer with with nice properties um so that's that's kind of like the hot swap and just like an image of it and we'll i'll i'll get back to some of the some of the details of like what needs to be done there a little bit later so uh just separately and be more concrete we need a client um and some of you may have seen this image before this is uh this was this is i take it think taken from like an older youth research post kind of originally proposing the idea for what this client looks like but um to to get there to get across the finish line today um we probably don't want to rebuild all of the great things that exist in each one clients around these this execution layer um that were i was talking about here instead maybe we can and this is the current plan we can uh combine we can define a clear rpc interface between each one and these two clients and have an ethq client driving an eth1 client as our new combined client that will um kind of like have the have all the things from both worlds that we want um so to do this you know there's there's some very detailed edge casey spec work which is a lot of what kind of you kind of merge r d has been focused on recently uh we need to make sure we can define that we define that rpc such that uh it it can work for every single eth one client interacting with every single eth2 client um you don't want you know once you once you start having like uh you know n squared number of possibilities you don't want to uh you want to interface that captures everything you don't want to have to deal with those cases individually so that's that's kind of technically but more uh kind of pragmatically you literally you literally need these people to start you know the people developing these clients to start you know working together um so a lot of a lot of what's been going on behind the scenes has has been sort of like the beginning of collaborations between folks on you know what previously were separate client teams um who knows what this will look like in the future um but you know there's there's a lot of cross communication that needs to happen and finally as i mentioned this is a very edge case oriented thing uh you you want to make sure that we want to make sure that this is very thoroughly tested um you know both kind of like unit tests and integration tests on the specs and clients themselves but also you know we want to run devnets and test nets and and and see that you know things don't break when you start putting them under load there's a lot of testing that still needs to be done um and and so you know just just the third the third item uh for the what how do we get there was kind of like a proof of stake chain like a consensus layer that we can kind of trust for uh to be the new home for this you know booming uh execution layer that has like all this cool stuff going on right now um and so this is you know separate from the merge itself we want the beacon chain you know the proof of stake chain to have like properties that you know make it make it uh uh make it resistant to capture resistant to you know like kind of like bugs and failures so we want you know we want uh things to be very thoroughly tested we want audits um we don't want uh a chain that is mostly run by a few parties we want uh a lot of hobbyists we want to know that kind of like uh that that part of the chain can fork and like you know kind of social and technical processes around forks work so you know we want to have experienced a hard fork in in that chain and so all of this stuff is stuff that's being worked on so yeah so that's that's like a high level you know three three three areas that you know are kind of being worked on in parallel and and so it's valuable to uh yeah i mean i think i'll now talk a little bit about where we are today which also kind of motivates what work is needed to to get to get to the end the uh the finch line so um i really like this image that's kind of like uh becoming sort of the norm for thinking about uh what's what the merge is um so so one of the reasons i like it is it sort of really clearly motivates like three separate places where work's happening um work needs to happen on the fir the top line you know the kind of like the proof of work line work needs to happen on the bottom line the kind of like the proof of state line and work needs to happen right at this intersection uh you know we need to figure out what needs to happen for this uh upgrade to to to kind of happen properly and so um i think in in terms of thinking about where we are now it's it's valid to look at like where are we here where are we here like where are we in the proof of work land where are we in the proof of stake land and where are we with registered merge so i'll start with start with merge so um uh shall we kind of like hinted at this and um i'm i'm not going to go into too much technical detail here i want to leave some room for questions but also like all of this uh is can be found on the um east to spec github which which actually is linked on the next slide but if you if you go read the spec um oh sorry if you go read the spec uh the way that the kind of like basic kind of like what this this this print that um the data structures are changing is actually quite simple uh we're just sticking in the block that exists in the beacon chain we're sticking in the information necessary uh to kind of process transactions so if you look at this execution payload um and if you've interfaced much with ethereum via like ethers or web3.js you'll probably kind of recognize this because this is basically what's necessary to what's necessary on on to to validate the transactions in the block um so literally just sticking that straight in to the to the beacon block it's quite straightforward um the there's still some you know some testing that needs to be done but like you can go to this link here um and actually get an idea of how this all works it's it's uh it's it it's it's hard to be convincing when you know i'm not going through all the details but it is pretty it is pretty straightforward to understand if you have a basic id basic understanding of how these two systems work so that's kind of the merge side so you know we there is there is a release out uh release cut for for the merge spec itself there's you know some additional kind of unit testing that needs to be done but we're uh basically there on the spec side uh next is nexus implementation uh so we haven't you know like that's uh proto-lambda is going to give a give a talk after me on ryanism the uh merge dev test net um and so that that's going to be you know so we're we're very close to starting to test the test the merge client in production um so on the proof of stake side i think i mentioned earlier that you know we want to kind of like know that we can hard fork um and the first uh hard forks spec release has already been cut and we are at the stage of implementing it in all of the clients which is really exciting uh so this is this is we're pretty close to full confidence on this front um and you want to learn more about what's in this hard fork uh check this link out one of you know kind of side note but one of the really cool things here i think is that um you know like clients committees i like the incentive to publish uh block headers for like clients as big will be baked into the beacon chain itself and this hard fork adds that functionality so you know yeah again slide slight side note but you know if you've uh if you're excited about light clients uh it's kind of fun to dig into and finally on the proof of work side um as we all know uh you know the berlin hardwork just happened with a bunch of you know important gas cost changes um you know sometime this summer we'll see uh london should happen and then after that um hopefully i think the merge the merge is the next hard fork for for the proof of work side all right so finally um i'll talk a little bit about um you know i think i think i've sort of set some parallel places where work is being done um i'll talk a little bit about where you can kind of get involved with the stuff today so uh i think there's broadly these three areas um one is kind of like testing of the actual thing like testing the merge itself like writing unit tests and stuff like that one is uh actually running the client and and seeing what happens um so that is um oh sorry one second uh so that is you know frodo's gonna talk about that after this but like um you know there's going to be a test net that you can participate in over the course of this hackathon um and actually really exciting i think if you think that the uh upgrade of the steak is like an interesting place like interesting period event in ethereum history i think participating in the cessnet is a pretty uh historic thing to be a part of and then finally uh just being involved with the beacon chain uh contribute to staking you know like run a validator get involved see see what the process is like see what doesn't work um and and and generally contribute to a to a healthy state ecosystem so on the on the spec test side um uh this is it's um i think i think you you kind of just need to go read the spec and like understand and i i linked it earlier uh where was it it was here um it's actually pretty like you can see that there's sort of three uh these three links for the changes it's it's not that long uh it's it's pretty straightforward to understand but i think what what we need people what we need is like people to read the like the spec uh and just like think about edge cases like think about like all right this this could happen right at the point of merge um so it's a very high level it's like there's a difficulty at which you know merge will trigger or or like at which uh beacon chain validators can start including that eth1 payload in in their blocks and there's probably edge cases uh there's probably things that we haven't thought of so uh go go just think of those things and add github issues for for things you think are missing um i think one area in particular just to flag something one area that could always use more love is like that exact point at which the fork should happen um you know like it it should be simple but yeah as anyone you know like who's worked on you know like uh software knows there's always weird edge cases you haven't thought about and um would be great to get more eyes on that um i'm also happy to help direct so my contact info will be at the end of this and feel free to reach out um next and and i don't want to talk too much about this because proto has been produced giving a whole talk on this um but uh there's a test that ongoing and uh it's it's a pretty cool opportunity to be a part of something quite historic which is the testing of the upgrade to proof of stake that you know we've been talking you know the community has been talking about since the beginning so uh yeah go go participate go run pick two clients that are may not be picking each one kind of needs to client that you know um aren't being tested by someone else and and see see what happens when you try to stick them together and then finally as i mentioned earlier uh contribute to a healthy stinking ecosystem so i i pulled this from um etherscan earlier today uh and it's it's kind of just to point out that um [Music] there there's a reasonable fraction of of uh beacon chain deposits um that are are exchanges um and uh it would be great if the miscellaneous category here were a little bit larger as well as like the segment category perhaps um so uh yeah just go run validators participate in the decentralized taking pools um help build tools that make it easier for other people to run validators i think this is something that here valuation is very interested in um you know from like a grant's perspective if you go try to run a validator and and you start running into some headaches it's you know we want it to be as easy as like one click if poss you know someday um so um if there are things that are standing between you and that you have ideas uh this is stuff that is very important for a healthy seat and you can use some of the stuff that the if your foundation is very interested in and uh that's it i think i think i blazed through my talk relative to the time a lot of me so we have a bunch of time for questions thanks watchman that was a really helpful review of everything that's happening in this ecosystem we do have a couple of questions i'll relay them um as we kind of get this and uh for those of you listening um we got we got a bunch of time for questions if you see this thing after 30 second delay type your questions and then we'll relay them back to lakshman so uh the first question is um you've talked about uh the need for a unified client that kind of bridges each one any two transactions so the question is what what does that actually look like in practice would that be something that behaves more like mist from the early days or or something else and then kind of just going deeper into that would be super helpful yeah yeah so i can i can kind of talk through what the kind of like control flow of this client is so it's a beacon chain client which does the sort of like growing of the blockchain that uses rpc to an eth1 client to figure out what transactions to add to a block and what transactions are valid so you can think of it basically as a beacon chain client with this additional evm attachment on it um it'll it'll probably feel and operate very much like a beacon chain client today but there will just kind of be this this other thing it talks to to figure out you know because we don't want to you know like transaction the transaction cool management is a whole thing right like there's a lot of hard software problems that go into that um the evm is a hard thing state size management is a hard thing these are things that just didn't didn't make sense to rewrite so all of those pieces will be kind of you can think of it like literally as a remote function call all those pieces will be accessible to the beacon so the beacon chain client will be like kind of building and growing the blockchain and calling out to that when it needs to figure out okay what transactions should exist in this block are these transactions valid according to the evm et cetera et cetera thanks for clarifying that and and i think there's another question that popped up from uh i think the slide that had destruct um they were looking at and uh the question is um why are transactions opaque but the payload isn't and uh this may not just be fully in context but uh i'm hoping that uh the question friends i'm subscription's opaque but the payload isn't yes oh ignore ignore the i mean i like i actually don't know the exact detail of why that struck that uh why that has the name opaque transaction i mean nothing's nothing so yeah okay proto says that he will answer in his presentation yeah that's better we'll have that question proto will know better than i will know so that's awesome uh and i think uh kind of the one last question then that will end is uh uh to kind of put you on the spot what's a realistic time timeline for uh for us to uh uh i mean i well first of all i'm definitely like i'm not i'm i'm not the person who is the best person to ask and also like i i think that uh i am also hyper aware of the uh sort of uh the dangers of mispredicting the future with regards to timelines um but let's say let's say within the next year from today i don't know i'm not going to say like when within that year um but within the next year soon tm right well watchman thanks so much again i think if you have more questions we'll leave them to you uh directly or if you i see you are hanging out on on the tv page as well so uh hopefully you can answer those questions directly so thanks again and with that we are ready to move on to our next talk and also get that opaque transaction questions answer