"Eth1 + Eth2 = Ethereum" - Danny Ryan

i'm really excited to actually welcome our next talk and uh for our next talk we're gonna cover what it means when we talk about eighth one each one point x seats two and how do all these things kind of merge together and how do we actually make sense of all of this to kind of really get to understanding how we build the theater for tomorrow and talking about all of this will be danny ryan who is a researcher at the ethereum foundation so uh i hope you enjoyed danny's talk on how to make sense of everything ethereum and i'll let him take it from here hello thank you appreciate the intro let me share my screen okay great um thanks for having me uh my name's danny ryan i work with the ethereum foundation on the research team i spend almost all of my time working on kind of core consensus protocol upgrades um and that project is kind of encapsulated in what we call e2 and today i wanted to give you some context on um what e2 is what each one is what these things are in relation to each other um and how they kind of come together to be the future of mainnet and if you tell if you take away one thing from this talk today uh it's that the terms each one and the term eth2 are a misnomer and imply a false sequentiality they plot imply one being deprecated for the other they imply some sort of migration they imply something that's not quite accurate instead they represent different layers of the stack different layers of focus different layers of optimization and r d and i'm going to get in and explain what these two layers are and how the vision of how they will come together in the near future so each one plus these two is ethereum talking about future mainnet today um so what are these ones each one is quite plain and simply the theorem we know and love today again if i could erase these terms and use something different i would maybe we'll maybe we'll try to in the future but each one is ethereum warts and all it uh is a single blockchain it is run by a proofwork mechanism it has tons of d5 activity nfc activity identity protocols all sorts of really cool stuff and it's why we're all here um but concretely uh we have implementations of this protocol kind of accesses into this protocol and gateways into this protocol called eastbound clients and really what is an each one client um obviously it's an implementation of variable but we'll dig a little bit deeper um for the purpose of this talk i'm going to view it as it's consensus and it's user layer its consensus is proof of work which is actually a relatively thin relatively simple consensus mechanism that's one of the beauties of proof-of-work is that it is indeed simple from an implementation and software perspective a lot of the complexity of the protocol randomness leader election dealing with the actual economic asset for the crypto economics that's all handled outside of the protocol in the sophisticated mining hardware and and so you end up with like relatively simple rules and protocol verify some hashes add up some difficulties and find the head of the chain um in addition to what i would argue is a thin consensus protocol in software is a highly optimized user layer when you think about all of the hard work that goes into optimizing ethereum and dealing with the complexities of state and transactions and all that that's really in the user layer and if you go and look at the diff and guess like there's probably almost been no work done on the proof of work uh consensus algorithm for years go look at it i haven't looked at it probably pretty thin on what's going on there whereas all the work is happening on state think state state management transaction optimizations evm execution and it's all really like the secret sauce here is how to get uh that user layer highly optimized um and so i mentioned some of these things some of the optimizations you hear people and some of the deep work being done on on equine clients or transaction mental management um there's you know riddled with security issues when people can send random transactions to dos vectors all sorts of stuff transaction gossip optimizations to optimize bandwidth consumption efficient evm state transitions and state accesses state rights read and rights and a big one is state think the state the active state and ethereum quite large and so managing that from uh how do you get it how do you share it how do you store it a lot of effort goes into these types of things this is a extremely oversimplified diagram that i've threw together of an e1 client you have the pdp network where you're communicating with peers and sharing information about the protocol transactions blocks state sync and you have this proof-of-work consensus which is at the top this is kind of the brain it's ultimately deciding what should be done at any given time with respect to finding the canonical chain which blocks should be included and viewed as canonical how should it manipulate the state with respect to that and if you were producing blocks if you're a minor how to go about that and then you have everything else and everything else is a lot and everything else is again where a lot of the hard work is done uh managing the evm state transitions managing the complex state in the state tree managing the mempool and all the complexities that come with that um and so that's definitely one client and the bulk of the work is not in that proof of work component um a little side tangent what do we mean when we talk about e1x um e1x is generally um a group of people on the internet like all these things we're just random people on the internet uh focusing on r d on the user layer so all those things it's not it's not an upgrade of that group of work instead it's really all those other things how do we manage state how do we manage uh transactions and execution and how can we make it more sustainable and how we make it a better experience for ethereum and the community so that's 1x focusing on that later to mention some of the things going on here they're looking at upgrading the state try from this hexary try to a binary try for some efficiency gains stateless block processing as a method for managing the complexities of having a large and growing state so that we can have spectrums of different types of clients interacting the network that don't necessarily have to have all of the state account abstraction uh to increase uh open up some like interesting types of applications um and some of the stuff i'm really excited about is some of the security implications of having count of traction being able to do tornado cash and that kind of stuff in a more secure fashion gas fee reform 1559 improving the ux of getting transactions included improving some of the security concerns around minor track destructible value again all in this this user layer so uh that's teeth one is one client any questions no i'm sure um 2 what is east 2 we talked about each one what do we mean when we say used to really what e2 is is a radical upgrade the consensus layer of ethereum when you look at all of the all of the research for the past five years that has culminated into this e2 project all the work going on at these two clients um and all of the work in this layer and in the sub section of the ethereum project is on consensus upgrades can we upgrade to a more secure more sustainable and more scalable consensus mechanism and that's what we mean by east too and so again different layers different separation of concern false sequentiality bad terms east one these two bad terms maybe they're here to stay but let's start thinking about them more as layers of the stack so it's ultimately uh this is back to that that scrappy diagram of an each one client um it's looking at that proof-of-work composite mechanism and saying can we do better again can we make it more secure can we make it more sustainable and can we make it more scalable past five years of research say a resounding yes um i just wanted to include this image because i thought it was pretty cool um pretty worthless slider than that but great to say um yes so the plan is to move to a proof-of-stake and charted uh system uh chartered consensus mechanism um and i'll talk to you a little bit about uh what that looks like um so what then is an east 2 client we have this piece of software called mate 1 client highly optimized user layer but what is an e2 client really it is it doesn't touch any of that user layer stuff it's really looking at that brain what was that proof of work brain and it's swapping it for it's building out a proof of stake and chartered consensus like i said proof of work we get a lot of stuff for three because it's handled in extra protocol complexity whereas with proof of stake and a started consistent mechanism there's a lot of in protocol a lot of software complexity to make that work and so all of the work that's gone on in each two clients in the past two years is to solve that problem is to bring is to handle and manage the complexity of a proof of safe and sharded consensus mechanism to zoom in into that that brain right there we solved that problem or solving that problem through the introduction of a system level chain called the beacon chain that manages validators manages the core consensus manages the system level state called the beacon state has its own p2p layer where it essentially manages gossip and communication of system level consensus level items such as beacon blocks at the stations validator exits that kind of stuff and ultimately manages is the brain uh for the consensus of rather than just one chain my cat of course is here um rather than just one chain each one as we know in the day uh managing the complexity and consensus of multiple chains um then we call them charging so that's what an e2 client is it doesn't touch the user layer it's come into consensus on the sophisticated mechanism becomes expenses on things um and obviously the stuff that we want to come to consensus on is things that are valuable to users and valuable user layer um but it is more of just a general mechanism becomes expensive um so then like i mentioned one of the things we want to come to consensus on is ease one is the ethereum chain that we have today and so the plan is to hot swap the consensus of each one at a point in the future with the consensus for consensus with the consensus of an e-2 client with that proof-of-stake sharded consensus um so i've kind of implied and alluded to the fact of where we're going uh but we have all this incredible work that's done in each one and on each one clients um guess is a machine like there's awesome stuff going on neither mind badass stuff going on all these clients lots of lots of great work going on in that user layer um and then in east 2 we have incredible work going on in this consensus layer and so the future of the ethereum stack from a node perspective is really the unification of these two pieces of software into a beautiful cohesive combination that each piece of software handles the sophistication of its designated layer so let's drill into this image and and take a look at what that actually is going to look like um in the prototypes of this merge that we've been working on this looks like the an e2 client and an eth one client or as we might call it in the future in each one engine uh running on the same computer uh connected via rpc so that efon engine that's that's that uh each one client that diagram that we had earlier instead of a little proof-of-work brain it's opened up its consensus to be controlled via rpc to be controlled by a remote protocol now these two clients running in parallel to this eth1 client and it's uh company consensus on things it's driving it's it's building these that system level chain it's building the shard chains and it's communicating to local e1 client the state of things the state of things from a system level perspective being like here is a new block rather than the proof of work saying here's a new block it's saying here's the new block oh there's actually been a minor reorg here update your internal state this is the new head and really that e2 client becomes the driver becomes the brain whereas the e1 client handles all of the user layer complexities transactions state sync all that kind of stuff this quick zoom in these are just the two diagrams the e2 client uh controlling to the e1 engine by this rpc and getting responses uh sending requests and responses the unidirection directional protocol the e2 client being the brain drives and that simplifies this relationship between the software um we've actually in the prototypes there are prototypes of this unification of software and eth1 existing as a shard today in these prototypes we have four rpc methods um block production block validation block insertion and setting of the head produce block is similar to the get work rpc call where on demand today an ease one where on demand you say uh given all the transactions in your mempool and given uh your current state give me give me an a good looking block to build on this chain validate block is um i've got this shard block i do some base validations on the system layer for example validate the proposer signature and some other things but then i say hey ease one client can you actually validate this with respect to all the user layer rules can you actually execute the transactions here and see that things turn out valid and if so return true great um insert block this is kind of a variant on validate block it's really like inserted into that local e1 block tree and updates the state and set head handles reorg so the e2 client being the controller of the consensus understands uh when the shard chains reorg and need to be reorg so uh when you need to update the internal state of what's going on in each one and on that user layer you can set the head there validate block can easily be modified to be stateless and some of this some of these methods can be stateless and there might be additional methods that make sense to add for example the east 2 client is aware of finality and so might tell the e5 engine that it's time to you can prune these things this has been finalized so these one engine can be a little more proactive in pruning state and things like that okay and uh so this this work this e22 merger work has largely been carried out thus far by mikhail from txrx at consensus and guillaume with the guest team at the ef they've been driving these prototypes it's pretty exciting to see these two pieces of software come together and to see that we can then by keeping the user layer intact we can leverage existing wallets existing tooling to interact with that eth1 chart so for example you can check it out mikhail has this out on twitter i'll reach post it later today uh but he has a video of demoing um the east 2 driving a multi-sharded consensus there being an east one shard and sending transactions with metamask unmodified metamod very cool so the vision here is really to um not modify not to make extremely minimal modifications to that user layer uh in this migration and upgrade of the consensus i got some cool stuff going on uh shout out to some of the quilt team it looks like they're going to get involved in this ethon e2 merger effort now that we have the base demo in place there's plenty of work to do a lot of fun to be had and if this stuff interests you like please reach out uh in the coming months there's plenty of fun stuff to do on the development and the research side uh so we're working on the proof-of-work proof-of-stake hot swap mechanic so essentially in on the live e1 chain how we can swap from proofwork stakes safely looks like mick isle is taking the lead on that and doing some r d and prototyping validator deposits and withdrawals once you have the unification of the e1 chain with the beacon chain we can handle validator deposits and withdrawals natively there's some stuff to think through there uh especially like on the withdrawals push first pull who pays gas that kind of stuff shard data fee market um in the at the initial merger there will be one uh shard chain of state and execution that's the e1 shard chain but there'll be many shard chains of layer one data and we expect this to be very valuable to applications especially in the roll-ups we need to have a good way to to pay for the data and essentially say i want this data to get on chain so working through those mechanics opcodes additions and reform due to the new consensus mechanism of beacon chain there will be some system level op codes that would be useful for user applications namely checking what's going on with validators or even especially looking into recent cross links from different shards you can make proofs against data and do cool rule stuff and some op code reform there are opcodes uh that will need to be thought through once they move into this new consensus mechanism for example there is a the proof-of-work difficulty opcode where you can peer into the chain and see what the difficulty is it's not quite clear what meaning that should take on once we've moved to proof of stake so we need to investigate some of the existing applications and usages and try to move forward with the minimally invasive uh path so a lot going on again if you're interested in stuff reach out there's a lot of work to do and a lot of fun stuff to work on so what about statelessness one of the other major upgrade projects and i mentioned this in the 1x stuff two ethereum is this major upgrade to the user layer into the state to move to stateless execution stateless blocks where essentially you don't have to have the entire state locally to process and follow the head of the ethereum chain instead bits of the state and proofs about them called the witnesses are included in blocks so that you can perform the execution without having the whole state and this opens up a lot of it opens up a sustainable path to state management ethereum um in terms of where this fits into the e2 merger these things because of the separation of layers can both drive forward kind of unimpeded by each other and there's not really a path dependency on one or the other the merger can happen and then statelessness or statements can happen and then mergers so we're driving forward on both um and then when one's ready we're gonna do it and once the other's ready we're gonna do it in terms of the long-term implications of uh not moving to statefulness uh on east two if we have uh if we have statelessness we could oh it opens up the opportunity to potentially have state and execution on multiple shards if we don't have statelessness we can't really open that up because all of a sudden you're going to be putting the burden of not just one state but the burden of many ethereum states on on all validators to be able to process and validate things so statelessness does open up opportunities and how uh the consensus layer and the user layer together grow but in the initial merge and the initial kind of immediate future they're they're past independent so what's the end game um i didn't know that vitalik was going to post that excellent article on i think it was the magician's forum this morning about a rollup centric map but something that i roll centric path for ethereum but this is something that i just want to talk a little bit about in the beginning of an e182 merge we have state and execution on one shard so we have ethereum like world on one chart and we have a lot of data a lot of l1 data coming from all the other shards and we expect it as i said to be very valuable to roll up since the scaling in the immediate and so we expect you know roll-ups today on ethereum and one shard might i mean one one just theorem chain might provide a few orders of magnitude scaling then once you have 63 shards of data uh you can get another couple of order magnitudes of scaling um and so once we get to this this merged place we're going to have many if we go down the path of bulbs we have many orders of magnitude of scaling which would probably be sufficient for a long time um so the question really becomes we have one shard of state and execution and many shards of data availability and thus terms of scaling is it really worth and the question this is a question i don't know the answer but is it really worth at that point adding additional l1 complexity to get another you know 10x 100x in l1 scalability um the end game for me is to get to this this e52 merger and get to statelessness as soon as possible and at the same time to allow the rolex ecosystem to grow and to mature i think at that point we'll have a lot better information on how if and how to expand uh native execution beyond that that initial one shard and so the end game might look like one state and execution shard many data availability shards maybe a few state and execution shards and many data availability shards or the full blown state execution everywhere but we'll see and i think the next 12 months as uh the user layer each one uh develops and as the consensus layer e2 develops and as l2 uh and the exciting rollouts develop we're just gonna the theorems become better better a lot of awesome cool stuff going on and we're gonna have better information on how to make good decisions good decisions beyond these initial developments um cool this is something near and dear to my heart the merge of each one ease 2 is i think one of the most important things going on right now it's exciting r d project hit me up if you want to get involved um and again if you didn't take anything else away from this talk east one is two misnomers false sequentiality instead they are separation of layers that's the better way to think about it and it's really the most informed way that we can talk about upgrades here talk about the different layers of the stack and and the software so thank you i appreciate it um i'm sad that we could not get together this year at devcon but i am excited one day we will all hang out again together take care everyone danny thank you so much for giving us an amazing overview uh looks like we have a few uh minutes before our next talk and now we got a handful of questions that people have asked on chat so i'll just kind of ask them to you here and we'll just relay them directly to you so the first question we have is uh i mean he did a really great job at clarifying the separation of these layers but if somebody holds eve today what do they have to do when it's two launches right so e2 is launching in phases uh this is to manage iteratively manage the complexity it's a major upgrade to the consensus it's a modification it's a it's a major upgrade of the kind of the core economics of the system so to do that safely we're doing that in phases what happens initially at phase zero and what we expect to launch imminently is that we bootstrap this new system level chain called the beacon chain in isolation from existing ethereum main net meaning that proof of work module that proof-of-work brain that's controlling uh ethereum continues to control ethereum for now and then uh when we're ready for the merge we're ready for each one to become a shard governed by e2 at that point the existing state the existing chain at this point before this point it's going to be governed by proof of work and proof of work miners are going to build the blocks and then right after the same state the same execution is going to be built uh and and governed by the proof of stake validators um and so in doing that there's nothing you have to do applications will continue to run your ether will continue to exist on this chain essentially it's that it's that swap i called it a hot swap of the consensus from proof of work to prove today so chill out nothing to do unless you're a validator and you want to get involved early and kind of bootstrap this new consensus mechanism that's great i think people are people are getting excited about hot swaps on on the chat here for uh for a live stream and uh and uh i know this is going to come up again so maybe just we'll ask this and clarify this one more time but if somebody is a dap developer do they have to do something special once each one is merging two needs two um the intention is is no um because of the way that the state will continue uninterrupted and because of the way that each one clients will actually still be the interface into the user layer um the the way that applications are built uh will will be uninterrupted will be the same and the way that applications interface into that user layer for example how you send transactions how um a web interface might uh use web3 libraries that is that is uninterrupted because you use the same uh user layer apis to interact with each one once it's integrated into these two something important to note is that uh cases although things can continue uninterrupted capabilities will expand so we'll have this new uh charted data availability layer which first and foremost we expect roll-ups to highly leverage and use for scalability but uh having a scalable data availability layer uh is kind of a very new thing in uh these decentralized protocols and so we expect there to be a lot of innovation just with that in general so um keeping your eye and your ear to the ground on developments there and kind of expanding what ethereum can do rather than like fundamentally changing the core is the goal here awesome and uh the last kind of two part or two related questions are how can somebody kind of keep up with what they have to do to stake for phase zero where can they find out more latest information and then on that same note for people who are interested in understanding how to keep up with what's happening with all of you too what is like the best place they can go to stay up to date yeah so on the staking front uh there are there's a testament called madasha long standing test net that's up and stable if you want to get involved you can stake on that we have another dress rehearsal test net called lincoln that will launch in about a week and a half if you want to get involved in a genesis event which is kind of like one of the more interesting parts of staking is really getting the chain up and running at the start and then if you want to follow i've been super excited about the eats baker community they started as a reddit community and they have a discord and they're just like super uh supportive like people are asking about questions people are answering questions people are talking about hardware people are talking about different software uh security practices all sorts of stuff uh check out like that's that should be the gateway go check out the east staker uh reddit and the east baker get discord if you want to get involved and the the second question sorry uh just uh for everything needs to on research or implementation or progress updates what is the best place to uh keep up to date on so i i do blog posts on the ethereum blog every um about 7 to 14 days depending on what's going on they're called east 2 quick update i try to keep you just informed on the little happenings if that's not enough then like if you jump into east baker they're gonna like that is a great place to follow on the kind of staking side what's going on and if you want to dig a little bit deeper um there's the east research forum uh and something that has been really exciting development become really fruitful over the course of 2020 is the east r d discord where there's all sorts of conversation uh talking about consensus talking about there's psycho um talking about consensus talking about user talking about eth1 talking about e2 and really like technical fun discussions uh it can probably be pretty intimidating at first but like lurk just lurk see what's going on um you know eventually uh you'll start figuring out what's going on and probably have something to contribute so yeah blog post east staker and ethernd now this is great i think your uh blockchain ethereum.org posts are super helpful because they kind of condense it for every day people just kind of understand what's going on without having their that's fully integrated in the technical pieces um and before we kind of end things one more then then edgington does what's new in east 2 and it's so good mine are like short little like i just say what's going on real quickly he gets some like deep thought about what's going on in youtube so check out benedict uh too i'm not mistaken that is a sub snack you can subscribe to is that correct probably i i trust you people post it on the chat for reference yes yes it is it is i forgot what that was good uh one last question that we got from our youtube chat uh before we move on to our next talk is uh can you tell us what is the role or the features related to the bls signatures it needs to and kind of what the significance of bls is over there yes so we um in a sharded proof of stake model uh we have to do random sampling uh to send parts of the validator set to different shards at any given time to essentially validate subsections of what's going on and in doing so we need a lot to get to make the random sampling safe we need to have a ton of validators so we expect from the consensus perspective for there to be like on the order of 100 000 of these little validator entities obviously a user might run a few of them or even many more but this helps with the random sampling but because of this random sampling because of how many validators we have we have tons of signatures like they're constantly saying like what they think is the state of the world um and so to uh do this efficiently and be able to verify and process everything efficiently we heavily rely on bls aggregation uh so anyone when people agree on something uh which if they agree on the head of the canonical chain and the state of bashar then we can aggregate their signatures you can actually aggregate things that don't agree but what we do is we aggregate signatures that agree and then we bring that all on chain and thus we have much smaller amounts of signatures to process it's a great cryptographic technique um obviously there's like a lot of networking complexity and stuff that comes out of here and like how we actually get all the signatures together and aggregate them but that's a different layer of stack but bls is awesome aggregation is awesome and it's been a really crucial feature to get this thing um to work in a sharded context awesome uh thank you so much danny and uh for uh for those of you who are interested in ben's newsletter it's uh that's actually e2.news that's the domain name so you can just go there directly and stay up today so thanks again danny for giving us a great overview of how all the ethereum players stack with respect to each other and we can't wait for uh eats too so