How The Merge Enables Greater Innovation at the Consensus Layer | Gregory Hill

[Music] uh thanks for joining me for the graveyard shift um my name is Greg and I've got a small talk on um the merge and how it enables uh Innovation at the consensus layer uh so a little bit about me um my name is Greg and I'm a senior software engineer at Inay working also on the buff stack running a hacker house this week on uh running on uh building on bitcoin um and if you want have want more info on that please come and chat to me uh mostly working with Bitcoin and ethereum sometimes poka do um and we have two power chains on polka do kusama U mostly again also in Rust um before I kind of get into the Nitty Gritty I want to kind of give a bit of motivation review uh why this talk came about and kind of where my stance is uh so looking at the landscapes Circa 2019 pre merge uh we had a few a few different clients uh G has always been the predominant um client for eth main net uh with that we have a few different consensus modes ethash and click um some of the other variants NE mind has Aura uh bezu has ibft 2.0 and qbf um so we have kind we had that sort of the lay of the land um um but then zooming out we can see there's like tons and tons of forks so gu itself over the years has been forked so many times uh it's just unmanageable at this point um the other clients of course have their own versions uh optimism has forked several to make its changes uh we have different versions of EV evm deployed on different chains both as Forks of gu and re-implementations uh a few of these on the list for example have been have used uh some common libraries some use G as a library um some use rust ethereum as a library um but if you go on to GitHub there's just there's just so much code out there um doing basically the same thing GE you can see at the top there has over 20k Forks admittedly some of those would be from contributors but uh by and large there's uh in my opinion too many uh so yeah that brings me to my controversial opinion I hate Forks uh but not Sports you know this is uh eth Denver after all and uh Sports have utility I I do like sports um the other thing that kind of motivated me recently was uh kind of Click was being removed from um cor gu due to obviously all the changes required for the beacon change and Beacon chain and eath to um and we had a project which I was discuss in a bit which made use of proof of authority so this kind of motivated me to kind of dig into this topic and kind of investigate the changes I'll outline soon uh and I what do I want and why am I sub better uh well I want I want the e one stat I love the eth1 stack it's tried and tested especially gu uh it's a classic and but we have so many different consensus mechanisms if we're limited purely to ethash or click or just whatever's running on eth2 then I think it's a shame because there's a lot of innovation happening out there and I think we can help that uh further uh and then of course I want fewer well-maintained evm implementations uh a bug in one spreads to all of them uh it's impossible to update and ultimately uh audits expensive so we can't we audit one we have to audit all of them um so let's fast forward now to the merge and the kind of text deck that brings with us uh we now have two different kind of uh clients a separation here so we have the execution layer that is called that's G um that's bringing us the smart contracts the execution the state transitions the um the uh state state management and it's a pretty well battle tested uh piece of kit uh then we have now the consensus layer this is what's driving the execution layer now with these two we have proof of stake that's handling rewards and penalties and attestations uh and again driving that block construction so we have this nice we separation typical example of the stack now kind of in the spirit of ethereum and Lego blocks we have gu and prism um we have WTH and Lighthouse you know cuz we love sticking to products coded in the same language together uh and we have this thing called client divers diversity maybe that's a fancy way just to say fork but uh it's nice to see um different things using the same bit of Kit throughout all of this um it's always usually G or W as the execution client so it's nice to see uh some reuse there um how this works is using kind of a nicely defined interface that I think a lot of people worked on for a long time called the engine API and it's kind of summarized in three steps uh Fork choice we can you can call that to update the head of the chain which block we're building on and where we're going with that uh what's finalized and what's safe to produce on top of uh get payload is basically used to tell G to or ref to construct the block and new playload simply Imports that block uh so you can see that if we kind of build a system around that it's super easy just to tie that together uh and build a a kind of new set of uh uh mechanisms uh to drive block production this is all locally so the the engine API operates on local node uh or it may operate remotely I guess but typically it's authenticated using uh uh Json web tokens uh and if you want to kind of a better more uh rounded explanation of that this uh blog here has some wonderful diagrams that kind of go into sort of the nitty-gritty of how that looks but how again does this help us uh we don't need to run uh the eth2 consensus layer we can as I said we can drive block production some other way uh and then with this we can now reuse uh com production ready components as I said like G and WTH although I guess reth isn't production ready uh G is obviously well used in the community and that as mentioned before handles all the transactions State Management and evm and all the the fun stuff and then maybe this means we don't have to F gu anymore gu can remain as it is uh with a nicely defined interface doing exactly what it's meant to do Lighthouse is one such uh consensus client but it also says the library I didn't view prism I assume it can also be used as a library but I like rust uh the lighthouse uh like library or application wraps the Json RPC API with a super easy wrapper we can even write this ourselves if we wanted to uh and we can use that to drive block production uh with lighthouse we can also use use it as a library to do some pretty other interesting things uh so for example in our client we're using the same BLS signature um signing algorithms um and we could do aggregation and data storage we can also reuse components from the lighthouse as the library uh using level DB for State Management uh so here's a nice succinct bit of code here basically again using the engine API to do exactly what I've said before uh we're telling it to update the fork build a block on it and then import it but we can see something at the top here we see these things called payload attributes and uh to go a bit further into that pay attributes is just defining what what we want gu to do so we have quite obviously the time stamp uh we have uh the Randal which is accumulated Randomness we need to submit and we have two things uh for I guess printing money as I say as I say here prin a go uh so the withdrawals is basically what we tell gu to Mint uh so to speak and this is kind of a hack I suppose in eth2 jargon actually this is what validators used to exit the system uh if you're not familiar in E2 there's a deposit contract which uses to lock lock up their eth and that's locked forever so you never get that out the way they get around this is by adding this withdrawal mechanism into gu via the payload attributes so users can withdraw their funds uh back to the uh the execution layer I suppose uh the one trade-off here is that the minimum Precision is 9 decimal so the interface is assumed that that's Inu which means if you are building a custom blockchain which needs to have uh smaller representation uh it's unfortunately not possible but it's kind of hacky how we get around this and that's just unfortunately the way it is uh we also have the fee recipient that's Set uh at the consensus layer uh to tell the execution layer who should receive the fee the block block fees um again that's uh uh that's based on the Genesis um so we like it might depend how those fees are distributed uh and also with the execution to consensus interface uh that interface isn't strictly defined so we aren't always expected that that fee recipient will be the one that receives the fees that actually may be overridden which is why you'll often see it written as suggested fee recipient uh because gu can uh have a configuration to override that but typically if you're setting up the stack Lo locally perhaps via the kosis package uh those components are well defined together and that won't be um overridden so this is kind of how we get away uh with kind of uh minting money if we were to write a new consensus layer using the execution layer the pre-existing one rather the issue is then we can't inject inject transactions which brings me to my next Point upstack have forked G I know I don't like Forks uh but they have introduced a few good mechanisms to help with this so taking advantage of the same kind of architecture uh they've added additional payload attributes so we can see in the snippet here they actually add uh transactions to be directly written to uh gu uh we we have a flag that tells the GU not to use the transaction pool this is particularly important for the way oback works uh it may operate in one of two modes either in sequencing or verification mode I guess and uh in order to distinguish those you basically tell it uh when it's constructing a block via the payload attributes not to include transactions for the mle so when when sequencing for uh the layer two and opack you say you tell it to includes transactions because you want to include the layer two transactions but when you're verifying simply that um everything submitted to the batch imp box on the layer one is correct uh you'll tell it simply to construct with the predefined set of transactions in these attributes uh it also defines a gas limit uh so I mentioned this is a a fork of Guth one nice thing is re actually has compile Flags so no more code to maintain it's all in one place uh and makes it super easy and uh so I want to now just spend the slide kind of defining what we're building here on Bob and why this is relevant so we're building a few different things so the reason actually I kind of went a little bit into oback is we're building a L2 rollup um with uh Bitcoin in mind using the vanilla upstack uh we're actually intending to add uh a new uh protocol based on merge mining to that um but right now it's just vanilla we're also working on a layer one Bitcoin side chain with a hybrid consensus mechanism again using merge mining uh this is based on current standards for merge mining including Ox namecoin oxal uh it uses new consensus well an old new consensus mechanism Aura which you're most aware of which I don't think works that well so we're planning to move to qbf soon uh and in this we use an unmodified version of gu which is super great because I don't need to touch anything to do with the evm it just works out the box and I can focus on the consensus algorithm uh and as I mentioned both of these will be merg mined with Bitcoin main net uh one interesting challenge I came across when building uh the layer one specifically was sort of hacking fees so we're already kind of doing a few hacks with the withrawal mechanism um but we actually did have an additional challenge which I would like to summarize so we had a requirement that uh with the merged mining protocol we were working on that the uh fees were distributed uh to The Miner when they submit the ox pal so not as with the payload attributes it's intended that each block you generate you set the fee recipient and they receive the blocks the fees from that block rather um so that's the the problem and we got around this was actually to burn all the fees so on each block instruction we would send all the fees to a burn address some contract or the dead address I guess um and then what we would do on Block construction is actually then fetch the receipts uh we would accumulate the fees and track those and then reimport via the withdrawal mechanism um so it's it's a bit of a hacky workaround but it worked for our use case um and just sort of summarize my personal views again too many Forks I think I've not said this enough uh I hate them and I want everyone to use uh more uh well-written codes and like the same code uh there needs to be more cohesion um development in general I think on E ethereum is limited to the eth2 effort uh Innovations happening at smart contracts and I really want to see uh more Innovation happening further down so this is why kind of my Outlook is uh to make it easy to build new maintainable l1s I want people to work on new consensus algorithms just as we have using and tested components uh like gu or W uh and go to market with that uh ideal would be if to there was exist a toolkit such as substrate for ethereum that could allow developers to quickly spin up a new network again using ref or GE uh and just work on the consensus uh I finished with a bit of time to spare which is great uh thanks for listening my name is Greg um please follow me on uh X I guess or Telegram and we're hiring so if you want to work with layer one consensus on ethereum uh please either speak to me after the talk or scan this QR code um and it would great to talk to you thank you again