Scaling Ethereum Summit | EIP-4844 in 44 minutes (or less) - Protolambda

foreign [Applause] [Music] next up I want to talk well I want Porto to come on and talk about eip4844 and that there's a lot of 4844 stuff going on here uh some of you may know how this helps us get you really really cheap data and transactions and there's a lot more that people don't talk about and we're going to take about 44 minutes maybe less to cover all those specifics so we'll get Porto to join us in in a second as we get to order the slides and in the meantime if you have any other questions that come up for Proto or Jesse or anybody else please let us know and we'll be able to kind of relate those questions to you and to the speakers and get those answers so without further Ado let's welcome Frodo and hi how's it going it's going very well thank you for interview for the introduction absolutely um profile I work at op lumps foreign okay so let's jump into this Ford prefer is an EIP it's about the year of Old Town to start it during Eve Denver and uh well you get started with the actual science this works moments there we go where it's technical issues it's like a curse starting with the presentation of golfing um so hello I'm I'm Proto um worked up elabs proto-dink sharding is kind of a meme it's this this name that's been attached to 44 what it really is it's the first iteration of the sharding roadmap of ethereum then uh this like this long-term vision is like inspired by the work by modern carts and by vitalik in other research team um however we cannot wait we will want to leave on the starting on ethereal today like these are multi-year research processors and so instead of trying to do everything at once we as Engineers we are looking to build this first iteration and this is a collaboration between OPI labs coinbase as well as all the different layer one client teams and the research team of course with Foundation that's putting together resources to implement the transaction tool changes the The Blob storage the syncing everything that's involved in this first iteration of sharing and so what I'm going to tell you about is really like what is this thing what this word for Fern why are we doing this like how does this work on layer one how does this work on Layer Two and then I'll talk a little bit about the fee structure of virtual firm and about the clients in the progress with like made so far so what does for eip4 do if it first for foreign increases the data availability for layer 2 to about half a megabytes per layer one block for context the current data availability is around 75 70-ish gigabytes and so this this jump up and that's availability increases the capacity that is there for Roll-Ups to post their data to if you've watched the talk from Kelvin about how these Roll-Ups work then you are now familiar with the concept of a data fatability layer the data availability layer this kind of hijacked on top of the Legacy ethereum we use call data which is meant to be in inputs to the evm but instead of using the score data what we intend to use is block data loop data is like special specialized resource so we can have more of this without putting a lot of more strain on this these layer bomb nuts and as you can see roller popularity has been rising like around and 2020 is when the idea of a roll up Centric roadmap starts to really take off around the launch of the beacon chain and since then the the rollups have taken more and more of data of layer one so half of the current capacity like half of the data inputs to the evm are not actually inputs to the evm their rollup contents so we want to change this and optimize so we can have more of these contents because why well 97 of the fees on layer 2 today are actually just paying for the call data on layer one and they're basically misusing this resource this resource is not efficiently utilized to derive the layer 2 Chain um well this is kind of what it looks like so this is where transaction fees are going and the thing we want to try and optimize like this could be this total could be much much smaller if we tackle the the largest cost Factor here like also Layer Two we're not asking a lot of fees it's primarily just data fees and we'd rather have a lot more users that can adopt ethereum through layer 2. and we can do this on by utilizing share data to host all the other content so this is what layer one looks like today merge is complete got to remove one slide leading up to this uh in like the modular blockchain stops we have a consensus layer and execution layer now that it's only the start of marginal blockchain but we are looking at today if we are ready it's kind of this hack where we have player 2 and it's attached to the core data of the ECM there's no clean separation and although we have this these fault proofs and functionality proofs to verify outputs of the evm we kind of have the strong coupling idea or like how our wallet should be secured it's not efficiently attached to to layer one and so what we want to do with 44 is to model it like this where we have a consensus layer we have an execution there and there's this layer on data which technically speaking it's parts of the consensus layer the consensus layer holds on to the data and thoughts on which data is available or not and then through the execution layer the data is registered and paid for but the status really meant to provide for these these Roll-Ups for layer 2. to scale and this is only the start like for it for fur is a is an incremental step towards Phil's Charlotte what we're looking at after first so far is Fielding sorting which basically means we can support a lot more of these data plumps and how do we plan to support this this can be supported with data availability sampling that just this this security technique for validators on the network to randomly poke holders on the network to see if that is available and the the poking here the sampling can be dramatically improved using redundancy and using these efficient proofs of the data so I'll talk a little bit more about how these blobs are structured they are structured in a very particular way where they are future compatible with this full sharding vision okay so how does emp4804 really work now there is this new blob transaction type and the blobs that are paid for these this this contents the data is held into the big note in the consensus layer for two weeks approximately and what this means is that B separate this ID of syncing and is ID of security syncing you should be able to fetch the layer 2 state from any place just as with layer one the critical part reduced this last two weeks of data and so as intuition you could think about a CDM or some surface where it's much much cheaper to provide hosts and the cache the data and distributed if the data doesn't change versus changing data and similarly like this this recent amount of data it's more unstable it's still being folded on it's in certain the layer one might propose it but not actually make it available to those that rely on it and what we want to secure already is that any verifier of layer 2 is able to get the data and then able to use the data on top of this this older state of layer 2. to get the very latest State the tip of the chain and then using the later States they can produce any like with filled data they can produce any proof that they might need to counter any false claims on their own or to produce a validity proof themselves on layer one and this is what secures the run-ups it's really this this recent data that's important to make available and then the older data we can optimize and so this task of ethereum changes to how can we make the data available to this these layer 2 networks that rely on it to secure all their their full assets Etc this is a this is special data it's not just that they cannot be can use regular storage Services it has to be available and this availability I'll talk more about this it's just a strong feature of sharding where this this type of resource that ethereum was lacking is dramatically increasing and then you ask well who's paying for this resource how does this work there's this new transaction type and similar to how you can run the evm you can now also attach data to ethereum to make it available and what this means is that there's a regular ap1559 transaction and the transaction contains some pointers and these printers they print to these separated blobs and the idea here is that the transaction registers the fee payments and is processed by the ethium where's the blobs they are separated and they go into the consensus layer into the beacon mode so here's the transaction in more detail the nice thing about this transaction is that it's structured using the latest encoding formats as you see that the beacon chain also uses and then the functionality is mostly the same as with erp1559 it's just strictly a small extension to express the fee payments and to register the data and so there's this Max fee per day the gas that's edit and there is a um a list of hashes that refer to the commitments that then refer to the fellow contents of the of the blob and the the pretty thing here is that the data hashes they are easily embedded into the transaction they don't take a lot of space they are 32 bytes long so they fit in the EFM very nicely whereas these commitments these are 48 bands they go into the big concern and they are kcg commitments kcg is this special commitment time type similar to what the Merkel 3 does except instead of a miracle proof you have an opening to prove any particular points or multiple points in this set of block data and this allows you to build an efficient proof for the contents of The Blob then you need it in the evm but otherwise basically optimizes away the the proof data and this is this is the key part for filtering sharding for this full fission of data availability sampling to have this type of commitment in there and so what this really means is that the transaction goes into the EFM like At first it's generalized it's abstracted us this message just like any other transaction the EFM doesn't really know about transaction tabs but this additional Ripper data the commitments and the blobs they don't go into the EFM they leave the execution layer and they go into the contents layer so how does this lifecycle work right I'm just layer 2 user I want to utilize the sharp data what does this mean though you just use ethereum as you would use it today it's a lot like just sending a layer on transaction except it's on Layer Two it's on our rollup and then the roll up operator or the sequencer the bundle the transactions and then the transaction bundle well that really happens there is that needs to be submitted to layer 1 to make it available for verifiers to see and like reproduce the layer 2 stats and so this bundle of transactions it's encoded it might be split across multiple layer 1 transactions but really the key here is that the data is encoded into blobs and these blobs base form with blob transactions go into the layer 1 transaction pool and then the big contain proposers can pick up on transactions from the transaction pool they can build their exclusion payouts and then along with the execution pedals there's now this new addition of a blob sidecar this is additional data that comes with the beacon block and there's a separate syncing thick syncing methods to distribute the Sidecar and the beacon chain distributes the data and then everybody on the beacon they alternate for a certain amount of time so they hold on to that for like two weeks maybe a month depends on the exact parameters in Eep voice for firm and then the execution parallels stays in there one with the feed payments but then the blobs eventually get pruned from layer on meaning that the resource usage from there one is bounded bounded resource usage is like this long-term goal of ethereum where we shouldn't always keep drawing the state and growing the chain history the event is sustainable and so this is a really important feature of 44 and of sharding in general that once you start adding a lot of shared data that we don't rapidly increase the resource requirements of regular layer 1 Lots and then it's for up to the layer 2 to persist data long term and this is not at risk anymore after the the proofs that security assets assets have already large skipping completed let's then up to the layer 2 Community the stakeholders of that that roll up that's the like the now fully agreed up on their two states is isn't there for others to sink and join this this layer to ecosystem similar to but we do another one so this is what it looks like in terms of the network I have to see concern and this transaction pool if you could think of it just these clouds of suggestions for the near one proposer to build a perfect block then the blobs and execution payloads make their way into the beacon chain and then the other nodes on the network they they verify the fee payments and the verify the data star but then the data is retrieved by another layer two nodes the layer 2 verifier to reproduce the layer 2 States using this data as an inputs to the robot again parts to talk about golfing to learn all about how the Roblox processes this data now how does this work between the layer one consensus layer and the layer bomb execution layer we used to have this API called the engine API and we have the same API right now except we are adding one method and one type we're adding this blobs bundle so that everything that comes with the transactions can be displayed for better transactions which then downloads together and separate it from the regular execution pair Dot and so the consensus layer can retrieve the execution payloads to embed in the beacon block and can retrieve the the bundle or blobs the Sidecar to then distribute two other bigger nodes for the the periods that data needs to be available so we talked about like how 44 works on there one and to complete the picture we also need to talk about how ep44 works on layer too because this data would be kind of useless if there were no all these many different Roll-Ups building ecosystems extensions of ethereum on top of this data and so what it means to update layer 2 it means to update the proof so that the inputs that we now change about the layer 2 make their way through the layer 2 State transition to produce the new layer 2 outputs this transition has to be proven on layer one and then we change the verifier cards to actually pick up on this data to process it in the first OS and we change the bits emitter to even submit the data before the verifier can pick up on it so it's really bad symmeter first then verifier and then the proof shows the full transition of processing blob data running the layer to stay transition and then producing the outputs that then is claimed or validators on their own so show a I'll show this simplified ZK validity Prof Zig overlaps they they all work a little bit different but if you generalize them a little bit what it looks like is that there is some form of computational layer one that verifies business data to show one one state roots and claim going to another claim and the proof shows that that this transition is felt so how this works is that they first want to prove that we have the correct data in our system and since not every roll up is every Z guy run up is built with kcg but some will have to do is this proof of equivalence to show that the case G commitments over the data is equivalent to the commitment that they prefer to use luckily because this is a polynomial commitment this can be done rather efficiently as part of the secret proof in a way and so there's this pre-compile that allows you to evaluate a a polynomial do a single parent evaluation and this can be used to do a random evaluation and run them in a way that you can securely verify it against another evaluation in a more another different kind of curve or some system that the zigarvelop refers to use and then once you have verified the inputs you can run your regular ZK uh validity proof using the the same form of inputs that the zika roll-up is already built for and then produce the the outputs or verify the outputs really and then all you need to do at the very end of this is to just persist the the outscope now um we also have optimistic grounds and they work differently however they use the same pre-compile so the additions to the layer on evm then we want to verify blob data for optimistic run-ups or for secret rooms they are the same if you can have this common utility to verify a single point of blob data except that we use the precompile slightly differently they're like an optimistic roll up they have the notion of this pre-image Oracle and the previous Oracle is used to access any history in the layer one and which could include globetizer and then depending if it's a regular hash or the special case D commitments the user hash function to verify the pre-image matches the the claimed hash or we use the parent of elevation pre-compile to make to verify a piece of blue data at a certain position matches the commitments to the set loop data and this this extended previous Oracle we can plug into the existing single step verifier and so it doesn't matter if you're like a an awesome proof system or a mips proof system or risk 5 proof system if you have this General ID about loading inputs through premature verification you can plug in just new type of pre-image effects web Loop data so this is kind of what it looks like to integrate this into the verifier so we've talked about the proof part of layer 2 we also have to modify the verifier and then the batch Smither is really just the reverse system of the verifier in the talk of coffin you you have seen that there's this input stream and there's output stream now this is the process that takes the input stream and converts it into the output stream in the op stack it's called the layer 2 verification pipeline and what it really does it traverses the their Mountain the chain of inputs it retrieves the data it processes the data it might use some layer 2 execution engine to process the data it's really only the retrieval that we have to adapt to this new type of data and then the batch emitter which does the reverse process then it takes a layer 2 block that has been sequenced it is going from layer to block it extracts the inputs and calls the inputs in a batch and then the batch needs to be encoded in layer 1 transactions and if layer on transaction is published now the plot data rather than call data okay so we talked about how layer one works down there two works if you really want to slot them together we need the layer 2 operator to pay for this resource of layer bomb this new type of resource and what this requires is new type of gas metering except that I wouldn't even call it metering in this burnt because the evm does all these complex computations whereas data is just static it's very easy to meter would look at the amount of data we can translate it to the amount of data gas that's required to pay for the data and now this data gas it works very similar to regular gas very similar to ab1559 where we burn the defense we burn the fees if there's an excess in usage and then in the blockadder of the execution layer 1 block the maintain a special variable that can track what kind of access there is in EP 159 we know this as the base fee the base fee goes up if there's an excess it goes down if there is not the excess you could express this in many different ways it doesn't have to be a like a a floating Point like variable that moves around it could just be more discreet and say you could say oh this there's there's this much gas that's being utilized over the expected amounts and so what this looks like is that the more excess there is the higher the price goes and notes the the y-axis here it's a logarithmic y-axis and so exponentially the cost goes up if there's more excess usage and so it balances itself out if the access usage goes up prices go up and then either users are willing to continue to pay elevated prices to continue to use the target usage or the common sense and they start using less for a little while and then the prices go down again and so the the base fee in ebm59 or the excess data I guess in AP 45 help balance out the um the resource utilization of the chain so we Target have and we affect maximum capacity of twice the target similar to eBay except now the data gas other than ethm gas out is how this is expressed in the Erp it's a little bit harder parse because the formula is different but it has the same properties so at just at the same rate the cp1559 a rate of 1.125 and it's it goes up exponentially this way or it goes down exponentially this depends on the resource research and the the limits is twice as much as the the targets and the only addition that we need to maintain the state is this one field Fields addition to the look atter to attract the success okay B things are going well so far we've talked about how fresher fireworks both in layer one in layer two out of fee structure works but if you are interested in like diving deeper into this to really understand it you should look at the specifications this is an EAP that spans across multiple layers of ethereum so there are consensus specs and then there are execution specs the consensus specs their schedules for their hard work they're isolated by hard work whereas in the execution specs in our EOP system they are directing just specific features of eips now there is a new exclusion specs process that describes the evm in Python expressing further I think will take a little bit longer that the specs are completes the Erp describes the execution changes and the constant specs have all the constants there Beacon changes and then what would things be if there were not other specs repositories so we talked about the like the state transition changes with consensus and exclusion there are also apis that have to be extended so there are the execution apis to support a new transaction type there's the engine API to pass along the blocks or sorry the blobs between the engine and the big amount and then there's the beacon API which has been modified to fetch the blob data this is the API that there two clients verifiers would now start to use to pull the diploops from the beacon knots into their their two notes and then they can cross verify it against the data hashes that have been registered in the transactions that they already pass as part of the the regular will not be traversal process now implementing all these specs is not easy that we're getting there we have five different constant Slayer teams and five different executionary teams actually more than this even implementing the specifications and participating in doesnet so we started with just prism and Gaff as we prototype a year ago during Eve Denver and since then we have improved the syncing methods improved the transaction pool they were still polishing various different things about the Erp the specs are stable and so clients are moving forward and now we have all clients participating on the same testnets we have this client interoperability and then it doesn't stop with just client modifications part of erp44 requires this new kcg commitment to really be forced compatible with the long-term sharding vision and this vision requires a trusted setup or the sketch G ceremony outputs and this thing has been running for like almost two months now I believe and also there are about only two days left so if you still want to participate there's still a chance um this thing is hosted online you can use various different clients to participants um showing you this this server graph because it's kind of insane that so far we've had over 70 000 contributions and when I was sharing about 44 just two weeks ago I think we were at fifty thousand this number is rapidly increasing and the more contributions we have the better because the the more we distributes this responsibility over like not refeeding all the the inputs to the ceremony and just one of the inputs has to stay like private hidden select toxic fast to secure the final output and then the final outputs is used to secure the data effectively and there are various fun clients there's not just the official clients they're also like meme clients so there's this front end that is a dog burst current teams I recommend just browsing around the casidy clients and to try and participate in film new guys and after this like two months of regular participation is offer there's also a special time allocated for teams and participants to do special contributions to mix in their randomness in a different ways to create more diversity to reduce the risk even further but yes like 70 000 participants scam of crazy this is like already 10 times if not 100 times more than most ceremonies are getting okay let's talk about progress so this Erp has been around for almost a year and since like last year in Denver we've achieved a lot we've worked some prototypes we've worked on the Erp drafts then transforms into more specs than other workshops more education definite started this is something where I really appreciate the help of coinbase contributors and like other early folks like Terence from Prism who like put a lot of time into these early prototypes to show that it works to show the viability to explore optimizations but now we're at the points where we're running defines more regularly and because the gym ceremony is almost coming to an end and in definite and sorry in Defcon we started definite or Sergio after Defcon you started definite free and then earlier this year at the interrupt event there was this definite that we put in all the clients uh where like you can see that it it's viable to ship in a hard Fork there's this interoperability problem with many AP changes where not all the clients always implement the same thing the clients are agreeing on the same blob data and with more testing more polishing we can ship this as part of a future layer one artwork so currently this is scheduled for the dev artwork which is the one after the withdrawals right now we have various different testnets behind us started with version zero the hackathon then there were very different death notes and increasingly we have been extending this from prototypes to something that's like adopting all the spec changes and the latest features as discussed during this interrupt testing and the latest features really there are the transactions the web the transaction pool hardening and a decoupling of blobs amongst these things are finished up then we're looking at the final stretch the final like Milestone towards like a production ready for it for and hopefully you can go live with the next hard Fork after the withdrawals so deflat 5 is coming up let's updates the transaction file primarily this is the same so much of a breaking change um the update here is that we're going to perfect the way the structure as a c transactions and get it right um if this doesn't work out we might refer to an ROP transaction but right now there are strong proposals for SC transactions to shape the way like we sign over transaction data in a more structured format which will be very good for day or two long term to prove any data about any transaction and then we have some Network optimization decoupling of blobs like what this really means is that the blobs and the beacon blocks are gossed separately and so the smaller the messages the more effective gossip sub is at propagating the data efficiently and without duplication so this all saves the smart bandwidth and then we have transaction pool hardening meaning that we don't want blobs to have a negative influence on the regular ECM transaction pool so we're trying to isolate the properties of these block transactions as much as possible I have a really strong implementation that can handle the data and so this likely will be like a pool on the mechanism where the avoids putting in large amount of data if we already have the data and then there's more testing there's always mirror testing if you want to contribute this is probably the best place to contribute we are building more test vectors there's work on hives interferes other things to Benchmark first or for to make sure it will run well on layer one so if this is still not enough for you look at the ep4h4 pluscom you have a question and answer section uh with vitalik answering things about foreign different resources you could read more abouts for fur and then if you want it gets into contacts if you want to really contribute actively to the implementation to like analysis of the cascg ceremony or like other post-processing or perhaps you just want to visualize the blob data or maybe you want to contribute to the new blog Explorer that's visualizes blobs either all these tooling efforts as well it's not exclusively client developments um during Defcon Rosa team building a a block Explorer like an experimental block Explorer specifically for this data and during the Eve online hackathon I also hope that more teams can build cool new interesting tools that utilize this new concept of blob data so yes any questions there's time lapse we can go through some things let's scale ethereum together I'm looking forward to everything that's being built during the second one thank you amazing thanks for now uh oh there's a lot of time left for questions but uh you were surprisingly on a 44 minute Mark so if I just verify this again after we shot this video but it was actually very informative and I think this is the only resource now that exists on the full depth of what 4844 is so thank you so much and this has been awesome yes my pleasure thank you for hosting this absolutely take care