Qi Zhou (QuarkChain) - Scaling Ethereum L1: Achieving 10 GigaGas/s EVM Execution with Block-Level Ac

Hello everyone. Um, it's my great pleasure to hear to present uh our latest work on Ethereum scaling, how to achieve 10 gigas per second in execution uh with block level access list. So, a quick intro of myself. Uh my name is Qi. I've been working in this space for about 8 years.

Before that, I was uh PhD and also engineer at Facebook, Google. I'm particularly interested in film infrastructure, especially scaling, and receive a couple of support from Ethereum Foundation including uh Metronic personal grant. So, uh why we want to do this uh work? Um, so we know that Ethereum is aims to scale dramatically by increasing the gas limit of Ethereum layer one. So, the current goal is to target for 100 X uh gas increase uh so that we can achieve about 5 gigas uh per block.

So, which means that in order to validate the blocks in about 4 seconds uh in the uh consensus time budget, um we will require at least about 1.6 gigas uh per second of the execution throughput. However, this create uh concern that whether we are able to handle this uh performance uh using current client, uh especially based on our commodity PC to ensure decentralization. So, here is number that this is we can get that now gas can only offer about 300 million uh uh 300 300 uh million gas per second, and also Rust is a little bit higher, close to 600 but still far from our target. So, why the scaling is so uh difficult?

The main reason is that there are couple of the limits imposed um which both in terms of the disk IO and also uh execution CPU execution latency. So, here is uh my one of my favorite diagram uh which is uh um uh generated by Flashbots a couple years ago. So, about this shows that the the disk IO opcodes like SLOAD and call opcodes takes about 70% of the execution time uh without any optimization like for example pre-warming or optimistic execution. So, and then block level access list basically remove this bottleneck from disk IO to CPU execution. So, Tony give a really excellent introduction of the block level access list.

So, if you need want to learn more, uh I will encourage you to uh watch uh his uh presentation yesterday. So, basically idea is to include what the uh transaction in a block will access uh including its account and also storage, smart contract, so that when we begin executing the transaction, we are able to preload all this uh data in parallel so that we can fully uh utilize the disk IO. So, on the right hand side is the diagram and shows the number of throughput versus the number of parallel IOs where we can show that the right now our disk can easily support more than 32 IOs uh concurrently so that we can easily achieve about 30X performance improvement in terms of disk IO IO. So, that now we can remove those random disk access to uh latency now to the current uh sequential execution. So, um with all these um uh ideas, now we are trying to study how BAL can also improve the parallelization in terms of CPU execution.

So, uh here is our setup. Uh we are running all these experiment on top of Ethereum mainnet blocks so that we can get some real numbers. And we are also running on top of uh AMD 16 core CPUs so that which takes about 100 euro per month uh to rent this machine. And also we are experimenting um so simplify the whole experiment so that everybody can easily reproduce our numbers. For example, all the state has been preloaded so that we don't need to worry about corresponding IO cost.

And we also recover all the senders which is uh very CPU intensive, and but it's not related to EVM execution. And lastly, we do not uh include uh state root computation uh which can be amortized by the uh large blocks that Ethereum is growing. So, the first number we get is to just run all this uh execution sequentially without any IO cost. And so, this is uh the number that we can fully such uh utilize a single CPU 100% utilization, and the number we can get which achieve about 1.2 gigas per second.

And then uh we by increasing multiple threads so that we can run corresponding transaction in parallel within a block, um we found some interesting number. So, we found when we increase the number of threads, we see some performance improvement, but it quickly capped at about four times for performance that is about 5 gigas per second. Um the reason is that um we found there are some of the transaction that can be hardly uh be uh optimized by parallelization. So, there is a famous call Amdahl's law which tells that even though we are able to uh improve the increase the throughput by parallelizing all these transactions, however, um the the whole execution time will be uh limited by the longest transaction that can be hardly be uh paralyzed. So, here is the famous equation that we can see this uh number called 1 minus time unoptimized.

That is the part that accounting for the uh longest uh transaction. So, in the Ethereum mainnet, we also do the experiment where we found that even we are able to increase um the number of the threads to up to improve the total time, but the longest time the transaction uh execution latency just keeps constant and which result in four times performance cap uh in terms of the uh parallelization uh threads. So, how we are able to solve this problem? The basic idea is how we are able to break the granularity so that we can have smaller granularity by two uh directions. One is to improve the block size block limit gas limit which is Ethereum is currently doing.

And second thing is to limit each transaction transaction uh gas limit so that the ratio between these two will be small and which means that we are able to increase the parallelization even um uh with uh with uh multiple threads like 32 threads. And so, this give us a uh really encouraging numbers whereby uh we merging the mainnet 50 blocks try to simulate the mainnet execution, but after that after for example 50 or 100 gas limit increase by merging these blocks so that we are scheduling all these parallelization among these 50 blocks rather than single block, which we can show almost linear scaling in terms of number threads. And the final number that we can achieve is about 40 gigas per second. Which means that we are able to achieve not only 100X but almost 1,000X of performance improvement uh by increasing the gas limit of Ethereum. That is really really encouraging number.

Um, also we do some study on how to uh for example changing the size of the megablocks from uh 1 to the like 200, and we found that right now 50 uh megablock is the sweet point which is uh pretty fit to the current Ethereum scaling plan to increase 100X block size um uh block gas limit. And so, um these numbers are really encouraging, but we still facing a couple of engineering challenges um if you really want to to put these numbers into the production. So, here I will be list a couple of the uh challenges we found during our experiments. The first one is the call uh sender recovery. We found that the sender recovery is significantly underpriced.

So, even we are able to fully for example computing all these sender recovery, but accounting all these recovery computation speed, it just the the performance significantly reduced to about 5 gigacap gas per second. Uh which means that we need a better way to uh solve these sender recovery either increase the gas or either we are able to using other more powerful computation devices like GPU. And there's also other things like for example we do not consider IO uh IOs um because like all the data has already preloaded into the memory, but actually the IO will also even we are able to do it in parallel, but still take some maybe takes considerable time. And also the BAL itself can take significant overhead in the in the block size. So, we need the other ones algorithms.

For example, P2P algorithms of EPBS and also peer DAS can also help to propagate those BAL information with for example pipeline execution so that we are able to fully utilize BAL without worry about the publication latency. Um another thing is like the BAL here is more experiment for the validators, but we are there's also limits from the builders how it is able to build this BAL quickly. So, this is the right now a really open questions. Maybe the idea is using super powerful builders and then using all in memory database all these high techniques, but still a lot of what's the performance limit if here is still open question and probably with our future research work. Lastly, it's about how we are able to compute the state rule for this direction.

I feel a bit more optimistic because we are able to amortize the cost with larger block gas limit. So, here I give a some quick result of how we are able to reduce the BAL size with the parallel IO considerations. So, the basic idea is that actually we have different types of BALs with the larger size of BAL may contains more information. For example, it may contains information of the not only the location that we are going to read, but also corresponding value. Also, we are able have to include a post transaction diff so that we are able to schedule the transaction execution early.

So, there are here we summarize three types of the BALs and we show that the parallel IO BAL which only includes the post transaction diffs will reduce the corresponding BAL size to about 1/3 with the most expensive full BAL which contains all the information IO information during the execution. And so, we try to understand suppose we are want to reduce the BAL size for example to 1/3, what is the performance difference there? And our result shows that by reducing this the the BAL size, we are able to still achieve some pretty good numbers especially I think that the number there is should be enough for our mid-term or near or mid-term um requirements on on the user transactions. So, it's about like achieve a 10 gig gas per second which is about six times of the even 100x of current gas even gas limit. So, the here is the summary of the today's talk BAL and by increasing the gas limit of Ethereum with by also imposing the per transaction gas limit, we are able to significantly improve EVM throughput with more than 10x or even more as demonstrated by our our experiments.

Um Um also, we are found some there's some limitations during this study like how we are able to improve the standard recovery maybe using GPU or by repricing how we are able to reduce the BAL size by using different types of BAL strategy. Also, how we are able to impose the per transaction gas limit by EIP 7825 so that we are able to easily optimize for our performance target. So, Um um that's all the talk I have today. So, I would love to answer any questions you have. Thank you very much.

You mentioned Amdahl's law that the the block is always the execution time is always constrained by the longest transaction. Yes. But is that like keeping in mind that that single transaction is executing sequentially or can you actually parallelize like the transactions themselves or like maybe certain function calls especially if they're like pure, you can actually parallelize them within the execution of the transaction itself? Yeah, that's a really great question. So, it is regarding how what is the granularity that parallelization that we can achieve.

So, definitely there's the idea that we are able to even within the transaction, we can split the transaction and have some intermediate state of each transaction and tell how we are able to early execute the transaction in the middle state in the same time and now we are able to run run those get them further gains. So, but this requires significant modification of current BAL standard because the BAL standard is more designed for the per transaction tells what what which transaction what kind of state you have changed or what kind of state you want to access. So, definitely there can be a future work so that we are able to say if this is really super long maybe a single transaction take over all the block execution and then we want to split it down so that we are able to fully like virtualize the parallelization, but I think yeah, that can be a lot of work can be done in the future research. Is it on? Now it is.

Great talk very clear as well. Um So, you talked about increasing gas limit. I was wondering if there's some effect of block size here. Um as you increase the block size, you're going to increase your communication network overheads. Is that is that something that's relevant or it does it nowhere near being the bottleneck?

So, the question is when we are for example increasing 100x block size, will that become the bottleneck of the for example P2P network, right? Is that right? Yeah. So, actually there are a couple of other solutions that are proposing to address this problem. So, I have some kind of like a my main game to address this problem.

First of all, first of all, we have kind of EPBS. There's an excellent talk by Terence yesterday tells how we are able to pipeline the corresponding the consensus so that now we are able to not only just require 4 seconds timing budget, but much longer time maybe 12 seconds timing budget to block to broadcast the the block and then execute this transaction in the next blocks. So, kind of a pipeline execution so that we are able to increase the corresponding time budget. But this just bring about 3x gain so it's still far away from 100x. So, there are a couple of ideas like for example peer DAS so that now we are able to also pack BAL as a block in the block itself not just the so the block will be encoded in the block.

There's a idea proposed by Vitalik saying that in the future all the blocks will be encoded in the block so that we are able to significantly increase to I think target is 32 megabytes per per block in the future or maybe even more. And so, with that idea so that together with pipeline execution because propagating and sampling is just guarantee the data is available and it's not guarantee I be able to validate the transaction. So, with pipeline transaction that tells that I I can delay and verify those transaction in the previous couple of blocks so that I can still able to obtain a large number of the block data with the much longer latency of course and be able to verify the transaction execution of the block is is correct is correct and then do the corresponding vote. So, there are a couple of ideas flying around to address this idea. But definitely they worth a lot of research work to to make it happen.

Any other questions? I think okay. Yeah, maybe one more question see. Okay. Then thanks everybody and if you have any for my for more questions, feel free to reach out to me and I love to see how we are able to make Ethereum the world computer.

Thank you.

[music]