'Catalyst - an Eth1 engine for Eth2' - Guillaume Ballet

uh guillaume is part of the go ethereum team at the ef and also is focusing on scaling and he will introduce catalyst or give us a talk about catalyst and eth1 engine for ease too welcome guillaume hello thank you uh wait i need to share my screen you went a bit fast uh here we go can you guys see it yes awesome uh yes so indeed i've been uh working for the guest team and they they asked me to uh to look into how to reuse this code that has been developed over the five uh the last five years maturing and and getting more secure uh how to use it into this new adventure of east 2 which is going to be a lot of new features and therefore more risky uh so i looked into uh into building uh building something that has some software that i call catalyst it's a reference to math mass effect oops that's a bit early um and uh yeah it's uh i actually don't quite know mass effect that that much but since geth where the bad guys in mass effect i i picked i picked that name as a way to accelerate uh the easter transition um so yeah i just uh mentioned uh briefly the the word east one engine uh so what is that compared to a east one client uh danny danny ryan two weeks ago i think made a very nice presentation where he explained that um east 2 is more focusing on like he's splitting a blockchain into several layers and uh he says like um east 2 is more focused on on the consensus part the data part and not so much the execution part is left to um to let's say to the execution engine engines so um the end result is that you have a lot of data and es2 really doesn't know what this data is about so it needs some kind of helper that knows how to process that data and this is the role of uh of catalyst or of any other eth1 engine so what it does is uh it keeps a local view of the chain and it also keeps the transaction pool and uh yeah the east east one chart client which is actually an easter component is connected to the eastern network and communicates with the eth1 engine via rpc um so they're still at the moment at least in catalyst a connection to the eth1 network which is used to update the transaction pool um so the transactions still come from the from the old uh from the old well the old network um and uh yes that's let's see how that enrolls so for example in the case of a block proposal you have uh the like the the client the e2 client determines it's uh it has to create a new block it determines that from from the connection uh to the from its connection to the network and so what it will do is contact the ether1 engine to say hey propose propose a block and what the eth1 engine will do it will look at the transaction it's got in its pool it will select some and use them to produce a block and it will add it uh where uh where the eth1 engine sorry where the the shard the client asked it to to add it so it can actually have several forks um because presumably uh because of the way the consensus uh engine works at the at the east to level um you can have several concurrent fork and forks and um the this one sorry the easter client will have to pick which fork but until it has decided it needs to be able to to handle several forks concurrently so once it decided uh like once it created that block it will return it to the short line and the sharp client will um will keep that until it's yeah well i mean presumably will not keep it but it could it could just decide to try several chains and ultimately it will return one block to the to the ethereum network so it will propose one block and so what happens on the other side on the other side of the coin it's the validator the validator receives the block from the network and it tells the eth1 engine okay you know what this is about you know tell me if this block is valid so the eth1 engine will because it knows it receives the block and it it knows what the parent block of that new block is so it tries to insert it in the chain and if it successfully managed manages this it will remove all the transaction that are in this pool and that are already taken by this block and then it will return um like the ghost signal say okay this block is valid and each one shard the client can also collect those responses and decide which one it wants to vote on so ultimately uh the the validator like the the ethernet chart client of the validator will send a a message sorry a vote on the on the eastern network uh yeah so basically that's uh that's roughly what catalyst is um it's some hack of geth at the moment with extra rpc calls and uh some like block propagation has been deactivated we built a demo to uh with uh with michael kalinin uh from from consensus and uh we uh so uh there's a simulated uh eth1 chart client that uh runs and communicates with uh with catalyst and we created several instances of this process so we could we can see uh the shark client telling catalyst to create a block catalyst producing this block it's being sent over to the other machines the other machines validating this block so it was really uh i mean it made us very happy we're so check it out there's a youtube video uh somewhere that mikhail did and uh very very fun uh the only i would say why why aren't we shipping this already uh it's because um we start using like the chains that we use the ethernet chains that we use are created for the purpose of the of this demo there's no historical data and so the the big question is uh like the big question to answer next is how do we answer how do we transfer from the proof of work chain to the proof of stake chain and that's a big question to answer because many things can go wrong um well first there can be a bug no one is is perfect and especially new code is particularly uh uh fragile in this in this respect um and then uh so it would be great to yes be able to if some if a problem is detected it would be nice to have a chain to fall back to because this is a billion dollar ecosystem so we want to make sure we don't uh yeah we don't just flip a switch and break everything um but the biggest risk apart from that are the miners uh miners they have uh they invested uh they invest electricity they invest in hardware they need to get some money back and the problem is um you know the expense of uh of electricity um is uh is good as long as they can expect the next block but if no more blocks are coming or if they cannot expect to be the next person minting the block um as the deadline with the fork with the merge what's called the merge approaches you can um you can think those people are not going to be as interested as spending uh energy on on this block and they can go and have a lot of strategies but the problem is yeah the problem is if they if they uh exit uh on mass it's um it's going to have some impacts on the network so there will be a slow down because the difficulty takes sometimes some time to adjust and of course if a lot of uh miners disappear what will happen is that you can have a situation when 51 percent of the network is is controlled by one sorry 51 percent the hash power is controlled by one entity maybe uh maybe a clique or um yeah but that there's a risk of collusion and we know the validators don't really want to um they're not really interested in becoming uh proof of i mean let me say okay early early feedback from validators indicate they're not that interested in becoming uh sorry early feedback from the miners uh is indicates they're not really interested in becoming validators it's a different kind of business so um so they might try several things to tr to prevent the merge for example uh doing uh reorgs attempting to do last-minute re-work so we are about to switch and then we realize the block we were building on is the wrong one i will i will explain this in more details later um they can try double spans they can try a lot of things and they can also try to uh put a short on the price of ether and then exit and so the block time starts spending so people realize oh the validators are leaving so they dumped the price of ether and they can break in by everything and um and come back uh and be bigger like increase their stakes in the new world so i'm going to cover a couple ideas that were floated around for how to yeah process or proceed with the transition the first one was to use the so it's this one is pretty self-contained it's uh how to use the the finality so uh as you might know uh eth2 has a finality so after a while uh the there's one branch or one block that is declared as final all the other uh branches that do not include this block are discarded and uh and so because the beacon chain on one hand can um can um indicate which one is the proper uh is the proper uh shard block that is uh correct and because before the merge the shard the like the the validators of the shard will be monitoring the eth1 chain you can um you can use this to introduce finality to uh to the to the proof of work chain so i realize i need to explain a bit what this uh what this diagram represents so squares are blocks and um the the left so the parent is uh on the left and the child is on the right and the reason why uh the child goes to uh the arrow goes from the child to the parent is a reference to um is a hash reference to the to the parent uh to symbolize proof of work i have used the pickaxe and unfortunately i couldn't find anything to represent uh steak properly so i i use this terrible pun i'm sorry uh but yeah that's that's the best i could come up with um so the idea is that after a while the beacon chain here will um like will validate will uh will indicate that this block here uh which is in the east is one chart or shard zero has been uh has been uh finalized and because um like the validators of this chain keep watching the proof of work chain uh they will also they can also get this information and discard everything that came before um so yeah that's um that's an approach that has some very a lot of advantages one of them is that like i said it's self-contained so even if uh if ethereum 2.0 stopped there um there would already be a benefit here and the benefit is that you can actually delete a lot of data that is no longer useful this come this comes at a price of course full sync is unlikely to happen you will need to start always you will need to find a trusted source that can give you what was the last finalized bug excuse me and uh sorry that's a bug the last finalized block and um and then you have to to synchronize from there um so this is an idea for example that works great in conjunction with uh with regenesis or even like clients uh but yeah that's that's also something that has some inconvenience one of them being the data download time if you start downloading uh so for example in guest right now you have the fast sync the fast sync like like fast forwards to the to a given block and then downloads the data for that block and the problem is that guest doesn't keep the data for that many historical blocks uh the state for that that many historical blocks so it starts downloading the the state but by the time but at some point the you get beyond those 256 blocks that uh for which guest keeps the state and then you don't have any more data to download so you have to reach like to fast forward to a new block and start the process again and hope that this time you can synchronize uh faster than the than the head advances so you would have maybe the same problem uh with this approach but on the plus side is very easy to implement all you need to do is when you in the real the real code is to say well if uh yeah if that goes beyond the last finalized block don't reorg um so yeah there's this idea that's of course a bit of a bummer but i think it's pretty interesting to consider it nonetheless it's uh simply to never do it any transition uh you just use um you just you like introduce the shard mechanism in the current chain the sharp communication mechanisms in the current chain and then you decide the short zero is always going to be uh is always gonna be the main uh like a proof of work shard and then you you start the other charm shards and uh the advantage of this is uh well um that you can actually leave it to the community to sort of vote uh whether they really want shards or not because if all those applications if all those accounts start migrating to other shards then uh well first you know shard actually brings value which we expect them to but it's nice to have a confirmation and the other thing is um if a lot of big contracts move away um then it's much easier to experiment and tinker and try to try to get the the proof of work shard to transfer at a later time because if you if there's a big mistake uh presumably only like the damage is only uh is is much more limited um so it's got some interesting aspects to it and of course yeah like i said you can you can try to roll out any other any other method of conversion later but at least you let people escape before uh before you start playing with the ship um and uh yeah the other advantage is that if the migration fails you can always um go back to to the main chart like the the boot is still afloat uh there was a project that i wanted to mention uh it's called the east one x64 um that precisely um wanted to like took interest into a short communication um and uh yeah that's uh that's that would be a great benefit to to this approach uh this one is uh kind of crazy but at the same time i i wanted to for for the sake of the argument i'm sharing it uh so the idea is to basically interweave like proof-of-work blocks with proof-of-stake blocks and try to increase the number of intermediate blocks over time this is an approach that is extremely complex but still uh interesting to play with um and uh so the idea is like uh you space out the number of the the amount of time between between two proof of uh proof-of-work blocks the biggest problem so the advantage first is that if it fails you can reduce the number of block and go back so it's pretty extensible um the question is we don't really know when miners are going to decide oh those two blocks are too far away let's just drop that um so yeah and the other problem is indeed like if you keep the pro the current uh main chain approach uh without the finality at least uh you will find yourself adding way more blocks and blocks are you know block headers block bodies are taking a lot of space so it's um there's a growth issue that we expect will be saved by uh will be fixed when we switch to east 2 but in the short in the short term that would make things worse um then there's the idea of the airdrop so you just it's actually independent of any method uh you just um like the idea is that you uh you keep the chain running and you you bribe the the miners to keep mining the chain even though it's going to disappear um and to the way you write them is every since the short zero is still monitoring the the proof of work chain they can perfectly attribute an extra reward when they see that this miner has produced a block like we hoped they would do and this block has been confirmed like has had six confirmations um a reward is uh is given to them on the new chain so even if the chain dies they will get the money they they should have gotten if the if the chain was still uh was still running uh yeah so it's uh it's pretty it's pretty safe as a technique it's not complex and uh you can also like as a user you can um you can try to send a transaction on both on both chains at the same time um like i don't think there will be any problem at least i don't know vision i think the same transaction should work on both uh the biggest problem is really uh was legal uh so it uh like the business of a uh of a miner is to uh like sorry a mining pool um they you know they put work and they get money for that work so that's perfectly fine but when you move to a staking model what you do is that you uh yeah it's like an investment you put some money in a bank and you get some investment back like more money back and that is something that is heavily regulated that is something that requires a lot of kyc um so there's a bit of a risk to to merge both of them um we discussed that like we don't think there will really be a problem but there's a difference between what's uh like what happens for real uh sorry what uh what the like the lawyers say and what the the mining pools think so there's a bit of a need to to clarify and to make sure they don't panic and and still accept that scheme um and there's the last uh method which the the one i call yolo and the idea is simple you you just you just fork in um and you hope for the best um so that was the let's say the the sarcastic way of presenting it but uh in in reality it has good chances of working it's it's really not that bad and the idea that is still being uh it's it's pretty fresh um but the the idea is that at the given block uh the validators um you know like the validators keep following the proof of work chain and i mean and they are validating the the shard number zero so um when this when you reach the predetermined block what they do is uh if everybody agrees with the what the last uh what the block number n was uh they will simply register this on the on the blockchain and uh sorry on the on the shard and therefore bootstrap the the shard with activate or um [Music] how do you call it the um launch like launch the the execution engine and because they they do this so basically they ignore everything happening in uh in the in the proof of work chain anymore uh they they launch it and they keep running and they keep running the execution engine um one thing they can also do is to uh is to basically uh also send that information so if the the chain uh the proof of work chain has and has a client that is able to understand a bit of what's going on in east 2 you could actually share the information that the the chain like the the execution engine on the eth1 chart has been bootstrapped and if so those uh those miners can keep uh producing empty blocks so they get that signal they know it's they only produce empty blocks so that there's no competition uh between the chains and um and yes afterward we use the same technique as before which is the the air drop or the payment and oh yeah one thing i forgot to say about the payment is that it could actually be done by uh instead of being done through the protocol it could also be done by the ethereum foundation for example which i think offers more flexibility because we know some people i mean it happened with the serum classic so uh the proof of work chain could become some kind of original recipe ether ethereum and those people might want to be paid in uh in original recipe ether um so they might like the human method even though it looks a bit more centralized uh i mean this centralized in a way um it would uh allow people to you know who really want to keep mining this chain to be paid into the the currency they care about but yeah um all right and i'm going to finish with the next steps um so what what remains to be implemented uh the finality i think that's the that's the lowest hanging fruit uh i started on that um so uh yeah that's the next the next delivery then there's the idea that uh i i alluded to in a previous slide be uh like let the easter client understand a bit more what is going on in these two sorry the east one client understand a bit more what is going on in the east 2 world um and maybe um like get this information even before the switch before the merge that could be used for example uh i know vitalik has [Music] made a few proposals where you could actually um try to read the beacon state and and do uh and do several operations even before uh the the whole phase one and phase two have been launched um do a bit of the same thing that shards do but through a contract on on the east one chain so there's a lot of uh yeah it would offer already uh an avenue for people to try um to try their uh or experiment with the east 2 ux before they they actually switch and uh yes so the last thing that remains to be done is to to make a regular release have a standard release process and uh and yeah that's uh that's all for me wow thank you that was a very insight for guillaume um a quick reminder to everybody if you want to ask the speakers so in that case guillaume a question please do put it in the chat so that i can relay it to him um the chat is on live.esonline.org and i also have a question for you already what has the process been like to communicate with miners and gorge their sentiment around the project and around the different options right a very good question um we are still looking for miners to so if you know any minor please uh that would like to answer those questions please contact us but uh so far we've been i mean i've been to uh to uh the the discord channel of the of go with her that the guy theorem discord channel try to find a few people uh but i'm always happy to talk to more of them because clearly uh there's uh yeah i only talk to a few so i'm more than happy to to expand my horizon okay then another question um will there be any sort of a test net that the public can join uh there will be at some point uh i mean the idea uh we haven't really discussed that yet um but from what i from what i see uh you could um yeah you need you need a fork of gurley i i i think it would make sense to to make a fork of gurley and try to connect it to whatever easter test net is running he's running phase one um so or we could just use the simulation like we did in the previous prototype and uh and use a fork of gurley so yeah there will be a public test net uh i think it's important okay cool and then lastly um is there any way for outsiders to get involved in catalyst uh prs yeah i haven't created that's a good question i haven't really thought of that because so far catalyst was a bit of a prototype um but yeah you can you can contact me i'm sure i'm sure there's i mean there's plenty of work to do so no um i yeah i give my branch you can always open pr's against my branch or you can contact me and i'll find i'll find you some work to do no problem amazing cool thanks so much kion thank you and