EPF6 Office Hours | Project Presentations 3

Okay, very well. So, welcome everyone. Okay, bit more people joining. Awesome. Um, so we can slowly get started.

So, thanks again for being here. Uh, this is our third and the last opportunity to uh present your project in from the fellows mentors public. Uh, we will have recording of this of course uh going public. So, um I hope you prepared some presentations. Uh we got uh five people signed up but I got a message from uh uh Vukasin that he suddenly still in hospital and he won't be able to make it today.

Um but coming from the previous call we got uh first in the list. So if you are ready if you're here um can feel free to kick it off with your presentation.

Uh yes sir.

Awesome. Let me just quickly share my screen. Go for it.

Uh, can you see my screen?

Okay, great. Uh, uh, hi everyone. uh my fellow Devon dean on discord uh and I've been working on a project as part of my uh work for the code 6 adding postgress DB support as a back end for the lighthouse be node. So essentially this work uh builds directly on the idea proposed by the lighthouse team and it addresses a real issue in uh production deployments. Um so I'll be walking you through the motivation, design, current progress and uh the road map for the project.

So uh let's start with why this project matters. Uh currently lighter supports two database uh back ends. There's the level DB and V DB. So while level DB is uh fast it is not designed for use over network file system uh and that can lead to corruption in real world setup. VD DB as well has similar limitations uh around concurrency uh and shared environments.

Although this hasn't been like tested extensively but it is implied by the team and here's where Postgress comes in. It is reliable production tested and designed for concurrent access. It also works well over NFS which makes it uh kind of suitable for uh the kind of solution that we want uh for this issue. The core idea of the project itself is to integrate posgress DB as an optional back end that conforms to lights out uh uh core trait abstraction the the key value store abstraction. So this implementation uh is or will be fully as sync uh to use uh SQL X uh for database access and is gated uh behind the feature a cargo feature flag.

So ultimately what this will do is like give users the ability to run lighthouse on on Postgress like they would do with level DB or red DB. uh this helps in gaining reliability and flexibility without changing the core behavior of uh of lighthouse itself. So uh let me go to uh what's been done uh in the first month uh which we just uh passed. Uh so far I've been able to design the tabby schema. I've also implemented get bytes put bytes.

Uh I've also been able to wire postgress back end into the codebase using feature flag. So you can uh select postgress currently uh uh like you would with sever DB or level DB and I've also been able to set up testing using postgress instance via docker. And also you can now test also I I was able to also test with uh uh the common test suit that is used for level DB and v DB and uh postgress didn't give any issue. uh so over the coming months uh I'll be expanding implementation uh complete atomic batch rights uh iterates of functionality I also focus on benchmarking uh configuration integration and in docs uh prepare a proper uh and submit like a proper pull request to let us uh rep I've submitted a a pair already through which I've been able get uh uh feedback from the reviewer and uh from my mentor. So uh by the fifth month, I intend to have uh fully finalized the merge, be able to uh publish uh docs and potentially explore optional improvements like performance tuning or observability or whatever the team thinks is best for me to uh focus on.

So what like challenges have I faced? Uh so first is working with async traits and SQL X. Uh this sort of has like a a sort of complex or should I say strict uh executor requirements. Uh then there's also translating lighthouse batch operations into SQL transactions. the way lighthouse already does it with the other databases being able to transfer that into uh posgress like SQL.

Uh I also need to like be able to make sure that posgress matches or exceed uh level DB and DB in terms of like performance at least be able to meet that before we can do any tuning. Uh and finally uh making sure that you know uh CI setup works smoothly with postgress. Uh yeah how I'll define success uh for this project. uh that would be like that posgress back end is fully integrated is functional and passes all existing test like it's currently doing without much hassle uh that it can be is selected via uh config and feature flag and runs uh reliably even in environments where level DB and red DB uh fail. uh also is that like I I think one would also be that the opportunity that it opens up for like future use cases like metric dashboards or SQL based uh uh SQL based query or like enterprise deployment patterns.

Uh yeah uh so yeah that's it for me. Uh thanks so much for listening. Uh I love any feedback. suggestions or ideas that can help improve the project. Yeah, happy to take questions.

Thank you. [Applause] Thank you, Maria. I don't have any questions coming from me but yeah thank you so much.

Thank you. Thank you everyone.

Yeah awesome. Um yeah, thanks for making the presentation for us. And um so next uh would be uh uh who couldn't make it so we need to skip him and there is Rahul Kuha Raul is next.

Okay, let me share my screen. I think it's visible, right? Okay. So, let's get started. Um so basically what I will be working on is implementing era file support in loadar mainly era one.

Um so let's let's let's see what it's all about. Uh so like a little background check on why we actually needed and what's the first all about. Um so for for us for uh for for a client to uh basically sync from Genesis, it has to get the data from somewhere and the and and CL clients basically serve that beacon chain history via P2P now so that uh people who want to sync from Genesis can do so. But if you don't want to do so then we have this alternative method called checkpoint sync uh which is based on we sub subjectivity we can use that to to sync faster uh but nevertheless uh the problem still is that uh postmerge uh the block size and history like expanded very much and recently there has been this uh EIP called EIP uh 44s where uh the CL where clients don't need to serve premerge history block via the P2P layer. So that's a relief.

So like what's the problem? So the problem is that we can't share this data in a standardized way and syncing from Genesis requires massive bandwidth and storage and also if the client uh if there is some problems with the client uh P2P spec or something related to the uh data loss uh it cannot be recovered. uh we have to reync from uh Genesis uh and also uh with uh four fours uh execution clients need to stop uh serving block bodies of uh after uh that are older than one year but still the data is a few hundred GBs. So one of the solutions that was presented is eraf files. There was also two other solutions.

One was uh portal network which was kind of disbanded by EF and there was there is also another EIP but this is the main solution that is being implemented. So what is error files? Error files is a flat type length value file format for finalized block and consensus state data. We have also error files for different types of data such as blobs or execution data. But uh era one which I will be working on is basically implementing the con beacon chain data.

Uh so it basically covers uh 8,192 slots which is about 27 hours and it can it is also verifiable. The files can be served via simple HTTP or uh torrents and it reduces the need for all the nodes to store the historical data. We can just uh get the files directly from a client and use that. So let's see what the existing implementations uh there are in the market. So uh currently only Nimbus has support for Nimbus and Basu has support from uh support for era 1 files and there is also some reference implementations of Python and NIM.

uh the era files are hosted in this URL which can be served uh which can be served via any HTTP service and also there is this uh script from a core div where we can use a bash shell to basically download all the data itself. So that's very easy to do and also uh my deliverables over the few months would be implementing the reading and production of error files probably a CLA to tool would be great and uh and also we are going to use the era files to serve beacon block by range uh and maybe we can also maybe I will also be working on the aerafile floor blobs but it's still in a draft PR by Nimbus. Uh yeah, I mean that's all. It's not a major feature, but uh yeah, I think it's worth working on. Thank you.

Um any questions from um yeah I would just add because you mentioned that uh there are the HTTP endpoints for downloading the error files but that cannot really be like a reliable source of the uh of the history and um the question is whether you are seeding uh the air files like there is the torrent file uh I have my uh it's like 400 gigs and I'm I'm seing it on my server server. So, um, another addition is that you can just, you know, uh, save the files as a as a on torrent.

Sure. Sure.

It's a good plausible liability, you know, if you use torrent and somebody your ISP complains about too many peer-to-peer connections. Oh, it's not movies, it's the error files.

Yes.

Um, yes. So, we just wanted to add that and yeah, uh, it looks good. Um, thank you so much. Um, any any more questions, comments? Anyone?

Um maybe a question is like whether you looked into the zigg um as part of um uh part of their codebase uh that maybe it could improve performance whether you're going to do it all in TypeScript or you looked into the possibilities of doing something in Zigg as well.

Yeah. So uh the current situation with the zig thing with loadar is that they are still deciding on which FFI to use like there is a discussion between whether we will use nappy or whether there is this other uh system that they might use. So mainly uh the mainly Cayman and all the Chads are working on that and like I I'm working on the library side. Uh but I think that comes a bit later because we are still finalizing on the main uh FFI layer that they are going to use. So yeah, I mean if we have something like that then there there is obviously a chance for performance and

okay cool. Yeah thanks for that. Uh okay thanks for um if there are no more questions I think we can move on then. Thank you.

Yeah thank you for your time. Yeah. Uh, next up we have uh Jimmy and the team uh Jimmy and Aman and Retesh. Um I'm already presented. So if you guys have uh Yeah.

Yeah. Uh I I I will just add a few words in in addition to amount presentations. Uh so I I previously uh I'm interested in the red uh CKVM benchmarking uh uh uh issue and I I submitted a PR proposal but then realized that the uh amount and also Ret is also working on it. So we are going to to join force together and we each of us will will pick a different CKVM uh uh to to work on and uh we actually greater has already done quite some work on on the benchmarking on uh risk zero and SP1 and so we need to work on other CKV uh VM only but uh like we are still looking in into the source codes and and trying to uh see see how how they actually put the state transition functions in into the uh guest in the C CKVM. Uh initially we were trying to work on uh OpenVM seem that there's the most uh documented well well doumented uh uh uh CKVM but then we realized that the core team of open VM is going to work on it.

So we can pick other other CGVM and I myself is going to uh pick uh brief pickle and then Aman and and Retish will like we'll make a decisions later. Yeah, that's that's the that's the updates on my site. And oh, by the way, I have to thanks a lot on another fellow Jun uh Jun Song and also Yuan. I'm not sure how to pronounce his name on on their works on doing the CKVM benchmarking on on rim. Uh I also study about the source codes and and learn a lot.

Is is Retish here? I wonder do you want to add a few words if you are here. Okay. Yeah, that that's my update. Okay.

Thank you. Yeah, I have a question. Can you hear me?

Uh yes, yes, yes, yes.

Yeah. So, um since the repository is not public yet, I mean, what is the progress that you made on the grantine uh codebase? I mean I would be interested to know uh or like learn from your ideas or understand something more than so I'm working on the rim side but I was curious and also when is it going to be public so that I can also read your your progress.

Okay. Uh I I haven't done much progress on it yet to be frank but I don't know like when they are going to make because that's not up to my decisions when when it's going to to make it public. I can check for you like like when they're going to make public.

Yeah, I'm not sure like what what is their consideration? Oh

yeah,

thanks. So

but I I can check for you and and and let you know in in later in this

Oh, no problem. No problem. I'm also in your group chat. And um so is it true that the main problem that you face or the problem that is posed by Saulus is just uh running the ZKVM with a very large number of uh validators. Uh what is the progress on that?

the uh the thing is they they they have a a scripts uh to to being able to like like generate the the large to generate the large amount of validators but I haven't able to successfully uh execute that script yet. So, I need to get back to you on this.

Yeah. Thanks. Thanks a lot. [Music] Hi. Hi everyone.

Um, I'm Ellie and today I want to uh talk about the FML implementation and uh phase two. It's a um it's a Java based um execution client and um okay about the presentation I I tried to I don't know I tried to make it simple to understand and maybe a little bit of vis visualization on it and uh so I thought maybe if uh at the end of day people can describe this FM test net with a few sentence then maybe maybe I would be happy. So I I turn off my camera and want to share the uh my slides. Um okay. how how to share my slides.

I I I see lots of here but um yeah this is the video guys. So,

okay. And do you do you see?

Yep, we can see it.

Uh, thank you. Thank you. Thank you. So, um, let's go to the how how I can Okay. Yes.

Oh no. Oh no. Um okay. The this is as I said this is the FMA implementation I'm betu and um okay um FM is a um yeah it's a it's a short life um test net public test net and actually it made me think of um a butterfly it has a short life um lifespan Right. So what is the FM?

Uh why, how, where? We are going to um talk about it a little bit. Um it's uh yeah it's it's EIP 6916. As I said, it's a short live public test net. And um yeah after a while it will it will automatically reset um any any cycle um now the cycle is 28 days and then um yeah it's like like other um other test net like other network it starts from a genesis file and uh there is a little bit difference here which uh I will talk about it um later.

Um so why why FM um how um how it benefits us? Yeah, it's it solves some problems with the with the um like other other kind of test nets long running test nets. Um one problem with those uh test nets is this state blood right and uh other one is the lack of test net funds uh which maybe most of us have run into that issue in the past. So uh also the consensus issue which uh is also important um uh and has happened and other um test nets um recently uh like um inactive validators and yeah other problems that comes with the this consensus. So and some some use case of it.

Um yeah it can be many but uh for now we can yeah we can we can use it as if someone is a running a um staking node. Um maybe maybe they want to uh test it before a real mainet um uh run. So they can they can try it on and a family uh would be very good option and also um it's um for for contracts for contracts that um that are applications uh that are uh going to be that's not important to be on the test net for a very long time. Yeah, I can say myself I can say for most cases it work because uh maybe it's very critical for um protocols that want to their um context be I don't know for a long time on a um on a test net but for other applications it's it's really not a um a big deal. Uh so you test it you maybe you just want your um your your coworker from other side of the world just see the uh see the parameters of the contract that it is working see the application test it uh yeah it's not a big deal being um being a um short time test net here I think it's very helpful actually so yeah and also you you you you you are not going to have um u this uh test net fans like with testing contracts and uh also yeah breaking uh changes in client implementation as we know client implementation yeah we we we test uh what we are going to implement on clients so um it's very good to uh run it on a test net test the um the client and with a very low cost cuz this uh this test net is not going to um it's not going to use your resource and your uh computer is the state blood is not going to happen.

I mean isn't it uh cool? Yeah, it's cool. And also um um it's it's public. I mean uh maybe maybe yeah you can you can for implementation you can run a um a private um network and maybe on some maybe I'll um client doesn't um provide a um private network but uh the private network is isolated uh but this this is maybe maybe this can be a better option and um Yeah. Um how how FMT how it's working a little bit more detail into it.

Yeah. It starts with a Genesis uh Genesis file. It has a Genesis zero file uh which is hard hardcoded in the client. the client who are going to support it have to has it um and and the client implementation and it is called genesis zero. So here is the difference the other network you you start from a genesis and you go on there's no other genesis I mean the yeah here the genesis meaning is is more flow is a little bit fluid every at at the start of every period you you start with a um brand new genesis uh which is which is not very different from the genesis zero actually only a few parameters are going to be changed, right?

And this few parameters uh you you derive you derive it from the genesis and um um yeah, you derive it from genesis zero and it's it's important to um know that this this the all genesis um between are not going to be initialized. These parameters are going to be calculated only by I don't know a sum a multiply divided by a number. Yeah. Easily cuz um for example here uh we want to know what's genesis one right so it's it's chain ID is going to be chain ID plus one right chain ID from genesis zero which we have it in in the client is hardcoded. So uh Genesis ID for this um the the next genesis going to be Genesis ID from Genesis 0 + one right.

Uh same same with the time stamp. Time stamp is going to be time stamp from the um previous genesis which here in this case is Genesis 0 plus 28 days. Right? And um so this is an example for uh Genesis 1. And uh what if what if we it's not Genesis 1 and we are at um some point in the time and we want to calculate Genesis n.

Um yeah we we know when now is now is we have a time stamp now right? So we can calculate the number n. We can the number n is calculated then you can calculate the chain ID. Chain ID is what is this the chain ID from um genesis 0 + n. You you already found them right.

So this is how we we we find the this parameters. So it's there's no um I mean there's no um heavy calculations like from Genesis 0 to Genesis N. No, it's just it's just as this simple that I'm describing it. So, time stamp time is going to be time stamp from Genesis 0 plus uh 28 um which is the period uh multiply by n and yeah um it's here I I I wrote it in days and um it's it's in seconds actually. So yeah it is uh so this this Genesis n is d from genesis zero and all the genesis um files between are not going to be initialized.

We just find the uh parameters of genesis n. Um yeah. So now imagine that we are uh on day three, okay, of the this per period, this 28 days period. And um we want to we want to um yeah, we want to like add a um black um on top of it. We want to build a black.

So we we know where when is now like the current uh that time stamp is now. Okay. It's we are on day three of um Genesis of some Genesis. We don't know cuz we are going to find it. Maybe maybe we need to find it.

Right. So here um we we we asked like if if the current genesis exist uh then if no then we drive it from um the genesis zero. So how I described it in the last um slide right if no uh if yes okay we have we have a genesis it's it's exist okay uh so is it the question is is it still valid like um we are on day three or we are on day like 29 uh 29 right so if yes if if still valid. In this case, we are on day three, right? It's valid.

Uh so we we we build a block uh like other test nets. Okay. Yeah. Sure. Sure.

Um it's it's not going to take too much time. If no, then it it's going to um yeah, this is this is actually I want to uh get to the point that there are uh what I'm going to implement, right? just uh so here if if it is no then the network is start to reset and uh and the reset we start to build this genesis and yeah uh how um yeah this this reset uh functionality is uh is the main part actually uh it's yeah we we drive the new genesis parameter uh with with the this reset when the network is going to reset this I I mean um the that's N right and this chain ID time stamp which I described so uh and also the um uh we initiate this uh this new genesis and uh we discard the old data the the state word the everything is going to be refreshed right um and it's going to be refreshed automatically so there's no there's no I'm going to um I don't know change something do something manually it should be has to be automatically uh so uh this state board should be um the old state board should be removed um and etc right so what uh verify I mean um is um ready for now or has been implemented uh yeah and and most um test net is is available uh whether it's manually or um yeah implemented almost completely. So uh I I uh yeah I I wrote the uh names of uh people from uh previous cohorts. So load load star is by holy from uh court 4.

Um lighthouse and rest by Terry from court 4. So maybe maybe you want to check out their work. Um and um if you want to continue and and the work uh that they've done and also BU and um TEU by Glory um cohort uh five uh which I'm going to continue on um this one like working on um BU and TU and um yeah BU and TU are the um pair um um client um Java Java base per client for uh yeah basis execution as I said take with the oh yeah take with consensus client I'm sorry for the mistake and um yeah so uh what what's my main task and this um um uh and this project is going to be the this functionality have automatically reset u and the database as I said it's going to be reset and yeah the state uh should be um refreshed uh my my timeline my timeline is um I'm not sure I mean I based on what I anticipate anticipated for now it's like maybe maybe I I I I wish to finish this reset functionality and by um I don't know by by big 10 11 and then um and work uh work on the TEU side right so for now I'm starting on BESU implementing Gan BU and then I I want to deep dive into TEU as well so um yeah this is the timeline I'm not I'm not still uh sure about it. I I don't know. I'm I'm very I'm I'm very at very beginning of it.

Okay. And also the um I don't know maybe maybe it needs to be developed parallel. I mean maybe maybe I'm just I'm just thinking that like with within week 10 it's going to be finished. Maybe it's not going to be finished. Maybe I should work on TEU a little bit, do some change on it and then come back on base two.

Uh so that's why I'm not I'm not very sure about the timeline. So uh the next step the next step I'm going to yeah I'm I'm going to see what's available for now. I mean in in in detail, right? Um I I I should run it and I should run run this test net and this to um clients and see see what's really available and uh yeah I've I've I've checked the um the code base of um BU uh but but no not not much on TEU uh so um yeah um I should I I should work on it. I mean to see uh how how the past is going to be right and yeah thanks for listening and if you have any question you can ask and also uh thank you Mario for your uh comments and my uh proposal h I will get back to it.

U yeah thank you so much. Any question please?

Thank you. Yeah, any comments, questions? Um, I would just uh point out in the last slide you had tech execution client. It's consensus client just

just a typo you had in the thing. Yeah. Um, yeah. Any any thoughts? Any comments?

Um, yeah, I already commented on your proposal, so I don't have to add now. But if there are any more thoughts, feel free to uh feel free to speak up. And also because it's our last presentation of today, uh we have a couple of minutes left. Uh if there are any um anyone else who wants to give any update or share about their plans, uh some of you haven't haven't done so yet. anyone?

Okay. Well, um if there are no more no more thoughts from anyone, no more comments, um no more questions, uh we can wrap it up. Uh I just wanted to mention again this was the last office hours dedicated to the project presentations and uh next week we will have AMA or next week we'll start office hours uh with AMA sessions we're going to have Barnabas from Pandops uh for the next session going to open the uh issue so you can uh learn some details post questions if you cannot make it there next week uh it will be u more focused on like local prototyping. You can get some tips about how to uh how to test your uh run your stuff when you are actually developing with the clients to get some get some uh thoughts on uh some some uh advice on uh the development process. So yeah, uh we'll see you next week Monday on the standup and Wednesday on the next office hours.

So that's it. Yeah, got u minutes left. Um cool. Any any questions, comments? anyone um was it was it everything otherwise?

Yeah, thank you so much everybody. Thank you for for the lovely presentations. Awesome.

Thank you.