Eth2.0 call #39

so there's a jinda testing and release updates I will start on that we a lot of my focus has been on p0m these are twelve the last thing really in that and the queue there is getting the upgraded BLS including that draft seven version to the spec for the new test vectors I'll be looking at today we're trying to get this out the next two whole days I know it's a little late but we've been the blocker there's been a number of things going on but getting this BLS is the last thing in there other than that there are a number of networking updates and modifications that came out of that networking Paul thank you for everyone for the input and review and also some increased testing is gonna come out there were some corner cases especially around handling multiple different operations in blocks that could result in some corner cases with respect to kind of modifying modifying state in the middle of state transitions so might catch some catching new bugs on your end cool so that is eminent any other updates on this cool let's run right into client updates first we'll have text sorry Danny yes just that if that's all right okay please no it's been a little while no all right so we actually pushed a blog post last week that details all the stuff that we've been busy with I'll just push it now on the chat before I forget and yeah it's been it's been pretty good we've done a lot of made a lot of progress on the structural fuzzing so implemented and derived the arbitrary trait on our in two types so we can now provide well form instances of custom in custom types from row by offers so it's a huge improvement in our fuzzing coverage so this has already allowed us to identify an integer under flow in an upstream dependency the snappy crap that we use so the maintainer confirmed the bug we pushed the PR but it's yet to be merged with the last few weeks we've been working with Jake who and Nimbus been really great working these guys raised a bunch of issues some of them could have been exploited off the wire there is are more hardening opportunities I guess the main ones there was a Luke in Turku when I said the coding bit lists without an end of list marker and on Nimbus there was a segfault future stack overflow final updates function I recall correctly updated the trophy's list of big and fuzz we're now up to 18 unique bugs she's pretty cool made some good progress with the goal and integration so just a quick reminder for everyone we've been experiencing a lot of issues integrating both serenity and prism and we actually lined up a call with prism in a few hours to see how we can better collaborate on on this one and forward yeah if you go through the blog post is that we're proposing a new architecture for the for beacon fuzz so it's all outlined in the blog post please feel free to give us feedback super keen to hear from from everyone on this basically going to move away from C++ and we implement they if I buy things in rust don't necessarily you want to go into details here but we're breaking down Beacon files into three separate tools each to fuzz which will be coverage guided fuzzing leveraging the structural fuzzing that we've been working on to generate interesting samples is to DIF that will allow us to replay samples those samples that of course all the implementations using the nice facilities that you guys have been building so P CLI LCL is the airline and so on and finally we have five bindings that's the core against differential fuzzing part that's because b2 there's a nice diagram at the bottom of the blog post if you're if you're interested yeah so please check it out super keen to get some feedback from everyone will also be pushing blogger images so that the community can help find bugs things a lot of people are asking how they can contribute to e to and that might be an interesting experiment kudos to Justin dragged for this question we should be wrapping this up next week and I guess finally we've been starting playing with lodestar I think we caught a few type errors on the SS said like pitch but you know we're not really JavaScript expert so I might reach out to came in next week to discuss this further because I guess most likely these are probably caught by the calling package yeah that's pretty much it awesome thanks very good yes there was a discussion three weeks ago about some beacon states that shouldn't be trusted so it involved everyone but apparently well last time I looked it was actually states that couldn't happen if we were in the natural function client so what's the latest news on that was yeah Anthes modified oh yes so that's the reason why we're splitting things up so that we can avoid this confusion you probably saw an issue or a PR from Danny I don't think it ended up being emerged to the respects repo but we were probably have to sync of beacon states at some point so the hope in my opinion the whole concept of you know beacon States the entrusted inputs we might not rely on this assumption for too long in fact we've we found a couple of overflows on my house when the Allegri you know invalid because states per se so the specs the spec has actually been clarified right like I think last week or two weeks ago now if you overflow in a state transition it's clear that the state transition is invalid which is good to see but yes we've been so the structural fuzzing help us you take the beacon States better so that we can actually not only have valid as it said containers are the even stateless isn't a nose but also valid even states as per the spec so it's yeah it's one of the reasons why we split up the the tool said that we have in three separate tools and yeah those conversations were super interesting in my opinions so it raised a lot of yeah interesting thoughts and yeah thanks thanks everyone for being involved and I guess one of the the issues that we had as well is heating the utilities NC Li for example was hitting directly the state transition and bypassing all the potential checks that are performed that the networking layer so we've now considered that and you know won't be raising such issues if they arise in in the future hopefully that makes sense right and I guess to clarify in the sinking there's there's probably two ways to sink a network safely once the networks run for more than say three weeks one is to have some have a check point for a given epoch and block sink from from Genesis and then make sure that the check point you reach is that checkpoint route that you had that wouldn't involve having to get an untrusted state from somewhere but if you instead the there's a much better UX around actually just starting from a state you know if you have this checkpoint that you that you want you want to actually just start from that state so at that point you're actually inputting a state and your there's a there's an avenue to input a state into your system generally you've got this from say a trusted source but there's a small likelihood that you got it from a nefarious trusted source and you have some sort of like tainted state that you're inputting into your system and so I think that's the that's the nuance there obviously you don't have to necessarily get from the PTP network like I had in that PR but the idea of getting a state from somewhere and putting it into your system is certainly a flow that I think we're gonna all walk because blocks Inc from Genesis is not it's not going to be the best UX you know once we get a few months into this thing yep spawn cool let me follow up on that or other questions from Eddie cool thank you glad to see all the progress there and the the trophies on that on your readme very exciting ok other testing items you that's disgust that's not separately yeah we're going to decline updates and then we'll go straight into test nuts great so with Taku perfect this is gem from techo so in the past couple of weeks we added tsunami compression over Gaza an RPC we also added support for pink and get made a metadata now we're randomly subscribing to persistent subnets we majorly reduced our memory usage while sinking and lastly we now have built-in support for syncing schlossie that's it cool what's that memory usage look like on sage lessee do you know I think it's now averaging around like nine hundred megabytes okay not hundred percent sure though it's been a while since yeah thanks let's start hey so with past few weeks we're finally starting to be able to sink as if we're starting we're sinking kind of stabili unless II haven't yet reached the head but we've synched a few thousand eight bucks worked through different gossips of bugs and sinking bugs and what have you still not really stable but hopefully the next few weeks we'll we'll kind of nail down some of that node level stability it still looks like our microbiota c-5 isn't running so we are you know stuck with our bootstrap peers and it's not really sustainable so different things like that we're still working through Thanksgiving Nimbus hi so like lodestar we had multiple sync Fink's fixes in the past three weeks in particular snappy which had the compatibility issue with lighthouses we now have a single makes lessee target to connect to Schley see sync is working slowly but steadily that's life and right now the main focus for Ashley she NSYNC is working on performance in particular on Windows besides that we had we have worked a lot on multiple memory leaks that were preventing or test net to last for a week somewhere coming from p2p some from a block caching system and we are we added several memory tracking tools and also since we are entering like we are focusing on bug fixing we have no tools to debug on discovery to debug value the topics and message received Thanks and this probably goes without being said but you know once we once we have fast state transitions it seems like the the next big culprit is memory usage I know everyone's kind of been attacking this from different angles but be sure not going to other stores I know there's a lot of pretty solid strategies to go off of now so you don't have to do this alone Trinity everyone not a huge update this week mainly we've been continuing our portkey the trio async framework we have updates to the latest beacon note and validator api's which was good to get into place we've also made some progress I'm bringing more full-time contributors to the project which should just really help out with everything we have going on cool thank you Alex another mind but just minor updates in the last week so we've updated because they open up a specification and they tested synchronizations that had some problems with the ultra networking not really much happening in the last two weeks got it thank you prism hey guys Terence here so over the last few weeks we've been working on topaz man tennis with fixing you Xbox and their users feedback as to get reporting the Topaz test net also fixing their water parts as they do reporting in the multi contest net so nothing really substantive from their regards just typical bug fixes we are also fully aligned to spec version 11 lemon - walking into a lonnie into version zero 12 right now and peter has been doing great work on optimizing in nature syncing so his latest experiment resulted in 100 watts per second to reduce your sinking and this is without at the station signature verification so we still need to optimize signature verification for your sinking basically what the lighthouse is doing I've been in shape in doing great water and slashing detection so Eric under reported to us that Wow his validator was earning more money than the rest it turns out his validator has included a special object lock so that means that our backhand slashing service is working that means that pops up of slash in network is working so it's pretty exciting let's the way be running climb for dodging readiness tests in particular stressed hands and then the inactivity permit finality tests so we updated a few internal matrix to better very super monitoring we're running a 16,000 elevators one section and in twenty notes and no and then and then no issue on that so we're starting the and him.t finality tests were within a few days to see the outcome so yeah what could you need it for stress on it and yep that's it nice are y'all running on shorts lot times on the the stresses yeah yeah yeah what we are doing one sec hearing them oh nice or do you know if you're seeing any degree of forking ah more than say on a normal 12 second slutting not necessary but we do see about 85% participation which is negative and that's and that's just due to timeout timeout from which angle so there are PC and otras then we have about like 2000 validators on one beaten notes though that on some RPC request within a second gotcha interesting okay cool glad that your other cushion on this thanks Terrence lighthouse that'll be me again so Paul's been busy implementing hierarchical key derivation for BLS so we're ensuring interoperability with each do by Jim McGee that's induced by prison so he's implemented the key derivation generation the BLS key store and the LS wallet quite excited to announce that we're kicking off the first external security review with twelve bits on Monday so we've wrapped up the slashing protection I spend a lot of time handling concurrency and to me city guarantees for database transactions and we've also changed the directory structure to better suit all dates in the future we've been running several 16k validator tests that some of the last couple weeks we've seen two panics one from an upstream package one from our code both have been fixed you were asking about memory usage Danny I've seen some great improvements there so we're looking at 300 megabytes of RAM for a node running at the keynote and volley better client with 2k validators which is quite cool you've been working to fixing consensus bugs as well that have been identified by Justin Drake so we've removed almost all paralyzation from our state transition code it's not really needed anymore since we we can do batch pls verification now we finished implementing full gossips of reification logic because a bunch of issues about test harnesses really looking forward to seeing how everyone should I see talk about Leslie we've we've defaulted our lighthouse to run on the chassis tested by default so the spec in the Genesis state are now back in to our binary I guess other miss updates we moved our disk refile implementation into standalone single prime repo and we've been upgrading to stable futures the entire code base bumping all the lighthouse dependencies to the latest versions and yes so we're really almost done with this massive upgrade that adrian has been busy with and we're hoping that trailer fix can tackle my house with these incorporated and yeah we've just been work on the RPC error handling as well and between integrated into our around reputation system cool thank you and the three hundred megabytes was that's a 16 K test net correct cool very good thank you thank you thank you I believe that was all let's move on to test nets we can start with a free yes sure thank you I can talk a little bit of about multi-client chestnuts a lot of stuff happened the last three weeks since our last call I did multiple attempts to create a multi-client chest net that unfortunately failed mainly due to network fragmentations but also due to beacon I was disconnecting and rejecting other peers each other for right limiting other reasons then remember it's a problem with kind of educates that is two times them can be less than minimum can assist time so we had different Genesis times calculated and prison meant lighthouse however a client gives a super responsive and I want to emphasize that I really appreciated apparently we managed to launch a multi-client chest net two weeks ago we talked about it's called Swayze that launched initially was to lighthouse belly datas at canada's and to prison validators in the beginning the finality you most horrible because some clients kept crashing and my belly datanodes I had a hard time to keep them updated alive but then again as I said client teams are super had full and responsive and we're doing an amazing job in fixing box and eventually now she has almost perfect final t8 like penis for more than a week after fixing the most important parts and I think everyone is surprised how stable this network is running after couple of days Tikku joints at estimate since they managed to successfully connect and shrink the network first but now I also know they run validators on easy so we have three full clients on crazy right now I know that the numbers we could shine client is also synchronizing I still present you're experienced networking and sync issues but I know the team is very close to fixing it I didn't manage to get to the shine head yet but I know it synchronizes and connects but maybe proto has more details because he mentioned earlier today that he managed to could do a foreign country busy I also know that loadstar managed to connect and synchronize at least some epochs on CBC but I didn't test that out yet so I'm aware of five clients right now at least was partially samples full support for chestnut I would be interested in learning more about where Trinity and the Court explained our regarding mobility or multi-client tested at ports and given the current stability of the z-test minute I would start working on outlining a coordinated multi-client test net soon something that's based on the Maynard spec ideally on targeting the version o dot 12 of specification with 16 K Genesis validators and maybe we can figure out a way to launch the chestnut with three different clients at Genesis that would be amazing but that's all up for discussion and also adds the idea might maybe if we if we do a more coordinated and more official multiplayer just met maybe we can also do a dry run test offer deposit contract ceremony on the chestnut but that's all to be discussed and I would carefully target maybe it the jewel 2021 state but I am still not very certain about how long it takes to implement the version 12 of the is to spec in all plans but I'm open for discussion here but I think we can start talking about this now that's it from my side yeah I think you probably have anything to fill in right so let me trying and experimenting with dog star inverse there are clients that are relatively new to the Russia does not load star has made some great progress in sensibility and they're thinking they're indeed in thinking many books like positive songs but 10% of them social stuff I think just ability to get in there and then moombas is close to very close to the head of the Jani about the hundred blocks distance this is where the sync mode changes I think like there's no stability issues if this special sigmod for those last few blocks but it's working and I've implemented sports for them to show up from chief stats so I become everyone can follow cool thank you yeah I'm gonna get a I'll put a diff up of what is going in to be zero twelve and outline those items so that we can get a better estimate on how long it's gonna take I think the sorry someone just messaged you I think the there are a number of networking changes most of these are very minor I think the big thing is going to be the support for BLS I know that we just got it on her roomie and I'm not sure the state of the Java implementation and the Milaca implementation does anybody anybody looked into those yet yeah we're good we're pretty much ready Chuck's just push the latest changes so we should be 0.12 compliant in terms of Bo that's already great yeah my mining felt planning to update the Java implementation this weekend I spent too much time messing around with Lacey this week that should be listening awesome and I know Python is also updated and we will have those test vectors output so I think a like coordinated start in June is certainly like right at the beginning in earnest is like for some sort of larger coordinated start will make a lot of sense and I think we'll even be able to do some smaller test runs in the weeks before but we'll I guess that'll be more on the client end to digest how much this p0 12 is going to take cool other conversation on test nets questions comments thoughts okay moving into research updates who wants to get us started if nobody else volunteers great I could go from the II wasn't perspective yes please um yeah we we have quite a bit to report because we haven't been a reporting yes frequently on these calls last I believe that was a few months ago so week ago we have released eat one x64 the first variant of it it's it's a write-up on eat research but we also have a repo under the he wasn't work on github which has a much longer spec and as well as it has examples written in solidity and both of these are linked in the in the write-up so this first variant uses receipts which are generated on the sending shard and need to be submitted on the receiving shirt and the the simple examples we have regarding this are tokens two kinds of tokens one of them is is wrapped tokens and with that example it is possible to you to accomplish having died for example on all the different shards and now on the next steps we are looking into new variants and it's not entirely clear which we're gonna do next but we we have two ideas on the plate one is to look more into yanking and there have been different similar proposals regarding yanking and as well as by writing the variant one if you look closely at the specification in one of the appendices there is something called rich transactions and we also use that to devise yet another version of yanking and the only realized it was retrospectively that it is kind of yanking so anyway I think we can either look into yanking next or something based on the eat transfer objects which I believe was mentioned by Casey as part of some phase two ideas earlier on at this point I also want to emphasize one of the the main reasons for this one x64 was not to to move phase two into EVM but rather to have a much smaller scale to experiment with these kinds of designs as well as to engage current app developers and give them some kind of and understanding what sharding could look like and that's why we have the solidity examples and we really want to get feedback from those deaf developers to guide us which design is something which which could be useful to them but eventually we do expect that the more useful designs would mean much larger changes in the EVM where it may not make sense anymore to keep EVM because the at least historically the the each one community has been really reluctant to accept radical idiom changes and if you would do radical EVM changes then you already lose the the benefit of the existing EVM tooling so you may as well just switch to to a different engine such as wasn't and then now maybe just switching over to some updates on the benchmarking work we have been doing so the benchmarking work we do because this is something we always mention and I think a lot of people just think they wasn't team is working in benchmarking but it's more like an on and off effort and in the past few months we have been looking at some new engines which seem to be performing one of them especially that wasn't tree is performing much better than any of the other engines but it seems to be a bit more complicated than the other interpreters and we have found some edge cases where a metering could be challenging on this engine and also we have looked at and another he wasn't compatible was an engine called SS VM but it doesn't really bring any speed benefits over over rabbit which was our main main engine so far and I also mentioned fizzy a few months ago probably in February which is an interpreter written by the II wasn't team and I'm happy to report that just today we managed to release the zero one the first version of it which passes a lot of the the official tests but it doesn't implement floating points and the reason for this release is because next week or the week after we plan to you truly 0.2 which has optimizations and and we just wanted this first you have like your the baseline against the optimizations and the sorry for taking so long but is just a lot of stuff okay which happened and now coming into may be a really interesting part and so as part of the benchmarking we have been looking at all the different well basically all the different pre comp eyes which exist on EVM on each one currently because if he would propose like it wasn't based system we wouldn't want to keep the pecan pies and we have reported this previously that the elliptic curve pre-comp eyes we got quite good results so that was with their to be and one to eight or the end to five four we got quite a good results with those including pairings and but it's required and what we call big integer host functions and so those are you could see those are kind of similar to pecan pies but and those are much more primitive operations than the pre compiles which exists on on each one so as an example it would be twenty fifty six bit addition because wasn't doesn't have it and but we also proposed montgomery multiplication on 256 bit numbers in this big integer API and videos we were able to achieve really good speeds on B N 1 to 8 and in the past month we have been looking into BLS 12 and again we managed to reach speeds very closely to the native speeds so first we have looked at and all of this is on interpreters so first we have looked at just a basic BLS felt implementation in rust which didn't really produce the speeds we expected or at least we hoped so just some random numbers here the this rust code compiled natively was taking roughly 5 milliseconds for a two point parent and the same compiler 2000 was roughly 500 milliseconds and then we reached out to wasn't snark Jordi and the team behind wasn't snark and because that has been the the optimized was an implementation we have been using for beyond one to eight and and they managed to implement support for be less twelve and that with that code with some more optimizations on there there the big entered integer API and we have been able to move to the speed down from 500 milliseconds to close to 14 and I think with one set of the optimizations we were able to get close to eight milliseconds so that's less than less than half to learn more than half the speed of native and so I would say this is really good news that even on BLS we could we could get to use BLS video pre-comp eyes it wasn't and the last part now is that we were also interested to to check if we can replicate these findings on a VM so in the past I think three weeks we have been Dingaan a small project called DVM 384 which we hope to release for tomorrow's all core devs and an under tests we have added three up codes to the EVM which are Butler addition or subtraction and Montgomery multiplication all these for it non 384-bit numbers and we have implemented just one building block of the parent corporation and made a more like a synthetic benchmark out of that to approximate the the extra implementation because obviously in two to three weeks time we don't have the capability to blend be less 12 on medium but with this synthetic implementation we got pretty close to the bottom numbers so Oh in the end what that means we may be able to even get rid of the Tbls rotary comprised for each one potentially it seems to be possible to to replicate that with just three primitives I think that's all and sorry it took so long but hope you found it interesting yeah good thank you and if you're interested in some of this stuff and you haven't read the x64 or the recent xtc4 post check it out any follow up or questions for x'q you other research updates and I got a handful of people that might want to talk italic [Music] looking into homework homomorphic encryption things more one thing that we this one that we discovered is that there's some use case for them private information retrieval so we'll probably ask more thing around more things about that producer and I'm also published a a post this morning basically kind of open calling cryptographers to see if they can solve our polynomial commitment problems not to my to much else in terms of and I've research research II things I guess spec side also I'm also looking into some phase one simplifications on the proof of custody side oh right if you have been following prefer custody stuff check out doctor it has a new post on each research that proposes removing the bit from the actual signature which is pretty cool yeah and I have a kind of meaningful thing that I haven't posted but the follow-up which basically is there and I've reduced the free the the frequency of key revealing so you don't have to worry about and if keeping track of reviewing this and it's just that if you don't work if you don't review on time then it's invalid yeah so between those two things it seems like we can cut through their complexity over the Perot custody by more than half maybe two three quarters yeah which is awesome yeah the thing where it turns out that we got a bit unlucky as well what went down this summer rabbit hole of trying to get self verifying proof of custody based on the Cate commitments and it's I mean lyric and there's efficiency and uncertainty is it would also bind us to using geek using K commitments for more clarification so there's mmm-hmm there's a challenges and going going down that path and don't exist if we just go down this kind of 0.001 bit approach yeah cool yeah check out that hosts pretty interesting very simple but might reduce complexity by a lot okay other research updates do one for tx/rx great home for eath 1 e to merge research mikail has just released an e3 search post about that I think just shortly after so call last time oh no actually probably about a week ago he released it and he started working on a draft e 20 to communication protocol - and and he's kind of like working on a PSC for face 1 as well for the network monitor they found that lighthouse was sending unsolicited UDP packets and so we open a PR for that regarding more choice tests we kind of we've been generating tests Alexes built this kind of transpiler for the tests and we found about already in TECO for the proto array implementation and a bit around the fortress tests made improvements to on a toll which is the high spec transpiler so now i could translate face one spec and found of three bugs or three widow and three PRS regarding a PI spec for phase one based on the results from that mmm and then implemented gossip 1.

1 on JVM Limpy to beef fantastic those are all awesome things thank you research updates Guillaume any success in making an RPC Consensus engine forget yeah I mean there's a recipe are in the works we're still kind of discussing the documentation or in the list of RPC calls with with Peter we had the meeting this morning and I think we just shared with you the document so I was going to ask for your input that's it it's called but overall you haven't say the skeleton is already there but we still need to yeah I ran out a couple a couple Peter's right cool thank you that's exciting other items other research items anything before we move on cool next up is networking we did have a call about eight days ago some really good conversation and items came out of that and we've also this has kind of been ongoing debugging and interfacing with networking but are there any other updates anything on your mind Felix or otherwise you you hey sorry I couldn't really run you didn't I'm sorry for not participating in the networking call last time so I said I'm working on still working on the new on the sort of like expect updates to the disk e5 spec which will improve the performance a little bit and also resolve this one error message that I guess if you've been running it on a test net you're also seeing it basically sometimes you can get packets which are seemingly which have seemingly wrong encoding but actually it's just the spec book so I'm still working on it and I will publish that person I'm actually not sure how to basically I'll have something for for feedback next couple days and then so would be kind of nice to get into a bit of a conversation with like all of the implementation teams to figure out like what is going to be the path of least resistance to upgrading because the discovery upgrading can be kind of complicated so we have to figure out if we actually wanted to try to do something soft update or just you know basically live with you know like half-broken discovery for a couple weeks until everyone has the right version or if so complicated with respect to live nuts yeah so if there if the network is live it can be complicated because they will basically be if there's a mismatch in the versions and they're in the versions are fundamentally incompatible then basically there is no way to work clean upgrade because basically just means notes won't find each other due to the v-0 twelve spec update with the BLS updates coming this it might just be best to wrap this in the same update so that because we're gonna have to research the nuts anyway okay that sounds very good so I mean depends on like I mean you were talking about like transitive June time line for the next test net so maybe we can just make it so the updates go in then and then we just launched a new test and with a new version of them yeah they'll be great I mean how long you expect these changes to need to be able to go back and forth and make it into the spec so I guess it's gonna take approximately like I don't know one one more week at least to like kind of get to like get to get the actual spec done and then I'll I can assist people with the implementation it's not gonna be like like there's gonna be one bigger change to the packet format but always basically gonna be fine okay all minor change cool so what I would say is reach out early to implementers to get feedback and input so we can just kind of streamline this but if we can get it done on that timeline then I think we can avoid the headache of upgrading these nuts in life okay thank you any questions for Felix great other networking items you okay general spec discussion I have seen those phase one PRS with the bugs I certainly the testing on phase one is minimal we're currently so I really appreciate those those bugs but I've been prioritizing the bees 0:12 but I'll get to those soon other spec items I imagine this this one's been pretty quiet for the past six months the spec item discussion but as we move into phase 1 into plantations I'm sure it will get a little more lively mMmmm our question already is a for Christmas we have them in 0.12 also point five point one right so Joseph what is the what is it the the state of the four choices that you all been working on are these generated off of the PI spec or he's generated off of the old harmony implementation and what's the format that they're outputting in and given that format is this something that we can enter paid into the canonical vectors for the next release um I'll let Alex who's likely on the call to speak to it but essentially what it does is it reads the PI spec and then trans files it to generate the tests but Alex are you on I don't think he is I'll knock on y'all's door and see if these tests can appropriately just be shoved into the next release because if they're already working for you they can work for others and definitely I think that's I think that was his goal was generally being able to just on an automated automated test generation for all the different clients cool okay I'm gonna knock on his door thanks I think shows you okay any other items here um hi and so about the POS um so before this car I just made an appeal to fix the face Nero face one the zero signature issue I put here and I hope that people who are interesting can take a look like in 24 hours so we can generate the POS test sure thank you yeah I will certainly take a look and others can as well thank you okay open discussion anything on anyone's minds sweet awesome work there's like a ton of moving parts right now and it's super super exciting thank you everyone and I will talk to you soon thank you just an x-file thanks everyone bye