Complexities in Aggregation at Scale by Mikhail Kalinin (Devcon5)

hello my name is Mikael Kalinin I'm from harmony team first of all I'd like to credit alex philosoph who has been investigating in these complexities in aggregation ok let's move on so here is the we have a station votes you are already familiar with them from yesterday's talks about beacon chain technical introduction so we have cross links to charts we have finality checkpoints and we have a fork choice vote as well so the attestation is a very important thing for the big and chain this is how we can chain makes the consensus happen let's see how it's progressing through a slot so at the beginning of the slot a proposer produces a block and send it to the wire so all the daters can receive it through the wire in the middle of the slot committees start to produce their other stations basically this could happen not in the middle of the slot one may produce at the station upon the receive in this block and important this to its chain after that a testers should deliver their added stations to the proposer of the next block to be included on chain and they are eager to do this because their rewards depend on that and if proposer of next slot for some reason fails to include some of those SATA stations each those at the station should be delivered to proposer of this lot of after the next one and so forth up to 64 slots ahead okay so what proposer does with those stations it aggregates them and includes them into a blog body aggregation reduces verification complexity and it means that instead of verifying hundreds of signatures you should in the best case you should verify only one signature pin very common T it also reduces size demand on the blog body this chart shows e y axis is megabytes of data and x axis is million of eath at stake so this green line is the aggregated case and orange line is not aggregated case and you can see how aggregation saves our lives so ok let's talk about attestation delivery how to deliver those at the stations to a proposer we might probably do direct delivery just send those other stations directly to let's say five or ten proposers of the next blocks whatever number we find reasonable but this this way is we're efficient and fast but in that case we are willing proposal rate the entity it means that adversary may match proposer public key and its network ID and organize a coordinated attack to this particular proposal kick them off of the network so we're not doing it that way instead of this we're using p2p overlay that includes all other daters of the beacon chain and gossiping the attestation x' through this overlay so everybody is happy everybody all proposers receiving those data stations nobody knows who are proposers who proposers are and these this p2p overlay is big enough and everybody knows that there are some valid errors in it but it's hard to to organize an attack on this because there are a lot just a lot of notes and nobody knows who is the proposer of the next lot or who are the participants of the committee internet workers network because identity is not reveal that gossiping to everybody gossiping in the p2p overlay is prone to Byzantine failures and to cope with Byzantine failures we have to keep the degree of the p2p mesh on a certain level and it brings a redundancy the degree is the parameter that is just a number of peers that your peer is connected to so let's say your peer is connected to ten peers and you are receiving the air to stay the same attestation message from five of them and send in this attestation propagating this attestation message to the other five so we have like redundancy by a factor of five and this redundancy is about inbound and outbound traffic and also nodes must do verification to prevent epidemic attacks where the network can be flooded by just bad at stations invalid and this brings a high demand on resources at scale so these numbers are a factor of redundancy in the p2p of the p2p mesh and this like triple redundancy is a blue line and if probably on the face zero we are okay we are here where is it ten million each what are the 18 but when we start to approach to 50 million eath we are having like mmm 50 megabits per second as a requirement and this is calculated for 60 seconds per slot what do we do with this instead of sending single attestation to this Peter Pan mesh where we want to partially aggregate them in relatively small overlays and only after that send it to the to the beacon attestation where everybody sign to this is where aggregate and overlays comes into play so what options do we have for those overlays we can build them for each people because we know what committees are in each chip o-64 slot beforehand it's like six minutes so we could probably just build them for each Ipoh or there is another option and to use short subnets these are like natural overlays for for the earth 2.0 so the requirements are to the overlay are like first of all it should be relatively small and efficient enough to do this partial aggregation in the fractions of slot because we need to send those aggregates to that be to be matched so it takes we are very restricted in time so it should be incentivized a the participants of this overlay should be incentivized to do the aggregation and the only one who are incentivize did the committee's that produce those data stations it should be Byzantine tolerant being relatively small it should be tolerant to kind of see bill and Eclipse attacks and if we want to build the overlay for each shaper we have a six point five minutes to do the so this time restriction okay so para Baja overlays are very efficient because you can build this overlay with whatever topology and properties that you want and it could be after that leveraged by your aggregation strategy but the drawback of that is that you need a reliable source of peers within a small amount of time and the topic discovery will probably not help here because if you don't have a reliable source of peers your overlay will be prone to civil and eclipse attacks so this is very difficult part of building this epoch very paja overlay we can use short subnets they already exist they are relatively weak and they are like more long plain network units and the hence they are resistant to kind of attacks like Eclipse and civil and but the problem is that one committee is spread across all the shard subnets so let's say you have 10 peers attend validators of one committee in one in charge subnet number one then in charge subnet number five and so forth and they just can't talk to each other that is the problem for the aggregation strategies so two strategies so one is coordinated it's like handle you probably heard about that it's very like optimal so you can get clean aggregates without overlapping with lower network demand but this kind of strategies requires those overlays to be built very poor and we have this difficulty with reliable source of peers and they are not be to be friendly it should be there should exist direct connection between validators and that's why we have let's we are getting back to this anonymity problem here again we can use chart subnets and to fire a gossip based strategy so it's just you know everything is ready for it but and you can send messages you can send at the station within one subnet make those aggregates and yeah well the dealers of the committee that produced those at the stations can aggregate them but it's requires higher Network demand because we're working in a p2p mesh and it's not that advice what is the threshold for the aggregation and when it's ready to be sent to the beacon attestation topic and that's probably the problem here and this problem is amplified by spreading committee occurs different chart subnets because you don't know how much of the participants in your charts net so you don't know how big aggregate you can combine within the resources that you have at far so let's summarize that we have to use p2p overlay to deliver at a station to proposer and it scale it puts some big requirements to bandwidth we we are considering a partial aggregation in a smaller overlays is a solution but it have a number of complexities there is also another possible solution to use onion wrote wrote in and similar techniques but they will bring their own complexity as well so need to invest it just investigating that and for a zero and probably for his one is not affected because it's not it's not it's not expected to have like big amount of eath validating so thank you so much I'm tom [Applause]