Data Availability in Blockchain

okay hello everybody my name is Anthony Blackburn I'm the founder of nebulous group a web 3 consulting firm I'm also the senior event producer at Suite thank you guys for for coming uh today we have a panel on data availability so I'll introduce my my panelists now uh first is Dr sriram Cannon founder of eigenlayer and director of the University of Washington Seattle blockchain research lab hello everybody give a round of applause for serum so a little bit about eigenlayer eigenlayer is a protocol built on ethereum that introduces restaking a new primitive in crypto Economic Security this primitive enables the rehypothecation of eth on the consensus layer users that stake eth can opt in to eigenlayer Smart contracts to restake their eth and extend crypto Economic Security to additional applications on the network next I have Nick white the CEO of Celestia Labs Round of Applause for Nick Celestia is a modular consensus and Data Network built to enable anyone to easily deploy their own blockchain with minimal overhead next I have David Barreto starkware developer advocate give a round of applause for David [Music] and starkware develops Stark x a permission layer 2 that uses validity proofs and start net a permissionless validity roll-up sometimes referred to as a ZK roll-up and finally we have karate the CEO a CTO of Liberty gaming please give a round of applause for krati [Applause] and Liberty gaming is an ecosystem that enables everyone game developers Gamers investors games Brands enthusiasts to access and experience the very best of what's available in web 3 gaming and the metaverse okay so before we begin we're going to do a little exercise to get everybody kind of loosened up so if you guys could stand up for me we're all just going to put our hands up in the air well one hand up in the air and please repeat after me haha okay thank you guys foreign is there a name for that uh so that's something that I picked up when I was in acting school um before we would do scenes we would do these breathing exercises to get everybody comfortable with speaking and just being themselves emoting so yeah that's where it kind of came from so I don't there's not really a name for it we can call it I don't know uh the Anthony the Anthony warm-up okay so let's dig in because we are on limited time so uh we're today's conversation is about data availability data availability is a Hot Topic in the blockchain space at the very at this moment and as we see web 3 adoption increasing and the astounding number of emerging use cases increase the urgency to address the data available data availability conversation is at an all-time high we're getting so popular how are we going to scale quickly and most importantly securely in this discussion our incredible guests will dig into new approaches to tackling the data availability dilemma so um I'd like to start with uh when we refer to data availability in blockchain uh what are we actually talking about and why do we need it and so we'll actually start with with karate here okay so um when new blocks are produced on the blockchain we need to ensure that data is complete it's available it's downloadable and second it is correct it can be verified and it can be used for further verifications now at the level of l1s and at the level of l2s this problem can be defined in different contexts for example if L1 layer the data is not complete then the other nodes will not be able to find out what is the correct data and based on that a lot of operations will be stopped so the blockchain itself will stall correctness so if the data isn't correct or for some some ways invalid data got inserted in some of the blocks we we want to find it out and we want to ensure that it does not result in stealing funds or any other kind of inconsistencies and at the layer of l2s because l2s are there to scale we want to ensure that when we are achieving scale we are not achieving it at the at the cost of security because we want the scale data to still be correct and hence all the l2s most of the l2s still depend on l1s for data availability so yeah that's the short of it okay wow thank you for that uh does anybody have anything to to add to I think that was a really great breakdown actually uh so we can actually jump into okay so why is data availability data availability so important um what could potentially go wrong if uh if the data isn't correct it's not valid um I'll leave the floor to you guys does anybody want to begin I mean the implication so I started being at Layer Two and being an ability roll up every time the layer 2 the state goes from State a to State B we send this validity proof to layer one to verify the computation but if we don't make the state B available to anyone to see then not only people wouldn't be able to use their asset and Layer Two it's just worst case scenario you cannot transition from State B to State C so the neighbor will get pretty much stuck at that point sure you wanted to yeah um so one way to think about why do we need blockchains right like let's go back to the very basics we need blockchains and why do we need decentralization in blockchains and if you look at it certain things like you know we just learned that you know in a blockchain you when you're verifying a block you're checking that the transactions are correct right and it turns out checking a transaction's correctness or the state correctness can be done really efficiently using cryptography right so that's what star choir Pioneers for example so when you're checking that execution is correct or not it is a mathematically verifiable Factor you can just check you know a piece of code if you run on a certain input what is the output that accrues to it but whether the inputs to that computation right like when you're doing any kind of like a proof of correctness there are inputs to the computation and you want to make sure that either the inputs to the computation are available or the outputs of the computation are available and how do we know objectively whether such uh inputs to the computation or output to the computation the data input or the data output is actually available um the way we do it in blockchains is by having a decentralized group of nodes bad testimony to this fact okay it's like you know one way to think about blockchains is you're actually freezing digital events in time right like so you have an ordering of digital events flowing in through the system and you want to freeze them and you know later look at them so that you have a history recorded there and data availability is one of those things which is not retroactively verifiable like I come and I claim that as today at this time I had actually published the data and you come in tomorrow and ask me hey did you act did you say no you didn't publish the data yesterday and you and I can keep arguing about this and we see this a lot right and normally when you talk about history there's always you know it's highly subjective and there is you know a lot of you know back and forth and he sheds he said she said and how do we solve this in blockchains in blockchains we solve this by actually having a decentralized quorum of nodes bad testimony to that fact so in normal history we use eyewitnesses right we you know you want to track what happened yesterday use an eyewitness and blockchains are basically decentralized eyewitness to The Happening of certain events and so that's why data availability is important we'll come to efficiency but maybe Nick wants to add something uh yeah I think I would I agree with most of that but I actually disagree that the way you solve data availability is by people bearing witness um like that that's if you're kind of assuming that people are not actually directly verifying the chain themselves but to me the whole point of blockchains is that they're verifiable ledgers so like me as a user I can actually verify for myself without trusting a committee of nodes uh that that this is true that the state is correct that the rules have been followed and so that's why uh to me um the way that you verified it availability is by either one downloading all the data yourself to prove that yes it is published because I have access to it and I can see directly that it's correct and everything else like that um but the problem with that approach is that it's not scalable so that's that's kind of to me what the data availability problem is is that if you want to increase the throughput of a blockchain you end up increasing the amount of data that people have to verify as available and that just means that you have to download more and more data but me as an end user I might only have a SmartPhone I can't download you know gigabytes of data so the way that you solve that problem is with this technique called Data availability sampling so I can actually verify all the data in a large block is available by downloading a very small subset of that data and that way I don't have to rely on the nodes attesting that it's available I actually know directly myself uh through this sort of cryptographic technique and so that to me is a data availability problem how do you increase the block size but still preserve the ability for end users to verify it directly great thank you guys so okay so you you let me write into my next question uh I think we've addressed the daily the data availability problem uh so what are your thoughts on on steps forward how how might we be able to solve this this scaling problem and what are the implications in terms of speed and security that might be what might need to be taken into account uh in the case again I was talking it of course we're looking forward to eap4844 that's a new way to you know post data on chain in a much lower price because today when we compare the the price of a layer 2 transaction 95 of the cost comes from the just sending data to layer one for data availability only the remaining five percent is just sending the proof and the verification uh so if with eip4844 really reduces drastically the cost of a storage or at least posting data uh it will have a significant impact on the cost of layer 2 transactions we're also looking into a different data availability mode for a standard called volition where basically the user gets the option to choose if they want to store data on chain or off chain and it's it's like a seamless transition you can have in the same smart contract some data store on chain some of chain so basically do this to decide how much you want to pay based on where you want to store data so we have to those two main approaches in the short term and of course long term we're exploring different solutions one of them actually being Celestia as well uh for now short term is a volition in erp44 what do you guys think uh I can add something on the scaling problem so when we talk about scaling we have to look at what the what we are scaling and uh if you look at it basically you you have each node has I would say four fundamental resources you have computational resources like how much like you can execute you have Ram right how much memory you have on random access you have networking like how much bandwidth do you have and then you have a storage long-term storage could be you know hardest magnetic whatever and one of the core realizations that the broader ethereum research Community came came to like several years back is that by separating what is happening by separating these four different types of resources how they are handled you can actually achieve very high scaling and what do I mean by that the whole Paradigm of say layer 2 Roll-Ups is the idea that you can actually uh offload computation and memory outside the network because a single node or like some group of nodes actually do the computation and memory management off chain and are able to prove to the consensus nodes that that has actually happened correctly and since it's a cryptographic proof that anybody can verify really easily so you are basically scaling computation and memory in this way so this is part of the modular layer 2 type architecture that ethereum pivoted towards sources there is Network bandwidth and then there is Storage first I'll handle the easy one the storage it so one one thing people can get confused when we talk about data availability is it's we are not talking about long-term data availability and for example you know there is an nft and like I published the nft and like two months later I want to have or like two years later I want to retrieve the nft uh this is uh called the archival storage problem and the archival storage problem is does not have a security bottleneck because anybody if they had the data even just one node had the data they can always uh prove that they have the right data relative to The Ledger because on The Ledger you record the hash of that data so the long term storage is also actually easy so finally the only scaling bottleneck left in when we talk about scaling of blockchains is Network bandwidth so when we are talking about like the data availability problem fundamentally we're talking about how to scale Network bandwidth and because the other resources have been taken care of in this manner and so the question is at least the way we think about this question is if you have n nodes in a blockchain each of them have a certain amount of network bandwidth what is the rate at which you can run the entire system okay so just to put these numbers in some context in ethereum node requirement the node bandwidth requirement for each node is something like two megabytes per second so that's the node bandwidth requirement and how many nodes are there depending on how you count it there are 500 000 validators or maybe 8 000 independent nodes so we have a lot of nodes right each node has like two megabytes per second bandwidth and you can ask how fast is the system running which means I take the ethereum blockchain the only thing I do on the ethereum blockchain is to just publish data that's all I do then you ask what is the rate at which you're able to publish data to ethereum you don't do any computation no unit swap nothing just use stock wire comes and buys off the entire block space just writing data to ethereum what is the rate at which you can do it on ethereum today that's 83 kilobytes per second here 83 kilobytes per second is the rate at which you can write data into the system each node in the system has like two needs two megabytes per second and there are like tens of thousands of nodes so the scaling to me is basically can we somehow not be limited to such a small number in the throughput can we somehow have the net system throughput increase linearly with the number of nodes in the system for example at 10 000 nodes and two megabytes per second if you multiply the two things we're talking about like in the high tens of gigabytes per second that is the theoretical limit for where we could be and a scalable data availability system would deliver us that so that's our view of what we think and where we think scaling Place one of the quotes from vitalik on this is like we know how to build a blockchain which has a non-scalable execution and scalable data availability and basically uh the layer 2 Roll-Ups offload the um scalable the execution so that you don't have to face it on the blockchain and the only thing that the blockchain needs to do is to scale the data available so that's all this is why these guys are heroes ladies and gentlemen without them we would be we'd be waiting about a week to get one transaction done so thank you guys um we're running low on time so I think we're just going to wrap this up with some final thoughts so uh hold on one second sorry just now oh so we can go along wait we can go longer okay we'll go longer until we get kicked off the stage okay so uh does anybody want to actually piggyback off what what true I'm just saying yeah sure I'll I'll add something I um I agree with sure um that um the property that we want our blockchains our data availability layers to have is that the more nodes there are in the network the more bandwidth the overall network has like that's without that uh they won't scale right and and actually traditional monolithic blockchains don't have that property you can add as many nodes as you want you have a fixed block size um and so that's really important and so far only data availability sampling is able to achieve that looks like we have run out of time my last thing is that scalability although I Define differently than like scaling scalability we Define as last year as the amount of transactions you can verify divided by the cost to verify those transactions so if you and that's why monolithic blockchains don't scale you can increase the block size but you need a bigger node to verify them and in a data availability sampling world you can increase the block size by and the cost to verify that block only increases as the square root of the block size so like it's not perfectly flat but it is sublinear which is really really important and like the key to all of this thank you Nick uh really sorry guys we we're gonna have to cut this short we're out of time but I hope that this panel was was insightful to everybody as an intro to data availability for those who don't know much about it thank you so much guys and I think we'll we'll end it off with just any final thoughts um from you guys and also um a little plug on on all of your projects just so that everybody knows how to find you uh yeah I'll leave it I think I'll go first because in the last question I wanted to say couple of things when you ask that uh scalability versus speed and you know when I look at all the projects that we are building here in each Denver and all the ethereum conferences they come to the same Crux of the problem which is um you want scale you want security you want decentralization and um you as part of scale you want speed and that speed is important for D5 projects that's important for gaming projects I mean I am working on a gaming SDK where game developers can use our Unity SDK and build smart contract wallets um omnichine and nft marketplaces but then at the end of the day if it takes 10 seconds to confirm a transaction no one is going to play those games hence we need 12 scale at every level at the layer at the layer one block Trail uh we want to solve it level two uh the layer 2 blockchains whether they are optimistic roll ups or they are uh ZK based roll ups and again the problem there of data availability also breaks down into two different parts where both the optimistic rollups and ZK based rollups they submit some kind of proof to the layer one that hey these transactions were done and they are all valid transactions so to prove that as well we might have to wrap up I'll cut it okay I'll cut it so basically everything that we are building is going to be around the same data availability problem yeah it affects all of us okay uh sorry we're getting kicked off stage you guys thank you everybody thank you guys for joining thank you if you guys have any questions these guys will be over here as well