Protocol Berg: Garrett MacDonald - Permanent Decentralized Storage Landscape in 2023

hello everybody good day um my name is G McDonald and my main mode of work is uh working at a venture fund called permanent Ventures where we invest in decentralized crypto infrastructure um with a particular focus on decentralized storage so today wanted to give you guys a more high level overview of the decentralized storage landscape um going into the various offerings that uh that exist um so the most popular ones that everybody talks about are obviously um yeah filecoin and arweave um there are a good amount of honorable mentions throughout time I mean storage being one of the first ones that gained popularity um you know 78 years ago and um and the rest there have been a lot of attempts um yeah I think just something to note when looking at all of the offerings that exist is that it's extremely trivial to artificially seed data to any of these networks especially the less expensive ones um so just looking at the total data stored as reported by the networks is is not the only metric that you should consider when um deciding which is the most popular so the way that we gauge kind of what is the most popular is the most Integrations with popular projects um and that puts both filecoin and rwe kind of at the the top of the list um storage although you know all of all these at least started out is very Noble efforts you know storage I would argue is turned kind of into a zombie like not doing any kind of significant innovation in the past you know 5 years or so um and Skynet and Zeus both kind of retreated into generalization um shifting away from you know dedicated on like storage offerings so I would say they're not all that honorable but if I remove those from my list I'd say those are the ones that are currently contenders to you know kind of get into um you know uh like the most useful out of All Storage protocols um so definitely worth looking at um and keeping track of although I would focus this talk today on both FAL coin and arweave as um they might be the most relevant for developers who um are actively building a production products right now um so topics of interest when comparing decentralized storage protocols uh both user journey and protocol perspectives um so on the leftand side is like the kind of process that a developer will go through to you know uh integrate protocol um and so on but today um we'll focus on the protocol perspective um and we'll go line by line through those items just doing a side by-side comparison of uh filecoin and rwe um just to give a good overview of what to expect as a developer uh when integrating either of the two um and don't worry about scanning this now I'll put it up again at the end um if you didn't get it um yeah so we'll start with nodes um so we'll um we can start with filecoin it has a distinct noge types um not notably the first two nodes are actually not blockchain nodes they're uh offchain nodes for storing data there's only one node type which is the the the blockchain node or the full node um which runs the blockchain network and the consensus which is not dependent on the storage at all actually um so the storage on filecoin you could say is kind of offchain and the onchain uh node is more of like a decentralized matchmaking Marketplace for uh storage providers and storage customers um there's a huge Advantage with filecoin node implementation though is that um you know uh there there's two implementations each in a very popular language one of them is focused most on like Chinese local and um yeah go and rust so it's very easy to get developers obviously for these um on arweave there's a single full node that supports you know Data Mining and gossiping and sharing and all that um data retrieval is done in a traditional blockchain style as on ethereum through uh either running a full node or if you don't want to run your own full node you run a Gateway um as in you know infura or or pocket Network and um there's there's a big downside where there's only one implementation in llang which is a very obscure language that today is used mostly by Telos because it's known to be very reliable and robust but the huge downside is there's not uh not many developers that actually know that language um so finding additional protocol developers is is is a thing so yeah that's it's obviously a big risk you want more than one protocol implementation um with a robust Network so it's a downside um so yeah proofs of storage uh rwe and filecoin go about this in quite similar ways actually they have uh small nuances there but uh effectively they're both accomplishing the same thing um they want to make sure that miners are not getting rewarded for storing the same copy of data um like multiple times they don't well they they want um It's Kind of a Funny wording but we want to make sure that one minor isn't or like 1,000 miners are not uh getting rewarded 1,000 times for storing one copy of data so AR we and filecoin each have this process of either sealing or packing where um where in you know in the file coin's case they create a ZK snark of the data set that's being stored and um and this ties that data set as like individually to a particular file coin mining address um rwe does the same thing um the difference there is you know file coin's using a ZK snark um which scales up very well with specialized Hardware um but it's very expensive to do that um but uh but it accomplishes that and then there's R weave which uses randomx which is geared toward using uh CPUs um randomx is the same algorithm that for instance Monero uses as their hashing algorithm for mining um to be kind of specialized Hardware resistant so they want to discourage use of gpus and fpgs and so on um mostly just to make it minable by by anybody who doesn't necessarily have that specialized Hardware um so then the proof of SpaceTime and the sync proofs of Random Access are you know two you know slightly different ways of accomplishing the same thing they both want to make sure that um the the storage miners actually have access to the data that they're getting rewarded for So Random requests to prove access to particular chunks of data uh come up and as long as the minor can prove an access to that particular chunk of data um then then it's allowed um rwe also has a verifiable delay function which essentially just makes it so um there's ample time to seed data to the network before a minor can get rewards for it so that way a minor can't create a bunch of data on their own uh on their own node and get rewards before everybody else has a chance to download and seal or or pack that data um yeah data persistence is a super interesting topic as well um the the two different ways of going about it um rwe does it on a protocol basis where they're trying to guarantee Persistence of the entire data set that's ever been stored up to rwe uh filecoin the user developer selects um and and defines their desires for replication um so you pay more if you want more replication less if you want less and um and this and and this you can also Define the amount of time you want to pay for um on R weave there's an assumption that's just blanket made across like anybody who wants to store data and that's your your basically just paying for 200 years of storage times 20 replications um and uh and and there's a kind of a key a key difference here um the there's an additional feature in R weave that kind of tries to enable that persistence and that's this notion of an endowment um so 95% of the transaction fee that you pay when you're paying that you know 200 years and 20 applications uh doesn't go to the minor it goes into this endowment which is held by the protocol it's not like a dow where there's actual people voting on it it's held by the protocol itself and this will automatically start adding funds to the um to the minor rewards if the amount of data sets uh replications of the entire network goes down to around 20 it'll slowly ramp up rewards coming from that endowment um of course if the endowment runs out of funds then um yeah it's kind of broken um but you know ideally that doesn't happen um and then on filecoin it's um it's it's using the same kind of proof of appication proof of space time um to ensure that a given contract is fulfilled that happens on chain through proofs being being submitted from the storage providers to the to the blockchain nodes um and Falin is also going to implement um long-term persistence VI smart contracts that will automatically make new deals with new storage providers over a period of time so that's an interesting offering which um isn't working yet but it should be um good to have like a an alternative offering for um persistent storage over long periods of time um yeah token economics um you know each Network have their own token um the uh rwe only has one demand driver which is storage um so you know people wanting to store things have to buy the token um file coin also has uh staking so it's a huge additional demand driver um where yeah if you want to actually mine filecoin you're required to stake funds um so this this uh this has the upside of creating more and the downside of um creating more sell pressure because the miners get you know additional rewards from the staking and uh that requires a higher inflation schedule um so yeah AR we is able to have much lower inflation but um also lower demand because it's not um it's not like uh boosted by the by the staking um yeah content addressing fairly similar although fcoin uses ipfs for Content addressing so you can upload the same same piece of data twice and uh it will have the same hash so you can re-upload data and and recreate a hash in case you know a storage contract expires um there's I no need for that in R weave because ideally you know if the system works then you upload a piece of data and it's there persistently um so each time you upload a piece of data on R weave it gets a unique hash and uh you can actually use that if you like to you know have additional uh persistence the network you're going to upload the same piece of data multiple times of multiple hashes so instead of having a default minimum 20 you have you know 40 and so on if you Gap that out over time then maybe that doesn't land on the same disc and then you have you know additional redundancy perhaps um so filecoin users self- select redundancy um by you know selecting how many copies of the of the data that they're uploading uh they wish to have stored on the network um and are weave focuses on you know protocol level redundancy where you know the protocol itself is is handling ensuring that all the data is stored across all the nodes which exist in the network um because of this you know are we if you lose a piece of data you can kind of just grab that back via gossip from you know one of your neighbor nodes um and so there's no like real need for protocol level um um kind of like data uh error correction but filecoin implements eraser coding which is kind of just a modern technique for you know same disk um error correction so you can think of it kind of like a raid that exists like dis as agnostically um and then yeah finally there's just um interoperability um yeah filecoin has really nice uh ipfs integration ethereum Bridges it's very well supported by the ethereum ecosystem uh because of ipfs being so ingrained with it um or we takes a more kind of General approach and has HTTP gateways which you can access all the data from whether it's read right um You can also call Storage via Smart contracts so um yeah you can have Smart contracts automatically uploading data without um you know exiting a blockchain um so just in case somebody wanted to take a picture of a kind of overview there um that's the summary of kind of the key points in all the slides um yeah can give you guys a minute to do that but um but yeah I think in general both both protocols have a lot of Merit um and uh yeah it's it's uh it's just uh important as a developer to consider you know which um which compromises you want to make um in order to um yeah achieve what you want so um yeah we're currently working on a full uh paper with uh all the links to the sources for the data in this in this presentation um that's my email and thanks very [Applause] much yeah and uh if there's any questions Happ you to Happ you to take those uh let's say in a year or five from now and these things are used heavily what kind of and these two there are no Alternatives these two are still there what kind of use cases you think would fit like one and what would fit or gravitate towards the other um yeah if you want less expensive storage for temporary purposes um yeah filecoin totally makes sense if the uh implementation of persistent storage on filecoin VI smart contracts comes about you can even have long-term there as well um by default uh you know R weave is always persistent so it's going to be more expensive um so it's only use cases in which you know the data is like important enough that you want to make sure it's around for a very long time um yeah I guess that I it's it's hard to get into like the individual use cases without taking a long time but uh I hope that helps with like a general response cool any other questions just one in the by the TV sorry all thank you very much first and foremost um so I'm wondering um about some other Alternatives and how they play into the you know your opinion about the whole landscape uh perhaps also Beyond storage if we're talking about compute bandwidth uh for example unigrid I'm not sure if you're familiar with this they're trying to set up an entirely decentralized internet you know in multiple you know sort of functional areas maybe a little bit megalan but you know good good on them uh but also um ipsf of course you mentioned but also Golem I think is still a thing maybe in the landscape um any take on that any particular reason why why you uh left those out of course if you don't know them yeah but Golem then at least for example yeah Golem um I think the main reason leaving it out is just not hearing from like a lot of developers that are not seeing Golem like integrated with many projects um I used to track that in the past but kind of less and less over time um yeah actually I just forgot to put it on the presentation um but uh yeah perhaps it's also should be should be there um but in general looking at the more generalized compute approaches um these these projects are super interesting but just so hard to pull off um yeah like uh for instance I really want to like herbit for example but um you know it's it's just so so difficult um to to pull off things like this or like I exac in the past was a super interesting attempt at like a decentralized compute platform but generalized decentralized compute is so difficult hey I um sorry um do you see any uh attempt from ipfs Fon to do something similar to what rwe has been pushing forward with uh content licensing via Universal data license for Via transactions and tax and so on a similar approach from the set of ipfs Falon Etc or how would they go towards these kind of challenges yeah I haven't seen that kind of attempt from them I guess it's more difficult because you have all these local data sets that you have to get to consensus um so where do you store that data first of all in like an easy access place I don't know I haven't thought about implementing like udl and those kind of R features to filecoin actually um but I'm not aware of those efforts but I also um would ask somebody at protocol Labs I guess yeah yeah cool okay well thank you all very much and have a nice day woo