IPFS & Filecoin 🛠️ Level up your storage with IPFS and Filecoin! - Sarah Thiam

foreign [Music] top uh Falcon fvm ipfs my name is Sarah let me just slideshow this right my name is Sarah I lead fvm developer experience but I'm also representing Falcon and we have various tools you can use for storage in in our ecosystem so I'm going to kind of run you through what you can use in this 2D hackathon also being super realistic I know how it's super stressful to finish a 2d hackathon and produce something great so I'm going to show you one easy demo and also give you options for more technical versions if you choose that you don't want to sleep tonight so it's really really a choice here right giving you all the options I'm available on Twitter here so if any questions you can always DM me and we have an fdm Dev Twitter account we've been featuring really great devs I've been partnering with us in the ecosystem like zondex and other partners and early builders that have been helping us out so yeah and if you want to tune in on a twitch we have occasionally live coding sessions where you can kind of see us try stuff out it breaks we fail and then we try to fix it and figure it out together cool all right so we're gonna get started all right so I'm gonna assume um you know you're kind of design your project it's already been a few hours and you're like I'm at what I do my storage right um so some simple options for you we're gonna run through them um so we're talking about leveling up right so I'm gonna get from simple uh super easy to deploy uh to get a little bit more um well not complex it's still really easy and then lastly uh super super complex but you know you get more programmability and more logic and more customization together with it so level one we have pinning on ipfs which is web3. storage I'm sure many of you know ipfs or fans of it usually show off hands ipfs use that see there you go there you go um so yeah all of you are familiar with that if you use webview.storage they have a super easy SDK I just did a demo at encode Hacker House um so I think I see some familiar faces here so I will then focus on the level two which is around Lighthouse so Lighthouse dot storage is similar to webview.storage but they have more functionality on the back end so webview.star which was really meant to be super super simple to use pitting on ipfs retrieving it um your data could disappear over time because it won't be pinned there forever Lighthouse or storage allows you to pin on ipfs they also back it up onto Falcon by making a storage deal for you on the networks and very very recently like literally two days ago after the last minute we have been fixing out our verifiable proof for small pieces of data um so you not only do you store your small image as part of a larger data deal on the Falcon network but you can also verify that your small piece of data is stored within a larger deal and exactly where it sits so you could rather than unsealing the entire data deal you can just try to look up for that one small JPEG that you want and then retrieve that so very cool functionality coming in there and then lastly level three we'll be starting on filecoin so you're starting it directly from a smart contract this is enabled by the Falcon Virtual Machine so anyone here has used the PowerPoint virtual machine yeah okay the numbers are increasing hackathon and hackathon okay awesome um so yeah if you use multiple virtual machine we are because of that we are fully ethereum compatible so for the first time you will be able to interact with the falcoin protocol uh prior to that for the past like seven years that we've been building our Network you couldn't really interact with it unless you are a storage provider and you want to run your own node make the deal yourself with the Falcon Virtual Machine we're not fully ethereum compatible you could start a storage deal from a smart contract and from a solidity smart contract so perfect for each Global it does take a little bit longer about 24 hours for a deal to be made but I have seen people submit projects where they show proof of concept of it working the deal doesn't get made and that's totally fine as judges because we know that it gets it takes like 24 hours or so right so but you can definitely include that in your project and the cool part here is that you will be able to add whatever you want to your solidity contract so you can write logic around native storage Primitives we have created a library called falcon.

solidity that will allow you to use solidity calls to then interact with the file coin storage primitive so you could do things like get deal status what is the you know how long is my deal being stored for can I renew my deal and and so on so a lot of things there um I have a if you head over to the booth later I don't think I have it in my slide this is the tutorial for our main contract that allows you to make a storage deal and you can play around with that but if you need more resources I will be at the booth and I have a cheat sheet for you that will have all the fem resources that you need so this is how you go from level two to level level one to level three um and yeah I've tried to simplify it as much as possible so you can kind of like see straight away like what are your options for the hackathon cool so I'm going to run through the architecture a little bit just a quick tldr for anyone who's new to our ecosystem I know that sometimes like for me I joined Falcon and for like three months I thought ipfs and Falcon were the same thing they were not the same thing so surprised um to me and um the difference here is the ipfs is a P2P Network so this is not actually running on the blockchain what you'll be doing is pinning your data to nodes that are hosted by a community or you host it yourself so these are these clusters show three different options that you can use when you use ipfs to make sure that your data is perpetually stored uh somewhere uh well technically it's not stored it's pinned right um You can run your own notes you could run notes of your community or you could pay a pinning service like pinata to do so for you however with the running your own notes and running with a community it's hard for you to guarantee that your storage is going to sit there forever because if you're if you break up your friends or you choose how to run your notes anymore your data could just disappear right so that's why we've created the Falcon protocol so TR they are for those who are new to this the way that it works if you look at the top over here which is the dollar box here you will make a deal by it with a client and a storage provider you would then handshake start make a storage deal and negotiate a deal how long you want to store it for how much you want to pay once the storage deals once the terms are agreed upon the storage provider will then make the storage deal and publish it on chain so on chain will show like your deals metadata right how long you want to start for how big is your file size and so on but the file transfer happens off chain and that is represented by the network you see over here which is All Storage providers here they would then represent themselves on Chain by periodically posting proof of your storage deal to show that hey my data is getting stored every single I think every 24 hours they have to prove they have to submit some kind of proof proof of space time and so it'll prove that your data is being stopped continuously right but then they'll store all your data off chain so it's really up to them how they want to store it and you can choose what kind of storage provider you want to work with based on the profile that you know so some might start in hard drive us honestly some might just print it out and store in a box but that's really you know your choice it's a preference right so that's how the file frame protocol works so it's because it's incentivized as a storage Marketplace it will then guarantee that your data is stored perpetually or for as long as your deal lasts right the way that we usually use it now so uh on a realistic note when you store your data you then want to retrieve it right if you're starting it on Falcon and you want to retrieve your data it does take longer for you to unseal the data that has been sealed cryptographically by the storage providers so it could if your deal takes the 24 hours to make it could then take multiple hours to then until and retrieve your data so um unless your storage provider has agreed to terms which some storage providers do to put an unsealed copy together with your seal copy so that's me that means that they store double the amount of data but that is their choice and we have a really easy tool called Lassie that you can then retrieve your data from so if you're using level three which I showed you just now the falcoin virtual machine client contract if you clicked on that tutorial you could then use Lessie as part of that it's within the tutorial you can use less YouTube than retrieve your data pretty quickly if it's unsealed and on the test net it is right but realistically and on the main net it probably most of the time is not so um the way that you will normally store data we do recommend that you use file coin together with ipfs so if you're planning to build a project um that's something to keep in mind so ipfs will allow you to pin a copy and then you start and back it up on the filecoin ipfs allows you to do quick retrievals so that's how you can quickly get your data however you need it so far I know it's like quite a lot all good everyone do with me yeah great great great great okay so um this is an example of the metadata of your storage deal um and I see a lot of words but I'm going to walk you through it so if you look at the backslashes you'll see peace size verified deals start Epoch and Epoch how long is being stored on chain how long your deal is supposed to last for storage price per airport how much you're going to pay and offer your storage provider and uh provider collateral is not for you client collateral is yeah how much you want to put out for the deal to then make the deal proposal and so on so these are all the meta metadata within your storage deal once like once it is accepted it gets posted on chain right so why did I show you this because this is kind of important for the next point so talking about fem so if any of you are new to the valvecoin virtual machine what it does is that it manipulates the storage deal metadata so all this metadata that you saw over here um that is what the Falcon Virtual Machine will be able to manipulate so you could write when you write logic within your solidity contract and you're using um our falcon.solidated library you are manipulating all the metadata of how you want the file to move around so you're not actually manipulating the data that start within the deal you're manipulating the metadata so I just want to make that point really clear um yeah and I'll talk about price structure in a bit but if you are looking for computing the actual data fvm is also really useful together with bakayal so bakayao is a decentralized compute framework that we have if you want to know more we are at the booth we have a mentor there from the backyard team what fbm will do is allow you to send you know by manipulating like hey I want the storage build to be sent as a job request to bakayal so bakaya will then do the decentralized computation off chain and then post the result back for you so that's how you use fvm and that's what fvm really unlocks there are a lot of a lot a lot of use cases that you can unlock with fvm and we're continuously improving the depth ecosystem to make it even easier for you to build Solutions on top of it right early days we just launched a match so you know a lot of good Integrations coming in so do definitely look forward to that um so I I just want to give you a little bit also the green pad the green circle over there that refers to how it's manipulating the metadata around the storage deal and then that enables you to then do whatever you all the magic that you want to do on the layer 2 on top of that so all the solutions that you want to build you can then build it uh with with those options there so that's how ipfs Falcon and fbm work together yeah so I this by the way this took me like six months to like digest and put into diagrams so I'm trying to give you like a really condensed version but um yeah if you still need product clarifications I'm at a booth you know a hackathon is to learn so if you want to learn more and then build later on that's totally cool we actually have a really big hackathon coming up as well end of August so you can participate in the open data hack there and like get you know all this like cool data stuff yeah all right so um before we get into the demo I don't know why I put this here I should put this at the end but the for the prizes we have like three uh we will give out grand prizes three grand prizes um anyone that's using falcon ipfs or fpm so as I talked through the demo and some of the use cases do have a think about what it is that you want to build um best project integrating fvm with bakl so I talked about Pacquiao a little bit earlier on that is the decentralized compute we'll be giving two teams two thousand five hundred dollars that would be a special price so definitely do check that out and run us runner-ups I think any project that impresses us and you know that is really really cool will give up to six teams part of the pool price which will be yeah really really fun so um have a think about these as I go into the demo okay so I'm going to demonstrate I demonstrated earlier at encode hacker house around the web 3 storage piece I'm going to demonstrate the lighthouse piece over here Lighthouse itself is really cool so I'm gonna if you if you scan this QR code it'll bring you to the documentation I'm gonna show you the page and kind of show you the options that you have and what it is that we will show today this is big enough yeah okay great great thanks um okay so if you head over to Lighthouse documentation uh remember do not use the CLI commands part that actually post the mean net not the test net so you want to use the SDK over here so if you are building your front-end uh it is JS you can actually easily click over here for code samples and then I will be demoing for the node.js backend yeah so you have options uh the cool thing about the difference between what three stars is that web3 storage is a simple storage of any data if you want to have encryption Access Control you can use lighthouse.storage they actually have it inbuilt within the SDK so it's super easy for you to use as part of your project and it shouldn't yeah it should just be on top earlier on top so it should be easy to use and Implement so we're gonna try this one out all right all right so we're going to start with this we're gonna do a really really easy one okay we're gonna create our npm package and um and then we're gonna npm install the web 3D storage file put on so I have to go is over here and you can very easily just copy this it's up to you I usually use this method um which is the wrong one whoops knock knock but they're ready to go hello yes [Music] what oh okay I found it I found it all right awesome thank you all right so we're gonna install this and once you get it set up um at the same time um they also have instructions for creating a lighthouse.storage wallet so you can then run um the storage deals that you want to run okay so we're gonna open up the folder trust it and I'm gonna get back up so over here I'm just gonna create a upload.

js and you just have to head into the into the chord and copy it over here so super simple and I'm gonna upload it from a file directly from my computer so I've recreated like a test file over here whoops you see um let me create a new I'll create a new text file over here so I guess I can see like hello oh sorry we're in friends uh bonjour okay great I'll save this um ESO ball awesome okay now we're gonna get the path all right okay I'm just gonna drop this in here for a second and uh we're gonna create the file from here so I have my I generated an API key earlier on um if you want this you can easily do this here so I do use this yellow commands for creating the API key I just find it easier um you can just easily copy this in and generate our API key to use it so what let me just check for this hold on okay great so I created a a wallet earlier on if you if you are new to this you would then have to keep oh head over here and create a new wallet as I missed that step um but I've already created that wallet so you create a wallet first once you get your wallet set up you can easily head over here and then get your API key so I have it over here which is great and let me just copy this and head back over here yeah but we're gonna let me key in my path first no no this one this one okay over here great and right so yeah if you want to upload any small documents that this should be a super easy way to use it and then what you want to do is add in your API key okay let me just check this first save it parents make key oh crap oh I did generate one already I'll see okay so I'm going to copy this again and I'm going to paste it back up here cool and I'll receive it and hope this works no this doesn't work but um because I will add in a type here so so yeah if you hit this Arrow definitely go in and hit this um this often gets me going and let's give it a shot oh see and yeah so once you have your file you have generated your Content ID so for ipf and default coin we I we ID our data not by location but by the content itself so this is how you generate your content ID and when you want to view it it's super easy to just head over to an ipss gateways I like using brave because they have ipfs plugins in there already so just key in here and you should be able to open up your files so once you download it um let's try it out great so that's a really just easy way to show you how to add storage to your project again they have many many different functions here that you can use there are different examples uh especially you see this used a lot for yeah for teams that want to use encryption so that's a really cool feature and that would be the lighthouse storage demo so um pretty similar for web storage you would have the tutorial link from the earlier QR code if yeah if you want to check it out this is the QR code from lighthouse.storage um some cool Innovation coming I just want to kind of show you guys a preview of what's coming up but um we have this so like once so basically what you saw just now was your file being pinned to ipss and you're retrieving it from an ipfs Gateway um but what it'll do on the back end is to it'll take like about five to ten minutes on the test net to generate the deal on the test net network but in in on the mean it'll take an hour or so but once your deal is generated you could actually retrieve proof for the small pieces of data like I mentioned earlier on this will be via API request uh we literally got this running like two days ago so I haven't really put this into my demo yet but to kind of give you guys a sense that you will be able to retrieve all your different pieces within a single deal so you can actually check that out and once we get it integrated with our Explorer you can actually just key in like your sub pcid so not just the CID for your deal but the sub pcid for your small image and you can actually view it and see where it is sitting within the deal itself yeah so just as a quick preview to not not available here you're using ZK verification via your solidity contract so if you head over to level three just now you would see again I have a starter kit for this using the level 3 version that you saw just now in the earlier slide so do check that out but you could then add ZK verification in to then verify that you are a legit contributor and you can upload your data via the client contract into your data set and once you do that you can also add in another layer of contract to then generate tokens and you can use your tokens to vote to then train an AI model for example so if you look at the top example there think about how you can connect sem to bakayal connect your data to a decentralized computation framework and you can get access to those two other prices yeah and there's also access control but you know you can use Lighthouse for that so just an example but yeah this is the data starter kit so if you want to use it for this hackathon super easy um piece there so I know it's a lot of information but yeah we have a lot of cool stuff for you guys to play around with great yeah that's more or less it I know we're almost at time so yeah thank you yeah uh any question yeah hi so did you say that Lighthouse is available right now or do you say it's available right now okay and uh so as far as I understand you upload to ipfs and then I didn't understand what Lighthouse is doing on top of that yeah so uh Lighthouse is like an aggregator kind of so we'll take your data um they provide a really easy SDK for you you upload your data they pin it to ipfs for you and while you are you know you can access the ipfs pin data it also takes out data and puts it into a storage deal and gets a storage deal made onto filecoin for you so backs up your data yeah okay so it pins it centralized by Lighthouse and then also makes a deal in filecoin which is decentralized yes yeah um just a FYI we're also working so Lighthouse is an example of a centralized aggregator uh one thing that we're working on in the next few months is a decentralized aggregator so we are providing we're writing specs right now for you to be able to build your own decentralized aggregator for uh any other questions around storage thank you very much okay cool thanks so much