IPFS 🛠️ IPFS in JS; Helia ramp-up - Russell Dempsey

foreign [Music] and welcome to the hack FS ipfs workshop ipfs and JS Helio ramp up joining us today is Russell Dempsey who will be taking us through the session and with that I'll pass it over to Russell to get the session started all right thanks okay so let me get my uh windows in the right order here Okay so yeah um today we're going to be talking about ipfs and JS but uh Helia the new implementation of JavaScript uh uh of the ipfs and JavaScript um my name is Russell Dempsey I go by Sergeant Pookie everywhere I'm the maintainer of ipfs web UI desktop companion with with another teammate of mine nashant who does most of the companion stuff um yeah I work on uh IP stewards team here at protocol labs and I help maintain the JS tools and guise and things like that um just click the intro there in in these slides I assume we can we can get a link to you know all the viewers um there are two videos that I think are really good for getting an intro to ipfs so if you're not familiar with with ipfs and and the basics and you know how the the content is is created and what cids are things like that um these are two great talks even if you do have a basic understanding I highly recommend watching these if you haven't um this first one is from ipfs camp in 2019. it's it's excellent It's really in-depth there's live questions um and then discordia and Ryan uh shared this one with me which is one he highly recommends so check those out if you haven't um I'm not trying to start that right now I'm trying to sorry about that go to the next Okay so um we've got some uh Helia resources where you can um you know check out uh migration guide and things like that I'll link to some of those in the demo here upcoming um in a few minutes but the links are here in the slides uh the wiki the wiki has a lot of things um the migrating from jsipfs is probably going to be the most useful for users who are familiar with um JS ipfs and then there's a bunch of other links here there are some recordings and presentations that have already been done by the the main maintainer um Alex and so yeah you can check those out here from those links um and uh that the Helia examples are great sort of like starter kits for for using your your favorite um where oh yeah I'm full screen here I'll just do this yeah they're great starter kits for using your favorite Library so we have uh like Helia in electron uh Helio in in like using it as in um CJs if you're not you know updated to esm um and Helia 101 which kind of walks you through some of the basics things like that a few other examples here later for my demo I'll be going through the Helia Vite I'll be using heliovite to kind of walk you guys through and then um mostly just walking through some of the other examples how you know what what the examples repo is like because that's where a lot of our learning is going to be if you're more Hands-On learner um and then yeah in general if you go to the ipfs slash Helia repo you can find a lot of information there um now what are the benefits of Helia um sorry I haven't been looking at you guys the whole time um so the benefits of Helia are really um you know they're they're multi-fold even even in web 2 you know there's lots of uh use case studies and benefits of ipfs and web 2 and you know a lot of those benefits are the same with Helia but Helia over JS ipfs or some other implementations um are kind of what I want to focus on here so um the P2P support via lib P2P like that's going to be the same with most ipfs implementations um but a lot of our users are in the browser and so if you're doing things in the browser or if you're Tech stack includes typescript or JavaScript Helia is the way forward that's what we're going to be developing you know we're we're in the process of deprecating JS ipfs so this is where things are going to be in the future um you know with with ipfs you get the negative bandwidth scaling costs and resilience against centralized outages and then um primarily with Helia the benefits uh are our developer experience it's much more modular it's an esm um and because it's modular it's it's extensible um but but yeah a lot of the same benefits you get with using um any ipfs implementation really so um pretty basic there um so yeah the demo I want to hop into the demo um that's kind of what this Workshop is so if if you all are out there I think you know we've got a few few viewers here yeah so you can go to this ipfs example slash Helia examples repo um this kind of you know tells tells you all the things but in short if you just go to this examples um folder in the repo and the code then you can see all the examples we have available we've got you know an issue open where we're going to be porting over all the old JS ipfs examples but um the basics are that you know the the each of these examples like we we consolidate them inside of this examples repo this is where the core source code is but then anytime they're updated here they're actually published to their own isolated repo that you can Fork from and then then sort of extend oh thanks Rory um so so yeah like if you're if you're going to start a new project and you want to use Vite or start a new project and you want to use you know es build or next.js parcel uh if you're using Vue or want to use webpack you know you would go to ipfs hyphen examples slash like heliovite or or you know Helia Dash whatever the name is here um so pretty basic so you know I started up some code and what I did was I you know copied the GitHub CLI code for um cloning repo and is this the right this is not the right one one second here this is the one I want and this is something else Okay so so yeah I've I've modified this um let me increase the size as here is that too big or can everybody see okay that looks good okay cool so um yeah I've got a ton of stuff set up stuff here so um you know I clone I cloned this repo and then um made a few changes just to export just to set the peer ID so we can see that on the screen um just clean up some of the code there a little bit um and then I'll talk about debugging a little bit because that that really helped when I first started using Helia and then I exposed it on the window um or globalness so that we can kind of toy around in the browser if we need to so if we go here um it's been running here and it's emitting some errors for connections to peers you know that don't support certain transports or you know there's lots of dials with uh lib P2P and ipfs when you're trying to connect to other peers um and so they're not all going to to succeed so those errors are normal um but yeah I created you know a little um text I'm going to copy that but like you can you can add text here and um it will generate a text as CID you know give you the CID let me just post that there and then with the CID like ideally what I see a lot of people do is they go to you know ipfs.io ipfs CID you know and they want to see that content now this one it's Fubar so there's probably been you know this CID probably exists on tens of thousands of developer you know nodes that they're running and testing and somebody's probably pinned it so so it's obviously accessible but the one thing that that you know people run into is they're generating some new content you know some new custom uh content that hasn't been created before or isn't um pinned right so like some content that that doesn't exist and then they try you know then they're they're hosting their you know their nodes running their ipfs node whether it's Helia JS ipfs whatnot is running and they try to access it here and you know the Gateway will time out because either you know the the Gateway that we've got configured isn't um you know it's got a timeout too short to find your node in the hundreds of thousands of nodes in the ipfs ecosystem or or whatever reason um it's it's not there and so there there's a tool that I wanted to to talk to for you guys while you're hacking that that is really really handy um for for trying to like debug why your content may not be available so like I just generated this one here's the CID um like the Gateway is going it's trying to fetch it but also you know I can I can go up here this is PL hyphen diagnose uh dot on fleek dot code this is a tool that Laurent one of our one of my teammates put together uh last year I think that's it's really handy for debugging things so you can put in your CID here and then this this is more of a tool for for like it's a debugging tool so unlike the Gateway which is intended to support people's like retrieving of images and things like that this this is going to search you know give a little bit more time and search the the DHT for certain content um so let me let me actually copy that um I'm just going to enter that so I can save that for later I'm going to fully refresh this is green so it's saying that heli is running I have this pure ID I'm going to paste this again should give me the same CID because that's how ipfs works right um and so Helia is like that's the wrong CID something else maybe I didn't paste the right one yeah so this this tool will help you figure out like is that CID actually available and then if this fails for some reason it's it's not you know your your node isn't configured properly so what we should be able to do is see now that our peer ID for this Helia node was found as the provider of this content and so there's some other there's some other um addresses here and ways of connecting to that peer so um let's see quick P2P um TCP won't work directly to the browser so we don't we don't want that browsers can't do TCP uh networking um so let's say do we have a there was a relay one I saw no there wasn't well yeah so this this isn't a valid peer-to-peer mult uh address um for this form but anyway we can see that the content is being hosted um by our by our peer so we know that the Helia node kind of published that content to the DHT and it's providing it we really just need the um peer ID so what we can do we have P2P circuit ap4 that one should P to p and then we want um is that copying that fully [Music] um so let's get this peer ID yep it's that one so that's going to be some relay stuff I believe that's providing that we don't want ip6 because that won't work on my machine see if we can do this one yeah so that's that's not working but what we can probably do um yeah we should we should be able to get the peer the the multi-address um if we see you later P2P dot peer ID which time do I have 10 minutes left I want to leave some time for some questions but but you know you can access with Helia there's there's the lib P2P object exposed um which is how you would do a lot of the networking stuff and you can like um dial the closest pier and do like DHT stuff live p2p.dht [Music] um dot get closest peers um for that you're going to need an actual like peer ID instance and you know um to get that and I don't have that this is from old code but um see if I don't think it works with strings I think you need an actual yeah yep but yeah you can access so you can you can toy around here with the lid P2P object um uh you can say get appears and see how many peers you are connected to if you type it correctly um can you guys see that yeah so I'm connected to four um peers right now so yeah it's doing it's it's doing stuff it is available even though you know this this particular CID is not accessible here right so so the content is in the DHD just not necessarily accessible from the Gateway so just while you're testing and things accessing from the Gateway is not the most reliable I did I did want to get that out there and then for developing Helia uh just know that like a lot of the I'll I should share that link that'd be P JS the b2p configuration so yeah this um if you're troubleshooting some networking stuff this is a very valuable resource um because that's that's how we you know do networking Helia is sort of like a um central place where like we're connecting live P2P and then your your um the blocks of the content for your cids and feel like I should move on but yeah that Link's very useful this link is very useful as well PL diagnosed very very very handy um let's see here so yeah I do want to go to I think I went over quite a bit actually um I do want to go to some project ideas just things that I threw together um could be dumb could be not whatever so kind of just a brain dump of ideas but um you know we have ipfs web UI which is based on like a UI representation of Kubo and so building like a helio web UI could be really cool so we could get those multi-addresses um easily for PL diagnose um and uh like being able to instead of running to um web UI Dot ipfs.

io and seeing this screen where like it needs to connect to some back-end node which it will connect for me in a second probably because I have a node running it might be on a different port right now but instead of seeing this screen like it would it would just work which would be really cool um foreign web apps so I worked on something a few weeks ago for ipfs thing and we essentially had um Helia service worker Helia in a service worker kind of working as a Gateway so you could actually like on EV on different web apps have a service worker that that automatically supports ipfs colon slash slash uh CID content and and then um by having this service worker it would enable all web apps to essentially have their own node running in the browser that can publish and retrieve content from from the network there's some gotchas around intercepting fetch calls with service workers and different different things there that you know could you could run into and then only one service worker per origin so for web apps that already have a service worker there's an issue for that but could be really really really cool uh problem for for moving web3 forward um conference Workshop code sharing there there is this Tool uh share share.ipfs.io this is using jsipfs but essentially like you can add files let me see if I um let's just do this you share these files and then people can scan this and ideally they'd be able to You Know download that easily but doing this in a way that um is fully offline would be really cool um see it's it's posted up above uh there's a question in Zoom so I'll repost it here you go um yeah this could be really cool the problem is like if you can connect appears on a lan like that's that's kind of a problem at a conference um just adding Helia support um into web apps to You Know download you know your maybe you have a game and you want to use Helia to to Cache content and then have your peers help you serve that content to your other peers um actually one tool that I kind of talked about uh or didn't really talk about a ton but is universal connectivity this is an app put together by the lib P2P team and a ton of others um I'll post this in the chat um that that kind of shows how browsers can can you know uh browsers can connect to each other so it's it's really cool expression of Technology this is not using the Helia but Helia is just you know it wraps live P2P so anything that lead P2P can do Helia can do and then do ipfs on top of um so so that could be really cool um reduce your bandwidth waste reduce you know increase your your resiliency the more users you have of your app and then uh there's if you're paying attention to Reddit at all you know there's issues with Reddit so doing like a a Reddit uh replacement using Helia could be really cool or any ipfs implementation could be really cool um you know there's there's yeah that's a big undertaking but doing like parts of Reddit you know using Helia could be really cool um I would I would love to see a Reddit replacement or or social network on ipfs that that um I would love to see that um other project ideas uh improving existing JS ipfs projects by migrating to Helia like um that could be that could be a good Avenue to go like if if you've ever tried something with jsipfs and it didn't quite work right um you know migrating that to Helia could be could be pretty cool or just getting ideas from existing like things that use Js ipfs um and then yeah q a is supposed to be 10 minutes I think we have four minutes but um yeah you can at me I think Discord just prompted me to create my new username so there's no longer a star 7787 it's just Sergeant Pookie on Discord uh uh yeah you can find me there or in the partner ipfs um channel in each Global Discord so any any questions yeah if you have any questions please feel free to take yourself off mute or to type them in the chat and as Russell said you know please definitely feel free to reach out in the Discord uh directly or in the partner channels one of the best ways you can get that sort of help in mentorship in case you get stuck on anything yeah and if if you have trouble using any of those examples um feel free Theresa the Discord it's uh on the screen it's a sergeant pookie yeah and if if you do want to DM me that's oh sorry um if you do want to DM me that's great but also if you ask in the partner ipfs channel that gives an opportunity for others who might have the same question to benefit from from your learning so if we can keep the questions in the public that's great if you have something you know you might not want to necessarily make public it's fine you know that's fine to DM me but if it's if it's something that I think others can benefit from I'll I'll point us back to the partner ipfs Channel yeah that's a good point and there was another question uh everything we can do with Kubo we can with uh Ilia right that is a very good question and I won't say everything but I'll say um you can do most things the majority of things yes um and then like if you're running Helia on the back end you can do TCP like if you're running it um as a back-end process and node then you know you do have access to that TCP transport um so so you know it can do a lot more um that way um and then there there are some different codecs and things that aren't uh supported right now like um let me see Helia uh let me just do it so like there's Helia Unix FS um excuse me um which you know is a library that we publish but if you look if you look at the at the commands in different like how how things are extended or maybe I'll do a dag Json yeah we'll do dag Json because this is It's one file it's super simple like if you wanted to add support for a codec that's not supported it's crazy easy to extend an add-on and it's in JavaScript like one of the most popular languages in the world or typescript um so we've got types um so really easy to modify if it doesn't do something that you need to do but yeah there's examples out there so like if you I had to do for I was migrating um the Explorer uh tool just as my personal work um got a PR out now but I needed some additional codec so I you know was able to modify that um very easy and add existing codecs that weren't you know ported for Helia syntax yet like very very easily it should be much easier to navigate than JS ipfs it might just if you're used to JS ipfs it might be just a little bit odd to get used to but that migration um migrating from JS ipfs should help and if it doesn't cover a scenario or API that you think it should um please like at me or or open an issue you know we we definitely want this to to you know help users start using Helia but this should cover a lot of them well we're at the top of the hour now so oh there was another question how difficult would it be to add clustering ipfs clusters only go right so it depends on what I I mean the the ipfs cluster I mean if you think of clustering as a you know group of nodes that's extremely easy um with Helia so so you would just um with your uh lib P2P bootstrapping with your lid P2P bootstrapping you would have you know your list of nodes your your you know dialable addresses that you would boot up like 10 Helia nodes on the back end and then connect them to each other or um like you you can do like a CL a cluster in the browser I guess but it doesn't really it wouldn't work that great because of browsers limitations of of you know the number of active network connections but yeah you would just like bootstrap to those other nodes um and we do so there is like swarm support I think there's an open issue on ipfs Helia right now for like how you would do swarm things um but yeah it's just a cluster is really just a single node or a single endpoint um that is fronting multiple different nodes right so so yeah you could you could do that fairly easily you'd expose using nginx or whatever you know your your front access to those singular nodes and then have those you know have that single node at the front like bootstrap to your back end ones or something like that but yeah I think ipfs cluster itself like is a is a library that protocol Labs put out that is just like for Kubo so that terminology like in that way that is only Kubo awesome what's yours I don't think we have any other questions thanks Russell for this great presentation and thank you all for attending with so a few more workshops uh left for the day but I hope everyone has a good rest of the day all right thanks cheers thank you bye