IPFS & Filecoin 🛠 201 Breaking Free from the Client Server Model

[Applause] [Music] good morning good afternoon good evening depending on where you are um welcome to like week two of hakka fest uh we have discordion back again for a second workshop for hackfest for um ipfs and filecoin um topic is 201 breaking free from the client server model uh as always questions go into the zoom chat and discordant we'll answer them in his workshop and with that being said i'll pass the mic off to discordian alrighty yeah thanks so hello everyone uh once again i'm discordion a community engineer from protocol labs um i'm here to give you a 201 talk on breaking free from the client server model this is meant to be a follow-up to the intro to ipfs filecoin workshop having knowledge on what a server is and some knowledge but ipfs filecoin will be a huge asset to following along but i've tried to structure this presentation in a way that it should be informative and hopefully enjoyable to anyone interested in our ecosystem or the distributed web in general if you have any questions during this workshop please feel free to ask them any question i don't get to during the workshop i'll do my best to get to afterwards alright so let's get started first by calling back all of the noise in the background and let's focus for a moment on what the client server model even is through this we'll rebuild the graphic and hopefully with it your understanding of the concepts at play as a whole so what is the client server model well today only a few companies are responsible for serving up most of the web we have google amazon facebook and vastly just to name a few these companies combined service our ads web pages social media videos images and allow us to host servers and services on their infrastructures to interact with these services we also have to use their apis sending some data to their server and eagerly waiting for response so we can process it and do whatever magic we need to do to create our apps and websites there are several problems with this design though from costs to privacy free services are often funded by ads further destroying our privacy as all our data is routed through a few central points paid for by our attention and private information for the most part all our data routes through their servers if you want to view a video send a message collaborate with remote teammates or do anything on the web at all early you're going through someone's central server this principle is fundamental to the client server model we use a client such as a web browser or chat app and communicate with a single entity and that's the server in the client server model relying on just one point for our source of truth also opens up a whole host of problems like trust currently we have ssl certificates to verify an entity is who they say they are however that doesn't do us much good if that service is compromised and even if we are sure of who the entity is it still doesn't help us know how advertisers or others are using our data or even what data is collected for the development side the source of truth in this instance could be the result of an http api call if for any reason the location of that api cannot be accessed your service could also be impacted today a popular way to build is to lean on the apis of another remote service which makes the web very fragile if the service you rely on goes down so does your service in the second half of this workshop i'll go over some other ways to communicate with outside entities in a distributed way to retrieve information without leaning on just a single entity too much in the client server model generally if you want some data you must know where it is if you want some image you generally need the full url to that image to retrieve and share it if that image is no longer available on that service you either need to search for it or upload a copy on another server to make it available again sharing the new location of that image this is assuming you even have a copy yourself this principle is called location location-based addressing and is how most of the world wide web functions today if you go to a website it has a url and all data from that website is addressed by the location of the data if the image analogy is too hard to follow think of how you would share a video on your favorite video platform you'd likely just share the url to the video also known as the location of the video by this point you might be wondering without addressing my location how would i even find the data we'll get to that i promise but for now i'm going to finish unraveling what the client server model is how it functions and some of the drawbacks to the approach as a whole for the wider web speaking of drawbacks a major drawback of the client server model is that a single outage can take entire services down think of when aws goes offline fastly facebook or youtube these events are treated as news and often slows or even halts productivity when one of these services goes offline it's much more impactful than just a single image or video being lost you've now lost access to your ability to even make new data available at all all the data you have hosted in these central hubs is temporarily completely inaccessible or worse permanently the location of the data simply cannot be accessed because it's offline you cannot simply ask for the data from the wider web either and expect to receive it even if another related entity has the very data you're looking for you could of course host your own service maybe distribute the locations of your servers in a way where outages are rare let's say you even come up with the next big idea for a website or platform where you're in control of all your own infrastructure not relying on some other central entity for your needs well that's my segue into my next point scaling can be very expensive these expenses don't always maybe not even usually happen linearly meaning an explosion of traffic can easily translate into an explosion of costs whether you host your own infrastructure or you pay someone else to you'll be paying for it if you want it to scale and run effectively the more users you have the more traffic that generates your cpu ram and bandwidth expenses ramp up on top of this if users end up depending on your service for one reason or another you're obligated to keep the service running and available if you want to retain users and keep them happy not everyone can afford these costs and they can be quite daunting especially if you want to create uh if you just want to create a fun project technology or even a blog a lot of companies as i mentioned previously turn to advertising to supplement these costs too resulting in much of humanity effectively selling their private information in exchange for free services online speaking of content creation content creators are often completely locked to a few platforms like youtube instagram and other similar platforms those can be subject to a whole host of rules and regulations as well far more restrictive than what your local government might allow and you have no say in when or how these rules change if you want to host any content at all on one of these central entities you must of course ensure you're abiding by their rules and regulations this can be quite unfortunate when popular content gets lost to rules changing or updating as is quite common with video and graphic media today as a content creator sometimes it can feel like an impossibly daunting task to solve this problem especially when people become so dependent on algorithms pushing their own or pushing their content to users sure one could set up their own services as i mentioned previously however not everyone knows how to do that effectively and they're not prepared for the time and costs associated with that from social media to blogs to forums anything in between anything on the web at all these are all the fundamental issues with the client server model to summarize the drawbacks seem to be one only a few companies decide the rules for what's allowed on the web and control the infrastructure surrounding the web itself two the source of truth comes from only the entity you're interacting with so if that entity or server is ever compromised there's little if anything you can do about it three data is addressed by location so for any reason at all if your data is no longer available in the same location it's effectively lost four an outage of any of these mentioned servers can not only remove your content from the web but can also completely prevent you from effectively publishing new content at all five scaling with this model can be very expensive and six it's not you or your government that decides what content is allowed to exist but instead a handful of companies i think we can do better breaking free from the client server model means rethinking how the web works as we know it today this is the stage of the presentation where we get to start talking about the fun stuff the client server model represents web 2 as we know it today love it or hate it i believe there's a clear and strong argument for how the web 2 model referring to the client server model can be improved what do we call this new model this new model is referring to the distributed web so let's talk about that and how ipfs fits in but first for the uninitiated let's briefly talk what the distributed web is and what it means to you whether you're a user or a developer you can benefit from the distributed web what i think of as web3 the client server model is a centralized model this is why we're always talking about central servers and single points of failure pictured on the left is this model you can see a central point in orange this is the server and then the spokes are the clients who are often the users this graphic should make it easy to see how if that central point is ever removed then the users can no longer communicate with one another before we jump into the distributed model though let's talk about the decentralized also known as the federated model with another visual so the decentralized model is a huge improvement too to the centralized model it still operates in a client server sort of way where you end up with hubs that serve users and if a hub goes out only a piece of the network is lost while this is matrix when those services have an outage it only affects a piece of the communities they represent instead of the whole social network or chat service while these are awesome let's see what the distributive model looks like as you can see on the left and on the right in the distributed web each user is also providing a piece of the network itself if a user goes offline the network functions as normal if a major node goes down the network can still function as well by leveraging the local peers it is the champion of resilience and the model i believe will take us forward some nodes might be bigger than others but no single outage can take the entire network down let's take a look at how ipfs fits into all of this and also dive deeper into the distributed model itself we talked a lot about location-based addressing so let's talk about one of the fundamental building blocks to an alternative called content addressing ipfs creates mathematically generated fingerprints for data called content identifiers or cids for short this step relates to something called ipld or interplanetary linked data and is fundamental to how ipfs works to give us content addressing breaking us free from location-based addressing pictured on the screen right now is the anatomy of a version 1 cid represented in binary let's break it down real quick on the far left of the graphic not pictured here would be the multi-base prefix this is actually omitted here because when working with binary there actually isn't a multi-base prefix so you can save that byte what the multi-base prefix does though is it allows us to know what base encoding was used to create the cid at least to represent the cid might be a better way to put it as ipld supports many codecs next we have the version identifier which is simply whatever version number we're working with in this case we're working with a version 1 cid so which is a variable sized integer and the list of supporting encodings are available in the github multi-formats repository and uh i have a link on there in the on the right side it got a little broken oh no am i freezing ah i'm sorry to hear that's happening i'll make sure to uh toss my router out the window after this and get a new one okay good good i'll uh i'll continue and hope it hope it gets better um next up is the multi-hash which includes three things a multi-hash algorithm a multi-hash length and then finally the hash digest itself you can see here we're working with a sha-2 hash of 32 bytes in length then the hash itself trails off the screen the multi-hash specification is also available in the multi-formats github repository i recommend checking it out if you're interested in all the options you have for generating a cid as we went over if you're requesting data by cid you can also verify it's the correct data as you have the hash of the data you're expecting baked right into the cid so you don't have to trust who's sending it to you as you can simply run the hash function in the case of the example that shout to on the data and verify it yourself ipfs uses cids by either looking up the cid in the distributed hash table also known as the dht or by using pit swap and asking their local peers do you have the cid with this it no longer matters where the data lives as we know exactly what we want so it doesn't matter who has it just that someone has it we have now broken free from location-based addressing does multiple files that are the same get cataloged differently uh no actually if um if you added a file on your node let's say using the default settings and then somebody else across the world adds the exact same file on their node with the default settings you'll generate the exact same cid so uh they don't get catalogued differently no problem so now if you want to share a web page an image a video or an article you know that if you send that cid to your friend they can download the exact same version of the data you also saw as long as at least your node or someone else's has a copy you can share it it wouldn't matter if the initial hoster removed the data only that you or someone else has a copy of the data through this data can live as long as there's someone who wants it to through filecoin you can even pay storage providers to guarantee your data will live for some time maybe even forever but that dive is for another talk in the meantime if you're looking for a service to persist your data i recommend checking out web3.storage nft.storage and lighthouse.storage they're all storage helpers eager to get your data available over ipfs and backed up the filecoin does it get duplicated we'll still refer to file one um actually yeah that's a great question so if you if you have a cid um and let's say uh i think a good example is if you have a file and then two folders one folder has that file in it and then you wanted to add the same file to folder two it won't duplicate the data it'll just add a reference to the exact same cid so this makes ipfs uh naturally it has deduplication as a result because there is no instance where you can have the same data twice because you can just refer to the cid and get the same data so you're really just dealing with handfuls of cids yeah no problem oh i like this question too could ipns be a way to version ipfs files how would that work and if so does it make sense to use that if you want to be able to update an nft or is there something more clean about using the ipfs address actually it sounds like you've already uh answered your own question ipns would be absolutely perfect for that because um with ipfs you won't be able to update the cid whatsoever but if you have ipns what you're doing is you're you're publishing and saying like i have this ipns key and i want this address to now po point to this new cid so that's really what will give you that mutability that you're looking for um you can create an ipns key um by installing something like uh kubo als uh previously known as go ipfs and then you run um ipfs key gen and then the final parameter is the name of the key that you want and then that's how you'll generate a new ipns key so now that we've broken free of location-based addressing and moved on to content addressing what does that mean for us well it means that now transparently the network can find new routes around problems problems in this case can be outages we know that if our node goes down for a period of time that we're in the clear if we've gotten our data onto some other nodes or even if a user running an ipfs node happens to have a copy of our data with ipfs your node will cache new data until it's garbage collected this aids at automatically strengthening the resilience of a cid this can work with entire internet outages as well in certain countries as long as some node on the local network has a copy of the data the data can still be served this shows that in a peer-to-peer network no single node outage or event can cause the service to go down can multiple people version and ipns um that might be something to play with i know if you have two nodes with the same ipns key each different node can update it but i'm not entirely sure all the nuance that comes into play when that happens but yeah i would like to see some more delegating kind of stuff to do with ips in the future uh we're just not quite at totally specking that out yet i don't think um so this knowledge is important when you're designing your application sure you can leverage a gateway and if you're tight on time in a hackathon that makes perfect sense however if you really want to create a resilient web3 peer-to-peer application then it's important to think about how to achieve resilience and to do that you must be utilizing the peer-to-peer nature of ipfs when you rely on a gateway especially just a single one then if that gateway slows or goes down your entire application will go with it where if you utilize ipfs directly leveraging its peer-to-peer capabilities you're on track to creating a virtually unstoppable service so we touched on how ipfs users help re-serve data they're interested in to nearby peers this happens when a user requests data from the network via cid their node caches it then makes it available to the rest of the network this aids in more than just resilience you can see it as the network pitching in to help you scale efficiently too think about the client server problem of when you create a popular app and your traffic explodes that situation is effectively flipped on its head with ipfs resulting in what we call negative bandwidth scaling costs if a bunch of nodes are attempting to download your cid they'll be automatically re-sharing that cid as well so while your traffic explodes from people sharing your cid to each other those very same people are helping send the data itself to their friends or colleagues automatically so effectively the more popular your cid is the easier it is for people to retrieve the data from some other node that might be more local to them this property is very important to the interplanetary aspect of ipfs if someone on mars had a piece of data that they originally retrieved from earth then another mars user shouldn't have to wait to retrieve the same data from earth the network should automatically figure out that there's another node on mars willing to serve that data content addressing in the distributed web helps us unlock such a future on the network level transparently if any of this presentation has sparked your imagination or gotten any gears turning then please come join our ecosystem to learn and build we have an awesome community comprised of builders who help us foster a positive and productive environment i highly encourage you to check out a resource we compiled of several tutorials currently available at tinyurl.com learn.

ipfs.filecoin i'm also available in these communities particularly on our discord but also our forums and stack overflow you'll find myself other develops and tons of knowledgeable people inside these communities so please don't hesitate to join in and say hi all right so when you say local do you mean geographical physical proximity is geographical location the primary determination of the proximity of nodes um sorry right now i don't think there's any geo-location done whatsoever um and local really depends on the context that i set it so when i'm talking about like network outages i'm talking about like countries where um they might uh cut or sometimes there'll be an issue connecting to the greater internet so in those instances local nodes can still communicate with each other just because they have the literal ability to address and find each other where otherwise uh if we're talking about like uh publishing something to the dht uh local is determined by key space not uh geographic um and then when we're talking about the instance of mars to earth um i guess that would be sort of uh geographical but it's mostly based on the fact that a mars node will begin initially serving blocks um theoretically nearly immediately where communicating with earth would take so much longer that you would you would know which one's closer just based on the time to first block um so how does using ipns to store dynamic data different than using ceramic are there trade-offs um so i'm not super familiar with using ceramics specifically directly but i believe uh ceramic does leverage ipfs so that would make me think it does leverage ipns as well so i think that's um like a case-by-case thing like if you're using ipns you're using um like probably more like bare bones low level type stuff where ceramic is another layer built on top of ipfs to try to help you give you extra apis and sdks to make things easier for you um so i guess yeah the trade-offs are totally totally up to you one might be better than another but it all depends on what you're trying to build and who you are no problem so i hope to see you all around um i'm also available at many hackathons so please ping me or send me a dm even if you're not in a hackathon i'd love to hear from you i'm discording on related discord servers and the file coinslock so aside from answering any unanswered questions that's it from me and i sincerely hope i helped inspire or teach you something and i hope you all have a lovely day happy hacking awesome thank you so much discordant i was super super helpful i know everyone really benefited from this and that was really successful for your first workshop of this type um so as discordant said um the team him in a team is available on slack and discord so feel free to reach out at any point if you have any uh questions about ipfs or need any help with your hack um and yeah uh thanks everyone for joining and thank you once again discordant for taking the time to do another workshop with hackfest um i hope everyone has a great rest of your day evening morning and i'll see you at the next workshop