IPFS 🛠 Breaking Free from the Client Server Model - Discordian

foreign [Applause] [Music] and welcome to eat San Francisco I hope you're all having a lovely time I'm discordian a developer Advocate with protocol labs and I'm here to give you a talk on Breaking Free from the client server model at points this talk might be a bit technical but I've tried to structure the presentation in a way that it should hopefully be informative and enjoyable to anyone interested in our ecosystem or the distributed web in general all right so let's get started first by calling back all the noise in the background and let's focus for a moment on what the client server model even is through this we'll rebuild the graphic and hopefully with it your understanding of the concepts at play as a whole so what is the client server model today only a few companies are responsible for serving up most of the web we have Google Amazon Facebook and fastly just to name a few these companies combined serve us our ads web pages social media videos images and allow us to host servers and services on their infrastructures to interact with these Services we also have to use their apis sending some data to their server and eagerly waiting for a response so we can process it and do whatever magic we need to do to create our apps and websites there are several problems with this design from cost to costs to privacy free services are often funded by ads further destroying our privacy as all our data is routed through a few Central points paid for by our by our attention and private information for the most part all our data routes through their servers if you want to view a video send a message collaborate with remote teammates or do anything on the web at all really you're going through someone's Central server this principle is fundamental to the client server model we use a client such as a web browser or chat app and communicate with a single entity and that's the server in the client server model relying on just one point for our source of Truth also opens up a whole host of problems like Trust currently we have SSL certificates to verify an entity is who they say they are however that doesn't do us much good if that service is compromised and even if we are sure of who the entity is it still doesn't help us know how advertisers or others are using our data or even what data is collected for the development side the source of Truth in this instance could be the results of an HTTP API call if for any reason the location of that API cannot be accessed your service could also be impacted today a popular way to build is to lean on the apis of another remote service which makes the web very fragile if the service you rely on goes down so does yours your service in the second half of this workshop I'll go over some other ways to communicate with outside entities in a distributed way to retrieve information without leaning on just a single entity too much in the client server model generally if you want some data you must know where it is if you want some image you generally will need the full URL to that image to retrieve and share it if that image is no longer available on that service you chose you either need to search for it or upload a copy on another server to make it available again also sharing the new location of that image assuming you even have a copy yourself this principle is called location-based addressing and is how most of the World Wide Web functions today you go to a website it has a URL and all the data from that website is addressed by the location of the data if the image analogy is too hard to follow think of how you would share a video on your favorite video platform you'd likely to share the URL to the video also known as the location of the video but by this point you might be wondering without addressing by location how would I even find the data I'm looking for we will get to that I promise but for now I'm going to finish unraveling with the client server model is how it functions and what some of the drawbacks to this approach as a whole is for the wider web So speaking of drawbacks a major drawback of the client server model is that a single outage can take entire Services down think of when AWS goes offline fastly Facebook or YouTube these events are treated as news and often slows or even halts productivity when one of these Services goes offline it's much more impactful than just a single image or video being lost you've now lost access to your ability to even make new data available to others at all all the data you have hosted in the central hubs is temporarily completely inaccessible or Worse permanently the location of the data simply cannot be accessed because it's offline you cannot simply ask for the data from The Wider web and expect to receive it even if another related entity has the very data you're looking for you can of course host your own service maybe distributed the locations of your servers in a way where outages are rare let's say you even come up with the next big idea for a website or platform where you're in control of all your infrastructure not relying on some other Central entity for your needs well that's actually my segue into my next point scaling can be very expensive these expenses don't always or maybe not even usually happen linearly meaning an explosion of traffic can easily translate into an explosion of costs whether you host your own infrastructure or you pay someone else to you'll be paying for it if you want to scale and run effectively the more users you have the more traffic that generates your CPU RAM and bandwidth expenses ramp up on top of this if users end up depending on your service for one reason or another you're obligated to keep this service running and available if you want to retain users and keep them happy not everyone can afford these costs and they can be quite daunting especially if you just want to create a fun project technology or even a Blog a lot of companies as I mentioned previously or sorry a lot of companies as I mentioned previously turned to advertising to supplement these costs too resulting in so much of humanity effectively selling their private information in exchange for free services online content creators are often completely locked to a few platforms like YouTube Instagram and a few other similar ones those can be subject to a whole host of rules and regulation as well far more restrictive than what your local government might allow and you have no say in when or how those rules might change if you want to host any content at all on one of these Central entities you must of course ensure you're abiding by their rules and regulations this can be quite unfortunate when popular content gets lost to rules changing or updating as is quite common with video and graphic media today as a content creator sometimes it can feel like an impossibly daunting task to solve this problem especially when people become so dependent on algorithms pushing their content to users sure one could set up their own Services as I mentioned previously however not everyone knows how to do that effectively and aren't prepared for the time and costs associated with doing that from social media to blogs to forums anything in between anything on the web at all these are all fundamental issues with the client server model to summarize the drawbacks seem to be one only a few companies decide the rules for what's allowed on the web and control infrastructure surrounding the web itself two the source of truth comes from Only The Entity you're interacting with so if that entity or server is ever compromised there's little if anything you can do about it three data is addressed by location so for any reason at all your data if your data is no longer available at the same location it's effectively lost four an outage of any of these mentioned servers can not only remove your content from the web but can completely prevent you from effectively publishing new content at all five scaling with this model can be very expensive and six it's not you or your government that decides what content is allowed to exist but instead a handful of companies I think we can do better Breaking Free from the client server model means rethinking how the web works as we know it today this is the stage of the presentation where we get to start talking about the fun stuff the client server model represents web 2 as we know it today love it or hate it I believe there is a clear and strong argument for how the web 2 model referring to the client server model can be improved what do we call this new model this new model is referring to the distributed web so let's talk about that and how ipfs fits in but first for the uninitiated let's briefly talk about what the distributed web is and what it means to you whether you're a user or developer you can benefit from the distributed web what I think of as web3 the client server model is a centralized model this is why we're always talking about Central servers and single points of failure pictured on the left is this model you can see a central point in Orange this is the server and then the spokes are the clients who are often the users this graphic should hopefully make it easy to see if that Central Point is ever removed then the users can no longer communicate with one another before we jump right to the distributed model though let's talk about the decentralized or Federated model with another visual so the decentralized model is a huge Improvement to the centralized model it still operates in a client server sort of way where you end up with hubs that serve users and if a hub goes out only a piece of the network is lost while this is a nice Improvement it still doesn't solve all the problems with the client server model or at least the problems that it carries with it today we will see this model used with many things or yeah sorry today we see this model used with many things like activity Pub and Matrix when those Services have an outage it only affects a piece of the communities they represent instead of the whole social network or chat service while these are awesome let's see what the distributed model looks like so as you can see on the left and on the right in the distributed web each user is providing a piece of the network itself if a user goes offline the network functions is normal if a major node goes down the network can still function as well by leveraging the local peers it is the champion of resilience and the model I believe will take us forward some nodes might be bigger than others but no single outage can take the entire network down let's take a look at how ipfs fits into all of this and also dive deeper into the distributed model itself we talked a lot about location-based addressing so let's talk about one of the fundamental building blocks to an alternative called content addressing ipfs creates mathematically generated fingerprints for data called content identifiers or cids for short this step relates to something called ipld or interplanetary linked data and is fundamental to how ipfs Works to give us content addressing breaking us free from location-based addressing pictured on the screen right now is the anatomy of a version 1 CID represented in binary let's break it down real quick on the far left not pictured here would be the multi-base prefix this is actually omitted here because when working with binary there actually isn't a multi-base prefix so you can save that byte with the multibig with the multibase prefix does is allows us to know what base encoding was used to create the CID as ipld supports many next we have the version identifier which is simply whatever version number we're working with in the case or in this case we're working with a version 1 CID now we're to the multicodec dagpv which is indicating that this dag directed The cyclic graph is protocol buffer encoded the multicodec field itself is an unsigned variant which is a variable integer and the list of supported encodings are available in the GitHub multi-format suppository which is linked on this slide next up is the multi-hash which includes three things a multi-hash algorithm a multi-hash length and then finally the hash digest itself you can see here that we're working with a sha2 hash of 32 bytes in length and then the hash itself Trails off the screen the multi-hash specification is also available in the multi-formats GitHub repository I highly recommend checking it out if you're interested in all the options you have for generating a CID as we went over if you're requesting data by its CID you can also verify it's the correct data as you have the hash of the data You're Expecting baked right into the CID so you don't have to trust who's sending it to you as you can simply run the hash function in the case of the example that shot to on the data and verify it yourself ipfs uses cids by either looking up the Cid in the distributed hash table also known as the DHT or by using bit Swap and asking their local peers do you have the CID with this it no longer matters where the data lives as we know exactly what we want so it doesn't matter who has it just that someone has it we've now broken free from location-based addressing now if you want to share a web page an image a video or an article you know if you send that CID to your friend they can download the same version of the data you also saw as long as at least your node or someone else's has a copy you can share it it wouldn't matter if the initial Hoster removed the data only that you or someone else has a copy of the data through this data can live as long as there's someone who wants it to through filecoin you can even pay storage providers to guarantee your data will live for some time maybe even forever but that dive is for another talk in the meantime if you're looking for a service to persist your data I recommend checking out web3.storage nft.storage and lighthouse.storage they're all storage helpers eager to get your data available over ipfs and backed up to filecoin we've gone over here how cids are immutable and verifiable and we've just talked about how we can keep the CID alive to host our data and websites via storage helpers if we have a website though this means sharing a new CID each time the website updates instead of doing that with ipns you can have a single mutable address which can be updated at will to point to a new CID with ipns AKA the interplanetary name system You Begin by generating a key pair with this key pair you can publish an ipns record to the DHT by using a multi-hashed version of your public key This Record contains what CID it's pointing to when it expires its version number and a cryptographic signature signed by the corresponding private key going even further you can use DNS link to link up a traditional DNS domain name to your ipns address giving you a human readable address you can share too with ipns a node needs to ensure it's somewhat regularly keeping the ipns record Alive by republishing it from time to time about every 24 hours or so just like storage helpers there are services to assist you with that too I highly recommend you check out web 3.storages web or W3 name Service as well as fleek.

co's service W3 name will give you a simple API and tool to allow you to publish a CID to an ipns address and they'll keep it alive for you fleek.co is very similar but with a hosting angle Fleek will host your website as well as give you the tools you need for ipns and DNS link so now that we've broken free of location-based addressing and moved on to content addressing what does that mean for us well it means that now transparently the network can find new routes around problems problems in this case can be outages we know that if our node goes down for a period of time that we're in the clear if we've gotten our data onto some other nodes or even if a user running an ipfs node happens to have a copy of our data with ipfs your node will cache new data until it's garbage collected this AIDS and automatically strengthening the resilience of a CID this can work with entire Internet outages as well in certain countries as long as some node on the local network has a copy of the data the data can still be served this shows that in a peer-to-peer network no single outage or event can cause the service to go down ipfs makes and maintains its connections through lib P2P for you but if you're interested in the whole networking side I highly recommend checking out learning resources involving libpdp which can be found at libp.io there's some cool stuff you can do with the tools like a published subscribe system for message passing called Pub sub and a whole host of transports to check out this knowledge is important when you're designing your application sure you can leverage a Gateway and if you're tight on time in a hackathon that makes perfect sense however if you really want to create a resilient web 3 peer-to-peer application then it's important to think about how to achieve resilience and to do that you must be utilizing the peer-to-peer nature of ipfs when you rely on a Gateway especially just a single one then if that Gateway slows or goes down your entire application will go with it where if you utilize ipfs directly or host your own Gateway as well leveraging its peer-to-peer capabilities you're on track to creating a virtually Unstoppable service so we touched on how ipfs users help Reserve data they're interested into nearby peers this happens when a user requests data from the network via a CID their node caches it and makes it available to the rest of the network this AIDS in more than just resilience you can see it as the network pitching in to help you scale effectively and efficiently too think about the client's server problem of when you create a popular app and your traffic explodes that situation is effectively flipped on its head with ipfs resulting in negative bandwidth scaling costs if a bunch of nodes are attempting to download your CID they'll be automatically re-sharing that CID as well so while your traffic explodes from people sharing your CID to each other those very same people are also helping send the data itself to their friends or colleagues automatically so effectively the more popular your CID is the easier it is for people to retrieve the data from some other node that might be more local to them this property is very important to the interplanetary aspect of ipfs if someone on Mars had a piece of data they originally retrieved from Earth then another Mars user shouldn't have to wait to retrieve the same data from Earth the network should automatically figure out that there's another node on Mars willing to serve that data content addressing in the distributed web helps us unlock such a future on the network level transparently if any of this presentation has sparked your imagination or gotten any gears turning then please come join our ecosystem to learn and build we have an awesome Community comprised of Builders who help us Foster a positive and productive environment I highly encourage you to check out a resource we compiled of several tutorials currently available at tinyurl.com learn Dash ipfs Dash filecoin I'm also available in these communities particularly on our slack but also our our forums stack Overflow and Discord you'll find myself other devrels and tons of knowledgeable people inside these communities so please don't hesitate to join in and say hi I hope to see you around and I'm available quickly on our slack so please ping me or send me a DM I'd love to hear from you I'm discording on related Discord servers and again the filecoin slack that's all from me I sincerely hope I helped Inspire or teach you something and I hope you all have a lovely day happy hacking