J1 SAY 03 Viktor Tron Infrastructure services in the swarm

today I would like to give a bit of an update on swarm and at the same time bring you a view on swarm in terms of a toolkit that we provide as a set of base layer infrastructure services for for the long coveted decentralized web that we termed web 3 so web 3 is the is the kind of this kind of idealized third stage of the internet when we go back to the old ideas of having some decent right services instead of the Facebook's and Google's of of today which is kind of a vision or a bit of a dream so so what is one originally the scope of swarm was to provide the data layer and like storage service and and content distribution layer to to the serum ecosystem so from very early on like the Athenian founders realized that the blockchain public block chain is the final final missing piece missing piece of the puzzle in the in the in the decentralized web because basically with the public blockchain and smart contracts during complete smart contracts you can program incentive systems which a lot of quite advanced peer-to-peer technologies needed to to to become self-sustaining to so to add economic incentive systems to make the maybe services self-sustaining so what is swarm swarm is basically an incentivize peer-to-peer AB it provides integrity protection for torrent and censorship resistant storage and it's it's a genuine cloud service in the sense that so from from a distance like swarm looks like BitTorrent on steroids and maybe maybe it's the easy way to explain to your friends what it is I will if you look closer it's not of a family of five sharing systems in the way is like others like ipfs and and and the others are it's a genuine loud service in the sense that you can generally upload content do it and and and Swami is very opinionated about where certain content is stored it's not that you open up your hard drive and you can you become cedar of a content that you already have so what happens is that when you put up a document on swarm its chunked into little pieces and and the little piece is based on their content address or fingerprint travel to areas of the network to nodes which are closer to to the to the content address of the chunk so it's basically distributing chunks based on their based on their address and the address lives in the same address spaces as the individual nodes that that formed the decentralized distributed storage network okay so what it what it gets us immediately is that because because we use content hashes as content addresses we immediately have like an automatic load balancing on the content and we have integrity protection because it's because the because the content addressing make sure that the content is immutable so if you have a hash that you know someone someone intended to show you a content to it then you can be sure that it's exactly what they wanted to show you so this there's no more hacking of web sites in in this paradigm now one important thing is that how you how you map it to to to real word memorable names so you use the DNS the etherium name service for domain name resolution and suam supports URL based look up how does it work you basically look up the domain name which matched to a hash that contains a kind of a data special data structure which maps paths to two two hashes basically maps you are a passes to documents and then and therefore you can access document collections this renders one basically able to to provide the server less hosting of websites because with these manifests you can basically create the sitemap and access the the individual documents on the site so it's basically server less by posting environment with the integrity protection censorship resistance and for tolerance so this this is this is the basic interaction to swap a bit of update so last week we we were fortunate to to be hosted by the research of Institute for art and technology in Vienna for a hack week where we managed to round off the the sprint for the very first POC three release which is which is going to come out soon this is going to bring major network resilience improvements and and some other features I'm happy to say that the swamp team has extended substantially in the in the recent past so team is on fire and we delivering new and new features we have working relationships and working groups collaborations with with some companies notably I would I would pick status mainframe work live fear and data fan who were supporting us and working with us on various aspects of the of the swarm so what am I going to talk about today it's it's so physically in Shanghai was the first time when when I kind of introduced a bit of a wider scope first one as being a toolkit for infrastructure like base layer infrastructure services for the web and that's where I first talked about PSS our this centralized messaging service for you know real live communication which uses the same same network in protocol and routing as basically as what it is responsible for cheong distribution and and request handling in in the in the swarm Network we also talked about how how the manifests idea so basically indexing certain keys to the content can be easily extended to handle various database related data structures and therefore it can serve as a database database layer for for for web 3 this is kind of important important for the mass adoption of all these decent right services because it helps developers to carry over and migrate some of the knowledge that word exists we back to so in current the current common method to build real-time interactive web applications is to use is more more and more common to use client-side client-side rendering and most of the most of the not so heavy computation but most websites have haven't have a database back-end which which the client-side render communicates with through an API and usually that databases is still sitting on on some server if you want to move on to a completely server last solution then we have to provide this and tries database services and this is exactly what we were working on bit with the help of walk us company and and this this will pave the way to finally put the blockchain and state and e theorem state data on swarm as well to help the diversity of light clients so what I would like to talk about is is so so in Shanghai and some previous talks I already introduced some of these components and and tells you about how how these components can be solutions to various infrastructure service problems and fell out of the fell out of the design of swarm and of this we kind of hit upon serendipitously and today I would like to take a totally different approach and take it take a top-down approach and take you and I kind of imaginary journey not so technical in which I kind of lay out the typical use case a rather banal use case of how we use the internet and digital services currently for a particular scenario and I would like to comment on what components we use today and how have these infrastructure services that we are building we have built or we are in the process of building on swarm can contribute to to replace and substitute these these these solutions so let me talk a little bit about this this this component but first first let me said the stress let me set the context so let's imagine that in a typical what you do you you came up with an idea that a group of friends is wants to wants to organize an event the event is like a panel discussion say about as the ethics in on the blockchain and there's a group of friends who organize this and but ultimately want the panel discussion to be broadcast live so what what kind of stages there are in this process well normally you would you would start some sort of chat group among these friends who are interested in organizing these events you would probably use Skype or well whatever group chat latitude that you typically use like slack and then you would you would start editing your document using using your Google Drive or Dropbox you would share something and then you would you would do collaborative editing tools to 20 play say then say the web page that you that you're gonna put up as a public face of this event you will you would use some sort of server to put up this content you would you would use the usual way to to do to associate a domain name with with with a current version by adding adding adding the name service to your to your domain name host and and in a later stage you probably how how will you create an event you probably go to Facebook create a Facebook event and there for me this make it searchable and then when when it comes to the event you probably would use Google Hangouts where some sort of Google service wave where you would do the same so what what tools are available that would help us replace this these solutions well first of all you could use you could use pss-pss messaging to organize the the chat in the group pß allows for encrypted messaging and and and and provides group chat there's already a product developed by mainframe because I think the current name is onyx it's enough product which basically implements implements the slack using PSS this will obviously be only production operational once we have PLC three lists release out and probably really reliable only when we have the production version of op small amount so this this looks like something like this so what are the components that we we might want to use so when when you when you share your document you could use instead of Dropbox you could use the the fuse extension of swamp which is basically a hostage with the back end for for a Dropbox application so what is fuse fuses is filesystem in userspace it's a it's a file system driver that allows the extension of operating system kernels to handle to write arbitrary file system drivers what does it mean we already have this this implemented it's it's been working for half a year so you basically can mount a swarm write or manifest as a file system on your local hard drive which means that you if you if you upload the directory and and mount it as a local PI system you can write into it and you can you can it behaves exactly like a dropbox directory now with using a hierarchical hierarchical access control encryption you can you can basically share you can have the possibility to share the directory so share share your one of your truck dropbox directors is as as public or or share it with with with the select number of people like that you want to share it with so it's exactly this feature that we like to use in in in Dropbox or Google Drive and so so the infrastructure is there already in swarm to to do that the encryption part is is currently being developed there's a talk by Donny on Friday that details our solution for for for access control and then encryption in general so what else is what else is needed for for for who had like for shared access content well if you if you want to update your your content from different devices or by different people you you brought you probably need to update the the changing immutable hash one on the theorem name service we also have another solution that which allows for more frequent and cheaper updates to to mutable resources it's called mutable resource update or more and there's going to be a talk by Lewis Holbrook on Saturday which details this solution and I'm not going to that the technical details now but the point is that with this kind of resource updates nodes have a way to to to check the particular versions of invertible resource in the same way as as they would be the ENS we do with the difference that they don't need to be necessarily synchronized with the blockchain you no need to they only need to retrieve particular chunks and and data from swarm to know whatever the current state of a mutable resources and most notably to make an update you just need to upload things to swarm and and you don't need to send a transaction to the rock chain but you can that's also important that any any such resource update which is obviously contains a signed it contains a signature from the resource owner it can be sent to the to the blockchain to TNS as a to consolidate the current state so by and then it can be sent by any third party so so let's let's let's move on to to our imaginary scenario what happens when when we we actually want to who collaborate Lea and collaboratively edit a document let's say we are a bunch of geeks and we actually want to Co edit the the HTML page that that will advertise our event so what do we do here's there's an interesting feature of PSS which allows not only like unstructured communication like chat but it actually allowed psst the the PSS messaging allows for for you to to mount their peer-to-peer protocol an extra structured protocol over this over this encrypted remote communication which means that for example you can use the decent rise version of the of the Google algorithm that that the tenders real time real time recording of documents you can mount that that protocol over PSS and you immediately have a have a dev server last step which which which allows you to to Co edit stuff and and you can you can have some sort of rule set which which determines who who's up who's updating the like the current version on on the on the resource update or or DNS and and and and that way kind of make make a kind of poor-man's version of consensus over where when you are in the indian what was that what's the most current version of course of course we consensus over this is only achievement when when when the when the hash is is registered and only NS on the blockchain so i mentioned that the next step would be to actually create an event so in a group we decided that we would have this panel discussion about ethics and the blockchain so in what what would you do normally we would create a Facebook event in this case what we can do is to use to use a decentralized database creation where we can create an entry for this particular event therefore make it make it immediately searchable and indexable by my venue and by time tour in the decentralized fashion using the using this this one DB extension and and when we get to the actual event what would happen we could use life peers media streaming solution for for we're for video conferencing and also for public broadcast of the of the video streams so these solutions are or either have been completed or were in the making and probably gonna reach production ready state with the with the POC for releasing and I I kind of predict that to be in the first quarter of 2019 when this when this or these components will become stable so let's just recap what we have at the moment we have we have the mutable resource updates implemented we have BSS messaging implemented we have the fewest driver as the as the backend for phone Dropbox implemented we are in the we are almost finished with with with the encryption implementation and what we what we still have for planned only for POC for instant is the database component and and the and the actual incentive layer for for storage insurance so if you want to have more information about these actual components then I encourage you to see some of the some of the videos from from DEFCON three because there's not a very good video about TSS and it is also about this mainframe application there's a talk there's a London meetup talk an hour-long talk which details de the generalized incentive framework that we using that we playfully term swaps parents window games this these are basically generalizing the like a framework in which you can program incentives and have have service contracts and and collateralized commitments of any kind of network service and also a peer-to-peer accounting by the swap system and and it's basically our glorified variant of us test of a state channel this the swap contracts and and they basically implements have payment channels to help scale a micro payments and micro accounting that is needed for bandwidth incentivization for example and for some various other services so so these components again fuse PSS swap silence window beautiful resource updates and so on databases are forming a kind of starting to shape up as a as a nice tool kit for websley basically your infrastructure and you can do you can use these components as kind of LEGO pieces to to put together module early high level applications or high level services which can ultimately in this dream but at least that I have replaced current solutions for example you know Facebook Google Drive and Dropbox and and hangouts etc so see that and finally I would like to mention that we have based on last year's success of the of the swamps I mean that we had in Berlin in June 2017 we're gonna have a new round so this is going to be kind of a mini conference and and and a few hack days this small morning summit one which is going to be in Ljubljana and this is the second week of May and if anyone is interested in presenting or were holding hands work shop or giving a talk on any topic related to to decentralized web especially especially storage and messaging then please submit your apps abstracts or just give us a shout that you want to participate thank you very much and I'm opening up with two cache questions now [Applause] sorry I was not unless I was not very agile in in handling the slides because I became a bit of my medication so probably people in the audience know that I have Parkinson's and sometimes I struggle with timing my medication for the talk so it's a bit it's a bit hard right now but so please go ahead with questions yes I mean I mean I kind of my my cheeky answer is that no and it shouldn't be anyway because it's kind of a stupid legacy of a system where well you know date was not globalized how I mean there are kind of far-fetched oceans in which in which you can you can form you can you can have like a set of set of loads that register and promise to be to hold that data in a particular location and they they had somehow put that commitment or promise upon the blockchain but if even that is just it's just kind of a positive promise that the data we had there there's no promise that other nodes outside that will not story so it's actually this actually this is actually a very difficult problem to do and system or at least I don't know a solution if you guys have any solution that I'm happy to hear out but in in in general I don't think we want to support such geographical restrictions because and I kind of even the concept is kind of alien to Toba distributed systems yes correct correct thank you for thank you for the nice synopsis is exactly the problem does there's two pillars of incentivization in swamp one is for the smooth operation of request handling that is basically corresponds to rewarding notes for their bandwidth contribution and the other one is rewarding notes for for the storage meaning that you have to compensate notes for the missed opportunity to serve our popular content and yet yet bandwidth incentives for them in favor of of keeping your I don't know your birth certificate or some some very private content that is typically not very popular or not not retrieved very often so for this kind of storage incentives who use a build an insurance system and that's going to be so the question was when it's going to be implemented so we had some some prototype implementations we have a we have an article or paper to work in progress which is called swaps and swindle games that describes this in a little bit more detail and and rough pictionary in from from riyadh started very implementing these contracts and it's going to have a lot of iterations of course and actually it has to be combined with the implementation of a proof of custody solution and and I I expect that to be ready only for POC for so so it's it's gonna be part of the but at the production release then and I I think some some some early version is probably going to be in about half a year in half a year and that but the time when you can expect like really resilience warm and no data loss that's basically only possible anyway even if the incentive is is implemented it's only possible if it's actually backed by real etherium tokens so when when the swarm actually is is it's going on the main net yeah because when the actual insurance system is it's properly paid with with ether or some or some other token yeah so that's that's when I expect resilient and and and and lossless use of swarm on in POC for this century question yes yes we are in contact with them sure and we have been we have been in contact with them from very early on and in fact a kind of we can see that them as a sister project or even mother project because they are the kind of the web 3 hub of protocols so we kind of have an agreement that swamp ashes will will be available on ipfs obviously they're gonna be part of the registry 9fs domain the main difference between swarming ipfs if you look closer kind of that's these two main differences one that i mentioned before that ipfs is kind of starting from the file file sharing family of both of decentralized storage solutions in the sense that you you open up your hard drive particular content and register that as an publish that content and and there's there's no genuine systematic way of distributing the chunks in a way that that that each chunk would have a specific space where it's supposed to be stored in swamp in swarm there is and exactly for this reason there's a genuine sense in which you can upload stuff to swarm even if you don't have any storage resources yourself now this has a lot of implications interval because in this in this sense you kind of as a swan what you forced to store other people's content so we have to have measures of totally plausible deniability in in which means that allowing tools to to forced or us to to deny that they they know anything about the content of what they store the other very important differences that five coin as far as I understand it is providing positive incentives ation only for minors which might work out but I have a strong suspicion that because it's it's basically socializes of socializes the risk of losing any particular file practically this kind of losses will happen or or if or if it doesn't happen it's it's probably after the goodwill or were lucky volume of available resources that it doesn't happen there's no strong guarantees in Swami we put a lot of effort into providing insurance solutions which which make the the stores of particular chunks individually accountable and therefore the threat of punitive measures is a strong disincentive against losing any particular chunk of file if this answer the question any more questions so what go to Danny's talk about encryption and lewis's talk about mutable resource updates thank you very much [Music] [Applause]