EthCC 2: Daniel A Nagy

okay so hello everyone and welcome so I'm here the ambassador of the swarm team so I'm going to give you a update about what we've been up to and what happened during the past year since we met here last year oh can you but it's too big can we do it full screen like f11 or something yeah so first I would like to give a little introduction about what swarm is so it started off as kind of the this distributed decentralized hard drive for the world computer that is aetherium but during the years it has grown to be a little bit more than that primarily it is still a peer-to-peer content address storage and delivery network but it also comes with an incentive and payment layer so if you're contributing resources namely bandwidth and storage to the network you can actually earn either with it and also it is providing real-time peer-to-peer messaging services for distributed applications well and it's also silly acronym so the just so that we speak the same language I would like to introduce the basic terms terminology that we use in swarm chunks hashes and manifests so chunks are these at most four kilobyte bits of like chunks of data which are the basic building blocks of the murk alized data structures of which upon which swarm depends they can hold arbitrary data and metadata and other swarm primitives they are the only things that the network layer knows and cares about so these chunks are the little bits of data that are being sent around the network and their addresses live in the same address space as the nodes of the swarm network and chunks tend to be stored by and served by nodes that are closest to them in this address space so this is a DHT and there's a metric a distance metric that it's it's a valid question and there's a number that answers this question is how far this chunk is from this node and of course the closer it is the more likely it is that you will find that chunk on that node hashes are the primary reference in this worm network so that's how you reference stuff by their hashes so that's the basis of content addressing these are binary Merkle tree hashes for individual chunks which means that in addition to integrity protecting the whole chunk they can also be used for proving to a smart contract that a particular piece of data is part of that chunk and they are also used for local tree representation of arbitrary data not only for individual chunks meaning that once you have the swarm hash of any piece of data than with a relatively small number of additional information you can actually prove that some small amount of data is actually part of your large connect collection sorry they have the usual properties of cryptographic hashes so they are collision free they are they can facilitate proofs of custody and just like the just like the chunk addresses they also shared the same address space so you can talk about hashes of collections you can talk about hashes of files and you can talk about hashes of chunks and they are all come from the same address space and they are the references that are used in swarm the primary references there are other references that I'm going to talk about so a hash of 128 chunks is a single hash and if you take hundred twenty-eight of these hashes they make up exactly a chunk so if you have a whole bunch of chunks then you have other chunks that hold hashes of these chunks and you build this Merkel tree all the way up to a single root hash and this is how you reference your entire collection of chunks at the and the Leafs of this tree hold the actual data and the so called intermediary chunks in between are the ones that are needed to both find and integrity protect that data and eventually if you work your way all the way up you're going to end up with a single root hash so because of this because of this merkel structure random access to data is supported by design so even if you have terabytes of data you can both access it in a random fashion so you can access any part of it and in addition to that you can prove with a relatively manageable witness size that that data is part of that big collection so what are manifests manifests are the metadata holding containers for collections of of any kind of objects so they are a mapping between arbitrary access paths basically strings similar to your eyes in in web in HTTP and they contain also the metadata so think for example of HTTP headers and they are also Merkel eyes data structure meaning that whatever you can finding them you can also prove that it is there and by proving I mean present a small witness to a smart contract so that a smart contract can automatically this decide on whether it is or it is not part of that data structure so just to bring up some better known analogies so these manifests they work as a index so they tell you where certain data can be found they are like a directory tree as well they are kind of like websites so we call them swarm sites so these are virtualized web servers for static content so you have both the HTTP headers and the HTTP payload described by these by these manifests so they are somewhat similar to HT access files on traditional web servers they are key value databases and they also work like filesystem directories so these are the kind of analogies that we can relate to when we are talking about swarm manifests so what has happened in 2000 since last year's community conference the encryption and access control that I have talked about how it was going to be implemented it has been implemented mostly along those lines so now we have it and you can try it we have PSS which is the Postal Service over swarm it's a real-time messaging service for adapts real-time means that both the sender and the receiver needs to be online but in addition to PSS we also have swarm feeds which are the offline version of messaging so this is a subscriber publisher model where publishers are publishing whatever they need to publish and subscribers can access that data even if they are if the publisher happens to be offline because the data is actually stored in swarm so BSS only uses the routing network of swarm where but feeds use both the storage and the routing Network so you can actually publish mutable content on swarm so it's a append-only kind of data structure it doesn't have consensus because we're not trying to replicate the functionality of the blockchain we have the blockchain for the consensus but the swarm feeds are structured in such a way that any user not only the publisher but any user can basically checkpoint the swarm feed at any point to the blockchain and hook it up to a DNS resolver and that way any any feed even though the feeds in swarm themselves are not do not provide consensus if consensus is needed it can be readily built by sending the feed a the check point of a feed to the blockchain access control in swarm so this is the part with which I've been involved most closely so again because we have a decentralized system access control works a little bit differently from how it works in a centralized system references are twice as long as hashes because they consist of a ciphertext hash so the hash of the ciphertext plus a decryption key a symmetric key for decryption so in total with 64 bytes you can reference any encrypted content in swarm but the API is exactly the same as for the unencrypted content with the sole exception that the reference is instead of the 32 byte ash are now are now these four 64 byte encrypted references otherwise the entire API remains the same and this is evidenced by the fact that we haven't changed any of the example applications they just word with the encrypted content now so the API remain the same the crypto the only cryptographic assumption upon which small content encryption relies is the security of sha-3 the same shot reversion upon which aetherium itself depends meaning that as long as char tree is secure you can trust that the encryption of swarm content is also cryptographically secure and if it is not secure anymore so if char tree gets broken then we will have much bigger problems than that because that breaks the theorem so we did not the takeaway lesson here is that we did not introduce any additional security assumptions beyond those upon which aetherium already depends access control so basically read access means that you do those who have read access have the ability to decrypt this is very different from centralized systems where the read access basically meant the ability to retrieve because the gatekeepers would not give you the content unless you have read permission so when in facebook you click that show this content only to my friends it means that Facebook would only provide access to it to your friends but in a decentralized system you cannot do that everything is out there but it's encrypted and read access means that you can actually decrypt it and write access is even trickier because write access in practice actually means that you have the right to update the root hash in some on chain or off chain registry typically this means E&S once you own the ENS name for your swarm site then you have the ability to update the route reference for that but if you want to create some complicated access control system which gives multiple parties right access then you need to encode it in the smart contract that governs that particular ENS names resolver so the right access control is with the resolver and the read access control is in swarm with swarming Christian and there are basically three access control strategies and swarm they are listed on this slide in the increasing level of sophistication the least sophisticated but also the simplest access control is using the pass phrase anybody who has the pass phrase can read that particular content this is very simple very easy to understand but it also has a drop a very important drawback namely that in order to give somebody the permission you also need to have that you need to have that pass phrase which means that you automatically has access to everything that that same identity has access to with other publishers so if as a publisher you need to give somebody access somebody read access through a passphrase you need to know that passphrase and in order to overcome that limitation because it's a fairly severe limitation the next level of sophistication is public keys so you can encrypt certain smarm content to a particular public key and that and the owner of the corresponding private key will be able to decrypt it now this is better in that sense that even if you know the public key you don't know the private key so you cannot access whatever has been made available to the same public key by other publishers but it still has a drawback namely that if you have some content that is made available to a certain public key and you need to change the permissions you would need to re-encrypt large parts of it so to solve that kind of problem we have the most sophisticated access control mechanism already built into swarm which is the AC DS the access control tries so this is the drive d3 version of the ACLS access control lists and in this case you have a set of undisclosed parties some of them can can be password based others can be publicly based and both of them can look up whether they have access to that particular content but they cannot determine who else does and actually what an act discloses is only a upper bound on how many parties can access that particular content but it's an upper bound because many of the act entries can actually be faked so you can stuff your act with some fake identities are not disclosed how many subscribers you actually have you can just provide an upper bound so these are the properties that I talked about so an act is referenced also by a swarm reference so it it it is a swarm object just like files or word directories or collections so it's no different in that respect you can use the same act for access control for different different content so you can reuse the acts multiple times and also it's it's been a very important design goal that we managed to fulfill that usual operations of changing access permissions do not require ever Dorien trip ssin of large amounts of data so whether you're granting access or retracting access to various parties you just need to re-encrypt a little bit of data basically keys and and you will immediately changed you will have immediately changed the the the act and when you're publishing new content encrypted with that act only those who are who have access according to the new Act will have access to the new content of course they will still have access to the old content because they have the old key so you cannot make somebody forget things which in a decentralized system is a given you cannot work around that and even in centralized systems if you if you remove the access control for some picture it doesn't mean that somebody hasn't saved it for them so you cannot so you cannot unpublish things that's simply impossible but you can stop paying for it being made available and that way the network will slowly but surely garbage collect that data so so the swarm swarm does forget but you cannot count it count on it very important so this has a separate slide please take note of it access controlled content can only be accessed on your local swarm node you cannot do it with public gateways reason being that if you could then the public gateways could spy on you so if you want to access accident if you want to access encrypted someone content run your own swarm node hopefully when swarm gets sufficiently mature it will be integrated so light swarm nodes will be integrated into f3 enabled browsers as an experimental feature most of most web 3 enable browsers already supports worm so you can already play with it but the vision is that every web tree enabled browser will double as a swarm node so you will actually access swarm content and decrypt it locally so swarm is an end-to-end encryption from the publisher to the consumer so PSS that's the Postal Service over swarm which reuses the routing of swarm and it kind of lets you decide on the trade-off between darkness or security or secrecy and efficiency so obviously etherium already has a secure messaging system which is whisper which kind of maxes out the darkness component and because of that it has some performance issues but that's the price that you pay for complete darkness but sometimes some distributed applications might not want to go all the way in darkness but they would like to be more efficient so you cannot stream video over over whisper but you can stream video over PSS at the expense of not being entirely anonymous both on the receiver and the and the sender side and depending on where you put that sliding control this set the anonymity set can be bigger or smaller so basically it still provides you with end-to-end encryption so we do not come from on that it has single-use session keys which have forward secrecy and it provides you with protocol encapsulation so you can actually channel any protocol through BSS meaning that you can have special protocol handlers and it can be basically your transport layer let's skip this one swarm feeds swarm feeds are published on swarm and they are stored by swarm nodes meaning that unlike in PSS the senders and the recipients of the data stored in swarm feeds do not necessarily need to be online at the same time so with swarm feeds you can actually publish feeds of data that's what they are and in order to post an update to your swarm feed you need two things you need the private key to sign your update and you need the topic so once you have the topic and the private key you can update your swarm feed and in order to read the swarm feed you need the etherium address of the publisher and you also need the topic so once you need once you have the topic and the etherium address you can access the swarm feed and you can even go back in time and you can see the entire history of that particular feed provided that it if somebody paid for its storage so it's not guarant it does not company automatic guarantee of availability we don't SPSS sorry swarm feeds do not give you that but swarm in the future will have that and it's not even necessarily needed for all applications so for example imagine that you have a blog and your subscribers who are interested in reading your blog also double as custodes custodians and they keep your content available meaning that in this kind of arrangement you are not subject to D platforming as long as you have loyal subscribers your content is also available I will talk about it in the next talk so please come to that talk as well so you can publish updates to ENS so swarm feeds can be the present state or the most recent state of this swarm feed can be anchored in ENS on the etherium blockchain so that's our version of eventual consistency there's a synchronous for messaging and actually swarm feeds are so performant that you can use them for IOT data feeds so even though you can publish every update to the blockchain you don't have to and if you don't then you can actually have massive amounts of data collected from you know sensor arrays or home appliances or whatever so that's where we are right now and our roadmap is the following so this year we would like to provide a stable test nut so those of you who have some experience playing with some arm test nut know that it's been a little bit flaky to to put it diplomatically so we would like to rectify that and have a very stable test nut this year hopefully within a few months we will have temporary files ending introductions so you will be able to send large files over swarm so basically somebody uploads the file sends over the handle to the recipient the recipient download the file so that's a use case that we would like to be production stable this year a reliable bandwidth accounting so that is swap that's how we call it and that means that by sharing your bandwidth you can earn and by using bandwidth without actually contributing you will need to pay and combining the two means that in order to use swarm you can pay for it in kind basically making your bandwidth available so as long as you have enough bandwidth you can use swarm for free meaning that you pay for your swarm usage by letting others use your bandwidth as well and one of the things that we would also like to achieve this year to have the entire aetherium state available through swarm because eventually if swarm fulfills its mission as a scalable available storage system we would like to actually be the storage and availability layer for etherium blockchain itself and that not only means etherium one as it is right now but also the scalable shard in aetherium - so just to summarize swarm is currently an experimental alpha stage project but we already have example applications we already have a test nut we are hoping to improve stability big time very soon we have fixed large number of bugs we have grown the team we have a wonderful ecosystem by many company is listed here contributing developers code publicity money so thank you so for everybody on this slide huge thank you from the swarm team you were great thank you so much for your support and thank you for all of you for listening and if you have any questions you can ask them from me right now or here are the ways to contact us and other members of this morning team will also be happy to answer your question thank you so much [Applause]