EthCC 2: Mike Goelzer

it's a networking protocol that many different um blockchains and as well as ecosystems are moving to so polka dot relies on this um i think ethereum 2.0 is moving towards this file coin which protocol labs is working on relies on this um so um yeah it will be fun to learn more about p2p okay thank you [Applause] all right hello everyone um i'm gonna try to present from here so but i need a way to advance my slides okay so um i think that'll work well uh so let me just give a little introduction like why is libpdp here what do we have to do uh with uh the you know the web3 track um and uh so basically i would put it like this um what fundamentally if you've never if you've never heard of lib p2p fundamentally what we are is a low-level building block for building peer-to-peer applications so there's not a is the sound working okay all right all right so fundamentally uh so we're not a blockchain uh we don't have any uh token any economic mechanisms in the p2p itself it's purely just an open source project that aims to build uh the the most useful and most robust uh system for building peer-to-peer applications on top of it and um as jack mentioned uh polka dot is one or substrate is one of them and polkadot relies on that uh ethereum 2.0 is also going to use lit p2p so that's another example um and we have we also have a lot of people building d apps that are not uh you know have nothing to do with blockchain so we're totally agnostic on what the users want to do we just want to build the best peer-to-peer stack uh so let me uh so i'll warn you that this talk is probably a little uh less technical than uh i usually give this talk to a less technical crowd so i'm gonna skip through some slides and what i'm hoping to do by that is uh create a little time at the end for some discussion i've already been discussing with some people in the audience some interesting ideas about uh different kinds of attacks against the data structures that live pdp and other similar systems used so like i think it would be fun to kind of dive into that and other people may have uh thoughtful things to say so um i normally start these talks with this dramatic image and i say you know look the where did the internet come from well you know it didn't come out of a vacuum it started as an idea about how would people communicate after a nuclear war uh most people would still be alive but how would you know if all of the centralized uh communication points in the telephone system were destroyed then how would they coordinate like a resistance or something and so this was an idea back in the 1950s and 60s like how do we solve this problem and that the roots of the internet were basically the answer to that question is build a decentralized network uh and then i talk a little bit about okay so you know where's uh with the internet today and it's basically a lot of centralized services they're not evil it's just that it's a lot easier to build something with a centralized database than it is with any kind of decentralized technology um i'm going to skip over these uh here are some things you might remember if you're if you're old that you use peer-to-peer um i just want to talk for a second about how lib p2p differs from like the original generation of peer-to-peer apps which were mostly like file share like you know illegal file sharing basically was that was their goal uh first off we're not focused on that use case secondly uh we lib p2p is built as a set of composable uh building blocks i'm going to talk more about this in a second but the idea is that you can you can quote unquote use the p2p in a way that uh does you know whatever suits your needs so if you want to use certain modules you can if you want to ignore other ones you can that's a big part of our design uh philosophy um uh so multi-platform and multi-device interoperability basically the idea there is that this is implemented in several different languages um uh we have a go implementation a rust implementation the the primary author of which i think is sitting in the back yes [Laughter] okay so uh and uh javascript implementation we're very close to having a python implementation we also have someone implementing uh p2p in kotlin which is a java dialect all right yeah kotlin fans uh yeah so um anyway the point is like fundamentally this is a standard and we want people to implement it in in many different languages and the the only thing we care about is that they they interoperate uh and so we have interoperability tests and we're continuing to build that out um i i want to talk about this side for a second because uh it gets into some things that um i think oh i guess i should be you know the camera okay yeah uh it gets into some things i think are important um so i've already talked about the multiple interoperability implementations um it's uh as i said it's very uh flexible it's not specific to a single use case uh we also use it as a platform to experiment with new ideas so you know right now we're trying to figure out like what should we do about uh the dht should we use code emily should we use another algorithm should we make it plugable so it's very much a platform where you can propose something and um you know we'll we'll seriously consider it and in many cases you can write your own plugins uh so i'll show more examples in a second uh oh a key principle for us uh is well there too um one is that peers are identified by solely by the hash of a public key so we're totally location independent you can have a device in a lit p2p network that it has you know it is uniquely identified by the hash of its public key it can move from network to network it can have multiple addresses it can have an ip4 address an ip6 address a bluetooth address we're completely agnostic on what the uh you know where the device is we we just know that if it has this uh identity then that's stable and that's not going to change and the private key is used for signing to prove uh to prove your identity the other thing is a concept that we like to call connectivity magic so basically we try to make a lot of things very easy so we have um features like connection multiplexing uh then within a like tcp connection i mean then within that connection we also we set up what we call streams which are means of communication those themselves have their own multiplexing mechanism uh so we we've built a lot and you know and connections can be upgraded automatically to be encrypted uh and so we've built in a lot of things that if you wanted to build a peer-to-peer network on your own you would have to build these uh or you know or it wouldn't it wouldn't uh be a very good one uh so uh that's kind of the direction we're going like we try to make a lot of things just sort of just work uh with all the details hidden behind the scenes uh so when i say that live p2p is made up of modular building blocks um what i'm referring to is the fact that um uh basically these are the these are the blocks so we have these concepts uh like um and these are all abstractions that exist in the code so we have this notion of a transport which connects to appears over uh you know some kind of link it could be tcp udp whatever tor other you know any connectivity uh mechanism um you know we have uh nat reversal mechanisms which is an area that we're trying to build out but we have some early stuff um working there uh peer discovery is another big one we give you ways to discover peers um that's where this concept of a distributed hash table tends to come in um so very briefly if you're not familiar uh most peer-to-peer networks use a distributed hash table called codemlia there are variants of code emilia but basically what kind emily does is define a distance metric between nodes and then connect to nodes that are quote-unquote close not physically but uh close logically based on the this logical distance metric and um it it uh it sets up connections that way and it grows it's its uh list of connections to other peers um in through this mechanism um we we're very interested in other ways of doing peer discovery um this is this is the one that's implemented right now but it's not uh it's not you know we want to have pluggable uh discovery mechanisms uh crypto channels uh so this is just a kind of marketing way or something of saying that uh you know we have encryption built in for streams uh and connections uh so you know you can um you can just basically call a function and your connect your connection will be upgraded to an encrypted connection um with you know without you really having to do anything um i'm gonna keep going oh yeah okay so for each of these we have some kind of documentation we're in a big docs improvement process right now so not everything is perfectly documented but basically we try to document the interfaces for you know these different building blocks that i showed you on the last slide and yeah and then uh you put these pieces together however you want you get lit p2p okay great um we also have a google doc that you can find in that github issue where we're trying to sketch out all of the ideas that we might pursue in the next you know several years uh some of them are pretty far out there some of them are kind of obvious uh you know quick wins i'm not going to go through that whole list but i i would encourage you to check it out and give comments about which would be useful to you um that's really useful to us people ask this question how does lip p2p how is it similar different to the osi seven layer model that appears in like networking textbooks and then never in real life uh so let me just let me just give a quick answer for that and then if people want to delve deeper we can um the this osi model first off it's not very accurate uh so then people propose other models that are more represent how the internet really works um however these models have limitations which i think yeah okay so um uh there's two big ones that we saw when we were designing led p2p uh one is that there is duplicated effort uh between different different levels in the stack so an example might be something like https running on a vpn an ipsec vpn so it's like there's encryption at the ip level but then there's also encryption at the application level and they have no way of explaining you know the presentation level the browser has no way of telling the ip level hey uh you know i'm already secure you don't you don't need to encrypt so you end up repeating operations um even sure you can probably think of a dozen more examples like this uh and then uh yeah related to that is what i was just saying there's this inefficiency between uh having having multiple layers uh so building blocks inlet p2p tend to span more than one uh you know layer they're they're kind of indifferent to this osi model um so lit p2p was born out of ipfs if you don't know what ipf is is the interplanetary file system uh basically think of like uh so something like uh you know uh get get meets bittorrent so it's content addressed uh but it's uh it's peer-to-peer in the dissemination of information uh and i'm not going to talk about it but it is a major um project for my employer protocol labs uh and it's been one of our biggest successes uh this one uh one way to look at it is that uh ipfs is a content address system um where you can find content anywhere on the network just based on its hash uh another way to look at and then another way an analogous way to look at lib p2p is that it is a method of process addressing so wherever the process is running it it's uh findable on the network um yeah so this i've already talked about this multiple language implementations um one cool thing about having a javascript implementation is that it runs it runs in browsers obviously um there's a limit to what we can do in browsers we can't open like udp sockets for example which is really really unfortunate but probably isn't going to change in like the next five years at least however we can do a lot of things a surprising number and so what this gives us is a benefit on on any mobile device has a web browser they all support the same standards so we can run lit p2p in web browsers using the js the p2p implementation we have a lot of contributors we have a lot of uh you know friends on github who've contributed or opened issues um i hope that this talk will help us make new friends uh let's see um so you know just as i mentioned um ethereum 2.0 is going to use libpet2p uh polkadot already uses it um ethereum ethereum 1.0 might even use lip22p that's still still up in the air uh yeah so here's um uh gavin woods saying positive things about us this is awesome here are some of the uh people who have built applications on top of the p2p uh live pier is cool it's like a peer-to-peer streaming system um open bazaar is really cool it's like ebay for the decentralized world and this is the obligatory you should join us slide i'm going to go through a lot of this fast if you want to learn more about lit p2p go to one of these urls as you can see they all follow exactly the same format the only difference is the uh whether it's uh js lib p2p go live pdp or wrestle lib p2p but either way you'll find a set of examples um and also github.

com p2p specs is where we keep the specification as i've mentioned the goal of p2p is to have an implementation have multiple implementations that interoperate based on a spec so these are good resources to check out uh okay so let me see where we are in time okay we have 15 minutes left so this is what i'm going to do i'm going to show you a short demo i'm not going to get into the code behind the demo but i'm just going to going to present it to you and i'll explain what it's doing it's it's a it's a canned pre-recorded demo so it won't fail um and then uh after that uh i'd like to use the last 10 minutes to try to just open this up and you know discussion questions criticisms uh i would i would love if people would would engage like that so um here's the idea behind this demo uh i wanted to write a demonstration application that uh used all four of the languages that were sorry all three of the languages that are kind of most complete so i i wrote a program that okay so this just illustrates that we have a lot of documentation behind this demo and you can go you can go to so you can so if you go to this repo you that's that's the readme you'll see and it explains in great detail how to run it and how to use it um fundamentally this is what the demo does is very simple uh there are three peers a go peer a javascript peer and a rust peer they all start out by connecting to a single bootstrap server which is also written in go but it is not an important detail and then once they're connected they use pub sub which is one of the abstractions we we provide to send just like a basic text string to each other so uh let's see if this video starts automatically or do i have to press like spacebar oh yeah okay so uh the top left window is the bootstrap node it's uh going to be it's going to start in just a second oh i guess it's going to compile first uh now it's going to be started um and uh this is again is just to give the other nodes a point to connect to um so okay so that has started uh there's a weird thing blocking some of the text but basically that's specifying a pre-generated key pair that's kind of all you need to know so okay so now we've got the bootstrap node running now we're going to go to the top right window and you're going to see the go the go pier which will be started it will connect to the bootstrap node and then the same thing is going to happen in the bottom left window which is the javascript here uh node.js not it's not a browser and then the fourth window is going to show you the rust pier and then they're all going to communicate so so for example what you see in the javascript here is it's specifying the address of the bootstrap node and as this goes you can see in the top left window that uh the bootstrap node is getting these connections and it can see who's who's connecting to it and then finally the rest here is connected so now all three are connected and one of them is i think the top right window is going to type some hello world kind of thing and you can see all four of the windows got that message uh that's because it's using uh they're all subscribing to the same topic so of course they all get the message including the bootstrapper and every other peer can also uh can also send a message that's seen by all of them uh there's nothing particularly special about pub sub like it's just one of many abstractions that we offer but i needed sort of something to demo and it's a pretty easy one to understand um all right i think that the video ends in a moment and then that's going to be the end of the part where i'm talking and all the code for this is available in that repo i showed you in the beginning so um you you can play with this and and it's it's extremely simple it's not uh uh it's works out just a tiny fraction of livp2p but it does show the three languages interoperating uh okay so i press this okay yeah and then it's just like thank you and then um what i would like to do now is have some we have 10 minutes left uh some time for questions discussion debate uh yes am i supposed to okay there's a gentleman back there coming up so in um uh okay so in tls 1.2 there were many uh downgrade attacks and things and you guys do use this um you guys use and well i know you use uh multi-hash in in lib in in the p2p and you may have other sorts of things like this would it be reasonable to sort of excise this from the code base so get rid of those kind of threats of downgrade attacks try and move to a protocol a versioned protocol instead of something with these this kind of craft lying around yeah that's an excellent question uh the answer is yes uh so i'm going to comment on this and then i also want to get raul is the tech lead for this project so i'll let him uh uh sp comment also but basically yes we uh our uh cryptography including the uh authentication part is uh currently implemented in a module that we wrote this is it's like roll your own crypto uh you know this this approach never never really works that well uh but it's called sec io we want to get rid of secio in favor of tls 1.3 which uh is not uh if i understand correctly it's not finalized yet but um the advantage of it is it works really well for peer-to-peer because it doesn't have this notion of a client and a server it's just like two machines can connect uh do you want you want to add anything so i just wanted to add that at the same like the problem is always the same which is the technical debt and uh backwards compatibility right so if from a project that is trying to define a canonical model for hashing and a dictionary for hashing and a plugable format for hashing and so on you remove hash functions that might be in use out there then you know you are you're going to see everest in runtime so removing those i agree that you know insecure stuff should definitely be removed but it should be done in a control manner and of course in a version manner and in terms of 1.3 yes there is a currently a proposal in the libya2p community to adopt tls 1.

3 and it does um some interesting stuff regarding how we generate a certificate and how we embed the you know approve of the peer id because the way that we model identity and keys and the p2p is different to the way that tls does it so we need to a way to link the identity behind the p2p with the way that tls allows you to transport an identity um there's a proposal um in the specs ripple and as well there are some efforts of looking into noise uh wireguard and so on and other things and i'd be curious to talk to you guys about that as well uh yeah so i'll let you choose jack i think he had his hand first the guy in the black shirt should i pass thank you um so there's a um i notice that there's a signaling server to do the handoff of the connection in between the peers right um is there uh a plan for setting up a system of uh signaling servers that would allow to do those handoffs um in a more decentralized manner uh i actually don't think there's a signaling server um yeah for hint oh oh for browsers yeah okay okay for webrtc you're talking about that specific case or yeah um the the bootstrap node that you used right oh yeah yeah sorry so okay so sorry yeah no no that's a good question this demo assumes a centralized bootstrapping node because it's the easiest thing to implement what we use in practice actually are centralized bootstrap nodes primarily but we we would like to add other discovery mechanisms that eliminate the need for any centralized infrastructure and um yeah it's kind of it's kind of a band-aid but at the same time it's uh it's very difficult to bootstrap yourself on a network if you don't trust anyone or don't know who you can trust um could you talk briefly about the state of specifications and i with that i mean sort of in terms of making a machine readable and let's say i want to write one implementation in nim like what sort of how high of a priority is that and sort of how do you think about it because there's lots of different components so how do you think about that then what's your plan for that yeah so um philosophically we would like to be spec first like we'd like to have really tight uh specifications and then implementers you know may must should whatever follow them uh in practice it's not where we are today i think the specs repo is a good uh sort of explanation of the p2p but it's not an rfc um so yeah we recognize that as a as a defect that needs to be addressed but we haven't yet yeah so ideally um it's completely what mike said uh we want to get to a point where you can just read the spec and and like understand what you need to implement first of all to interrupt interoperate with other nodes and second to embody the spirit of liby2p in terms of you know the principles of the b2b so uh so that you can call something libby to be following the model of liby2p in terms of likability in terms of uh you know the system uh that's also something that that that we need to define libby libit2p as mike said the history of libby2b is that it was extracted from ibfs so we are at that point where we're trying to now libby to b is definitely a top-level project that is being used independently in many other downstream projects and we are at a point where hey we now recognizing that this needs to be specked out right it's not a so it's kind of like bridging that gap um and we're we're we're getting there yeah it's difficult it's a big backlog of stuff that we need to speak out in fact short question i want do you have um any primitives in the leapt p2p to build up rpc servers maybe then then let me just uh as uh ask the second question in your example in a demo the javascript client they had the ipfs parameter if you can comment on that so yes building in an rp some kind of rpc system is something that we're very interested in doing uh it does not exist yet it's it's one of many ideas that's on that in that google doc if you uh go back and watch the the video of this it's issue 60 in lib p2p slash p2p um on the the second point um so so the question is what is the status of rpc i would say it's at the idea stage uh no no specking or or implementation has been done yet uh and sorry now i forgot yours oh you were asking in in the demo yes sorry i forgot your second yes yes yes uh that's a historical relic um we all mult so we use multi-adders to identify a peer and this is a format that protocol labs created originally for ipfs so right now ipfs is embedded in every multi-adder we want to switch to slash p2p a generic one uh for demos like this that have nothing to do with ipfs so it's kind of uh it was a good catch on your part but it's kind of unfortunate ah good question uh so we have this notion of multi-formats which are self-describing uh byte formats where there's a header that identifies what the thing is and then there's some stream of bytes that are interpreted depending on what the header says um so we we have addresses uh in this format so you know there's a byte at the beginning that's like this is an address uh and then yeah and okay and so what you saw was actually the textual representation of a multi-adder the real representation is compact binary yeah the idea the idea behind a multi-adder is that the protocol doesn't need to state that a particular field is an ipv4 address and it will remain that way forever because that can change in a world where transports are pluggable and you can find peers in across different communication transport transport methods whether it's ip networks whether it's bluetooth whether whether it's radio whether it's you know a different uh mechanism um that part the way that you address a peer that you locate up here should allow should be flexible enough so that you know that can change over time and it can change dynamically as well so we're not multi-adders is a way of not embedding is a particular field as ipv4 oh and then later we realized that maybe we want to deal with ipv6 so we need to create another field because we can't extend the previous one and then suddenly we want to incorporate bluetooth in the protocol so now you know so it's it gives you that flexibility essentially of self def self describing locators on a network um i have a feeling we're out of time because jack is like typing oh you're fine okay okay so just i mean just quick oh okay quick note i mean i work a lot with ietf and i think lib pdp would be a great ietf spec and tls 1.3 is done but noise may be more interesting for your particular use case that being said so it sounds like the big advantage of lib p2p is which is the correct design is we've separated the identifier of the node it's a hash of a key from whatever however we've discovered it and whatever transport we're using do you have any key management like what if i want to change my key how does that work in lib p2p that's always really tricky so so far uh that is left up to the application that's using lippy to be uh so lippy to be uh when you start when you when you call out to lippy to be and you construct a b2b you pass in a key um that will be your identity so it's left up to the application itself yeah but i'd be curious if this is something that could be a service that libby to be offers in a plugable manner it could be a facility you know if it's commonplace then it could be something that potentially could make it into libby to be for standardization yes okay so it's been it's been 30 minutes by my account um but i guess they're having trouble with the laptop by the way sorry just going back to the uh the nym question for a second i just wanted to add um uh i think we'll have a chance to talk to you guys more but we would really we would really like to help and if that means like leveling up the specs you know we can do that um basically the the de facto spec right now is the go implementation it's like the reference implementation yeah so uh most people uh who implement lib p2p basically end up reading the go code all right keep going okay uh one more question anyone oh uh this gentleman yeah uh just a quick feedback i just wanted to confirm what was said earlier about this pure discovery uh as somebody who's like pretty new to networking and like if you want to make a simple demo it's really useful to have a simple pure discovery mechanism and especially like if you're trying to show the centralization going through a bootstrapping node is that so great although i do understand like why you would want that like for security reasons but yeah yeah that's all i wanted to say like discovery is important yeah i mean so we've just booted up this documentation project that's going to work on um usage examples um better highlighting the the examples we already have uh better explaining them and so pure discovery is some is something we would want to work on i mean my thinking is that basically we should the documentation should be uh robust enough that you know you can go from basically zero knowledge to being able to to write a program on top of the p2p i think that's how most other libraries are so also i wanted to say for a demo for a simple demo like this you're probably on the same machine you probably could use local dns for example for discovery or mdns for discovery but there are of course like if you go out to the network if you want a bootstrap at the end of the day you do unfortunately you haven't found any discovery mechanism so far that is addressing peers across networks that doesn't rely on a bootstrap mechanism somehow over time you could avoid that booster mechanism if you've had a successful boot up at one time right in the previous point in time and you store peers that you've discovered in the network then you don't you can the next time you boot up you don't rely on a on a bootstrap uh on a on a bootstrap mechanism on a bootstrap server but it wouldn't be possible to just like flood all the networks with like a call like like hello like that's ending so the reply was that's mdns okay i guess this is a sign we should keep going so does anyone else have a question or do you want a break okay okay all right well thank you very much for coming uh if you have more questions come up and talk to me and raul pierre from parity implemented uh rustling p2p almost entirely by himself so he's a great person to talk to as well thank you very much [Applause] okay great next we're going to have