Demystifying libp2p Gossipsub: A Scalable and Extensible p2p Gossip Protocol by Raúl Kripalani

[Applause] [Music] on gossip so there's been a lot of interest in gossip so uh there are like 130 people signed up for this particular talk right i don't think everybody will fit in this work if they happen to show up uh but yeah we have a very little time and a lot of garage to cover so i'm going to get started i'm raoul i'm the techie for the b2b project from multiple labs and this is what we're going to be talking about demystifying glasses so conceptual is a scalable an extensible peer-to-peer gossip protocol and our agenda for today is first of all i want to make sure that we all have kind of like a common baseline uh to talk about these uh technical concepts so i'm gonna do a very quick introduction to the beer to beer box up who here is familiar with with these concepts like maybe a five and above in a scale of ten fantastic okay so i'm gonna move through this pretty quickly um then we're gonna talk about what is custom sub we're gonna talk about how glossary operates and what's up next uh because as you'll see gossip serve is a um a stepping stone in a road map that uh includes a lot more functionality that's uh that's coming that's coming in the future so oh my god all right so let's talk about what is up is a message oriented communication pattern right this isn't stop contrast with stream orientation so here we're not interested with pops up we're normally interested in sending transactional messages and small units that are actually messages and not streams right over the wire between a bunch of peers that are not necessarily talking to each other so the interaction model of pub sub is m to n this means that n peers are publishing messages about a particular subject and a lot of other peers in the network are interested in consuming uh messages from that from that particular subject right this is as i said in contrast to rbc or reply uh request reply patterns which are mostly uh one to one now bumps up communication is asynchronous and decoupled in nature right this means that whenever i send a message as a producer i have no guarantee first of all when who is actually going to receive that message because we don't have peers don't have a complete view of the network and a complete view of the subscription state and it's also it's also a synchronous because there is no time dependency um i have i received no explicit confirmation from any peer that they've actually received my message right processes act as publishers and consumers of messages and then appear to be your network this is important because the process can act as both things um and they congregate around topics of interest now uh pub service is widely popular in enterprise software uh and if you've heard of products and projects like apache and mq or kafka rather than you then that's it but in those uh contexts usually centralized around brokers in private networks and it's definitely not these approaches are not suitable for for public message practice so what does decentralized uh appear to be your website first of all it's brokerness it means that we have no central authorities to keep track of subscribers publishers topics it's self-regulating uh whenever we deal with peer-to-peer networks we have to moderate current right and peer in networks can rotate in presence uh very rapidly so there is no global knowledge there is no single peer that is on authority that has a complete view of the network and the network behavior actually emerges from small tiny local decisions that peers are taking along the way right so peer-to-peer pub sub is usually constructed as an overlay network with peers collaborating to ensure message deliverability right and when we say overlay there's usually another right so it's an underlay network that actually makes sure that messages and traffic gets routed and usually algorithms will assume that but the underlay is well connected so this means that i can always find a path from a to b and if i can't that's a topology so just to set a baseline for terminology a topic is a subject around which beers congregate and they commit to receive messages subscription is the act of expressing interest on a particular topic publishing is the act of creating a message and pushing it out to the network forwarding is the act of propagating a message to other peers on the topic and ambient discovery is something i'll talk about in a second so the desirable properties in peer-to-peer pops up are reliability resilience effect efficient dissemination altruism free routing this means that only those peers that are actually participating in one in a particular topic will um will actually contribute to forwarding that topic and we're not burdening burdening the rest of the peers on the network that are not invested in a particular topic and disseminating that it has to be scalable across various access yeah so we're talking about number of nodes number of topics messages subscriptions subscribers all of these are different access for scalability there has to be a low overhead for maintenance as i say as i say we are relying on peers making local decisions so that messages are globally routed in the network and ideally the protocol that is simple and easy to understand so we'll see if we've actually managed to do this one customer all right so what is gossip so the key elements of gossip serve is that first of all there's a scalable and extensible pubs out so gossip stuff in itself is not necessarily intended to be a final usage of a peer-to-peer of appear to be your pub sub protocol it is a protocol on top of which we can overlay different routers to take different decisions and pops up and sorry and gossip sub actually provides the primitives to make sure that first of all messages are delivered and second there's enough gossip metadata start canadian on the network so that different and alternative uh decisions can be taken along the way it was envisioned and specified initially by a developer um called viso he's an elevatory core team and as i said it provides a base layer and primitives to build adaptive self-optimizing routing algorithms capable of repair and rearrangement in the network in the presence of churn and so on so the key aspect of gossip sub is that it combines stable reciprocal meshes with gossip about message from metadata we'll dig into this in a second and it says it's a protocol that we had in the living tb stack which is called flat sub and it's the basically the dumbest protocol uh in terms of uh period of year pops up you disseminate the message to every peer that you're connected to then you know it's interesting on topic right which has uh which is great because it's robust it's resilient and you know it can take minimal latency parts because you just push the message out and it will those peers that are are close closely connected to you and have a fast thing with you will receive a first but it also creates a lot of right amplification and it basically clogs clogs up uh the bandwidth because you're creating a lot of duplication in the network as i said gossip serve is the stepping stone towards the grand mission of msr which is i'll introduce it a bit later but basically concocts ideas from plum tree high party or go past warning the stickiness of gossip is currently alpha right and this is very important for people to uh to understand it is a protocol that we're gonna continue iterating on over and over again and as i said as a stepping stone towards this more stable and feature feature complete protocol called there are three implementations there is uh the reference implementation and go that the liby2p protein produced there is a implementation by the sigma prime guys who are just here raise your hands please get some credit fantastic uh it's currently sitting in a fork and i don't know if the chainsaw guys are here i don't think they are but they also contributed and they created a gossip server yes cool so how does it work i'm going to go through different layers of the of the protocol together like do a walkthrough so the first thing that you need to understand is that gossip sub is actually a hybrid of two different networks right that are on the same protocol that are on the same monday so there is a full message network uh formed by receivable links through which messages are folded and this is reciprocal is the key the word for circle is very important because this is supposed to be a stable mesh where peers are actually exchanging messages with one another so if i if i'm pra and there is i'm interacting with peer b in in a topic these two peers will actually be exchanging messages with one another consistent and then there is a metadata only network that carries gossips about which peers what peers have actually seen so as i said the full message network we call it mesh is a sparsely connected network of reciprocal links between peers that are actively consuming and interested and therefore forwarding a particular topic the construction of the mesh is actually randomized and is self-step stabilizing because it has a number of stabilizing parameters particularly in the degree so peers will are configured the mesh is configured with a target degree which by default is six there is a high and low water mark which by default is four and twelve and peers will strive to maintain the connectedness for a particular topic um at an optimal level of sex taking compensating action whenever it goes about below the low watermark or above the high water this is how we keep that that um that leeway to avoid drastic you know like down cutting and not regulating this is uh so this table mesh is complemented by a metadata only network which is a densely connected network that augments the mesh with gossip propagation about which messages are actually moving so this enables writing decisions to be made if we if i have so the the easy way to put this is in a particular topic if i have 50 peers that are interested that are consuming so 50 neighbors 15 neighbors of mine are interested in a particular topic um only allowed to keep six right in theory based on the on on the stable mesh the remaining so the remaining nine will turn into gossip years so i will not exchange messages for them but i will i will collaborate with them telling them what messages i've seen without pushing actually without actually pushing messages we'll go through this in a second so how does topic membership work as i said a peer can be subscribed and re-emitting messages on various topics artworks so peers can participate in many topics either as subscribers publishers or both and peers keep a view of all neighbor subscriptions whether they overlap with ours or not so if i'm subscribed to topic a and another peer is subscribed to topic c but i'm not interested in c i will still know that the other players subscribe to topics and subscribers as i said only engage in message propagation and not set forth virginia topic so i will only actually participate and this is one of the key properties if you remember that are desirable and pops up systems and peer-to-peer pub sub systems which is only those fears that i invested and the topic should be making an effort towards disseminating it so that so lots of stuff does satisfy them so we have a concept that's called subscription weakening which is basically on every new connection peers will exchange the topics that they subscribe to and as uh the application that is using the livvy to be stack joins or leaves topics they will send updates to all other peers so as i'm participating as i become a participant in a new topic uh i will send an update to to all of the peers now we don't want to be very chatty so we've introduced the concept of piggybacking right so this means that we will try to coalesce um updates metadata updates to peers on top of messages that we anyway need to send them right such as graphs and rules and so on as you can see in this kind of weakening enables us to uh create about metadata flows and to um and to do also a federal topic setting which is also called fanart um makes me a business so this uh with regards to topic membership there is two key concepts to to to understand here there is the concept of grafting improvement i said that um the stable mesh is actually a self-regulating mesh and and these two messages are key to regulate that uh to regulate that mesh so graph control messages a rough control message proposes a two-way um full message stable link with a peer so it basically when as one appears sends a graph to another peer is basically telling it hey i want to be part of your reciprocal mesh for this topic right this means that i'm going to send you messages and you promise to send me messages as well right now the other peer can reject that because they're already beyond their target degree right and if they're rejected uh they send back however the other peer knows that i'm interested in that topic right because a a subscription to that topic therefore they're even though i'm not part of a stable mesh that will send me metadata right so this is where the asp these two networks start making sense right now um a proven control message dissolves a two-way message a two-way stable stable mesh uh stable between two layers and this can be uh and this can be centered respond to in response to a graph that we want to reject or if i'm down regulating my degree right now uh one key another key aspect to to understand is that by itself gossip sub right now does not include a mechanism for peer discovery right so if we to whenever we want to subscribe to a topic we need to find a set of peers that are already subscribed to that topic and this and the specification is called ambient period discovery right we rely on immediate discovery mechanism which is present in the ambulance right this is going to change as we go forward uh where there's currently a pr uh to add active peer discovery so that whenever you the application subscribes to a topic uh there will be a pops up we'll actually do a lookup of peers in that topic for them for the application and we also plan on introducing something that we called collaborative peer exchange now let's talk about message flow when a peer wants to send a message as a message source they just push the message to all's table to to all peers and then mention their full message now right super simple now peers that receive that message are actually message routes so whenever they receive that message they will forward that message to all other peers that are part of that mesh of that of that view of the panel right and we follow a heuristic to break sadness because we don't want to send the message back and forward back and forth right now we can also publish messages on topics that we don't subscribe to right and this is interesting for things like for example ipns the interplanetary naming service where we want to send updates from whatever name the mapping of a name changes to a different app for example right so you don't necessarily the peer that's watching for example a dns server a dns name server somewhere unless you notice things change doesn't necessarily necessarily want to be subscribed to those topics right so that's the possibility of sending those messages and when we do that we basically are remember that we keep in track of what peers are actually subscribed to the network we're not for forming stable meshes with them but we keep in trouble so if if the local application tries to send that message we will immediately form what we call a finite group from those peers that we know are already subscribed to that topic right and this is going to be a different location so once we form that group we push out the message those peers we know are already part of the mesh therefore they will forward that message internally because the message has arrived inside the mesh for the topic and after two minutes um we just dissolved that fundamentally because you know it was just an informal set so we don't want to keep it along for too long because that would be aggregating state that we don't um we also have the concept of topic validators and message signatures so uh we can so when appear when an application sends a message uh sorry the application can configure it can attach predicates validation predicates to topics right and ether2.0 for example is using this uh for validating signatures in in blocks that are getting propagated right and transactions so when a message is published locally or before falling a message this predicate will be called if it succeeds then the message will continue dissemination otherwise it will just be dropped we have the concept of synchronous validators for heavier logic and gossip help can also be configured to sign outbound messages into um and just strictly validate panning signatures now how do we regulate the mesh there's a concept called heartbeat which by default happens uh every second this is a very popular concept in distributed systems uh and this basically performs mesh and gossip maintenance functions right there's three things that happen we regulate the mesh degree for every topic we emit concept and we slide the message questions so this is kind of like how degree recognition works as i already explained it if we're below a certain threshold we will try to add peers to the mesh for that particular forum so for that particular topic if we're above a certain threshold then we will try we will prune uh periods from that topic um to try to keep the degree at a stable constant to be able to do this we keep a bunch of local states or topics that are subscribed to topics that are recently sent a message to this is the founder groups uh peers that we currently connected to um and what kind of links we maintain with them whether it's a full link message data link or if the fan appears and what messages would you recently see why is this important this this last bit because we send business so this is where the gossip comes in so for every single peer that we know is interested in a particular topic that is not part of our stable mesh we will collaborate in the network by by gossip and gossiping what messages we've seen in that particular topic so we don't send a message to them but what we do is with every tick and every heartbeat we will uh we with the message we keep in a message cache and we keep it under like a sliding window the messages that we've seen in the last 30 seconds two minutes this is configurable we will send them a list of all the message of the ids of the messages that we've seen right and this usually happens uh as well we we try to think back this because we want we don't want to create excess uh excess overhead in the network that's ending this makes a daytime around every time that we see a message so we try to make it back if we have a message to send them anyway then we try to bring back uh these control messages now if a peer spots a message that they haven't seen right they can ask for that message they can pull that message by uh sending an i want control message so in that i want control message they will be listing the messages that they want us to push to them right why is this important this is important because it allows us to take routing decisions so imagine the case where you have a stable mesh right you're in a topic you had a bunch of peers that are sending you messages and another peer which is gossiping with you tells you about a message that you haven't seen yet this means that your that that deliverability towards you is not 100 right because in your in your mesh uh you haven't received that message yet right so you can use that as a sign as a hint to try to repair that pathology right by pruning peers that you believe are unhealthy and grafting with beers with the peer that sends you a message who probably has a better route to the rest of the mesh than you do right you have one meaning nick okay fantastic um i think i'm almost done um so so yeah this is kind of like a bit of a schematic view of what's on the wire by the way all of this all of these images and so on uh you can find them on the web so don't worry uh i'll just show you where they are now i just want to talk quickly about what's next uh for a gossip sub first of all we want to formalize concepts up into an academic paper um we want to harden the reference implementation and go this is for us very very important for the vision of concept of itself and formalizing what's upside into an academic paper is gonna be also a robust step in allowing other downstream projects to actually uh build on top of that analyze it and to get all this work peer-reviewed because it's essentially a new contribution to the state-of-the-art uh we want to integrate discovery mechanisms and there are various uh benchmarking efforts the way underway important aspect is said um gossip sir is not the final destination and gossip self is a step along the way really the key aspect here is and and where we want to get to is a protocol that we call it exam which is a combination of gumtree um high power view for membership maintenance and goal cost for the proximity awareness meshing and so on and this will allow us essentially to model the kind of behavior that i described so if we so uh with episome we'll be able to build dynamic dissemination trees um that are mutating and that are adapting to the conditions of the network by keeping track of the peers that are better connected to the network and as well those fears that happen to present pathologies with us right so whenever we find messages that we haven't seen that are gossip that are incoming gossip from another peer we can rebalance the tree and make sure that we always have a good connection to them right we wanna so this is what we also call the self-healing mesh we wanna introduce adaptive cache windows and things like collaborative and passive membership views so something that's that's key here is the ability to send advisories over the network of what peers are available in which time so ideally you want to use a ambient discovery mechanism like the dht for example to find a seed a limited number of peers that acts as a seed to bind to that topic and once you have those those few peers which are probably overloaded because they are being advertised right you want to enquire them for other members that they know about in the mesh and you want you want to create gossip not just about the messages that are available but about what peers are present in that topic right so this is something that is going to be cool that's going to be that's coming down the pike it also has some notions of being exploited cool so as i said all of these diagrams and these schematic views which are pretty nice and very beautiful they are available under the docs website so make sure that you check out the new snazzy dogs for gossip sub which we just released one week ago just in time [Applause] you