Hetu: A P2P Communication Layer With Causality Graphs

[Music] all right um so yeah today I'm very excited to uh share with you guys uh our hat to protocol um so forgive me I'm uh I just recently got a pretty bad code so my voice is a bit like a bit shaky uh but hope hope everybody can hear me is it good oh good okay cool all right so uh first a quick introduction uh so my name is Jalen Lee uh so I'm currently an assistant professor at the National University of Singapore and also I'm representing uh at the V Labs uh to uh to tell you more about our uh very recent hat2 protocol uh where we try to generalize uh this generalized notion of distributor consistency uh for building uh large scale decentralized applications all right um so let me first start with I guess all of you heers are uh our developers uh in the web stre space so uh you you guys use or have been uh developing infrastructure or use large scale peer-to-peer infrastructures um but so far there are very limited uh number of options in terms of what kind of consistency you get uh or kind of security you get from this infrastructure right so I here I gave you sort of like a graphical illustration the kind of spectrum of consistency levels or the kind of trust you get from this infrastructures so on the very uh left far end of the spectrum you get things like traditional TCP IP stack or Li P2P or things like that where you can build very fast um very fast networking right uh communication between different peers in a uh in a peer-to-peer Network right but the problem is that you don't really get a lot of security or consistency from these infrastructures right so you kind of get uh the bare minimum kind of security if you do end to endend encryption right so that gives you things like authent authentication Integrity of your messages but you don't get much more than that right uh so if you build distributed applications or decentralized applications would require a lot more security or consistency uh instead what you do is you choose the far right end of the spectrum where you use probably a blockchain use a uh permission bft protocol or you use any other form of infrastructure like Layer Two rops reaking and other forms of instructure that gives you very very strong security right so you have a totally ordered uh uh set of blocks right so everybody agrees on the order so there's Global consensus right uh so security wise is really good uh it's a very strong consistency uh but the problem is it's very slow it doesn't really scale right so for many kind of applications it's not probably not ideal to use this infrastructure as well right but we can see that this is a spectrum right so you don't only have two end points there's a lot of ground in the middle right so all these middle uh for Security Options and consistency options they're currently not a lot of uh for you infrastructure wise for to use right um but actually we see that there are a lot of applications which can actually benefit from a more uh slightly more relaxed level consistencies or trust models um to to scale them right so here I just gave you a sense of uh example application but here definitely there are not a uh exhaustive list right so here things like social applications right things like communication between different AI agents right uh like Games social games um uh and more recent like uh uh decentralized physical infrastructure and so on so forth right so all these applications right they are so they're they're proper one of the main properties that very interactive right so there are a lot of interactions between different parties in the network a lot of communications going on right so putting them on chain is probably not the best option because of the scalability bottleneck so even though you have more scalability options these days uh but eventually because the cost and then there's still limited throughput and scalability is still not ideal for you to to to build these applications on top on the other hand these application do require some level of security and trust and also consistency right so for example if you're building social applications you want to have some kind of consistency so that your followers right your tweets right so there are there there there the follow your followers right so these are your your your uh your metadata changes there there are some kind of consistency guarantees right so so there's so at this moment you either choose right the uh the very uh very uh so infrastructure that they don't give you anything or you you choose like blockchains uh so which are fairly Limited in terms of scalability right uh so in this talk so we're going to introduce HE2 uh which does provide this kind of Middle Ground where there are offers a lot more kind of consistency and Security Options for you to build distributed or decentralized applications right so HE2 is a peer-to-peer communication layer right so but uh so compared to existing peer-to-peer infrastructures or communication infrastructure uh it additionally provides this what we call trusted event ordering uh naturally from this communication layer right uh and then underlying it we are leveraging a novel um form of verifiable logical clock uh which we I'm going to elaborate uh in a few minutes um so the so the so the so the nice thing about HE2 is that it is very low cost and then very highly scalable um but also it gives provides the opportunity for distributed or decentralized application to build various consistency levels on top of it uh and then so it has really really wide kind of applic applicability right so all the applications I just show you uh will will benefit from our communication stack so let me just elaborate by starting from a little bit background right so uh so when you build any kind of applications distributed systems right so the order of events is a really important aspect right so no matter what you do you care about some kind of ordering right either is like between different blogs or not between a uh a social social media post and then some other activities right so these are all events and you care about the ordering so how do you order events in a system right so the most traditional way so for example if you come to uh this e Denver event where you look at all the events uh in this in this uh in on the schedule so you clearly Mark right what is what time is each event happening right so that's the most traditional way of ordering events right so my talk starts at 1:45 and then the previous stock start at like 1 uh 1 p.m. or 1:30 p.m. or something right so by looking at the different time span time stamp you can know exactly which event happened first so but the problem is that in a truly distributed or decentralized system this kind of physical clock or commonly agreed physical clock is not present so there are multiple reasons why you don't have that in a decentralized system so first from a theoretical perspective uh if you guys uh have heard or learned about relativity right so in relativity right so different objects uh which have different relative motions they actually don't have the same common clock right uh so this phen phenomena is called time dilation so for example uh here I show uh on the left is the international uh space uh station right so and then so because it's moving with a relative motion with for example this is the location of e dver right because if if there is relative motion so the time actually disagree with each other right so they don't have the common time so the other problem is that even if you have if you don't have relative motion among different objects because your clocks devices they are not perfect right so they cannot track your physical time um very accurately right um so for example depending on what kind of clock technology you use if you use U small atomic clocks right versus very uh so sorry very sophisticated atomic clocks versus a simple clock you can find your computer like a crystal oscillator so they have very different clock drifts right so it's very hard for you have to have perfectly synchronized clocks among different parties right so if we can't have like a a common agree upon physical clock what can we do to have uh to to order events in a distributed system so here we look at uh look back in the literature in distributed systems so back in 1978 uh so a famous distributed system researcher uh his name is Lassie Lamport uh who later got theer award for some of this work including this so uh he published a seminal work called time clocks and The Ordering of events in distributed system where he exactly talk about this problem so if you don't have physical clock in the system how can you order events so he argued that the only trusted way of ordering events in a distributed system is to look at The Logical ordering of different events or another way of saying it is called the Cal causal ordering of events in a system so what is causal event ordering in a system so here I show if you have multiple processes in a distributed system right so they talk to each other through messaging right so you don't have a clock so you don't know which event happened first but you know for sure that if I send the message from process P to process Q the sending of the event must happen before the receiving of the message now I establish a causal dependency between these events and then you can use this if I I receive a message and later send another event uh and sorry I send another message I know the sending of the next message causally depending on I received the previous message right because the sending May cly depend on actually receive the previous message right and then you can do a transitive closure of these events then you get this what they called a happened before relationship which captured the causal orological dependency between different events so that you don't require any physical clock and then that is the true ordering of the the system that nobody can disagree with right so that's a pretty cool and a pretty strong kind of uh uh event ordering property and then uh let's lample argue that now we can develop a new kind of clock that doesn't depend on physical clock but instead on a logical clock this logical clock essentially track this causal uh causal dependency between different events by looking at this causal clock sorry logical Clause you can infer what's the causal dependency between different events right so so this paper and the notion of logical clock on causal dependency have a lot of implication a huge implication on the development of a lot of distributed and decentralized uh protocols to this state um so we are uh taking this protocol and this notion a step forward so we are looking at how do we build logical clock in a decentralized environment right so a peer-to-peer environment one of the biggest challenges in this peer-to-peer envir environment is the presence of potentially malicious or adversaries users right so pantin knows or another way of saying it right so because traditional logical clock depends on all the protoc all the all the participant in the system follow exactly the protocol of updating The Logical clock right so if there are any pantin node in the system it will break the break the entire notion of logical clock and the caal dependency guarantees right so what we do is we have done some uh recent re research and and then we found or we developed a new way a new logical clock called verifiable logical clock uh that ensures that the The Logical Clause pass in the system can be verified by any third party right so that means nobody can can uh create a false logical clock that breaks Lo uh causal dependencies right and then so I mean I'm not going to have the time to give you all the technical background uh but a hell of a gist is that we leverage this new in noal application of zero knowledge proof so this is very recent uh development in the space uh so this new technique is called incrementally verifiable computation or IVC that allows us to have multiple parties in the system by passing these messages build a trusted logical clock with a verifiable proofs right and by applying this uh this novel kind of verifiable logical clock we're able to build a lot of interesting applications and infrastructure on top of it here I just give you a some quick uh examples that we recently developed uh but you can certainly think of using this verifiable logical clock for a lot of more interesting applications right so for example we can use uh this verifiable logical clock to build decentralized Mutual exclusion or the other way of saying is like m a a lock right uh a synchronization primitive between different participant in the system without centralized trust or centralized server for log Services right and you can build as I I hinted in the beginning of the talk right so there are a lot of application that I want something uh consistency levels not that are not like strictly consistent right not linearizable in a more technical sense right so here we are able to use this uh verifiable logical clock to build causally and eventually consistent data stores and then uh uh things like virtual machines right so your virtual machine State doesn't need to be always synchronized across each other but we guarantee either eventually synchronized or if there's a c dependency between the different updates we make sure everybody observed them in a causally consistent order right and then we can use this logical verifiable logical clock to build a shared sequencer for example right here we're able to use the partial ordering property of this verifi logical clock to easily scale a shared sequencing layer right uh and then so so far we have built a so from this verif a logical clock we build a HE2 stack uh so this HE2 stack as I mentioned before is a peer-to-peer communication layer right um so sorry that the the the picture might be a little bit blurry and small uh So currently we we can structure as three three kind of layers so the first layer is underlying the the lowest layer we have this messaging layer where um so we can the messaging layer creates the uh the proof and then generate generate proofs and then verify uh the for the for this logical clocks and verify this logical clock while generating messaging and then this layer also uh uh build build a traditional p-to-p communic layer things like cadamia uh DHT right and then on top of it we have a a middle what we call the shim layer where we build a sort of like a a relay Network where there's a large scale uh peer-to-peer network with a lot of participants messaging each other uh with uh the underlying shared sorry underlying a verifiable logical clock in this each messages and on top of it expose a simple to use API to the application layer where the application can use uh the The Logical clocks from the underlying layers to build their uh desired consistent levels and their applications so that's the current hat2 protocol uh hat to stack uh and then we are a very Community Driven approach right so so we uh uh we develop a fairly interesting and pretty novel uh protocol called the hat to Improvement uh proposals this is like the EIP system right where everybody not just the our core development team can propose um different Improvement on top of K to protocol or a components of the hatter protocol right so here we gave you some we roughly coarsely developed into three different layers so the first layer is called hiip a layer so this is the like the fundamental layers of the the HE2 protocol so here so we build the fundamental technology behind HE2 things like logical clocks uh the peer-to-peer Network and then uh the incrementally verifiable verifiable computation right and then on top of it there is the hip B layer where we there's the middle uh middleware that people can build things like the relay Network I mentioned about and also things like sequencers and then random number generators and then there are many developers who care more about the upper level applications and then they can propose uh things like Messengers for social applications uh decentralized games or things like social other kind of social applications on top of it right so we open up uh this hip system to all the developers in the system uh in the in the community right everybody can propose right and then once it's approved uh they can be merged into uh the Hato stack right so currently we have a proof of concept right so if you're interested feel free to check uh check out our uh GitHub page right so currently we have a few of these components currently already built things like The verifiable Logical clock so the underlying peer-to-peer communication Library uh and then we also developed some a few applications on top of it already as a demo right things like uh a social application called Z social and then a random number generation right so um so this is our prep proposed timeline for our uh HE2 stack and HE2 protocol uh so I don't know if you guys can read it but we are currently roughly here so we're uh we develop our current propal concept POC and then our goal is to have the test net some sometime uh in the middle of this year right so which is here uh and then later this year and by the end of this year or early next year where our plan is to row out the main net of hat2 uh so we welcome everybody to uh contribute uh to talk to us give us feedback um so uh so many of our team members are here at East Denver so if you are interest in our protocol or our stack feel free to talk us uh so uh so aasha is here in this I think somewhere here uh that's me and then I think Sean is also here and then we have many other people here from the uh the hat2 team uh and then uh also uh follow us on Twitter uh and then we welcome all your feedback and then yeah I think that's it and thanks everybody bu