Juan Benet I InterPlanetary Consensus I Protocol Labs I Pragma Denver 2024

[Music] all right we are ready for our next Fireside discussion and for this talk what Bron W founder partical Labs we're going to talk about interplanetary consensus IPC so please join me and welcom in Ju on stage thanks for having me so there's a lot of interesting things happening here at the protocol Labs universe so let's uh let's start with what is IPC I feel like you would have called it uh work done when the interplanetary file system existed but we got to get to the consensus level what's uh what's going on yeah so um as we were building things like ipfs and lipop and filecoin um we got a very good sense of all the scalability problems that this kind of tech um has and we've run into a very clear like an obvious present problem with the entire industry which is there's no single strong consumer app that in the web fre space right like today when you use your normal um phones or computers for doing your daily tasks almost nothing none of the applications that you use on a day-to-day basis actually use web trate and um the crypto space and that's fundamentally because the scalability of the tech is not there yet meaning you cannot actually build good social networks you cannot actually build good productivity software you cannot build um any of the normal types of applications that normal humans use with current web3 Tech there's a huge gap in the scalability of these Technologies to actually be able to give the right to tools to developers to build those kinds of things so IPC is a a technology that we're building to solve that problem specifically we think the current generation of like the current next generation of um of crypto networks are improving the scalability story significantly but they're very very far away from actually reaching the full traditional web to level um of scalability meaning we need to be hitting um trillions or quadrillions of objects tracked and transactions um not you know kind of like the current like 10,000 DPS or like 100,000 DPS we need need to be hitting billions of transactions per second like that's what it means to be at true large scale uh and so in order to get there uh blets are going to have to change a lot in terms of their structure um and we found like a a a good straightforward path to go from where we are today to that kind of future um with much higher scalability and that's what we're we're kind of building through IPC it's called IPC interplanetary consensus because it builds on the same principle that ipfs buils uh which is the interplanetary principle if you design a system a distributed system to work across planets and you force yourself to to have it to make it have like good real time guarantees um in that setting like really imagine your the thing that you're building working on Earth and working on Mars and working across the two planets and just to make it clear that this is a specifically the variable being just a time delay for the information sinking yeah exactly so that so you know between Earth and Mars there's a huge time delay because it takes a long time for you know signals to um pass uh through space it's like an 8 minute delay between um Earth and Mars if you have that in a normal normal HTP request uh you know imagine that you try to load a web page and you have to wait 8 minutes for the server to come back to come back like that's obviously doesn't work right um that's kind of the the fundamental same principle that that um I used to build ipfs similar with blockchains um if you think about a system that has to work at that range of scalability you end up with a very different architecture and kind of the reason why this principle is helpful is that the kind of like intuition that you have about a latency of like 8 minutes is the same kind of intuition that you should be using for thinking about the difference between your phone or your laptop talking to a data center or for one data dat Center talking to another data center so the the latency difference between two data data centers talking to each other or for your phone talking to a data center is roughly the same kind of orders of magnitude as you know thinking of thinking of your application today here on Earth talking to Something in in Mars uh just in terms of the the relative latencies between programs in some devices or programs that are collocated in that space right so if one program in one data center is talking to another program in the same data center that's like at one millisecond to to 4 millisecond delay right like millisecond this is like 1,000 of a second like super fast so if you design systems that take into account that kind of latency um separation you end up with a dramatically better system design that it ends up being much more um highly scalable that it ends up being ends up being better for applications that need to be collocated to be um you know kind of operating high performance and so on and so IPC is an approach of taking that same concept and saying if you had to design blockchains to operate really well um you know across planets or conversely here on Earth um between data centers and mobile phones and browsers and so on uh what would it look like and so the the structure that we that we arrive at is a a hierarchical network where you have subnets you you make it extremely easy to generate and build um and derive subnets from a from a root network uh and every single level uh it is recursive meaning you should be able to generate subnets at every part of the um part of the tree and each subnet is a full independent chain that should be partition tolerant meaning it should be possible to have full finality and settlement in a sub region of that tree so instead of thinking of like you know one massive L1 with maybe a bunch of rollups that are like super connected and um where availability of the whole thing still relies on availability of the of the L1 imagine a set where you have a tree of chains where at any point a subtree can disconnect from the main Network and continue operating and finalizing things in its Subspace um now of course it couldn't transact with the rest of the network until you get you know you heal that partition but then when you do it you can you continue communicating this kind of structure lets you do this really neat thing of saying hey in a subp part of the tree we're going to do things like gate the membership by some properties like Geographic locality right like you can you can say that in a particular subnet only certain um nodes in a particular data center are going to be joining that that uh that subnet and that lets you make the block time super short like you could have a 10 millisecond or 100 millisecond block time and now you can go you can go at proper you know highly scalable data set our speeds so yeah that's actually great intuition um what about finality and any kind of fork Choice differences even in those partitions so how do you actually kind of have a global consistent system that way yeah so um the Insight here is in using the the parent child relationship to bootstrap the security so you the the things in the in the higher parts of the tree are more secure than the things in the lower parts of the tree and you want to use checkpoints that give you verifiability through the parent right so for example a child subnet would operate for a while using its own consensus and relying on the security guarantees there but at a periodic interval it would checkpoint its state into the parent to gain the security of the parent and you can just recurse this all the way down and make it infinitely scalable you know you need more you need faster um uh things or you need like um to paralyze you just either scale horizontally by adding more siblings or scale y yeah exactly for developers who've used GI branches to their extremes this is yeah and the cloud operates this way right so if you think about the how the data centers work and you know you have like a deep network of hierarchy there you have like the data centers themselves you have you know whole racks where like you know the latencies between machines within a single rack versus from that rack to another rack is a totally different landscape um and then you have in a specific physical server you then have hypervisors with like multiple VMS on top and then each VM then has you know containers potentially and so on so we're taking those same kind of developer Concepts and Primitives to build a blockchain platform and a blockchain system um you know in set of clients and tools and Frameworks to enable developers to actually get that level of you know customization and scalability in their in in their Tech um and the customization part kind what I mean is um through filecoin and fbm we built this really cool structure to be able to um Define both Mount multiple VMS on top of of the FM so the FM is this hypervisor layer you can mount multiple VMS like evm or you know whatever else you you want to use um and you can create a network that's um that's specific to the application you're trying to solve so instead of saying a general purpose public network where anybody can add any contract imagine generating um special purpose networks where you specifically pick the set of programs that are going to run in that Network um and you and you PR so the notes that you're organizing are just taking on that job so this is precisely what a lot of groups like say deepen networks need where you know they they want a they want an orchestrator to be able to like coordinate thousands of nodes and they want to very precisely run a very specific set of programs um they don't want to just run arbitrary smart contracts they want to run a specific set of programs um and so with this you can like take the blockchain which is effectively this really nice kind of distributed systems orchestrator with like strong security um and then just run um run those programs that way uh and use kind of the subnet architecture to like help these these Network scale out I think there's some very interesting overlap with how you already touch on this actually so the way kind of we've seen ethereum do L2 systems uh this is kind of a a slight Twist on how rollups effectively are being done because now you actually have parallel rollups that are in a tree structure conversing on each other yeah yeah can we go a bit deeper into how does different rups or other techniques for actually kind of bash processing transactions yeah so a key difference here is that rollups don't have their own data availability they have to rely on some other network to give data availability and they still rely on Final settlement in the L1 um and so a rollup can like accumulate a bunch of operations but really it doesn't really happen until it's clears in the L1 and that's not the case here so in in this kind of tree structure each Noe in that tree is a full blockchain with finality so you can write an application that um where a subp part of the application just cares about that so like let's work through a super concrete example imagine that you're you're making an MMO and in your MMO you have multiple worlds and in each world you have multiple regions of space and you have to deal with things like you know items and players fighting each other and whatnot so things like items and you know an auction house or something like that see where a lot of values being exchanged requires a lot more security and you probably want to put that you know higher in the tree um but you can partition the entire application by by F creating a topology where you put every World in its own subnet you put then specific regions of that world as children subnets um and then you can run the individual logic of the game server like the actual game server itself with like the hit you know like the players you know casting spells at each other or whatever directly on a chain um that's moving extremely quickly right like for things like MMOs and fpss you need you know 30 to 60 operations per second like no chain today gives you that um and that's kind of like what you need to be able to settle um with finality right like if a player is like playing a game and like they're casting a spell they expect finality they don't want like a random roll back of like oh never mind like the chain like did this other thing or like oh wait wait 7 Seconds until your spell like commits like like absolutely that that's that's not how how um how those things work so so that way you know the actual you know fight in a region or whatever doesn't need the same level of security that say like the items in an auction house or something like that require and so you can compute all that very quickly you can still kind of settle it and finalize it for all the clients that that care about it and then over time kind of externalize the results up the tree um see I don't know if that it actually did so it obviously kind of it's very easy to understand that like you can solve these deep end cases or or the gaming case you talked about or like even now social networks are possible are there any trade-offs that you have to make for these networks to talk to each other like can the one game understand the assets on the other game side and what is that complexity yeah that's a great question so um you know I we think that the world like already it's kind of funny like a few like many years ago like I I used to kind of like have talk about like how the world would have a ton of chains and people are like no no no there's going to be like one single chain that's going to like rule everything it's like I don't think that's going to happen um and you know right now we have like what a few thousand chains already um you you can expect that number to go up to tens of thousands and millions eventually um and so in that world uh you need extremely good calling conventions between this entire world and today the bridge Tech is like atrocious right like uh it's extremely difficult to um move State between these things it's extremely difficult to like write programs that span multiple chains it's extremely difficult to like interact this is very similar to the internet before things like tcpip um and so we need like an extremely good set of calling conventions across this entire entire landscape um to reason about the state moving and that's one of the areas that like we're putting a ton of effort into is like designing the calling conventions across this like Universe of subnets right if you if you are designing for world where you have millions of little networks like millions of little blockchains with their own State how do you reason about um State operations and changes across all of these different logs um and how do you get like strong finality um guarantees over thing asset exchanges that are really valuable right like um how do you reason about moving that state up the tree when you need more security or how do you need how do you deal with like you know some subnet some program deeply nested calling to some other program deeply nested in some other in some other network yeah how uh you just uh what what I'm talking about is like you need you need programming language Primitives that enable yes so so the inside the inside is like it has to be inherently a property of the network and not a facilitation enhancements is that is that no you can do it as a facilitation enhancement but it's much easier if you make it a property of the network so my sens is Network are gonna enhance themselves um once they see how much better it is to to have that but you can do it with any with any network um but what you want what the the core primitive that you want here is the same thing that we've built for the traditional cloud e solve or create with yeah so um you know kind of like when I started working in the space building you know the beginnings of ipfs and uh fcoin and so on uh I wanted to build an institution that where I could go and build open source projects um and kind of scale them at the time when I kind of looked around um I couldn't find an organization that could support building a project like ffs to do like the long-term R&D that you need in order to build that kind of thing um and there's a lot of reasons why like large companies don't quite work for this there's a lot of reasons why like Academia certainly doesn't work um and so I built protocol ABS to be uh an organization focused on the R&D pipeline itself to try and figure out like what is it about R&D itself that is difficult um how to make it go faster how to like do really good technology translation um so on very interesting like problem here is like when you when you think about like new new technologies and like advancing um the world and kind of like building the future uh those new technologies tend to come from uh critical scientific discoveries uh and usually people tend to think like okay great like you do some science you develop some Discovery and then magically you like have a technology and you put it into a product and you kind of like grow the product um and there's a huge gap in between those two that actually has a lot more to do with research and kind of um what B Labs used to call fundamental development and not kind of like the traditional product development that we do now and that fundamental development stage is really about kind of advancing a scientific concept like the early Theory work and do a whole bunch of De um uh research and development steps to figure out how to use that that concept that new knowledge to then produce what eventually becomes a technology that you can then put into a product and that part is by far the most expensive part in the entire system and it is tragically underfunded so for example expensive not just on time but also on actual Capital yeah on yeah huge hugely expensive in capital so um by my calculation there's kind of like a 10 to 20x difference between the cost of doing the fundamental science work to get to the conceptual results um between that step and then the step of translating the conceptual results into a technology that you can put into a product this is before a startup like this is before you have something that you can actually ready enough to to be productized independent of the viability of it yeah exactly so like making concrete like this is the difference between you know you do the conceptual research to invent the transistor now that you like have one big prototype you need to like miniaturize that you need to figure out what materials you're going to make it out of how you're going to manufacture it what applications does it have and so on until you eventually come up with like a tiny little transistor that you can then put into an actual product um and then that that's the thing that you then like build into into a product and sell and so like that entire process is ridiculously expensive and underfunded because there's no kind of like public goods funding for it really like governments send to fund the scientific research part but not that part and VC can't fund that uh it's way too early so VC's can invest in startups but they cannot invest in that earlier space funny story here like when I went around pitching PL to and you know IPS and to people um I used to kind of like have to describe like you know the business model of like creating a network and so on creating um this is like 2014 2015 um yeah 2014 and they used to kind of like reference the theum at the time and then usually the VC answer was oh ethereum is a science project like that's not going to work and um wants a world computer that'll never work yeah and like it seemed like very you know hayy and people like doing a bunch of like again fundamental development that wasn't yet productized right and so like that part like cannot be funded by VC today um it could be funded by VC fund that was like maybe you know one or two times um one or two times larger than the biggest ones today and maybe with a 30-year Outlook instead of a 15year Outlook but yeah but the interesting part here is that there's clearly that Gap you've identified and we have a an a growing public goods funding Concept in in the world of web three year and that ends up effectively starting a lot of these initiatives and you know play a pretty big role in that too yeah and that's what I'm most excited about in terms of public G funding is the ability to use pgf structures to fund that that fundamental development part that is extremely expensive and underfunded if we can like route um capital in that direction that would be that would be pretty great uh the way that I think about it with PL is like if we can kind of generate significant returns from um both investing in startups and creating new projects and new startups to then um fund that stage then we can kind of like um accelerate this entire process quite a bit uh so that's the kind of like model of like what PL is the pl is a network that enables this kind of like regen Loop through um the building of new technologies new startups and so on you already answered my last question I was like well looks like protocol laabs is no longer the company that makes fcoin and uh and that's because you're trying to do so much more here in terms of advancing actual Technologies to get to life and be produ high yeah we we sort of transitioned over the last three years we transitioned from kind of a single Le company into a network of organizations and now we have you know probably over 600 companies in the network um and there's a whole bunch of like systems and structures that support like Founders and Builders um to kind of like develop their their projects and Tech and so on and one of the things that like I'm most um kind of excited about and trying to figure out is like how do you create like the the the super compelling culture of like you know the the most exciting kind of like high-tech startups but in an open-ended environment that isn't kind of like blocked with like the big thick cell wall right like if you think of a place like um you know Google in the old days or like you know you think of like has SpaceX today or whatever um they they created an amazing environment inside that looks a little bit different than the rest of the world um but you have to go through this like really thick cell wall to like enter that environment and when you enter that environment you cannot actually collaborate with groups in another cell right and so what PLS trying to do is kind of like imagine creating that kind of an environment but without a very thick cell wall to to be able to create like a like a nutrient-rich environment to enable the building of these kinds of projects but just using the tools of public Capital like building companies building organizations um blockchain structures um communities lean into those kinds of like public Primitives um in in a very open permissionless way um instead of like forcing this kind of like thick barrier of you must join as the single Corporation uh and we you know very kind of Open Source and Crypton native oriented way we think like this could be like super super powerful in creating like a um a very generative environment amazing well why I want to thank you for this amazing chat and uh can't wait to see IPC at even a bigger scale yeah uh billions of transactions per second you quote me on that thank you so much