The Future is Modular What it Means for the Application Stack Lane Rettig

hello hello all right let's how's it going whoo here we are saturday yeah we're almost halfway there um this is the best event of the year i'm so stoked to be here um i'm lane i am a core developer at a project called space mesh and i'm really excited to be here today talking to you guys about a topic that i've been kind of deep diving on recently uh it's quite a recent development in the blockchain ecosystem and that topic is modular blockchains i know you you have heard a number of talks on this topic here the past couple days i'm going to take a slightly different perspective look at kind of high level what modularity is why it's relevant in this context and talk about implications for the application stack specifically so let's kick off uh quick outline of where we're going uh we're going to talk again about what modularity what modularity is why it matters we're going to talk about modularity in the context of software specifically we're gonna do a very quick um introduction looking at kind of uh the origins of blockchain and and how we're moving towards modularity in blockchains we're gonna talk about the four key features that blockchains give you and how modularity fits in we're going to look at the development of blockchain roadmaps like the ethereum roadmap and look at where we're going with respect to modularity we're going to discuss the benefits and disadvantages of modularity and then finally we're going to conclude by talking about implications for web3 applications as i said so i think it's good to start at the beginning what is modularity this is the best definition i saw hello ninja turtle donatello right a module east denver folks a modular system is one that gives more importance to parts than holes parts are conceived as equivalent and hence interchangeable cumulative and or recombinable and i think that if you think about how blockchains work about the different pieces of them that we're going to look at you'll recognize that this definition makes a lot of sense because we do have interchangeable components in blockchain we're moving in this direction another definition is from the lens of graph theory right so many kind of problem spaces in computer science and in the world in general can be broken down and analyzed along the lines of graphs and networks like this and i think a great definition of modularity is you have these sort of sub-sections of a graph where there's great internal kind of cohesive cohesiveness cohesion um and so there's kind of like these modular shapes that emerge so why do we need modularity in the first place right wouldn't it be cool if everything could be monolithic and just be kind of more simple well there seems to be a fundamental law of nature which dictates that in order to scale things beyond a certain point we need specialization um scale is a very interesting topic right you see it in biology you see it in architecture i'll show a couple examples of this in a moment there are limits to the [Music] how do i put it there are limits to kind of like the size that a single cohesive system can grow to right so if you think about biology you think about kind of the development of multicellular organisms about about cells and how they kind of scale into into organs and into larger organisms and into communities from there um the only way that's able to happen is by specialization right so you have different cells that specialize in different things you have different organs in a system that specialize again this seems to be a fundamental law of nature and i think blockchain is no different once you kind of start paying attention you'll notice that modularity is everywhere so when i first think of modularity the first thing that comes to my mind is something kind of like this so this is like a modular prototype of a modular phone uh google and a few other companies have released prototypes i'm not sure if any of these have actually come to market but i expect eventually they will you can kind of think about some of the benefits of having a phone that works this way right if if you care a lot about battery life maybe you're willing to sacrifice uh you know a camera or a touch screen or i don't know a gps or some other element of that device uh to snap in an extra battery uh so it allows kind of customization it also allows reuse right so if one individual component on this device fails needs to be replaced you know in in modern phones modern smartphones you can't even replace the batteries typically you have to replace the entire device it's very wasteful so another benefit of modularity is that it means that individual components can be upgraded or swapped or replaced architecture is also very modular right so as we have um developed more complex systems uh more complex um you know architectural systems so so large buildings and large structures and large skyscrapers things like that um it's not the case that these buildings are uh are monolithic there's increasingly many many pieces um that can be kind of individually fabricated and snapped together again you have this kind of uh replaceability as well uh you have customizability so you can develop these individual pieces and deploy them in a number of different ways and if you are a software developer this should sound very familiar because this is also how we build software of course so music is also modular this is an example from a totally different space um there was a very brief period of my life where i tried composing and arranging music i was really bad at it but i did notice that it's a little bit like software in the sense that you have these reusable pieces these melodies these harmonies that can kind of be you know snapped together in in creative ways or turned around or flipped upside down and of course there's a lot of patterns on a lot of math in music i spoke a moment ago to biology so of course you know the human body any complex organism is very modular as i said we have single cell organisms that developed into multicellular organisms in a more complex biology um specialization around cells and then over time into into organs and into entire systems and i think what's interesting about this is if you really zoom out from a kind of social lens you can actually see even at the level of communities of people of societies as well that there's modularity there of course even so it's kind of fractal in a way right so the organs themselves are also modular so the brain of course being the most complex organ has many many different pieces and there are of course very interesting examples from history where people experience traumatic brain damage for one reason or another and uh you know let's say they lose their ability to to create new memories right because that's a specific portion of the brain but all the rest of the brain still functions um so that's another benefit of modularity university curriculums curricula are also modular i think this is kind of a cool example um you know not everybody wants to learn the same thing and as universities have developed and evolved and become more complex over the years curricula have also become more complex but if you think about it you know you have these this idea of kind of credit units or units right and you can kind of like snap them together in various creative ways to build a curriculum that makes sense to you or that allows you to fulfill a requirement for a particular career that you want to go into um so yeah again just just re-emphasizing the fact that when you really look for it modularity is everywhere i think the best example and i alluded to this a moment ago is at the social level society right um i i if you've heard me talk before you've probably heard me talk about this idea of social scalability which was originally nixabo's idea i think it's a very interesting idea right it's the idea that if we want to scale our society from you know kind of dunbar's number and these small tribes into these big fantastically complex societies and institutions that we have today things like nation states you know kind of continent-wide governance things like this the only way to do this is to have specialization and i think again this is a great example of that universal law of nature that in order to scale things and to grow things in complexity we need specialization so we have different groups of people who specialize in different things and um it's about kind of cognitive burden right the idea is if you flew here on an airplane like many of us did you don't need to understand how that airplane works you just need to sort of trust the institutional structure that allowed that airplane to be built and serviced and operated so that's a great example of of modularity and specialization leading to modularity so let's move into talking about software because of course blockchains are software how does modularity apply in software so this is quite interesting i actually wasn't aware of this before preparing for this talk in the late 1960s early 1970s there was an event that occurred called the software crisis uh some great articles out there about it i encourage you to look this up and study it so this was kind of like 1968 to 1972. so what happened was around that time computers were getting much more complex and much more powerful than the ones that existed previously and as a result developers were trying to build and deploy much more complex applications to tackle much larger problems larger databases larger code bases things like that but the tools were not yet in place to build these systems in a manageable way so for example object oriented programming and many of the other paradigms that we're familiar with even modular programming it didn't exist at that time it emerged in the late 1960s in response to this challenge this is a great quote from dykstra many of you will be familiar with dykstra if you've studied computer science as long as there were no machines programming was no problem at all when we had a few weak computers programming became a mild problem and now that we have gigantic computers programming has become an equally gigantic problem i think if you replace the word computer with blockchain here this would make just as much sense right so when there were no blockchains we had no issue the era that we've been in the past 10 years is a few weak blockchains and programming was not particularly hard you know we have bitcoin script we have solidity we have ebm things like that but we're rapidly moving into a world of much much much more complexity in what we're trying to use blockchains to achieve and that's why we need modularity to get there so what happened in computing the first thing that happened was the hardware itself got modularized these are of course graphics cards graphic processing units um i'm old enough to remember the the days of the 386 and 486 and pentium computers when there were no graphics cards early on right all the graphics were handled by the cpu but again specialization if you want to be able to run 3d applications very compute intensive high frame rate etc not to mention of course things like um proof of work mining algorithms ai big data et cetera you need specialized hardware to get there and of course in the blockchain world we have like asics and fpgas and things like that and of course the software and the application stack also became modularized if you've spent time building web 2 applications or even even earlier than that even pre-web applications you're probably familiar with this osi model right so this is a sort of abstraction a set of abstractions that allows application developers to focus on a single piece of the puzzle from the kind of hardware electricity bits physical layer at the very bottom all the way up to the application layer at the top most of us who have built mobile applications or web applications are working primarily at the top layer but i mentioned many times this idea of a cognitive burden right these application models allow us to reduce the cognitive burden because we only need to think about one layer at a time uh and this as you can imagine you know blockchain is going in the same direction you'll see similar slides in a moment if you've written software in the past 10 years you'll be familiar with with tools like npm right of course npm is the largest package repository in the javascript ecosystem uh there are equivalents to this in every ecosystem basically in rust and in go and in c and in python et cetera et cetera and there are literally millions upon millions upon millions of packages or modules of course different programming languages different ecosystems use different terminology to refer to these right but they're published there and you can download them and just use them and you don't need to understand how they work internally they just have well-defined apis there's of course challenges associated with this with relying upon kind of untrusted code but it's it's very powerful that you can snap together all these pieces very quickly and easily and build applications orders of magnitude faster than you ever could before um as the application stack has gotten more complex what's emerged is not only individual packages and package managers but also um these frameworks right so again these are these are javascript frameworks i have up here things like angular react that i'm sure you're familiar with javascript is a very mature ecosystem much more mature than blockchain it's been around for for quite a while um but the beautiful thing about these these frameworks is that they abstract away a lot of the complexity right if you're building a front-end application you don't need to worry so much about where the data comes from so you don't come from you don't need to worry so much about passing data back and forth between client and server these frameworks make that very easy uh make presentation very easy as well the next thing that happened in sort of the web 2 world was the development of the cloud so again i'm old enough to remember building monolithic web applications in the 90s using things like cgi okay cgi script is a single monolithic piece of code that runs on a server typically a linux server and um it does everything right so it handles the data it handles the business logic it handles absolutely everything um you know over the past 20 years we've seen the emergence of not only cloud but also this idea of micro services built on top of it so yeah the the kind of web 2 infrastructure has very rapidly embraced modularity for all the reasons i talked about before you can mix and match different things it's much cheaper it's much more cost effective it's much faster to get started in developing using these tools all right so we've talked about modularity at large we've talked about modularity uh in software let's zoom in now in the final portion uh in modularity in blockchain specifically i'm going to leave this amazing coin telegraph slide up here for a sec so in the beginning there was bitcoin and ethereum and uh they are of course quite different however what they have in common is that they are both monolithic right what does that mean it means that if you want to execute a transaction in bitcoin or ethereum you don't need to touch anything else you just spin up a node or or talk to a node over rpc using something like infuria send a transaction that transaction enters the mempool is confirmed lands in a block and a message gets sent back to the front end over the rpc to let you know your transactions confirm pretty straightforward right so these blockchains do basically everything so there are four primary features or functions of a blockchain and in order to understand modularity we need to break them down so the first one is data availability the second is settlement third is security and the fourth is execution so we'll briefly talk about each of these so data availability what is this well as the name suggests it means that the data necessary to uh sort of compute the current state of the blockchain of the applications on the blockchain are stored and accessible basically via you know some some interface but it's a little bit more complicated than that the difference between a a web 2 database and a web 3 kind of blockchain that offers data availability has to do with things like cryptoeconomics right so we have these powerful cryptoeconomic tools that allow us to not only offer data but to guarantee that it's there using very strong sets of incentives cryptoeconomic incentives so things like fraud proofs punishments data availability sampling etc so data availability includes all of that extra baggage the next thing is settlement i like to think of settlement as kind of like the the court system right so it's kind of like um enforcement of contracts so to speak both in you know kind of in the smart contract sense or in the legal sense of the word again this includes things like fraud proof so um basically um this is where the the systems of incentives or the punishments and rewards are are finalized are settled in the settlement layer and of course there's a financial component to this as well the third is security so security has to do with consensus consensus mechanisms guarantee certain things like consistency and availability right so these mean that the nodes in a blockchain always converge on something right that's liveness and security means that um that no two honest actors ever um agree about different things and so security guarantees that we have coherence across all the actors in the network all the nodes and that it's very expensive to attack the network and to attack consensus and finally execution execution is the fun part that's the application layer that's where you get to build smart contracts and things on top of it and basically at the end of the day all these other things are there to serve execution right because without execution without applications what's the point of having a blockchain so let's look at how the ethereum roadmap has evolved over time so this is what the ethereum ethereum 2 roadmap used to look like a few years ago right the original idea was that there would be multiple phases and that there would be many many shards there's a beacon chain there's many shards um each of those shards has a vm state execution is what's referred to here using something like ewasm or potentially evm inside of them and they would kind of each of them look and feel like ethereum today of course unless you've been living under a rock you know that the ethereum roadmap has changed very rapidly and in particular there's been a recognition that um the how do i put it that the landscape has shifted right and um so this this comes from an article on the ethereum foundation blog which you should check out the idea is that uh east 2 uh there is no more ease too effectively right so what's being developed now is actually just a consensus layer um and there's an execution layer that sits on top of it and it's going to be merged in through this merge process that happens this year the um the key point here is that this was the initial step towards modular blockchain was this development of the ethereum roadmap over the past few years so between 2019 and 2020 is when this kind of happened there was some initial conversation in 2019 about well maybe we want to simplify this east to roadmap maybe we don't want to try to do execution maybe we want to just jump ahead and offer data availability and let people use things like uh sorry use things like rollups on top of it so this is a great post on the east magicians forum from vitalik uh from october 2020 um and again just kind of recognizing that there's a lot of stuff happening in the roll-up ecosystem you've heard about a lot of it on stage today uh and at this event um and so yeah so the ethereum roadmap has evolved in recognition of the fact that the roll-ups are going to be king going forward along with a few other uses of kind of plasma and state channels so if each one is this monolithic architecture where it offers execution settlement security data availability all in a single in a single client a single monolithic application the vision for ethereum going forward is breaking it apart into these modules these components so on the bottom you have this new consensus layer right that's going to give you data availability right the shards are going to store the actual raw transaction data the blocks and guarantee security through proof of stake right then you're going to have a layer above that settlement we talked about that that's enforcement of fraud proofs and incentives using existing evm right so this is the existing ethereum chain that's going to be merged in the eth into sorry into the beacon chain and then finally you have execution that's being outsourced to roll-ups like arbitrum um but the really cool thing is that again because this is modular all these components can can be snapped in and snapped out and rotated and changed so you don't have to use arbitrary as your execution layer you can use um any of the other um any of the other roll-ups you can use optimistic dk roll ups et cetera and on the bottom layer as well you can also use different data availability layers in the extreme case you could even imagine using something like filecoin or ipfs or who knows even like a centralized data storage layer like aws or google cloud or something that's very exciting so uh one project that is at the forefront of this is is called celestia i believe someone from the celestia project was here on stage yesterday talking about it i think it's it's very exciting right they're the ones pushing this modular blockchain hypothesis forward very aggressively and so they're positing that the future again is not going to be monolithic it's going to have this modular architecture and celestia is a data availability layer that's specialized on only data availability right so going back to this idea of scale and modularity and specialization the idea is if you're highly specialized and you just do one thing if you just do data availability you can do it much more cost effectively much faster much cheaper in a much more robust way and again you can you can then outsource execution on top of that to other blockchains to rollups to all sorts of things it's not that simple right so um this is a really fun uh chart that i found which shows various kind of permutations of what the modular blockchain future might look like so of course monolithic blockchains that's layer one execution layer one security layer one data availability that's how ethereum works today um roll ups we talked about that that's where you kind of move execution off to a layer two but there's many many many other takes on this and different types of architecture validiums volitions multi-volitions validiums with external data availability et cetera et cetera et cetera and and the the person who put this together also hypothesized the existence of things that don't actually exist yet right so multi-volition or you know this l1 with external data availability this doesn't exist but it's hypothetically possible and i expect that we'll see different projects launched to kind of fill in some of these gaps and test some of these different combinations um so it's going to be very complicated uh for the next couple years while we kind of figure all this out but the future is modular so we kind of have no choice but to get our heads around this and kind of um get on board with the direction we're going so uh it's i think it'd be interesting to talk about the benefits and disadvantages of the modular blockchain architecture model because it's mostly good but there are always some some trade-offs everything is a trade-off so um specialization scale we talked about this if you want to really scale blockchains transaction throughput etc the only way to achieve it is in a modular fashion less cognitive overload we've talked about this as well um as someone who builds blockchain full node implementations for a living i can tell you that it's an extraordinarily complicated piece of software right there there are tens or hundreds of thousands of lines of code maybe as many as a million lines of code in the extreme case um they're they're extraordinarily complicated applications and by being able to break them apart so that the developers working on them can focus on specific pieces and do really really uh deep dives and own particular pieces of that architecture the software's gonna get much more mature very quickly there's a concept in in software called separation of concerns so by separating these different concerns um the ones we talked about into modules you get these really nice features reusability swap ability upgradability parallelization right so you can do things like upgrading a smart contract or your business logic without necessarily needing to upgrade the entire blockchain or layer one right today you can't really do that um on ethereum without a hard fork and parallelization is another good example by having many rollups for example that share the same data availability layer all those rollups are actually processing transactions in parallel it makes testing easier right if you're a software developer you understand that having small clearly defined interfaces small modules makes it much much much easier to make assertions about those modules and test them in a robust way my favorite is creativity right so it opens the door to not only experimentation with kind of different languages different frameworks right in a mod in a sorry in a monolithic blockchain the entire stack is written in go or rust or something like that in a modular world again you can have different pieces that are written in different ways and um and and you can you know you'll realize that people use the modules that you create in ways you didn't expect which is how modularity works in software let's look at the disadvantages so there's a concept also in software called impedance mismatch the extreme case here is when you're passing data back and forth between two applications using a format like xml or json right so there's some cost to translating data structures into those formats back and forth rather than you know passing them around in a native kind of in-memory in-process fashion which you can do in a monolithic architecture however this is more than outweighed by the gains that we get from the other aspects of modular blockchain i think early on we'll see less stability and this should be obvious right because we have many different moving parts so there's kind of like more things that can go wrong more points of failure but i expect that over time all of these positive features will combine to produce much more stable blockchain infrastructure um redundancy things like that this is an interesting one it's a bit of a nuance it makes things like defining trust relationships and governance a little more complicated right again in the case of a modular blockchain at the end of the day there's only ethereum right in the case of eth1 and there's a single kind of governance process multi-faceted governance process that would cover data availability execution upgrades to the roadmap etc in a modular blockchain world um governance gets more complex you have to have different governance for different pieces of the system but i think this is positive long term it can also make testing harder and the reason is because from the high level kind of black box testing perspective there's more different permutations and combinations you might have to test right so while you can test individual components in a clean fashion if you want to test an entire kind of architecture top to bottom a whole stack you may have to have some very complex devops to do those tests and there's a risk that if we don't get the apis right between these interfaces we may get locked into really crappy apis so let's not do that all right in the final couple of minutes i want to speak a little bit as i promised to what this means for application developers so point number one i have three points point number one it's going to get worse before it gets better okay if you build an application today a web 3 application adapt whatever there aren't that many choices you have to make assuming you're using ethereum out of the box right you're using solidity you're using maybe hard hat you're using inferior truffle there's a relatively small set of tools and there's not that many options you use web3.js you fire off a transaction you wait till you get an event back you you know know the transaction was confirmed in a modular blockchain world this is going to get really really wacky for a while uh finality is not super well defined right you're going to have a lot of asynchronous calls uh it's you know you you can do things in a somewhat synchronous fashion in in um in the current world in the monolithic world but that will no longer be possible in a more complex modular world you're going to having things moving across different kind of infrastructure boundaries et cetera et cetera also you're going to have to think more carefully about your kind of application and data model right like you may have different classes of data in your application and you may need to figure out okay this data is consensus critical this data is not consensus critical this data can live um you know in browser storage or you know on aws or some centralized thing or something but like this other piece of data we need to sort of hone in on it and that needs to go into into a consensus data availability layer things like that so these are all positive trends but it's going to be challenging it's going to be bumpy in the beginning you saw this slide earlier these are these javascript frameworks we really desperately need these frameworks in web3 and the beautiful thing as i said in the javascript web 2 world is that these frameworks hide away all the complexity under the hood so the application developer again can just focus on building the best experience possible and they don't need to worry about um you know things like finality and async and this and that so these don't really exist they're kind of just beginning to emerge but i'm very bullish on these and if you're interested in working on these please come talk to me finally the good news point three once we get over some of these initial hiccups things are going to get much much much faster we are moving into a world uh we're moving from a world of a handful or maybe a dozen a couple dozen transactions per second you know maybe on the order of hundreds or thousands in the case of uh blockchains like solana into a future where we're gonna have hundreds of thousands or millions of transactions per second right and that opens up entire application use cases that were never possible before gaming is a great example so i really believe that we're moving into a future thanks to the modular blockchain architecture where applications are going to look and feel much more familiar and much more like the kind of applications we're used to before that is all i've got so these are three resources i want to leave you with these are the places i learned about modular blockchain so starkware many of you probably know this project they have a lot of great resources out there on on this idea they're a big rollup platform there's a blog polynia.medium.com really really excellent work there lots of amazing links you can spend hours going down that rabbit hole check it out and i mentioned the project so last year they also have a lot of great documentation on their blog that's it my twitter is already my dms are open if this stuff is interesting hit me up i'd love to chat and let's roll up our sleeves and get back to work and start building the tools we need thank you [Applause]