Stateless Ethereum: Paving the Way for the Migration into ETH 2.0 – Piper Mirriam

how's everybody today Oh anybody in the back of the room want to come hear something interesting now yes something cool I am here today to talk to you guys about stateless aetherium um this is the most exciting thing maybe that I don't know everything I work on it's exciting but like this is on par with maybe eath too exciting so listen I'm Piper Miriam I work for the etherium foundation I run a team of snake charmers also known as Python engineers we make open-source software Trinity the etherium client is our kind of primary focus but we do a bunch of stuff cool so this is the roadmap for this talk I'm going to give you just like the high-level who what when why how this list so what do I mean when I say stateless etherium what is this we're gonna drop down to kind of like a simple concept of what is the etherium state so the etherium state is this every account or address or contracts depending on the word you want to use has an entry in a database that looks roughly like this you've got bet your balance and if you're a contract you have code and maybe some storage plus to store your token balances things like that we take all of these accounts we put them into a try this is what we called the etherium state it has this fancy thing called a state root at the top which is how we refer to it and this is the etherium state so when I say the state this is what I am talking about in the blockchain we also have blocks blocks are made up of transactions and transactions do stuff to the state so here alice is sending a transaction to transfer one ether to Bob this modifies the state so we've got Alice's account here it gets the balance that's deducted there Bob's account over there the balance gets added so so yes this is what I'm talking about when I say the state okay in stateless lis theorem we have this concept that we call a witness the word doesn't really matter that much but we just refer to this concept of a witness as the pieces of the state that you need to execute a transaction or maybe even a whole block so a witness is just a collection of pieces of this state it's like a subset of it so when I talk about stateful aetherium clients and stateless aetherium clients this is what I'm talking about a stateful client has a database of all of the state and so when it executes a block and it needs to say read Alice's balance and write Bob's balance or whatever it just uses its local state database to do this a stateless client gets that from somewhere else and typically it is a witness a stateless client will generally not have all of the state it could have some of the state but all of those little semantics don't matter that much this is the high-level concept and difference between statefulness and statelessness currently all aetherium clients are stateful so when I say stateless aetherium what am I talking about this is a set of protocol changes that a bunch of us are working on to really strongly support the idea of this kind of stateless paradigm in the etherium network you can't really do stateless well today we did actually prove that you can do it today on the network but it is but but it can't be done well and it can't be done very efficiently and it is very much attackable in its current state so stateless aetherium is the effort going on right now to do a bunch of protocol changes that help us support this model so why are we doing this so the storage model in E theorem is this you pay once when you pit when you run your transaction you pay some gas to write to the state and then you never have to pay again and that data just stays there forever it's a really bad business model it we call this unbounded state growth you've probably heard this talked about if you're deep in any of the conversation but there are no current economic incentives to limit or restrict the upper bounds of the state size which means that it'll just keep growing there's arguments to be made about like is hardware going to outpace the growth of the thing and whatever but every time the gas limit goes up this problem gets a little worse or if we actually solve the scaling thing right then this gets a lot worse because the more usage the more state writes so this is a problem that has been known for a very long time and that now we're finally actually doing something about it um to really like drive home why this is a problem the state gets bigger that means that you need a bigger hardware to run nodes that means that it is harder to run a node which means fewer people do it which means we end up depending on centralized providers more like in fira or things like that and while i love in fira I would love for us to not need in fira and in and in the current model it'll just trend towards centralized providers and that's like against our ethos or I don't know we get pitched for itself which is maybe what was going to happen a year and a half ago so alexey is one of the people working on me on this working with me on this and state rent was like the first stab at let's try to fix this this started at DEFCON two years ago and the idea was well let's charge rent for this storage and ultimately this just never ended up going anywhere i have my own opinions about ten of why i think it probably would have been a contentious hard fork this would have been a backwards incompatible change it would have really changed a fundamental kind of paradigm of smart contract development it would have broken existing contracts on the network in an unrepairable way all of these things are you know probably or something but it was like really I don't know medicine and like the 80s tasted bad my kids love it today it's like candy and they're like SiC give me medicine but like state rent was like bad tasting medicine that that while that would have made the chain better it would have been really hard to swallow so it ended up not going anywhere and kind of fizzling um so but all of this is sort of just talking about the problem statement I will tell you why stateless aetherium and why it's better but I'm just gonna gloss over it for a second here but the general gist is that stateless aetherium addresses this problem and also does a bunch of other really cool things that I'll get to so who's working on this here's an incomplete list because I didn't finish making this slide but the gist that I'm trying to get across I'm glad or no name-dropping here maybe there are a lot of really talented core people who are working on this full-time or near full time this is not something that's kind of like struggling along this is something that has been picking up steam for a few months and like we are running at full speed now and we have a bunch of really bright smart people from all over the protocol working on this which is important because this is going to touch large parts of the protocol um the how so like what are we doing to fix this um I am NOT going to tell you today because it's very complicated but what I will tell you is that it touches a lot of stuff and I'm gonna go run through a couple of those real quick it's looking a lot like we're going to transition from a hex or a tree to a binary tree for the storage model this is to reduce witness sizes it looks like we're gonna make changes to the account model to actually Merkel eyes contract code so that you only have to pay for the parts that you use rather than having to ship the whole thing around that's gonna need us to actually change the model by which we store accounts in a certain way which means touching every account in the state tree we are gonna have to make protocol changes themselves to actually just support these things at the networking level and we're gonna have to make changes to like our assumptions about the networking because this is going to change the network topology itself we're gonna have to make EDM changes to adjust of the gas schedules to account for witness size in gas expenditures and things like that so this is a very like multi-discipline effort that's going on that's going to touch large parts of these things and if all of these things sound low-level and kind of like out of your view and things you don't care about you're right but the effect that comes from this stuff has really positive things on probably everybody in this community and I will get to those momentarily so when I don't think this is overly optimistic um we have been having regular calls we are getting together for our first like major in-person summits coming up here in March and like our roadmap and and you know we have these lists right things we know things we think we know things we know we don't know and then this like list that we can't find if the things we don't know we don't know and everything is really steadily moving towards the like we know this now we we we have a much more concrete roadmap than we did at the start we have a list of changes that we know we need to make and we are generally getting to the point where we know how to make all of those changes so we probably have six more months of rnd twelve more months after that of roll out six more months of you know just standard delays but I think it is reasonable to expect us to really fully deliver on this within two years but this is not one of those things where you wait a full two years and then you get all of the good stuff our roadmap is specifically designed to deliver value all the way along the way so there will be rollouts and features that will be rolling out within the next few months even under this kind of six-month schedule that start adding these increments incremental improvements okay so the why the main impetus for doing this is stay growth you've got this you can think of it as a spectrum of statelessness it's really not a like binary choice you can shove yourself right in the middle and have half of the state if that's what you want to do but the thing is that as you slide that slider along the kind of spit statefulness spectra so do the hardware requirements for your node change so a full node has kind of the highest hardware requirements and as you slide that slider all the way to the stateless side your hardware requirements really drop off you you don't really need to manage the database of the state as much anymore and we are doing other things to also just reduce the requirements on hardware and the goal and we'll get to the goals here but this is the primary thing that without this we probably wouldn't be doing this effort another really powerful thing is this gives us a path to come into the 'if to world eath to while the execution model is still under heavy research is going to be a primarily a stateless protocol and without getting eath one ins a kind of this paradigm the transition would be a lot harder this gives us a much clearer way to migrate the eath one chain into the e to world and that is a pretty big deal cool this is one of my own like kind of personal missions which is improving client diversity pretty much the entire history of the etherium chain has been the gaff and parody show we have not really seen other than maybe very recently any new clients really show up and be part of the network and that is because it is really really difficult to write a client I've been working on writing one for a couple of years now and it is hard they are complicated pieces of software you need deep understandings of pretty much the entire part of the protocol stack to be able to to write a client and even then it is actually just really hardware intensive to make a client work death is full of very like lots and lots of optimizations to make it work as well as it does and it takes years to build to build that technology up and then on the other side of this we want to make it easier to actually participate in the network um I don't run a here iam not-it's too hard and annoying most of the time if I turn it off then a couple days later when I actually want to use it then I have to like wait a while for it to sink again and and the UX is hard and and it's a heavy piece of software to run and it eats resources on my machine and I imagine that there's a lot of common like feelings and other developers or just people in general um so we want to make it easier to run a client there's like something in the six five to seven thousand nodes on the main net today and I want that to be in the like hundreds of thousands or millions range within the next few years and the only way that we're going to get there is by making it easier to run a client and the other part is we need different kinds of clients right now there is only one type of client the full node right and it doesn't necessarily fit everybody this woman on the left is how I felt about conference t-shirts for a while I like extra-large right I want clients that are specific for tailored like use cases app specific clients and the ability to only track the state for the stuff you care about because in the stateless paradigm you can do that something about Nimbus which is the name of my dog but also a client um yeah so in the stateless paradigm you can have clients that say only track the addresses that you use or only track the contract state that you use or even just do like basic caching and just cache some of the state for the last you know 2000 blocks or the last week but the nice part about stateless clients is that it gives you this choice um and it also opens up a gate for a scalable Elliott Lite client infrastructure right now we have the LS protocol but it doesn't actually work that well conceptually it can work quite well but what really ends up happening is that it's hard to find light servers and even when you do they're often being crushed by all of the other people who are leaning on them too do le s protocol things the stateless paradigm in the the way the network works in the stateless paradigm kind of naturally supports a scalable like client style infrastructure and I'm really excited about that there are a bunch of other really nice improvements to clients the one primary one that I am not going to go deep into is just the UX for a client today you download a client you turn it on and then you go wait until tomorrow while that runs for a while and syncs with the network and then tomorrow hopefully it's done and then you can use it or you had it synced yesterday but it's been offline for a week and you're going to have to wait up to like maybe an hour or hours before it actually catches back up with the network in the stateless paradigm you are operating as a full node and have full node functionality at the moment you have your first peer connection which means that in the stateless world you can pop on and off of the network and every time you pop on as soon as you have a connection to another peer in the network you are essentially working as a full client with full client functionality and that is a powerful UX change it means that those of that if you want to use meta mask against your own node you can spin it up real quick and then have meta mask connect rather than needing to kind of maintain this infrastructure yourself cool this is the call to action slide there is a forum where we are talking about some of this stuff but right now most of our conversations happen in a telegram Channel we are working on migrating away from that it works very well for us but it isn't going to scale well so we're working on talking about this stuff more publicly and making it lower barrier entry for people to join we have calls on monthly basis they are advertised here it you should just come lurk in a call if you want to kind of hear the latest of what's going on you can come talk to me after this I love talking about this stuff but this last one is is really the most important one so I do this as a job I get to make open source software full-time and get paid for it and that's really cool and I got here by just showing up and starting doing things there's this thing called impostor syndrome that a lot of us have heard about where we like feel like we aren't very good at stuff I'm pretty good at what I do but I also regularly feel like I don't like why do I get to do this and and other people don't and maybe the reason is that you didn't just show up and start trying I don't know I also got lucky and I've been in the right places at the right times but ultimately what I want to point out is with a little anecdote is that a couple of years ago when Donald Trump got elected my wife got very angry at me for not being angry enough I just don't really pay attention to the world and what we eventually figured out was that she along with I think a lot of other people had this need to act to do something to fix the world to like address this huge problem that just got elected and I didn't feel that mean because I was already doing the biggest thing that I could do to make a make some sort of difference in the world like I was already like my this is the biggest lever I have so if you want to do this full-time you you can you can join a product company and have a like 99% chance that you'll fail or you could just come show up and do this open-source thing with us show your work and we will find more things for you to do and we will find ways to get you paid to do them but show up and show your work and like come join us and make this thing happen I think that's it there's Phin yep and questions anybody want to know what I had for breakfast I live here without sinking so the idea is that as a client executes seblak and it finds that it needs to read something from the state typically that means reaching out into a database in the stateless model we use this concept of a witness which is a package if every one of those things that you would need to touch in order to do the execution you could think of it as a proof but it really just is it is both a proof that the data that you're looking at is part of the the state tree so it's it's provable data but it also is just the data that you need to execute the block and so rather than using a database you use the witness as the database no because the idea is that you acquire that witness from the network on kind of on demand at the time that you receive the block as well so you didn't have it in fact if the witness turns out to be wrong you kind of just have to throw your hands up in the air and say like I don't know what to do next because I don't have the data that I need to execute this block and those are some of the challenges that we have to work on to make this work well hi so I know that with rent being dropped the idea of an in protocol mechanism to pay for the witness data also went away these are any thought being given to an API around being able to like buy witnesses from other nodes and kind of get again at sensitisation schemed of running a full node so so the question is what are we thinking about in terms of making sure the witness data is available and then since its schemes for running full nodes so one of the things that we've discussed is that we are reasonably confident that there will always be a cannot strong economic incentives for people to run full nodes miners have economic incentives to have all of the state because not having it means getting paid less centralized providers like in fira have strong incentives to have all of the state because it's part of their business model and companies like exchanges and things like that who need kind of broad access to the state also have their own financial incentives in their business to run full nodes so we're not too concerned with with with full nodes disappearing from the network what we are concerned about is all of these other nodes crushing them with all of their on-demand state requests and this is an ongoing piece of research but we have ideas and I will be happy to talk about them over there can you explain why a binary tree would be reducing the size of the witnesses yes the question is why does the binary tree reduce the size of the witnesses so we use a hex or a tree right now which means that at each level there are 6 up to 16 things on that level of the tree so and then each of those branch down and there's 16 more things so as you provide that witness which is kind of like both the data and the proof that the data that you're giving them is part of the state root you have to include all of those sibling nodes as well which means which means you're walking down the tree and and you you have the kind of like singular path but at each step there's up to 16 other things you have to include with that in order to prove that it for it to be a provable piece of data whereas in a binary tree you just have the one sibling so it essentially just cuts down on all of this intermediate tree data that is part of the proof and it makes the proofs much smaller oh sorry I started staring at the wrong person yes there are mechanisms to eliminate those intermediate layers but to save it like this is already bit this is probably the most researched part of our roadmap and we've demonstrated objectively that the the theoretical savings for binary trees are something like 4 X and the real savings that we end up seeing or something in the like 2 to 3 X size reduction those are numbers pulled out of this soup up here so it's the idea that the witness data is provided by the person who submits the transaction or by the miner no it actually is really problematic to try to have transaction submitters provide witnesses upfront because if the witness needs can change right depending on transaction ordering you can end up touching different pieces of the state we have dynamic jumps and things like that that could just change the execution entirely so the idea is that if you execute a block then you have already actually done all the work to build a witness because you've touched it all in your database so what we're looking at doing is a couple of things one is probably putting a hash of the witness in the header that makes it just compulsory that you build a witness rather than an optional and then yes the idea is that those full nodes they'll like center of the network is maybe one way to think about it are the ones that generate witnesses and they they propagate out with the blocks through the network cool if nobody else has questions I will happily talk to any of you that want to ramble about this somewhere towards the back of the room thank you [Applause] [Music] you