Introduction to Yul+ by Nick Dodson

in etherium far too long I've aged significantly and it has been a wild journey and yeah I've been using solidity I think since it started or since the very very early early earliest versions so it's been a very wild roller coaster of an experience and and so in lots of different apps lots different stuff happens but but effectively after a little while you know getting deeper and deeper into solidity and getting deeper and deeper into how the VM actually works you know you want to go you want to go a little deeper and a little further and thanks to you know Chris and Alex you know you'll has been an interesting pathway so yet today I'm gonna present you know my extension to Yule Yule plus you call it a new language but it's effectively he's mostly you'll but with some extras that's the way you can think about it so for the agenda we're just gonna do a little overview of you'll you know what what I really like as a damn about you'll and and what it is and as well the motivations for why there might why there's justification for maybe an extension to you will you know objectives for this new language a feature overview roadmap and you know where we might want to take you'll you'll plus and and then lastly I have a special reveal of a project so you guys are gonna see well kind of the reason why I want all these things and the reason this sort of the result of doing it so yeah that's sort of sort of the agenda so yeah with you'll you'll is just a an intermediate language it helps Chris and Alex and the rest of the solidity team sort of builds and and act as a nice sort of intermediate layer between assembly and some kind of reasonable language and the things the things that really stand out to me about you Liz just simplistic grammar there really isn't a large grammar there it's functional notation its low level it manages all the stack work for you and if you've ever tried I don't know computer science class or something managing stack it's it's a goddamn nightmare so it's great it's easier to audit and formally verify in primarily at the assembly levels so so when I want to take a build that I've done and have people who do you know form of application work with it I find the produced assembly is much easier to you know build proofs out of then then solidity so I think this is a big this is a big minute benefit and I know there's tools that are on the way and that are you know constructed per per solidity and form verification but with you'll we we can really do it right at the assembly level and just the produce bill this is really clean so the compiler is simplistic I love that I can open up the solidity compiler and look at the you'll section and actually understand exactly how it's breaking everything down and how it's going to how it's going to ship up to to EVM code I think that that's a huge benefit for the community in the language to be able to have this this really simple step solidity is obviously got a ton of features and it's working with a lot of different concepts and this is this of course makes the compiler complexity increases the compiler complexity significantly whereas with with you'll you know you're really doing more simplistic operations and this is this is - the the yule optimizer which adds a bit more complexity but of course you can just turn that off and then it produces is really efficient build so you know the because it's just managing stack and you're effectively still using step right up assembly you get these really tiny clean builds of contracts and I and I really do like that of course you're not going to get the safety you want the language and compiler level safety that you get from languages like solidity but you do get he significantly increased build efficiency so the deployment cost is very small shared memory and function macros so you can actually using you'll and I'll show cases of that but one one big aspect is just sharing the same memory across many different functions and I know solidity can probably and eventually do this using some some macro terminology or if it if it already does I don't know I haven't worked with solidity recently but but effectively this is one of the big reasons and you'll see why in a second and as well just more precise that control and management so just being able to more front in a fine-grain way build applications and low-level code and get a better idea as you build you know how the stack is actually going to be managed because you know the you'll compile is actually very dumb it's not doing anything special so so if you're if you're loading up a huge functional statement then you know you're gonna be pushing the stack you further and that's basically you know the the precision you get with you'll so some motivations for the extension tool so so I love you Ava nila I the second I saw it and the second I started to work with it I immediately loved it and given that I've spent a long time with solidity I felt like as a developer was sort of reasonable for me to be able to extend my capacity and go further down closer to the the EVM metal you for experiments and for for developments and everything else and and I loved you all for various purposes and I built the fuel optimistic roll-up which is when I'm currently building most of the contract that 80% of it is written in Yule so it's one function written in Yule and then there's the few solidity functions and in our second build which we'll be releasing this quarter but the entire contracts built with you'll Plus effectively you will but you will plus so the lessons I've learned from that I've kind of taken into this and said okay well there's definitely a few things I would love you know in Ewald but it may not make sense to have annual so first thing is is management so building large code bases with Yule becomes very verbose repetitive obviously because it's low level and it's going to not do a lot for you so it comes with a lot of mental management as working with you know memory on a low level would always come with there's not a lot of sugar so there's many instances where you just wish there was a little bit of syntactic sugar and and adding it would not be significantly less efficient so it's not like we're adding on these really complex statements that you know have big concepts around them or anything like that they'd be simplistic concepts and I just add significant notational value without you know increasing build complexity as well a I think you know you you can add sugar to something like Yule because it has such a simplistic sort of base grammar but that doesn't mean that it should be an Yule because the reason why I think I love it and why other people will love you will for certain cases is going to be because of its simplistic grammar and that there isn't all these new concepts or expressions or anything on top of it it's very simplistic so this becomes kind of a motivation to say hey maybe there should be a little sugar or extension on top of this that's going to pepper in little things that just make building apps with this easier and I will say to that I don't think it was the expressed motivation the you'll would be used for application development or for things like scaling and in roll ups but it's really fits the bill in terms of what I'm looking for as a more lower-level developer with with aetherium and for the expressed reasons above so so of course the design intention was not to put something really developer facing in front of everyone with Yul was more you know how can this be a nice intermediate step between assembly and the development of solidity and or other languages so that's that's just to note that so as for safety Yul comes with little safety nets of course it's just it's effectively a simplistic compiler over over the assembly over assembly so and then lastly extensibility so making you know an open-source layer on top of you'll that allows for a lot of custom language variants and plugins so this you'll see is kind of something we're looking looking at and and it's something that I will we're already kind of using with with d-type which is you know a an interesting new typing system so these are the sort of motivations and as well for for fuel labs and you know the optimistic roll up a game it's it's primarily about gas sensitivity and you'll you'll see why in a second and then as well it's it's better for roll up contracts because roll up contracts require a significant amount of you know low-level gas sensitive work in terms of you know breaking down proofs a lot of cases where memory would be shared and where if you're copying fresh memory every time that's going to really significantly add up so so you want to be able to have a deeper level of control over what's happening in execution and then is lastly I'll show it's fantastic for building stateless contracts which are sort of I don't think a concept that people are really working yet because we're not really dealing with with problems of scale just yet but as you'll see stateless contracts are what we believe is the future of decentralized app development and you know the future of well everything so so you know pretty pretty important okay so objectives for the new language and we're almost through my dense point slides so just hang in there if you're if you're you're getting bored so objectives for the new language so philosophy don't do too much value just add sugar where you need it don't you know where it's only going to add maximal value don't don't add all this fun sugar for nothing you will first so produce valid you'll build grammar keep the grammar as close to you as possible so that we're not doing too much once again be extensible so allow others to add in different layers on top you know I'm not gonna know every every reason or feature that people want but I do know that javascript is a really easy system to add and remove things so there's a really easy language for others to contribute and you socially contribute so you know it comes a lot of interesting potential experimentation and then as well influence so if a lot of developers are using a certain feature and you'll plus we could see maybe can that be carried over into Yule or solidity in some format so just being able to influence other other languages so the features of you a plus after that huge rants are as follows so just all existing you'll features enums constants and a theory maybe i generation the hash generation safe math so you can basically get safe math injected for free and you can turn that off as well any of these things you'll be able to turn off in the production compiler injected methods so em slice and require and memory structures which were the more sophisticated addition and we'll be expanded to support a bi encoding and a few other different kinds of structures d-type structures etc so as I said before you will plus just a layer on the the you'll you know sole compiler so we've written you'll plus and nearly which is actually there which is a nice sort of grammar parser so the grammar is constructed in this grammar language which is sort of a hybrid between JavaScript and their own language and then we break down the yule code that way add a layer of features on top that gets compiled to you'll and that's it we're really not doing anything more than that so to cover some of the features so here's what imams look like we've kind of kept most of the notation similar to you will so you just enum colors etc SMB zero one two and is well constants so just adding the Const specifier so this does two things one in with constants it it will if it's a literal ie a number hex value something like that it will just do injection across the build if it's not a literal so it needs to be run in execution then that is going to be set as a LED statement but we'll look out for other things trying to reassign and and so that's this the current way we're handling it I'm sure there's there's more that can be done there but that's effectively the the basic idea ABI signature generation and topic generation so still all vanilla you'll but you can do fun things like write sig and then a effectively a solidity function which is how it's getting parsed this is thanks to ethers J s is a solidity function to a bi parser so you can effectively write what you know you believe that function is in vanilla solidity it's sort of a header and then that's going to get compiled down to its its signature and do all the checking etc same thing for topics and in topic captions so it'll do the topic hash generation as well so this just makes handling readability all that sort of stuff it just cleans up a lot of a lot of jargon that you're gonna get if you just tried to do it with hashes because you'd really have to pre specify like this hash is this thing so these are nice little generations and that's just going to do a literal injection there which is quite nice so save math by default so if you're just using add subdivide etc it's going to basically change that method name in execution and inject the save map methods above so it'll switch ad over to save that and then you know then you're running that however you can turn it off and you can use features like unsafe add so you're just specifying and they'll in the literal sense hey this is I want unsafe here Sookie Beyonce and of course you can just turn it off that right if you if you don't want it it's really flexible on on the basic things like safe math and overflow and underflow checking and you know I with this addition I wanted to be as hands-off as possible I don't want to be maintaining this magnificent addition to you all I just prefer that it gives us the flexibility to have what we need cray low-level builds and then move on with their lives so some methods getting injected and another feature so M slice require so all M slices it's just is effectively just shifting stripping right by certain amount of bits so you know here I'm just taking the first four bytes signature just grabbing it from the call data that's copied to memory so doing little things like that M slice is great for it's just I use it so much in my builds that just had to go into injection however these methods can be changed in the compiler settings the injection settings and you can turn them off and you'll be able to basically specify like okay or you'll be able to go hey I don't use M slice at all so it'll never get injected so have many different options with what's being added onto the yule language gear and require as well so you'll be able to specify like hey this statement whether it's true or not and then you can specify an error message and we've chosen a simple error system here we're in the air basically you could put al it's just going to hash that string take the first four bytes and then use the first four bytes in an injected literal in the required statements so that's that's all it's doing and then it at compile it will give you it will give you all the errors in there hashes and it'll give you all the signatures and their hashes so you still get all the sort of luxuries that you would get from you know a solidity like compiler environment using sig and error and these sort of injected systems but they just come in little clean packets at compile time of course we can standardize notation there make sure that you know all the builds are up to suspect and they produce nice nice arrays and everything else of all the api's but but even right now just with this basic injection we've been able to do quite a bit and then lastly the most sophisticated thing we've done and this is going to change over time right now you'll in the later versions of solidity compiler support types and their basic types but there's still types so right now you'll is just all it's typed but it's all at least in the builds that we're producing it's it's all the same type so it's once only one type but of course as we go along we're going to add in that typing and we're going to add in as well some custom typing with the enums M structs and you'll be able to do some basic custom typing so I think that'll be a really nice addition it's really not that hard it's just haven't gotten the time yet to switch it so right now this M struct notation is sort of in an on type setting but it will be in a type setting after so so basically what we're doing here is we're just specifying and describing an existing structure and member or a structure that may come into existence in memory so right here we have some complex data maybe that complex data has a signature it's four bytes maybe it's got a value that's 32 bytes maybe it's got on a rail egg structure with the dynamic length and in an array where you have 10 bytes for each item so this this allows you to describe a lot of what structures would be popping up in Abyei however this doesn't have offsets yet where you can specify dynamic offsets for each property but that will come as well so you'll be able to get a nice sort of recursive offset 'add memory structure description and I think that that's going to add a significant amount of value to you'll plus so so right now it's it's more just keep it basic and just see how this can be used and this is what we're gonna build a lot of our roll-up with but in the future you're going to have full fledged ABI and not only that you'll be able to specify your own ABI encodings and you'll be able to specify on typing as well so these would turn into you know instead of 4 + 32 + 32 + 10 it would go to something like you know like a bytes for or it would go to like a you you 32 in YouTube to d6 etc so that's really you know where the typing is going to come in and then here we have the usage so the usage is really cool it's just the structure name so complex data dot so the dot notation is already supported and you'll so we're not doing too much here from a sugar perspective and then signature so there and then we're just putting in the start of where complex data as a structure begins so this makes managing memory at a low level just really really really easy you you you just laying out the structure and saying hey here's the structure and then you're just picking it apart and and it comes with all these sort of generated functions so you can say give me the position of this particular value you could say give me array should be are there you can say hey give me this item within this array at this position you can say give me the array length you could say hash this particular value or hash the entire complex structure it'll just do all the positioning for you and it'll inject those functions so it's really just a function generator and what we want is a system where you can tune that generator the way you you want it because we can't expect everyone to you know use all of the same formatting however ABI encoding at present does most of what we need unfortunately when you get into roll ups the ABI encoding falls short because it's not tight packed so this is something we can go over in the in the discussion but that's effectively what's up there so roadmap and future or language exploration and and by the way what am i doing for time my my ranting too much and no you could you have five minutes to go and then anyways go into the your discussion so see ok so I'll get through this as quick as I can so so basically we're gonna add this ABI in string specifier so this will actually generate an instructive on the ABI you specify and then you'll be able to you know then use those structures and existing memory which is really cool so this is great for call data in parse and call data and doing all that sort of work as well you'll be able to it's extensible to be able to plug in your own typing system like D type the objects will be extensible so we'll introduce some basic extensibility it'll just be copy and paste extensibility so they'll just follow the Express ordering and and just inject it there'll be tiny packages so we'll do a tiny little package manager and it might support as well the the EPM package manager because I know it it can handle this sort of thing better CLI supports and low-level customized version of remix so remix is great and and I and I like the way it's configured but I think yet we could have yet another lower level oriented version that I think expresses what's changing and execution over time with better gas metering and so forth so and then the big thing as well types types basic custom types and memory pointer types which is what we're going to discuss next so that's the roadmap so now for a special reveal of some stuff that we're working on here at fuel labs and we think will be a game-changer and I'll try to present it in four minutes so here we go so using our you'll plus tech and our low-level code we were able to roll up ENS so we now have a version of an ENS registrar and Nick Johnson knows all about this and we'll be releasing this soon and basically it costs cuts the cost of ENS registration it currently in half but that will be even more as we get it more efficient and as we kind of go further it's a system that is got a lot of features and and this this is sort of the usage of why you'll and you'll Plus these lower level languages is so important because as aetherium scales as we go to 2.0 as we go to a world where we're in a more stateless environment using a lot less state rights all of this becomes very very critical you won't be able to use state the same way so you're gonna have to get creative so so sparse ENS so what is it stateless verifiable and cheap ENS registrations so basically we've knocked the cost of registering a name down at good gas prices to about 0.001 cents per name so 10 10k gas and right now we're sitting at about 45 k k gas per game per name so this is you know pretty significant savings and name registrations can be back like I really love like getting face to face time if you can call this that but yeah I don't think it's like the end world--if oh sorry sorry whoever is talking usual time right now okay Nick I muted everybody and now you have to unmute yourself again there we go great and it should work great uh okay can you guys see yes okay great still got it okay great so so the way this works is using a sparse morkul tree and some kind of encoding this is uh well you can get sixteen characters in with compression but you lose certain names we do that it's the IP 712 compliant and it stayed optimized so the entire thing by the entire contract it uses one state mod so you're literally registering potentially millions of combinations of names and we're only doing one state right on etherium so we believe this kind of design is the way that a theorem apps and EVM app should work we shouldn't be using of millions of state writes on a theorem it's it's not sustainable and the way that the client manages state as we've seen is a huge bottleneck because of random access state memory so this sort of state optimize design where you can have millions of ENS names being registered under a single state law is sort of the way we think it should go so here you can see we have our 1s store that we use for the root and then you're deriving the names and a name is broken down by an encoding scheme to a specific to specific number which is going to be your number of where this isn't included in the sparse Merkle tree and then that's going to resolve to your name data so this simple use of a sparse Merkle tree to increase the efficiency of things like ens will present you know massive value at I think as we move closer to scale and just making things a lot cheaper I mean instead of paying something like one cent per name or things like that on a bad gas that you're paying a lot less for that and you know for people like wallet developers and stuff this is a pretty big deal so you know I can go into this a little further but effectively looking at some of the sparse ENS con all this will be open-source soon under Apache - and you'll be able to really check it out but here I'm using the enums you can see I'm using em slice you can see I'm using the cig donator you can see I'm using that em struct formation specifier so I'm really all these things that we're building into language we're really using all the time here's more em struct usage you know taking in the registration and I'm doing the batch work and as well adding in safe math where necessary adding in these nice little multi n store bits of sugar and then doing the require of statements and so forth so instead of doing M store 0 M store 0 + 32 M store 0 or whatever you know you know how M store it's going to function so you can just do some nice quick sugar there and it just makes the notation so much easier we know it's going to do so considerations so I'll just try to wrap this up so basically as we move into what we believe is a new era contract development which is stateless contract development or using one to two state rights for an entire system so imagine you imagine a Dex using only one to two state rights something like that you know we basically you know we believe that scalability is going to be key designing for scalability is going to be key reducing s stores reducing X loads and in getting to a point where the execution is very clean it doesn't cost a lot and it can last a long time and be formally verified so yeah stateless contracts whoo and yeah our sparse DNS system should go up in about a week or so so you can get a demo and you can try it out on on robson Gourley and you know I think that's really to set the bar and and the reason why we're doing things like ule plus and kind of focusing on that stuff so if you want to use you'll you'll plus just save install you'll PMPM there's a there's a you plus plug-in which is awesome thanks to the d-type team and you'll plus IDE which will build a bench into a lower level system github discussion etc all the links are there I can post these slides somewhere so I went a little over I went three minutes over but yeah that's presentation thank you no voice and thank you so much especially for the info on the enesta you can stop sharing your screen now if you like and other than that are there any questions for Nick either and you can post your stats by the way on our solidity Geetha because we're using the guitar to shed during the conference so let's check in this room first any questions for Nick on your class for the newbies here you can use the raise your hand feature to raise your hand so that I can see that you want you to say something and that's also look so there was a discussion going on in the guitar channel about basically the problem that like when your class is a nice project I'm a bit worried about promoting developers to write low level code some of the civilities mitigations are going to be missing and people will make mistakes and my honest opinion this should come with huge warnings like what happens if someone is unlucky and writes to functions that generate the same function ID with solidity he is going to be safe but with you I suppose there will be no warning there was already a little discussion on that but I wanted to give you the chance to also comment yeah yeah so so I think safety is a is a big reason why solidity needs to exist and the development safety that comes with having a compiler focused on new developers coming to a theorem and new developers coming to to the ecosystem and I think that that's critical but the thing is as you start to build these more complex builds and as you start doing you know development that requires a lot of sensitivity to to gas and opcodes and you know exactly what you're doing yes it has it takes expert or will consider to be I think expert knowledge to do there's not a lot of places to go to and there's not a lot of places to find languages that will support you you know we had Lisp for a while and things like that which offered some you know language like options for working with assembly but you'll really station itself nicely as an intermediate ground and then you'll plus is a sort of addition to that that makes it more human to use so I wouldn't say this is something to be promoted for general developers I wouldn't say that at all well we can add a lot of the same safety protections into the yule plus compiler just just because humans are humans and even I make a ton of mistakes as I'm building I think this is more for people who want to go a little deeper they want to be a little more sensitive they they might really have a deeper knowledge of the EVM and they just need something with more precision but they also don't want to be dealing with the baggage of working with just you'll that's more designed to build languages at least at least it was intended by defaults to be that so I think that's my justification for this and why I think it's important and as well as aetherium devs moved to more stateless out designs they will need the sensitivity and the precision and that's where I think you'll Plus will really shine so we'll help to get enough safety in there for for even more general-purpose people to to use it our general-purpose builds but right now it is really more specifically for scaling stateless apps and sort of deeper EBM development so that's my rant I think you're wrong but I think you're right yeah okay we're running a bit over time but I've seen address we might want to comment on this again okay thanks since for the presentation yeah my point is that like you a new Prius like that's a great project and when you're going to use it that you know what you're doing and like from your expectation like only Pervis some expenses are going to use it but my point is that you should kind of also when you are promoting and presenting you work a new trees had some huge warning just to be sure that like people that are going to look at you and you'll please most understand that they need like some level of expertise to use this language yeah I completely agree I know that some people are going to you know like with your blog post or like with some provocation and see oh your place is great I'm going to read you like the ghost ghost and everything and there might not understand that they need like this level of expertise so it might be just a a notion of having more like whining or having more like promoting needs with like this huge red flag you need to know what you're doing when you use price does make sense yeah I completely agree I think I think we can slap massive warnings on everything from a developer perspective so that they know that that's the case yeah and even even you'll - in many areas says the same thing so because it's not who really is for more experienced people I mean and I wouldn't even and just to wrap this up you'd like you know when you're starting out even with coding you you would recommend like a Python you wouldn't give them a C++ right away and say like hey you know this is great this does like everything you could ever want use this you know so I completely agree yep