New Features of Solidity 0.5.0 - Christian Reitwiessner

okay wonderful yeah thanks for coming on this day actually I expected perhaps two or three people to turn up here the rest is that at the lake or in a cocktail bar or whatever things you do you usually should do on such days so thanks for coming yeah so this talk this talk I tried to explain some new features of solidity zero-five-zero this is special because we we follow the semver versioning scheme and this means zero-five-zero is a breaking change and the last breaking change was I think in something around October 2016 so quite some things happened since then I will mostly talk about the actual breaking chains and not the actual breaking changes and not the features that happen in the I think 25 releases we've had since then so yeah for zero-five-zero or for the breaking changes of survived see we mostly focused on safety so we noticed that it's often better to force the user to be much more explicit about some things we will see examples later and we will also remove some things which are yeah can be interpreted in different ways so we'd like to everyone be clear about the semantics of the language and we will also add some more runtime checks adding runtime checks is always a tricky because the more checks you do the more expensive the operations will be and you kind of fight with users who want their smart contracts to be cheap and then you fight with other users who want their smart contracts to be safe and yeah I always have to find a middle ground there okay let's go through some of the changes we start with visibility constructors and events so these are things which you might have noticed already in the recent non-breaking versions because we so most of the changes we gradually introduced by the way of adding warnings if you still use the old non ranking recommended way and so this is a simple smart contract in the zero four zero syntax and if you know the language a little then you notice at least one bug and the bug is that this initialize function is meant to initialize the smart contract so it should only be called from the constructor only from a privileged user and the reason is it sets the owner but since there is no visibility specified here it's publicly visible it can be called by anyone so anyone can change the owner of this contract and yeah most of you perhaps know that this is something that actually happened and because of that we now require visibility to be specified explicitly and yeah in current versions the compiler produces warning from zero-five-zero arm it will be an error so now this bug is fixed and the other something that also happened is that the constructor is defined by using by defining a function with the same name as the contract and now of course over the course of development you might change your mind about the name of the contract and you change the name of the contract but you forget to change the name of the constructor because of that we now force the constructor we specified in this new syntax where you don't use the function keyword or the special construct a keyword and now you can change the name of the smart contract without having to change the name of the constructor and then events so here we here we call an event it looks very similar to a function call but events are rather different they don't actually execute any code they just log information and because of that to make it explicit before we require an image keyword to be used together with events so that's not a big security thing but it's just nice to have I think good then the next thing is to force more yeah force users to more explicitly mention the types they want to use and yeah here this is also something that actually happened the VAR keyword can be used to declare a variable when you do not want to specify the type explicitly and the compiler tries to deduce the type and it does that from the first assignment that is done to the variable and here in this case the variables designed from 0 and now what is the type of this 0 literal constant literal constant always have the smallest possible type where the constants constant fits inside because of that this is an unsigned editor with 8 bits and now of course the problem is when you pass an array which has more than 250 556 whatever elements then this loop will not terminate because I is always smaller than length length because the largest value it can hold is 255 and you do not have overflow checks here so even if we had overflow checks this is not the behavior would want because you would not be able to pass in an array with one and two and fifty six elements okay long introduction know the easy thing here is just this allow bar so we had a warning for exactly this case because that happened a lot so we have a for loop and do not declare the type here we already had a warning for a long time but now we said yeah I mean just force users to specify types explicitly that that is also a rule we follow or we more or less follow inside the compiler itself so I think that's it's good to have some people complain that in some cases this results in quite robust code but we're trying to introduce elias's or type type definitions that might help in these cases then another thing which is perhaps not too visible here but in other cases a is a reference type a is an array a reference type which so a itself points somewhere and it can point either to memory or to storage or to call data and in many situations I always fall into this trap from time to time in many situations you make the wrong assumptions about where it actually points and because of that we force the memory keyword to be used always when you declare or when you use a type for yeah we need a clear variable yeah so then the next thing storage pointers so this is also this is something that is also a bit improved with having to specify the data locations so the local variable P here is of this struct type and by default it points to storage now since we force a data location specify here this doesn't happen too often because people then think oh yeah it's memory so you put memory here and yeah perhaps I should not go too much into detail here so storage is statically allocated so it's impossible to create new storage variable storage objects in a certain way and this leads to this variable actually having in having an invalid value so it has to be initialized with something but there's actually no nothing the compiler could initialize it with to be valid and because of that we now force the user to always assign P assigned something to P when it is declared ok and because of that it's better to completely rewrite this thing so you see peace peace defined here and it has to be directly assigned from somewhere yeah and this way it's much safer because there's no way for a storage pointer to be invalid unless yeah there are there some cases you might find out that it's much more much harder to get it to such a point good that storage point is now something that was also requested quite often is changing a change in the scoping rules mmm some of you probably don't even know but solidity inherited its scoping rules from JavaScript and JavaScript scoping routes are really really weird [Music] the thing is JavaScript scopes are always functions so usually so when you declare a variable here in most languages it can only be used inside this block which is delimited by curly braces but in JavaScript a variable District idea that is declared somewhere is always visible in the whole function and moreover it is also visible before it was even declared and even visible outside of whatever so this is valid solidity code and a version of it will also be valid JavaScript and it returns three yeah with zero five zero this is not possible anymore and we have regular block scope variables as you would expect from something like C++ or Java so even this Coast code is valid that this is not this cannot be done in zero for zero because you would declare a variable here whose scope is the full function you declare a variable the same name here which has the same scope so you have a clash of variables it doesn't work but this is of course very idiomatic code to write a for loop so yeah and with the new rules this code compiles and works correctly so okay block scope means as I said variables are visible inside of the curly braces block they were declared in with the single exception that if you have a for loop and declare the variable in this first initializer block it is visible in the for loop but not after it and another advantage of block scope variables especially in solidity is that it helps a little improving this at most sixteen variables for function limitation because variables can be removed from the stack when their scope ends previously the scope was the full function so you could really remove it and now you can remove functions when their scope and so you can actually have more than sixteen variables in a single function as long as you don't have sixteen five variables more than sixteen variables visible at the same time okay so if we already have fifteen functions in this fifteen variables in this functions declaring this I variable twice doesn't hurt because they are unrelated good we're almost done we also have some functionality improvements there is no way to remove the last element of a storage array very efficiently and idiomatically then we expose the ABI encoding functions so there's a bi dot encode which does regular ABI encoding and returns a byte array a bi dot encode pact which does pact encoding that's the encoding that is used by the hash functions for example and there is also encode with selector and encode with signature those can be used to create so ABI during code just creates the binary that follows the four byte function selector am I talking crap here or are you following this is really really deeply Technica so I'm really sorry about people who are already asleep so maybe I don't encode just result returns the actual ABI encoded payload data and if you want to use that with the raw function call you have to prefix that with four bytes that tell which function to actually execute and because it would be so prefixing data onto byte arrays is kind of weird so we have two additional functions that can do that in one run so the first argument is this for piped function selector and then the actual data arguments follow we will also support enums instruction interfaces that will request it quite often because that actually allows you to define yeah interfaces for smart contracts or for class of smart contracts in a useful way then so this these were three functionality changes we have tons more most of them were introduced already earlier so you might already know about that then some minor safety changes so call the call function so address dot call has always been a very weird thing with with weird behavior most people didn't know about and that's why we decided to simplify it it now only takes a single bytes parameter so just raw bytes and these are forwarded and that's it previously and so if the first argument would was a bytes for had bytes for type then we would encode differently and also the encoding of call was very weird it wasn't either a bi encoding nor pact a bi encoding but something else and whatever so now you can use call and you would use these a bi encoding functions to create the data of call and then just this of course applies to delegate call and all the others also the hash functions only take a single bytes parameter if you want to get the old behavior you use a 3 of a bi dot encode backpacked off and then the actual data while cut triple assignments are removed this is when a function returns multiple values but you are only interested in the first you could do a comma equals function call and because that was easy to get wrong now the left hand side has to have the same number of components as the right hand side so you would do a comma comma comma equals the interface functions have to be external that's the only reasonable way of for interface functions because yeah and then the next thing is also interesting so the bytes types are yeah byte arrays of up to 32 bytes and they're very similar to the unsigned integer types which also come in various flavors up to 32 bytes the main difference is that byte arrays are left aligned in the world and integers are right-aligned in the world now if you convert between byte arrays and integers of the same size what would you expect to happen how does it convert so convert the number 8 which is a you into 8 to a bytes 1 so a byte array of length 1 what happens okay say it again how many suitors in the beginning and I in hexoren binary yeah right so it it takes the single bite which was right aligned and just shifts it to the left exactly and so I hope this is what everyone would expect and it gets tricky if you convert between these two when the number of bytes are different because so okay if you invert if you convert between you into eight and you in sixteen then you would expect nothing to happen it's just the the the range is increased and if you convert between bytes 1 and bytes - it would just add a zero byte now the question is what happens when you convert between you and eight and bytes to does it first does it does it go through you in 16 or does it go through you into eight so does it first where does it add the zero on the left one right right so that's the tricky part and that's why we disallow that and you still of course can get exactly the same behavior by just having two conversions but you explicitly specify in which order you convert and so it should be visible what happens right shift and sign is it is this is actually my mistake I thought oh yeah shifts are just divisions by powers of two or multiplications by powers of two which is of course incorrect for signed integers and with 0 5 Siri fix it this will M so this is only right shifts on signed integers which probably nobody actually uses since it's not just a division by a power of 2 it's a little bit more expensive because the evm does not have native shift operations but i think that's fine if nobody uses it anyway yeah and then yeah constant state variables were very weird because they were kind of usable as macros so you so state variables are these variables which are declared a contract level and if they are constant they don't have to be concert in zero for zero but instead you would assign them from an expression and then this expression is copied everywhere where this variable is used and if it's not concept than it results in a different value and which of kosta finds the purpose of a constant and now we're forced it to be pure which means compile-time constant and thus it is still so the the the way the code is generated is still the same the expression is copied everywhere but at least the value would be always the same good then the major safety changes view and so since byzantium we have the static call up code which forces so which enforces at the level of the EVM that you cannot change the state of the blockchain and view and pure functions are disallowed to make any modifications to the state view can read from the state but pure functions cannot even read from the state and we cannot enforce so at the level of the EVM we cannot enforce the difference between view and view and pure but we can enforce the difference between regular functions and view or pure functions and that is why we use steady call it might happen that you compile code solidity code and then run it on a virtual machine which does not implements that e call so on so sometimes test nets if you if you run a client in test net mode then sometimes it uses the very old frontier release and in that case you have to manually set your EVM to the right version tor to Byzantium to do that if you if it's impossible to set it to Byzantium then you can specify so in a compiler setting you can say this is the tower evm version i want to compile for and then it would not use static call and then so this is also something that happened some i think it was in exchange implemented the ABI encoder incorrectly and this would so this is again this bites alignment thing so what happened was that if you so you specify an address to send your money to and if the address has leading zeros then this encoding library would just shift it to the left so it would instead it would not write align it but it would left align it and that yeah that of course results in the money ending up in the wrong and the wrong address but the good thing is that the way that was implemented was that it actually did not write padded with zeros but the data was just one byte short and [Music] yeah i'm really going to do this but eyes are still openness that's good so when you read from call data then everything is zero padded so the data that is passed on to a smart contracts actually ends in an infinite number of zeros and because there it still worked so if you if you call a small contract and give it not enough data then it still works because it just fills with zeros but this is yeah most in 99% of all cases not desired i would say and because of that we now check whether the data that is applied to the smart contract is long enough to hold actually did yeah to be decoded to be decodable yeah so this might cause some calls to fail if you do low level stuff proxy contracts or something like that so yeah please take care there and then inline assembly in the very beginning it was very yeah you were able to do a lot of things almost all thing that is possible with bytecode and due to several reasons we want to restrict that one reason is because we want to compile to web assembly and not only to EVM and for that we disallow basically everything that messes with the stack or messes with the program counter this means explicit jumps or disallowed you have to use the control control flow statements we offer so switch if for yeah and also direct access to the stack is to sellout and this actually turns inline assembly into a language that doesn't really assume that there is a stack or that is yeah they're just variables and you can call functions and assign to variables and that's it good and then there's another yeah weird thing which is different in other languages which perhaps I'm also to blame because I didn't find it in the review so do-while so the Dubai loop is basically a similar to a while loop so you the control for enters here and then runs this and the difference the while loop is that the condition is checked at the end and if the condition is true it jumps back to the beginning runs it again and check the condition again and if the condition is false then it exits the loop here now there are these statements called continue and break continue just skips the rest of the loop and continues with the next iteration and break exits the whole loop exits loop completely and yes somehow in solidity continue was implemented in a way that if you have continued here it jumps to the beginning of the loop and skips checking the condition because I mean the condition is it Daddy and then at the end so yeah yeah that's of course not very useful and not the way to while is implemented in other languages so now continued does not jump to the beginning of the loop but to the beginning of the check okay that's about it do you still have questions yeah I didn't mention that because it's not finished 4:05 server actually so are you talking about this new API decoder yeah that's still experimental and the reason it's still experimental is because we wrote it in a way which is extremely modular so every tiny little function yeah every tiny little thing that is to be done inside this decoder encoder has its own function and most of the time these functions only consist of a single opcode or sometimes they even do nothing they just return their input and the idea was that it's much easier to implement it in the correct way because you implement every single thing only just once and then call it from everywhere but it will of course result in very inefficient code because you have tons of function calls and jumps and because of that we implement an optimizer a new optimizer just for you just for inline assembly and [Music] we're not yet confident in that optimizer that's the reason so if you use a bi encoder v2 now it is that the optimizer switched off so you can use it but it will be much more expensive so it's basically the same as maybe i dot encode okay so there there's encode with signature and encode with selector the difference is that one of them I think it's selector one of them takes a bytes for which is just the hash the beginning of the hash of the function signature and the other one takes the signature itself and then hashes it yes so for the one who would pass F opening parenthesis closing parenthesis in a string and the other would get four hex bytes [Music] so I mean I'm not because of the storage layout so George there is something that is very critical and if we change it at some point that will be big big warning signs perhaps even a runtime check or something like that so so and the reason it is critical is because through libraries can access storage of other contracts and of course you can you can link against the library that was compiled with a different compiler version because that we actually cannot ever change storage yeah remember Leila is different so memory Laird is changed all the time actually we did change so there's this so at the position 0 X 4 0 there is a special value that points the first free space in memory so if you want to write something into memory without overwriting something then you would write it there and then increment or increase this this pointer and what we changed was not the location of this pointer but its initial value so and this was done an unbreaking change because I hope it didn't break anything so yes yes yeah sure I mean we're aware that assembly or there are libraries that use in an assembly and that's also how it should be of course if you ever change that and they will also be big warning signs so memory layout itself is not changed it was just this changed where the free memory starts so we changed it from so its initial value was series six zero so right after the pointer itself and now it points to eight zero and the reason is there is in this gap it's a single a single 32 byte cap and we assume this gap to be always 0 and we use this a selection to have more efficient initial values for dynamic memory erase previously if you if you if you declare a variable of memory type with a dynamic array it would create a new memory array of length 0 which occupies 32 bytes a memory for this length prefix and the length prefix is 0 so what we now do is we just initialize it with a pointer pointing to this gap and because we know or we hope that the value in this gap will always be 0 all memory erase actually point at the same value and 0 so the name [Music] there is scratch and scratch in space and that's the first 64 bytes so this is always can always be used unless you're inside and of course you can use you can also use the point where the free memory pointer points a scratch space because you don't have to increment it if you don't want this data to live on exactly yes we have you know I think all of what I just said is documented perhaps not the part that you can expect the free memory pointer to point to zero field memory so it's internals of the compiler somewhere in the in the solidity documentation because at the very end of the in-depth part so I can sitting it yeah there's a plan to make it return to values a boolean success value and memory byte array with the return data and the reason we didn't include it now was because it's not really useful because you cannot we don't have ABI decoding functions and so since we don't have any ID coding functions you have to resort resort to in that assembly anyway and then you can just do the call within an assembly to begin with and the reason we don't have ABI decoding functions is mainly because we don't have a nice syntax so the difference between ABI decoding and encoding functions is that the so in both of them you need to specify the type somehow you want to decode and but for encode just you to just take the types of the arguments you put into that and for decode you kind of that doesn't really work so you can look at the types you want to pull out but that yeah so and the syntax of solidity doesn't really allow for a way to specify the types in a nice way so we thought about using I don't know perhaps [Music] how do you call it angle brackets yeah but the parser doesn't really support that yet so it yeah will be in zero six zero at the earlier stages more questions yeah one of them is the call datasize check another one is at Mott and mall-mart these are these is a built-in function that take three parameters and two modular addition and multiplication and they check for the third parameter to be zero it was probably on something would encounter but yeah this is another one so something we've already added and was perhaps not such a good idea to add it or in now was check the size of the return data when you call a function that cost some some some tokens that claim to be EOC 20 tokens to break that's another one time check we're currently also experimenting on overflow checks what I said in the beginning so if overflow checks are nice to have but they will be more expensive so we're currently evaluating how expensive they will be and regardless there will be a keyword to mark an area to be unchecked so I think that's also something that c-sharp has yeah where were these overflow checks would be switched off I didn't get moving into what and we didn't really have type deduction anyway it just took the type of the value I'm not sure if you could call a type deduction so the expressive type conversions are something if should need to do anyway unless you do some little bit fiddly so yeah we try to kind of make that bit more visible that these explicit type conversions so when people try to do that but if you have some examples to share them please contact me so but yeah implicit type conversions are always possible when the value range grows or is the same you don't need to do a conversion they just assign it to the value at a specific converts so if you okay if you change yeah so if you for example as we had earlier given date to bytes one that has to be an explicit conversion even though that the value of range doesn't really increase so you won't notice the difference okay but that's disallowed anyway now so yeah all right you won't notice the difference between converting from you in sixty you and eight two bytes one and converting from you in sixteen to you into eight for example so this ii actually truncates data and the first is not truncate data just changes the alignment so having a more fine-grained distinction between those cooked something we could be something we should add but again it's not clear how to do that in a syntax which syntax to use for this distinction yeah yeah I hope that the switch to the new intermediate language you'll will be something I can focus on soon and the optimizer that comes with it not sure how to get that faster so and this is also something the user won't really notice so it was hard to justify working on that but in the end we have to do it because I think we have to move to something like webassembly or also yeah get kind of safer cogeneration because it's much hard to make mistakes in this intermediate language and also the optimizer might do much more things that weren't possible before yeah and and then of course the the SMT components so the the static static SMT based analyzer formal verification component we're pretty far there already so it now accepts storage so it can handle storage variables it even handles branches and recombines the control for branches later on the main thing we will have to add there next is handling modifiers and when we're at that stage I think we can have a proof-of-concept release of that component yeah that's something I want to avoid because you can't rely on us until checker or I don't want to rely on them there are some things that the optimizer can do but we have to be really careful I mean the SMT checker is just a black box anything can happen inside there anything can change between words and chip version changes and so on so yeah huh no it's just something like yeah someone else is doing it and they you don't know how stable it is I mean usually they don't have box they only have boxing components which were recently added but [Applause] we see winter so say something about the next meetup or