Workshop: Fuzzing: Like a Security Engineer | Trail of Bits | ETHTaipei 2023

I'm not here in the morning so I'm going to repeat this um there's a workshop channel in this and then that will ease you know I'm not only to send out the legs and resources to you guys so make sure to join this is that uh Workshop channel uh this microphone okay I was just like yeah yeah so join the channel and then we'll send them please uh the channel so go to the eatsafe organization and there's a Discord right so join the Discord first and there should be a workshop thank you foreign repositories and to be here early and here on Linux through next otherwise the first roll rock and roll yeah like three four and second one five feet thank you foreign [Music] along the way you know five questions all right foreign to install it and then after we start the exercise there's a link in the description oh yeah I got it [Music] so I'm Matt I'm from Australia trailer beds where we mainly do a bit of Security reviews as uh and that goes into not just blockchain but also across applications so that's including like application security cryptography and a variety of other things thank you you might have heard of Trail events through some more tooling we do have a lot of Open Source software some of that is static analysis further buzzing with Echidna as well as support for other blockchains in in some you'll see tealer algorithm Forest Park at Cairo and socilla select core kind of tooling for the ethereum space where do I start I just want to get kind of energy of the audience who kind of who's here so from I guess raise your hand if you are a developer contracts I'm writing videos so I guess for those of you who raise your hands this um for those of you who raise your hand this is kind of this tutorial of kind of this Workshop is going to be very heavily solidity based and so this is a chance for you this is a chance for you to kind of follow along if you want from um in terms of causing going to start us with kind of a very high level how do we find bugs in smart contracts and what we look for when we Implement property we're also going to look at how to find invariants and what we look at when we start looking at how to implement system invariants and then we're going to go into some live demos and interactive opportunities for you to get a chance to actually write invariants of your own and be able to kind of get a feel for how how you can actually implement so to start off um I started off with kind of this first slide of installing a kidnapp so Echidna you can install if you if you're on a Mac you can just install it using Brew I'm just using that First Command Brew install and this should work for the recent m1mt people if you're on Linux you can install it through next and if there are some windows binaries that are also available that are available on critic I'll just pause for a minute or so give you a chance to spell it first I also posted a link to the building secure contracts branch that has these slides as well as the exercises that we're going to be going through in the workshop Discord if you want to follow along the slides you can there as well as actually running the examples that will be going through today all right let's do a quick gut check um how many of you have uh you sell the kids up I'll just wait another 30 seconds or so then we can get started oh yeah let's do something you know the first letters capital with a question mark all right so from that Discord link um I did also link a GitHub repository which has the examples that we'll be working off of it's not ideal that this is going to slide but if you clone that repository and check out um the five pay Workshop grants this will allow you to follow up this is this is also the same link as the Discord so if you just want to plug directly from GitHub there that works great how many people have cloned the GitHub repo all right um and I'm going to get started into kind of how to find bugs in smart contracts and how to use tooling that's available within the industry in order to maximize the amount or the number of bugs that we find and the complexity of so if we take a step back and look at how how we can find bugs with code in the first place a lot of that ties back into kind of the like testing through either unit testing or manual analysis so unit testing is kind of the expectation that you have like an event tests and integration tests on your code base whereas manual analysis and automated analyzes kind of take a different look at your code at different View so with fully automated testing what you'll find is that this is something that you can run on your code base and it will show you many different findings this is something that is generally something that likes leather that you don't necessarily need to run your code it will just analyze your code for you and spit out a bunch of different findings which requires you as a developer or a security researcher to be able to go through and hit and kind of verify the output as well as what the results are which means that from from the perspective of actually looking through these results it often requires triaging and going through going through a bunch of results it also requires you to actually understand the system and kind of the the context that you're going to be using that smart contracted this is one example of fully automated testing in which you can run Slither on a smart contract and it will show you for example all the results all the results in the smart contract that might result in rapes whether or not they're actually exploitable that's something that as a developer or researcher you need to go through and kind of manually promote what's realistic and what's not other side of automated analysis is semi-automated so this is um this is really good for kind of identifying potential bugs that result in with as a result or as a result of logic errors or arithmetic for example but it does require kind of someone to be able to go in and write those invariants specify what is expected of a system and it requires you to think a little bit differently about your code at least in a different way um Echidna is going to be the the topic of today's conversation in this Workshop before I go in more into kind of property-based testing test um any questions about I guess the different types of analysis seeds that you can do on a smartphone check so far or I guess I'll go to kidnap as well all right perfect so in terms of fuzzing this is what we're going to be focusing on today and if we take a very high level look at what fuzzing is it's basically stressing a program with random inputs so if you were to take the most simplest implementation of a buzzer it would basically be like a cat typing on a keyboard and seeing you know how your code handles that random input and whether it behaves so another example of this that's that doesn't involve a cat is like randomizing input in like dude this would essentially this is somewhat similar to this to the similar concept of basically taking brand of inputs throwing up a smart contract and seeing is it possible for invariants to break in some way so buzzing in general is something that has been implemented fairly commonly in traditional software you can see that with like especially lip fuzzer and Gophers you can see this for other languages the only thing is they usually tests for crashes so this is when your program just can't handle input and therefore it fails don't necessarily have crashes in smart contracts smart contracts just don't allow you um this don't allow your disability so with property-based testing it's slightly different because as a user we essentially need to Define invariance we need to specify what is expected Behavior so that we can actually tell a fuzzer like Echidna what is expected and what is not expected and so from a from the perspective of the buzzer we're basically going to be generating random inputs in order to detect fins that we've specified so if we go back to the example a unit test then it's basically like unit tests on stairways because with unit tests we test kind of a subset subset in smaller area of the system but as long as we increase the randomness aspect that we can we're able to hit more pointer pieces so that brings an interesting question as to how are we supposed to think about code bases and how do we actually specify invariance um well that that that kind of requires us to know what an invariant is and how how we can even think of it because we don't usually think about code in terms of invariants we think of code in terms of this function supposed to do this and so this is just this is the expected behavior of a function when in fact we might be able to actually completely change the way that we think about that in order to kind of specify in a variant when we're talking about it so an invariant is basically something in a system that's that should always be true it's something that should hold true regardless of user input regardless of how a user interacts with your system so that's a little bit vague um I'm going to be going more into examples of the variants and how to think about that in different code bases but using Echidna we can basically take the concept of how is the code expected to behave and executed on time to select our contracts in order to find potential deviations Echidna is our smart contract buzzer it is open sourced and it's used in a lot of kind of mature code bases ones who have a fair number of unit tests and and kind of they know what the expected behavior of their system is also has some examples of more complicated integration with kitna in case you want to take kind of take a closer look at how complex test Suites so I'm just going to start off with the first exercise that's in that GitHub repo so exercise one is basically a walk through kind of the the code um and what what what we want to check first so let's take a if we take a look at that code what it's implementing is basically a token implementing kind of the the underlying expectation the it's operating on a token and one potential invariant that we can say with respect to token balances or user balances in general is that a user balance should never actually exceed the total amount of the total number of tokens in circulation if it ever does then we know there's definitely some kind of arithmetic issue um or there is a logic issue Within so if we think about this token as if we think about this token and the invariant the invariant is basically that the user balance should never exceed the total supply of tokens in the market so one potential one thing that we need to kind of analyze here okay all right so this is going to walk through this group as well so this the exercise already has code for the token and the token is basically implementing this transfer function when we create this s token or basically initiating it with an initial Supply in this case of 10 000. when we write properties in Echidna one of the ways to do it is by just creating a function in solidity with this Echidna underscore prefix and what this will do is basically tell a kitten that there is a property that is expected to run here and to telekinet to look out for something so if I were to run this with Echidna right now it's just going to tell me that there are no tests that are found because the body of this function is empty so I'll give you uh 10 minutes just to try to figure out how um considering considering you bright and variance using um assuming that you have an Echidna prefix Echidna is basically going to look for a return value of pulse and that's going to essentially identify that there is a property violation so this is so assuming we did just return false here a kid is essentially just going to immediately feel like dysfunction is an invariant failed but instead of returning false we want to make use of the variables within this token contract and this test token to basically add the property that user balance right so um in order to run this file you can basically go into that directory and then assuming you've already had Echidna installed um and run this directly on template in that template this also does rely on you having so C installed for a certain version so this one assumes version 0.5 which we can run folksies collectives basically allows you to gain Focus key versions uh so in here we're basically changing um the global so cue 0.5 and then we can run [Music] directly on the template this is going to be the output of Echidna of what it looks like right now when we just return cells it's basically we're going to say no transactions remains no transactions were made because it immediately found the false return value and then it just halted because it effectively already found a property violation foreign oh yeah [Music] foreign thank you if you were getting a selfie select error you can run a PS3 install Super C select and this will essentially download the the soxi select Library and then I'll post the other two commands in the workshop Channel um if you are running into the arrow what does yours look like mine okay sorry is anyone else getting compilationers that are different to like soapsy select areas when you run it how did you run a kid now okay I did see the exercise one and then okay I still didn't get them I did not have like that so I just can't wait okay less than equal to yours no I think it doesn't matter thank you wow okay that's weird hmm whatever I think it's are you doing the next ones thank you is amazing all right um did everyone who installed it kidnap get this working and running okay perfect and I know we kind of provided the solutions to this already but I do want to kind of kind of explain a little bit more context as to why and and how how this how this invariant works within Echidna and and kind of what we're looking at we'll open this up to kind of the audience to kind of help me fill in what this property should actually say um so for those of you who got this property running and working what was the return value for this function so this is awesome any ideas as to or how did you come up with this property in the first place like how what was your intuition um for kind of converting the the sentence into perseverant the sum of the Holdings of individual accounts should be less than the football supplier yeah exactly so this is a really good example of a simple invariant that we found or that we thought about in a token converted from English into code and we can basically convert it verbatim um so we use we did say that the user balance should never exceed total Supply which means that if it does this function is basically going to return false and find a property violation so this is a very small and kind of simple example of how we can use invariance and and how we can use tools like Echidna to help us find these cases so the essentially the um the property was basically just to retrieve the total the the balance of the user Make sure it was less than or equal to Total Supply so what does it actually mean for in terms of Echidna like how how does this proper idea that we specify how does it actually or first how does echidna look at it and then what does the output from Echidna actually mean so from a kidney standpoint we specify the smart contract code so this is you know our Target system in this case with our token we specify the invariant that we wanted to uphold and again this is regardless of how what do you what is what a user does with your system so remember this invariant is something that we require of the system to be true regardless of whether a user does a transfer has a mint has a burden it doesn't really matter to be honest we just we need this property to actually hold so from Fitness standpoint it's basically looking for any combination of function calls into your smart contract that can make that invariant false so what it's going to do is just it's going to be that monkey or that cat on the keyboard and just randomly throwing inputs at your smart contracts in order to see whether an invariant can be so if we go back to output from running a good note any ideas as to why this property failed in the first place so if we take if we take a look at the token code this is our transfer function considering it's using that line that you had to run for your silk C select that's basically changing your self seat compiler version to 0.5 this is the version that the code is running on so this is the version that is flipped into underflow um what it basically does is underflows balance of the Thunder which is why a transfer of this amount causes our invariant to break from echidna's point of view it doesn't necessarily matter what the inputs were because Echidna was able to find a front contract call that would essentially result in the failure of being very upholding Edna will also tell us um the exact arguments that are required in order to create to break it and therefore under so in this particular case it's just saying transcription random address um transfer thumb amount you'll notice here that the arguments within this box are different this is another thing to keep in mind with buzzing or these randomized values is that there is a in order to make it reuse the same values that it has explored before but for the purposes of live demos is never going to actually match um it's it's going to essentially be random it will depend on what Echidna was able to explore then in this case a transfer to the zero address of the amount one able to make it under so if your numbers don't necessarily match the ones that I'm showing on the screen that's also normal before I move on to the next section any questions about Echidna and kind of how that exercise translated to co Co for Highway kit Networks the solidity compiler version okay um so we can actually try that live so if we change the token as well as the template 0.

8 I'm gonna change the silk C select version to it as well and when we run it this is what you're going to see with the kinda it's going to try by default 50 000 runs you can of course also change this at the limit but it's this pass is basically saying within 50 000 runs it couldn't find a failure you'll notice that this this will also depend on kind of your your aspiration and how your fuzzer works so if you as I mentioned earlier you are using that config setting in order to rely on a previous Corpus like previous buzzing runs then you'll likely be able to explore more state but this past is basically saying that Within These 50 000 runs it could I couldn't find a property violation know what functions to test when we Define this template if we're using Echidna in this mode a prefix all it's looking for is a publicly publicly followable functions that return movements with the prefix so as we add more functions as long as we uphold with the prefix and the Boolean return that essentially adds new properties to call so in terms of how a kidney determines what internal like in this case the token functions call it's because the Echidna contract inherits from token by that it can basically call any public function within token um which which in this case essentially will operate on top of the output in order to API in order to figure out so any public external functions but when it's view it doesn't bother it can call view as well um it's like you can use a view function within like a property but it's because it's not going to change State then it unless it unless your property relies on that function like randomize the coordinator or yeah so um public holiday function um it it still that's a challenging one because it um Finn has the like if you're open contract has a fallback it will still execute but a lot slower and so you'll you'll see generally with kind of larger projects where we do rely on callbacks we'll just manually basically specify this is like a callback that we expect and this is kind of the log function that we expect to get them to call so Echidna can explore that it just it's going to take a lot longer and so there are ways that we can guide it could not into kind of executing more efficiently and that's what you'll see in more complex also try like for example Force funding the contract with like a self-destructible and a contract it has the ability to basically check if the it has some functionality to check whether the target can be self-destructed but that's something you can turn on and off so it's it's basically if you want to turn it on then Echidna will tell you if it's possible to self-destruct right so I'm just going to go into kind of how to find good invariance because we've seen an example of one invariant allowing us to find a bug but when you get into really complex systems and kind of more obscure behavior is going to be much harder to actually pin down pin down what is expected to happen so one thing that um we generally start with is trying to start small with your ear contacts a lot of people especially if you start to look at the Integrations in big code bases what you'll see is like an integration or an Echidna test on maybe a library or like a mass function so that's pure math no State changes that's generally a really good place to start because you can basically isolate a very small part of your system and be able to think about you know what how is the system expected to behave so when we identify invariance what does that actually look like it's a lot of not writing code unfortunately it's a lot of looking at code bases and figuring out if I call this function what is the expected output what is behavior and just writing that out in plain English that allows you something that you can for me as a security researcher go back to a client and say these are assumptions that I think are true in your code base can you help me verify if these are actually correct and for a developer team to kind of be able to have those ideas and bounce them between your team because your invariants are only as good as you define them in English you can't you can't possibly write a really good invariant test without actually knowing what your invariants are and this is one of the hardest steps of the process because we don't think about invariance very often you think of um or we don't think of it quite in this way when we write unit tests we think of I send it this input this is the expected output that's that's basically a one-to-one mapping you for example you call transfer okay you expect the balances to change afterwards maybe you check total supplied has increased but you don't necessarily think about system-wide invariants that include something like or change as opposed so this is one example of how we can pull out the mathematical libraries into something that is actually digestible easy to test easy to kind of get an invariant weight up for it something like a math Library if you're taking two numbers you're implementing addition then if you send it two inputs regardless of what order you send it in the output should be the same with identity if you're multiplying something by one the output should be itself and the same thing for inverse if you do one operation and then you do something that's basically the opposite so position subtraction you know multiplication division the output essentially should be zero so system-wide invariant is basically that the system shouldn't have changed you look at the system kind of before this operation and afterwards the states should be identical I found it through you in the deep end for this token one because that kind of gave you the invariant to test but this is or these are other examples in which we can actually look at um how how to think about invariance obviously the no user should have a balance that is used total Supply that's that's one that we've we've parked on for a little bit another one is for transfer um but the sender balance under balance should decrease and the recipient balance should increase now this one is pretty intuitive if I'm transferring funds my back my wallet My Wallet balance should not be less than it was before again if I try to transfer funds to myself the balance should be the same so these are you'll notice that um as I kind of talk about invariance there's there's a difference in kind of the the type of invariance because some of them are function specific some of them are kind of global to the system so that brings me into identifying what your invariant is actually supposed to Target so if you're looking at a functional invariant this is I call the add function with two numbers and this should be identical to add regardless of what the order is another example is um if I were to test a math Library I pretty much generally any math library is is generally a functional level experience because it's because it's something that's stateless we can very easily craft a scenario basically setting up a scenario in order to test kind of the expect now the other type of variant is a system level variant this is something for example like user balance and total Supply and kind of their relationship these are generally stateful by staple I mean literally Changing ethereum State something like balances for something like a token that contract is relatively small um but as you start getting into bigger code bases um and especially the bigger they are the more complicated your system level of variants are going to be so for unusual any ideas on what a system level and variant might be for unisol so uniswap relies on basically a concept of the constant product so it relies on essentially after every swap after every Opera after every swap it's going to check it's going to check an invariant that's basically K that's kind of the colloquial term going into a whole lot of detail about how K is calculated uniswap has operates completely under the assumption that K is maintained to be monotonically increasing for every for every operation this is just expected behavior of uni Swap this is something that's built into how they are expected to function and this is something that needs to be withheld in their entire system this is another kind of example of invariance and kind of being able to pick out what is expected Behavior abuse so if we look at if we take a closer look at what a functional level invariant is a functional level invariant is basically allows us this is a code example of the commutative adding property in the a plus b this equals a plus b is equal to B plus a this you'll notice in this case inherits from let's just say a math Library it doesn't really matter what kind of math Library it is but it inherits from a math library and these tests Community function this test looks slightly different to the one that you were interacting with before the one before had the Echidna prefix and returned a Boolean this uh way of specifying invariance in echidna is basically to specifying variance through this assertion if you're not familiar with assertions and solidity they basically require statements but in terms of developer in terms of development pools like Echidna will look for assertion failures so essentially think of this as a if Echidna finds a failure within this community of added commutative adding property the kidney will throw an error similar to the one that we saw in our terminal earlier so this is just another flavor of interacting with Echidna where we can essentially set it to assertion now one benefit of assertion mode over um kind of the property mode that we saw before uh before it was the Echidna prefix assertion mode allows us to test multiple things within a single function so if we look at for example if we look at that example with a testing token transfers then we can with the Echidna prefix we would be limited to only testing something like total Supply whereas with assertions we can now test Thunder balance recipient balance and and kind of add on additional properties now system level invariance as I mentioned before this is where things are start to get a little bit more complicated and this is definitely not going to be the focus of today's Workshop the there are many ways that you can kind of implement system level in variants and a lot of them do hinge on either construction so you're basically setting up the system something like like the uniswap core unit swap Factory within the Constructor or you're relying on deployment unit tests to basically set up your system and then start testing on top of it so with all of these in all of these cases Echidna is going to look for that public keyword within um basically all the the functions that are public and Echidna is basically going to call them um and trying to in trying to integrate any variation of calls that will result in a system failure this is one's small example of a system level of variance is the one that we did related to um now you'll notice this is this is showcasing a slightly different way to use mychidna in that assertion mode and it allows us a little bit more room kind of to specify what and how we want to continue them so you can run that we can basically run this uh with the as well just with so if we were to set uh remove the public return basically it's very then when we run echidna refer this back to your eyes we just add a test mode thing this tells it to look for a search assert statement instead of the default and we're need to run [Music] it's basically going to find a similar output now the only other thing it's going to run assertions on all of your publicly callable functions which is why you also see additional function s down below so this is basically looking out for assertions in all of the functions all the publicly callable functions Within that were inherited from the token but I've talked a lot uh any questions about kind of those invariants and then how Echidna works for example you mentioned for the constant product so the staff complete or is the uh different Behavior so that's a great question um so the question was whether Echidna could handle or whether Echidna handles something like a rebasing token for like uniswap um if hidden will only explore the paths which you define so if you define if you specify if you set up the entire system working with a traditional non-rebasing air C20 then the Assumption there is that a kidma cannot find um potential invariant failures that being said if you do rework kind of the deployments and set it up with the rebasing token it will also depend on a variety of different factors like um especially with something like unusual if it's something that you'll want to essentially see a state every time you run um and um it will take quite a lot of coverage in States in order to find those invariance so I think it kind of goes back to their only really as good as as good as you set up the system so it's not going to find anything outside of you four statement to the uh it's in the pipeline yep let's say there are most low contracts for each other and I with the system level and go and I totally hit my conscious explore according this contract also try calling methods but it only ever try running yeah that's a great question um so in in the previous examples like for example for this it's only going to explore publicly Public public functions on the token if you do set up Echidna with like a deployment script in the women's script say creates like a factory in like multiple pools and whatever the goal depends on and then Echidna will essentially require anything that's public or external on any of the contract um now that being said especially if you're running in an assertion mode the output that you're going to get is what is going to be way bigger um than these assertions that you see um and so it can also has the ability to spend we also have the ability to basically say ignore buzzing like these certain questions so that you can kind of isolate the system ways that you produce so it will basically if you do set it up with a deployment script then it will follow any particularly any publicly available can I specify like uh if I was in the test mode you have like a sterometer which is randomly from the whole base I guess and I specify should constraints for example let's say one kilometer address and the only one is like certain emphasis to be tried or once the second number yeah um so yeah there's there's a variety of different configurations that we can specify one thing that we'll be looking at with exercise two is is how to bound your input ranges but just to provide a sneak peek um you can also use a traditional solidity required statements or just like modular checks here so one option is you can just add a required a if you have a function that takes an argument um let's say U.N balance um you can actually found it a buyer balance is like less than that for example so you can use essentially your a test to bound I could like if you want it to be you know one to two values or you want it to be certain addresses then you can formulate essentially what those inputs are before you actually invoke and this is this is a slightly different example than like the positions that you would do something like uh call like the transfer function afterwards for example with like the balance any other questions foreign [Music] as well so you can do that using uh solidity as a try catch statement um assuming that that's one way to do it you can basically wrap your call in a try catch and if you expect something to revert then you can actually just assert a false in the case where it was successful um the other option is if you're using like a low-level call invoke the call and [Music] then you can check on success as well all right so exercise two is going to be a bit more involved a little bit more Hands-On um I'll walk through kind of what uh what exactly the contracts you're checking right now and then I'll give you an exercise fill in as well so before we get started I want to kind of walk through this this Library um this library is based on um kind of a preserved Library you don't need to need you don't need to know a whole lot of details other than it holds the reserve two tokens as well as liquidity of has the ability to swap basically if you take one token you can essentially depending on which direction you want to swap it's going to either decrease or increase the amount that you walk with it has the ability to add funds so again this is a library so we can make the assumption that there is a wrapper or some other place where it actually takes the input of tokens this is just a a stateful library that basically just changes internal balances um what it does is calculates essentially how much how much liquidity to add it takes the minimum of either one of the two tokens and then it changes balances within the storage so by this we can basically say Okay add funds does some rough calculation and then increases the amounts of both of your tokens as well as your liquid remove funds basically does the exact opposite it calculates how much given an amount of liquidity that you want to take out how much of each token should be adjusted and then adequately decrease so if we open up template um one thing that we can kind of think about within kind of how this even just add funds function works is trying to think about is to think about what are the preconditions um what is the action and what are the post conditions of this ad funds go back to liquidity tracker let's take a look at ad funds um what is a potential post condition of this function what is expected to happen once this function runs reserves increase are there any preconditions of this function usually tell by the number of required statements in it foreign statements in this function there are no preconditions this function can be called at any point all right that's that's a good point um reserves Reserve risky and reserved stable cannot be zero um because once we once we operate on this division uh it will essentially refer with divide by zero error so to an extent um we can basically say the preconditions are that um reserved reserves are not zero action is basically we can call the liquidity trucker in the ad funds function basically the we expect to be reserves to increase so if we were to be more specific about that we're basically saying that they preserve risky stable and liquidity should be we'll give you 10 minutes to fill in these preconditions um and these post conditions the way that you run this is the same as the last one the one thing that you might need to do is install the latest version of foxy reflect or foxy for 0.8 um version if you don't have it installed already and then you should be able to do foreign running this version is going to be slightly different than the previous one just because it's in a different API mode so I'll just post the exact command to run it but we're basically invoking Echidna on the file and specifying that we want to to run this specific library with a test mode of assertion but I'll post this in Discord to see if the ability just oh and if you pull the latest version you'll also find those commands um to use how to run a kitten yeah so if you have it in the same function it'll be within like that same transaction but if you have it if you have it like split up across an alternative then a kid that can explore it like across blocks yep so as long as your post conditions are in the same function I would be able to press that is there a way to like step um [Music] coinbase I would have to check on that I know there are configurations for like um sender addresses for sure for block numbers I'm not 100 sure I can check with that generally you can manipulate um generally manipulate the VM it's pretty easily so yeah any testimonies okay um well I should say you can get event emission but a kidney is not going to like test what like the output of an event hello oh actually I haven't been trying to do it because I feel like I haven't been doing the exercise oh it's okay my brain's a bit down now I think I know you you you're right I did yesterday right yeah three a.m yeah I think I need coffee before the next okay give you about three more minutes and then we can walk through the solution and then also kind of walk through a bit more a few more features of Echidna that might be useful for debugging foreign will allow you to essentially hold these functions across many different blocks and so you can test really well to get an exploits without really needing to do a whole instead of um also the kitchen supports like Frank [Music] before immigration for those of you working on those post conditions um do you want more time to kind of iterate over those all right I'll go through it in about two or three minutes then foreign yeah Escape yeah and then yeah your number so another thing to note if you do want to increase your test limit for for these iterations you can also you could do that through your CLI through your test event and then a number and this will essentially increase your runs up to the number you specified thank you all right just to walk through those preconditions and post conditions our preconditions was basically that the reserves should be initialized with some value you will find this set of Reserve function already there so what the set of Reserve function does is basically in sets up these three values with some number and sets the is set up to true means we can use this to essentially determine um whether the system has been set up yet I'll come back to the precondition for this because the fear afterwards but if we start looking at the action start looking at the action what we want to check is basically their reserves have increased that the reserved risky stable and liquidity of increase so how we can do that is basically by first saving a state of what exactly are the liquidity values before in the transaction so given it's using liquidity tracker data we can [Music] say this is the free um gives us a snapshot of what the reserves are before the ad funds is executed now how do we check whether reserves have actually what would the syntax be for that we have an assert what should we be asserting um thought he greater than three or three we have no yeah and then we can basically repeat that for pretty much all of the functions or all of the state variables including research now if we go back to the reserves are non-zero we can actually make use of the Boolean that we provide whether the system is set up or not so the simplest way to do it is if the system hasn't been set up then just calls out of here this is what I mean by making a craft this is what I mean by we can craft scenarios that essentially allow us to test a setup system easily so in this case we don't necessarily need to have that initial transaction to increase your reserves we can essentially assume that that has already happened through something like this so if we run that so while that runs any questions about the implementation or the syntax um all right he post that comment um one of these methods to return it like outside um yeah that's a good question in in these functions we're returning values and in the previous one we weren't this is just a sneak peek for kind of an ex a future exercise in in the current um right now we can actually just remove everything after Wards and essentially it would behave the same way um because we're specifying assertion mode Makita is not actually going to look at any of the return values and what their values are it's only going to look for restriction failures um time for the war room right so if we take that it's here our Echidna tests um we can see that in this case we had a failed assertion um and it failed because when we called add funds with zero now this doesn't give us a whole lot of information to go off of um chosen now here we have two zeros which presents zero like I don't know certain values like Mexico yeah so um randomized is kind of simpler than it actually is um it will start with kind of the the end of your input so generally zero your max or your type and it will depends if you do have constants to find in your code base it will also use those um as as test cases first um so it will essentially test on the the known balance first before it starts to kind of iterate so again this Reserve at funds doesn't give us a whole lot of information as well why it's failing and what caused it well um and so for that we can actually make use of um we can make use of this assertion failed object or event which is defined in a kid novel essentially a look for instances and give us some concrete information so if we take that then we can basically just convert assertions into a slightly different syntax so if these are true that we can admit a certain field even just like with the message specify Reserve risky and like the free uh reserved whiskey and we can basically do that for all of these which allow us like a little bit more information as to why exactly specifying reserved specify liquidity the basically if we if we run that and then we get a lot more useful input that's that's kind of different and more intuitive than just zero zero what we can see is kind of two failure events because Echidna is looking for instances where we're emitting this assertion failed event it will essentially halt whenever it sees the event and tell us about a failure but more importantly what it gives us is the event stack trace of the spring that we specified we wanted this to emit the pre-balance post balance and who the senator was Ascender doesn't really matter for our purposes but basically this is this this test case is basically saying we added a one second and from the output the post balance of the sample token didn't actually change any questions about kind of this Outfitter any ideas as to why this actually happens foreign yeah exactly um so this this is one case in Echidna where it essentially notified us of a rounding error um now again this is probably something that could have been detected through menu review maybe manual analysis but this gives us a very easy output that we can actually see what what what the expected Behavior was and how it was tampered so this is another example in which we could actually make use of these are variants in order to to find kind of those small rounding or they look like small round hairs but um eventually they will Auto and and couldn't result in protocol loss no is a few other functions within the reserve removed and then the addition of funds we're probably not going to have time to go through these exercises today but I do recommend at least trying out for yourself kind of writing those preconditions in those post conditions for the rest of these two functions because it gives you a better intuition for this small code base how to write them so that you can apply that that knowledge that to other code bases as well now I did also provide solutions to to kind of the ad funds and that's where the return value for the ad funds function comes in you can actually make use of those two functions when you write the invariance with this last one so I'll open it up to questions just about fuzzing manikidna before I enter the the last section thank you yep that is the next section it's now again it brought like the zero one so one is also a special value package for us so so Echidna relies on heuristics um in terms of like how to find those inputs um so it'll it'll generally try out the bounds to zero given to 56 and then start entering into a closer um so again it kind of changes if there are like finding your code so if you say Define like a constant um one common example would be a number of decimals for a token that would be another magic value that a kidnapp would be to use in order to call your code in order to call your countries so it kind of relies on on a variety of different factors in order to kind of maximize the number of tests that you're that you're running when you run that closing campaign um the other thing to keep in mind is um if you enable essentially coverage saving within Echidna um the every time you run and so every time um okay now once it's done it always says campaign complete press escape to exit if you enable the Corpus like coverage saving another step that it does at this stage is basically save the coverage state of all of your contracts as well as like magic values that um it found like useful or productive so that the next time that you run it Echidna can essentially execute off of existing exist in state specific [Music] I did not happening problem like finding that even though that I could have had like have you said it is completely obvious but it doesn't it doesn't use anything that you use in your tests or divine variables that we said foreign like finds those inputs and generally it works better on ranges in bettering ranges than like specific values um that being said there are like certain cases and like complex deployment in which um some of the like those those constants that you define should help the seed um and if they don't that's likely a bug within a kid though that it's not actually picking up um like the the values that you've defined so in terms of kind of execution um the the idea is that we should be essentially relying on clients that are defined all right so going into kind of a comparison between other similar tools and I think generally a lot of the questions that we get are kind of Echidna versus Foundry and in kind of different areas of program analysis on Smart contracts so in terms of Foundry in front of other filters one area where Echidna kind of helps is that it doesn't really necessarily matter what development framework you're using it will work with pretty much any existing framework out there so if your tests are iridium Foundry your deployment scripts are already in Foundry Echidna will work with that out of the box it's possible that something like that dream might be easier um kind of easier to pick up when you first write your tests and kind of your especially your your stateless tests um that being said Echidna also allows you to essentially run your Echidna has a Foundry mode that allows you to basically run existing fuzz tests that work in Foundry on a kitchen as well and one benefit that you get out of that is that you can you can basically use like the vm.crank ETC of what of syntax in Foundry and with Echidna you'll be able to test not just per transaction in a block but you'll be able to test across blocks so that's that kind of gives you a little bit more power in terms of cases and Corner cases to look out for when you're passing your program so um the other approach is kind of the formal methods approach into something you'll see generally with Manticore kdbms Victoria and for a lot of these they will provide proofs they will you know provide kind of concrete proof that that your code behaves as you expect that being said it is generally more difficult to use and this generally takes a lot more ramp up whereas fuzzing is something that you can basically pick up what's because you already know solidity it's relatively easier to kind of extend um your solidity like unit tests or just buzzing in general given that's it's within the sample Echidna also has the ability to for you to test either functions that consume a significant amount of gas be able to differentiate which functions especially especially our gas costly and be able to essentially analyze where a lot of that gas used it supports differential fuzzing in that you can test against like a reference implementation or implementations in other languages and generally works with any compilation framework so behind the scenes Echidna relies on output from Slither and credit compiled these are both trailer bits external open source tools as well and what those do is basically kind of provide the input kitten would need to execute it also has a variety of apis we saw two of them today with the Boolean property with that prefix and then the return value it has the assertion mode which we saw like an example the assertion mode is kind of something that gives you the ability to to test multiple kind of assertions within the same function and then it has adapt tests and manager mode as well which you can essentially match a syntax with them so I guess in conclusion um the hidden we have quite a lot more resources and kind of examples that you can continue to work off of and kind of get a feel for not just how to write invariants but how to think of these systems in general that second example is just one part of something of invariance that we saw and I definitely recommend going back and kind of implementing the post conditions for the rest of those functions and kind of seeing how you can not just Implement post conditions but also how you can make use of multiple functions when you define these invariants um generally because invariants are so complicated in terms of just thinking about your code differently and kind of requiring a slightly different mindset um one way to get started with that is just to write down experience what only the expected behavior of your code is expecting behavior of your function start with just writing it down in terms of words and then work on essentially validating and verifying that those invariants are in fact true before you even start thinking about touching code one thing I do a lot is sit down and start hammering at ingredients only to realize that you got the invariance wrong and you have to rework it now that's also another thing that there are cases in which you'll be really confident and invariant you have a failure and then you find out it's because we were missing a precondition or that we didn't set up we didn't set up the system properly so running invariance in general is kind of that iterative process of really getting a chance to kind of iterate over first defining it writing it code and then kind of if you do go to failure a friend of trying to figure out why exactly it failed um and then if it's something with your test adjusting those tests so I'd highly recommend continuing to build off of exercise too um in the the two other functions as well as try out aketna on on your projects and also to take a look at okay so if you also want to see a more complex use cases of Echidna um this these property testing Suites um it's a really good resource to use so that you can kind of see how these how these large projects have picked up Echidna and what their invariants look like wow any questions about closing or anything contracts people yeah exercises yes aside from building secure contracts there are some exercises there there's also I had no streaming series I can post a link to this as well I've been out into these slides in here we have six Echidna streams all of them essentially they kind of start off from the basics of how to fuzz how to think about invariants up until how to essentially Implement a kidney on a complex code base all of these have exercises associated with them um there are a few other repositories that also have kind of more Hands-On experiences with writing and variants um about drill bits process like what it process do you yeah it's really like a Central Park definitely um so the question was around kind of trailer bets's um kind of Security review process um so a lot of our security reviews we try to use Echidna and kind of these tools as much as we can throughout kind of the audit generally what we'll start with is running something like static analysis on the code base like Slither and that kind of gives us an idea of just looking through just something that again it's very verbose so it tells you a lot of information but it requires you to kind of go through and check whether whether they're actually valid and naturally relevant to the code once we once we do kind of the the static analysis piece a lot of it goes into manual review and kind of understanding okay if we're looking at a system how is it expected to behave and this is generally when we can start thinking about invariants football so the man you're reviewing kind of the invariant finding uh generally something that I at least do tandem um and then depending on kind of the the infrastructure and time what we'll either do is deliver a list of invariants so these are the list of areas that we came up with during kind of our review and these are something that we can kind of provides support for in terms of integrating kidnap but not necessarily necessary during the specific audit but in in generally a longer audience we'll do the setup and essentially have be able to give that to a client so that they can continue to have that running like a CI every time they start to add code and change code so that that's something that they build off of there's there's not necessarily a clear delineation between when the manure of who stops and when when vehicle starts um generally we'll really bounce between the two um sometimes we'll kind of go into a kidney and realize we started too early because we don't actually understand the system yet and then we'll go back to manual review and kind of pick up the program analysis piece of Pizza so it's um it does take a little bit of practice in terms of identifying what area to start with with the kidney especially when you when you start looking at um when you start looking at these complex projects the the biggest challenge is is really trying to identify um what you should be testing with kidnap and when um is is kind of the the big piece on the agenda to have like support for running against like a certain state of a maintenance whatever alcohol is this because for me it seemed like if I have a protocol that's maybe integrates with unisoft then police has to work I have to do a lot of money or set up in my tests um so the the current infrastructure for foreign [Music] like the factories and the pools and in front of everything associated with it um that's kind of a work in progress in terms of just how to kind of formulate an end kind of setup of everything in the package one thing we didn't necessarily talk about today was the ability to see it off of a deployment scripts so like if you think about any kind of project that has like the essentially either testing deployment or like contract deployment um what Echidna can do is basically snapshot the blockchain after so assuming that you have like a local blockchain running you can run the deployment scripts against that liquid washing and then a Kidman will save the Snapchat right afterwards and so from that you can basically start testing kind of more um locally that's one thing that like already exists so that's about something that we use that pretty frequently but the the kind of more complex integration requires a lot more um kind of not just um requires a lot more kind of configuration and and details there you said that you can also integrate with like the rainfalls from celebrity example or set storage to manipulate like foreign [Music] so pretty much any any of you uh boundary tests you would be able to run in a statement if they couldn't as well very good question so uh you might have already covered that recently you can uh rolled out uh um in general like what is the best yeah yeah we're I think we're still uh so so you can point to connect to the main Network um we still need to work on kind of additional documentation on kind of how to uh how to use that the most effectively in general it's it's it is kind of the the first version that we've pushed out for magnet and this is uh kind of still a bit work in progress um so it's definitely something that we need to flush out a little bit more [Applause] thank you that's enough um