A Developer's Guide to Self-Auditing | Joe Van Loon - Audit Wizard

[Music] all right hello out there uh my name's Joe Van Lon I'm the CEO and co-founder of Auto wizard so today I'm going to be talking a little bit about security my talk is called a developer's guide to self- auditing easy methods and strategies to not get hacked so if you're a developer uh wondering what you need to do to stay secure or get the most out of an audit you're going to get this the right place all right a little bit about me I worked as a security engineer in web 2 for about eight years worked at some large tech companies uh did a lot of Security reviews in that time did some research as well on um uh empowering developers to be better at security to self- audit um and now I'm building a platform for Auditors and devs to easily analyze the security of smart contracts all right so why self audit I like to think of security as a probability function you basically have an expected value based on the controls you put in place the standard calculus in security is risk equals probability times impact when you sum your security risks and apply your layers of Defense you get an expected value of the security of your app of your smart contracts so this value can never be zero But ultimately it's your goal to get that number as small as possible keep it there there's also a model in Risk Management called The Swiss Cheese model I have a nice little graphic here of that but essentially um each layer of Defense you have has holes in it um when you stack them up you know those holes ideally will not overlap you get enough layers and you've got a pretty good uh set of layers of Defense right ideally those layers are complimentary and they can reduce the overall holes in the system so when thinking about the security of your project um your goal is to take a series of approaches that ideally have their own unique probability distribution so the holes don't line up audits are great um and you can get multiple Audits and sort of Stack those up but there's a lot of other stuff you need to do um that have their own sort of unique probability distribution that can ALS also have a good impact um and yeah and security doesn't start an end with an audit sorry security doesn't start an end with an audit it's an ongoing process you should practice it at every stage of development so being critical of audits they are definitely not a silver bullet I have a nice uh graphic here you can see of the cumulative amount lost over time by audit status so the orange line is unaudited the blue line is audited and you can see that blue line is not zero so why is that um I'll start by saying the incentive for Auditors is not they're not necessarily aligned with yours right an audit firm is their goal is to produce a report that's satisfactory um that does a good job is the best effort and then move on to the next one your goal is to not get hacked um and yeah Auditors have a reputation to uphold uh but no audit is perfect right even bigname firms have experienced hacks on contracts they've audited um some vulnerabilities are quite complex it's totally understandable that not every auditor would catch every vulnerability um so one experiment you can run is if you get an audit from two different firms you'll find that the reports they produce are not exactly identical so as I said stacking up audits um is a good standard practice it's a standard advice to get two or three audits at least but you should also augment those layers of security with your own um and also in addition to audits from firms you should explore things like audit contests Bounty programs those can help fill in the gaps as well all right so if you're prepping for an audit uh my first piece of advice is to make the job of your Auditors as hard as possible you want them to struggle to find vulnerabilities because they absolutely do not want to turn in a sparse report so if you can get all the loow hanging fruit out of the way if you can front run uh essentially the Auditors on the work that they're going to do you're going to force them to look a little bit deeper to produce a meaningful report um Auditors hate to turn in you know a set of five informational vulnerabilities and that's it doesn't look good on them so they're going to dig deeper and deeper until they find something that they're proud of so the number one thing here is that if you have insecure code you're going to get an audit they're going to find all the you know vulnerabilities then they're going to say okay fix them come back for around two right um usually that means getting another audit sometimes you know it's part of the initial audit deal but if your code is very insecure um you're going to be wasting a lot of time and even so uh if you have that time Auditors are going to be you know spending their effort working on that low hanging fruit that you could have easily gotten out of the way instead of doing more uh complex research on what could really go wrong with some of the more involved systems in your smart contracts so getting that low hanging fruit out of the way is a really good way to make audits uh worth your money the other thing I'll say is Auditors the first thing they're going to do is read through your docs they're going to run your tests they're going to Fork your tests and then they're going to um kind of modify them and play around with them so having good docks good tests will save Auditors tons of time um it will leave more time for them to really think critically about the review to explore more complex vulnerabilities those hidden criticals which are really the things you're after you don't want one of those popping up you know 2 3 years down the line after you've gotten four or so audits that no one ever caught it and essentially you can think of you know you're hiring Auditors for one thing they're critical thinking security expertise if you can get as much of that overhead of the audit process out of the way they can focus on that one thing that they're good at if they have to write a bunch of boilerplate tests because you have no tests that's a waste of their time that's a waste of your money um my last tip here is to make use of the tools out there static code analysis tools um you know they're usually Limited in value because they target low hanging fruit but if you can't pass the static code analysis audit then your code is not ready for a human audit so it's a good step one all right so auditing yourself if you don't want to get an audit or um if you're not ready for an audit there's still a lot you can do um and actually developers and auditor skill sets overlap a lot more than you might think as a professional develop ER you're actually not far from the skill set of an auditor um you only need to take the right approach to reasoning about Security in order to successfully perform an audit one of the key things there is threat modeling if you've never heard that term before um that's okay I have a slide for it coming up we'll go through it in detail but it's one of the most powerful tools uh or techniques an auditor puts to use so you want to at least do some basic threat modeling some basic assessment of your own security from different perspectives you know think like an outsider think like an attacker not just someone who's been hammering away at this code for a year or so um yeah threat modeling involves thinking of anything and everything that can go wrong and uh very importantly also includes things you believe you've already mitigated so the key thing there is to challenge your assumption um trust verifies what we say um so one example that I have here of thinking of everything that can go wrong and testing that Viper had a particularly nasty bug in their reentrancy guard due to it not working in some of the cases that they hadn't tested so essentially the reiny guard uh variable was scoped to single functions um and if you re-entered in a contract in a separate function um you know that was actually valid the rency guard wouldn't block that so what went wrong Vib didn't challenge the Assumption of uh their reentrancy guard uh you know working for multi-function reines so things can go go very wrong if you make assumptions and don't verify them so one other tip I have here for any Dev team out there is that putting at least 5% of someone on your team's time to think about security is much much better than 0% um so that's always been my my advice actually my my advice is to assign someone to be sort of this security champion of the project um but you know if you're moving fast and you have a lot of uh development overhead you can't sacrifice a lot of time but you could sacrifice at least 5% to not get hacked and then last thing I recommend here again is another set of tools for fuzz testing um so if you're not aware of what fuzz testing is essentially is fruit forcing your contracts with different inputs to see what the behavior is like it's a pretty good way to uh reason about or to uh I guess detect security issues and the main thing is there it doesn't take a lot of human overhead um it's kind of point and shoot there's some things you can do to set it up and optimize it but you're mostly relying on a computer to bruteforce this for you so it means you don't have to sacrifice a lot of Dev resources okay okay so threat modeling uh when I first started as an auditor in the web two space I would often ask the question in my reviews how do I know that I'm done how do I know that I found everything what if I missed something um no one really had a good answer right uh but the the technique for addressing that problem is having a very exhaustive threat model you know as I said before modeling is just imagining anything and everything can go wrong with security um and it's much easier to reason about the completeness of a Security review at a high level than to sort of aimlessly bug hunt um so there's a ton of strategies for threat modeling but the basics you need to know are the CIA Triad that's confidentiality Integrity availability um all risks can sort of be cator categor I into one of these three buckets to further expand on that we have the stride model and I have a nice diagram here of that um you can see what it stands for there but essentially gives you uh a good lens to model potential attack or through and to think of the various vulnerabilities that might be present right um you could think of how your uh assets might be tampered with um how a user might spoof another user um you know how in information disclosure might happen right so you can use those uh models to help you build up a significant threat model um to think about everything that could have gone wrong lastly every auditor relies pretty heavily on a vocabulary of sorts of common security vulnerabilities you know some something they do day in and day out is look for these vulnerabilities they know them pretty well um and they start to you know get to know the common ones and know what they should look for first obviously as a developer you don't have that vocabulary built up a good place to start with that is just looking at the top 10 smart contract vulnerabilities and asking yourself how could I be vulnerable to this um obviously there's more than 10 that you want to look at but the top 10 is a good place to start all right so OPC um I'm sure we've all seen quite a few obsc failures in web 3 so far um the main thing here is security is broader than just code uh you can have the most secure code in the world deploy it with you know a certain wallet and lose that wallet right or you know someone gets access to your keys and it's all over so you can't think of security in just one domain you have to think of it more holistically so there's a lot to cover in OBS I think that's its own talk in itself but there's a few key things we can take away um few basic principles one the principle of leas privilege right so for all your assets just asking yourself who really needs to have access to this the ability to do this thing um anyone that doesn't absolutely need that you know if you could survive having taken it away then do so and then multiart multi-party approvals so you can have one bad actor on your team you could have one uh compromised laptop one malicious employee if you will um but you significantly reduce the probability of them being able to do something when you pair them up with others in the organization right so if you want to do a smart contract deployment rather than everyone on the team being able to just do that you need uh you know three or five people to agree on it and they'll all you know be a filter for anything that goes wrong TFA saves lives SMS account recovery as we've seen kill um even vitalic is not immune from this right I think we've seen on Twitter or X quite a few times now where people's accounts got hacked how does that happen almost all the time it's because they have their phone number tied to their account they have some sort of SMS account recovery mechanism they get SIM swapped then their account's gone um so it's all about permissions right think critically about your permissions review them often um look at things like Discord for example Discord servers um actually have very complex uh sets of permissions easy to mess it up but make sure you're looking at everything that your company does that your project does um making sure you're secure there all right so I only had 20 minutes for this talk there is so much more I would love to cover um so much more you need to know but hopefully that was a good primer for the basics so you know where you need to dig into further um a few takeaways I wanted to highlight one assign one person on your team at least to Champion security thoughts initiatives even as little as 5% of their time goes a long way um and then Empower them to actually make changes that matter to um audit your own code right especially before a professional audit no one knows your code better than you you have all the context um you just need to employ some auditing techniques uh and you'll be better than any other auditor out there maintain security as you develop so don't wait till the very end to start thinking about security because you'll have dug yourself many holes that are difficult to dig out of um always follow good opsec principles um definitely include that in your review don't just go get a smart contract audit and think you're Totally Secure you need to think holistically about everything that could go wrong and lastly um let's check out audit wizard so I have a small clip that I'm going to show you but audit wizard is a platform for auditing your own smart contracts reasoning about security it can help you with everything I've talked about today uh thank you for listening I have a few uh links up here few things I recommend looking at oasp have some really good resources um the Wikipedia entry for stride oddly enough does a really good job of uh detailing uh threat modeling and a couple other ones there and I wanted to end in showing you just a little bit in the last minute or so of audit wizard um essentially it's the platform that we've been building at Auto Wizard and you can load your contracts in there you can run all sorts of different tools we're in an open Beta right now so feel free to come test it out give us your feedback uh we're adding to it continuously we have many many more features ahead of us uh and definitely looking for your feedback to know what's important to you all right that's my talk thank you everyone