Martinet Lee: So what is an audit anyway? | 14th Oct 2023 | Sunway University | ETHKL 2023

K thank you thank you sir uh and yeah good afternoon everyone so uh I am Martin Lee so today I'm going to talk about uh what is an audit so it's more of a beginner session um Al thr in here bits and tips for uh like developers as well but most of all it's like odit 101 and clarifying some misconceptions so let's go um so a bit of background for myself martinet and then I'm a head of developer relation at Quant stamp uh my experience I actually start as a security auditor so I have audited well more than 50 projects defi nft to later later ones some stuff you might have heard of including Ur uh eth itself so prison implementation uh and then Avalanche as well uh our firm did a lot more other stuff then I continue Ser as a PM standardizing a lot of like different stuff so kind of like all stack on the uh auditing side and so today I'm going to talk about like you know what audit is and audit spoiler alert audit may not be what you think it is and then kind of like uh from our point of view what the goal of an audit is and kind of like okay what's the tip of reducing audit cost and enhancing audit like efficiency efficacy so the first thing is I think when people think about the term audit then this is often times what's being like popped in their mind like oh Financial audit and uh you know they they consider they link these two terms together but it is it a correct notion I would say no And so there's a lot of text there I know but we're just going to look at the most important part so this is kind of like the weaky definition of financial audit and you can see one thing the first line is like oh it's talking about a financial assessment of an entity as its whole right so an entity and whether it's profit oriented or not the second line here is as required by law that's another highlight I want to have here now the third one here is well Auditors do consider propositions before them you know obtain evidence this one we can talk about a a little bit later the last one is uh free from Material misstatement right so this is kind of like the uh the stuff you think when you're like thinking about likei financial audits now let's look at it uh from the looks of web free security audit still a lot of stuff but let's just zoom in into parts of stuff so here instead of the whole entity we see it's a part of project defined by the developers right uh and whether it's security so from profit oriented to security oriented or not so this part seems okay right so it's marked in green but the kind of problem is like it's a part of project not the whole project that's number one and secondly it was defined by the developers not someone else right the second point is the most uh different thing here there is no law that is like requiring people to get audits requiring them and then like standardizing how to get an audit right so uh and it's also kind of Impractical how would you uh have a law like determine how you're going to implement your code that's very different from bookkeeping like bookkeeping is relatively simple activity whereas well uh coding is a t and complete activity and that is very complex right so that's why it's given a very complex name to in complete um the third and fourth is sort of similar and this is why the audit term is sort of still uh acceptable so we auditors web free security audits we do consider the propositions that comes before them which we look at the user document that was you know out facing and we verify whether those are correct or not um if and this is kind of similar like oh free from Material misstatement right so these are kind of like similar and that's where kind of the notion of audit works but the notion of audit don't work is the one like the 0.1 and0 two uh so the this is one thing I want to point out the Impractical goal of a secured audit if you have this thing in mind then that's not the correct notion like oh I'm hiring an expert or experts to ensure there's no bug uh we if you're a developer you probably know this is very hard to achieve almost impossible so this is really the wrong mindset and I always like want to take the opportunity to generalize this uh to the general en public what is actually the goal of a security audit a lot of words a couple things one audit itself is a Time limited activity right and what we're trying to do here is to assist the project assist the developers into finding inconsistencies and vulnerabilities these two things and here the last thing is you what you defined of your pro right so this is for the developers uh vulnerability inconsistency let me expand a little bit there vulnerabilities is like hey I have this design I think it's awesome it's Innovative and when Auditors are looking at it we say like oh this design has flaws because we can exploit it kind of like the business logic right and you don't want this consequence so that's vulnerabilities in my opinion inconsistency is another big part of audit so you say you wanted to do it one way but your implementation is actually doing another thing uh as an example uh we have previously audited a protocol uh options protocol right um some people may have background but options in simple terms there are like two types of very different options one is European option another is American option and they implemented European and not the American but they intended to do American right completely different uh but they don't they didn't realize that until we audited it and this is inconsistencies right so it's different because it's not really vulnerable uh it's inconsistent it's yeah so this is the goal of a security audit I kind of like to do this visual comparison uh with some other kind of certificates or stuff that we typically see so the left one uh or yeah the left one for you guys is the typical Financial audit or bookkeeping kind of stuff so you have the government and Define the scope and Define the rules of how you do this audit right and you have the test subject and demonstrate these relevant data and then the auditor then comes in and examine these things right so this is how like audit Financial audit work now on the right side there these are also two common other types of certificates so the one above is uh let's say like a TOEFL test or GRE test or any test right The Examiner defines a scope and tells you like hey this is the scope and then you like kind of like show them the data the your ability and then they will examine assess the output right so that's uh the upper right the upper uh sorry the down right one is uh another kind of like certificate sometimes we we say like oh this this entity has this credibility and basically how it does is like okay the test subject there's no external entity that defines the scope but the test subject just publishes report publishes data and then there's like a independent examiner publishes a report based on whatever is published outside right right and but they they themselves Define and collect the data so these are all very different but very closer to our real life kind of like certificate auditing experience what audit is in web free space is this the developer team defined the scope okay so this is completely different the developer team defined the scope and auditor accept the scope and assess only that scope right um so of course there are a lot of like Notions that comes out of this uh a very simple one is if I get an audit does that mean that I'm secure or not that's something that everyone wants to know and the short answer is no and I can translate it in the very simple terms um if I take a TOEFL test does that mean that my English is great no right because I need to like see the assessment that's one thing second if I can Define my own scope then hey too test let's only test my writing ability and I Define the test uh I Define the scope and I I only take the test that way and I get a good assessment does that mean that my English is great and not necessarily right and so there are like these Notions that comes out of this so that's why we going back security audit's real goal is to help developers right it is from the developer for the developer helping them to see what flaws there are and what they wanted to see uh this is the current state of the audit industry uh in web free so the audit is not being enforced by government there's no government involved here the project is the one paying for it requesting for it and it's up to the project to publish the report or not because well they pay for it right and this the third thing is the main goal that I've been keep trying to convey audit firms is to present findings to improve def teams code okay we do write as fairly as possible because we do know that these things can be used as marketing material uh from the dev team to the general public and so we do write these things you know not only having the dev team in mind but also like oh if they publish this what notion does it like what kind of message does it convey so we we are sort of uh aware of this and then we we are um careful about it but the main goal is always to improve Dev teams code now uh I'm going to the third part which is actually uh talking about how to reduce cost for your Audits and and how to improve the security but before that it's NE sort of necessary to understand what's the auditor's thought process and this is sort of like roughly okay uh let's see so the green part background knowledge that is already inside Auditor's brain right uh there's no additional information the purple part is documentation comments and code this all comes from developer side the project side and these of course requires the input from the developer um now it all goes to a couple things so Auditors tries to do free things when they're kind of assessing a project they want to understand your intention the intention of the project they want to understand the implementation right and also like just from their background knowledge they know like oh some of the past tack vectors or some of the common errors that they've seen those kind of stuff so they have this like list relevant attack vectors and fret modeling going on so these are the fre things they do in parallel the ones I listed as kind of like painted as yellow means that it is affected by user or Project's input their documentation their comments their code quality um what we try to find after understanding this this is this is like the understanding phase then there's the Attack Phase kind of uh basically we ask very high level these questions does the intention make sense because if it doesn't then well there's a vulnerability in your design uh secondly is are there discrepancies between the intention and implementation right and this is the inconsistency that I mentioned earlier well the third thing is uh there's the common attack vectors and then we just like check against them this is more or less simple for us but it's kind of like the learning curve love thing and all these things compile into findings but as you can see a lot of these are determined by or affected by user or the Project's input which is the documentation comments and code now uh as you may have remember auditing itself is a Time limited fashion right so we we think about how much time we need to understand this thing and then attack the thing so the kind of trick to reduce your audit costs is also simple actually is to reduce the time that Auditors your Auditors need to understand your project as simple as that so having higher code quality documentation quality better design pattern coding style test quality code readability and don't over optimization these will you know become make your project basically more readable more understandable reduce the time right now the great news is it is also a way to optimize for security because it's generally just more clear right um I want to like give example it might seem ridiculous but this does happen in the industry I've been scoping out a project and all its variable naming or contract naming are human names like contract Joseph uh and then contract Jennifer I'm like okay so Joseph do swap Jennifer and I was like what and so like just by reading that you're like getting headaches right and this this goes into this part the purple part like don't don't mess up your code don't do these things uh the other this is the ridiculous side of things then there's a well it's a spectrum right the other common misconception is like if I write my code so well you should be able to read my code and understand it uh unfortunately no that is not the case um there's different reasons for that so we need to guess the intention like we need to guess your intention and then read your implementation and if you don't have a separate documentation or comments or all those kind of things then we we have to guess we cannot um like then there's no inconsistencies to be found and we can only kind of like guess the uh inconsistencies and that makes it harder so generally these things I would say is can reduce your audit cost and also optimized for security because for the same time frame we get to understand it deeper and then attack it attack the protocol in like more creative ways because we understand it deeper now audit itself is not the end of security that's how R viw it and this is what what I want to like uh if you're going to take away another message from this talk this is the other thing like AUD getting audit itself doesn't mean it's secure right and even if you're auditing the whole protocol it still doesn't mean it's secure why because the evm itself may change right there there could be changes like one old example was that one of the gas uh I think for SL load or s s store changed and then it broke some of the contracts like through the fallback function right and and that's the evm part that was changed and so what you need to do actually like for for this from the security uh firms kind of like uh perspective we consider this like a defense in depth so there's a lot of layers the ultimate goal is security of users funds and if that is our ultimate goal then there are a lot of things that we can do and some of the things are actually doable just from the developer side which is developing using best practices which we already talked about which will reduce your cost and all that you can also do automated scanning and testing that's a good Dev would do and of course security auditing is one thing but after security auditing there's monitoring incidence response because we don't know what we don't know right and evm may change and all that that's one thing then the last thing is if the unfortunate things happen in those events you still want to have prevent things from happening at least prevent loss of funds for your user then there's sort of like insurance right so this is how we see it like there is a lot of these layers that we can prevent and enhance our security uh rather than like oh I got an noded it's secure perfect no uh this is how we see it so also G gonna quickly talk about the different audit types so the one that people are more familiar about our code review audits basically we review the code of the smart contract or the the node software um we're also doing infrastructure audits so basically looking like right now they're like a lot of like these staking facilities they run AWS Etc how is their security posture right how how are they're managing their keys Etc so this is infrastructure audience there's deployment verification so the code itself although it may be secure by its implementation its configuration May lead to vulnerabilities right uh I guess one it's okay that's a little bit different but um I have an example which is a contract is secure but it's it should only be used with a decimal token decimal of 18 right let's there could be such a contract and the team that we interacted they didn't uh think about it and then they linked it with I think usdc was eight or six right it's just not 18 and then that messed up the decimals so this is like where the deployment verification comes from U making sure that you actually configured everything right of course there are like Consulting engagements like before the code is M uh for insurance monitoring Quant stamp because we're thinking this way of course we didn't offer such stuff uh this is still like ongoing uh chain Pro is a subsidiary of us I think the interesting thing for me at least is that we are the first legal uh insurer in this space uh I think we it's under like some sort of legal framework and then uh reinsured by Munich re so Munich re is the largest reinsurer in the world uh monitoring wise uh or let's say in incident response wise we're partnering with Hyper native uh hyper native is an awesome tool I would um recommend you guys to check it out a lot of times what they do as a as a Dev I'm actually quite Hye uh they have machine learning models and when you deploy or when an attacker deploy a malicious contract they determine that oh it may be malicious and then have an alert out and then also determine the potential victims as well and oftentimes that is like five minutes before the actual attack happened so it's pretty cool um in theory in theory if we catch those and imp PA it at that time then we could have prevented the love the losses um but there are obviously like some um you know progress needs be made on false positive Etc but at least they have these things and it has proven like um being detecting these attack beforehand before the actual attack happened so pretty cool to see them doing that uh yeah so that's the talk uh you know know so there's the contact of mine uh feel free to reach out to me and uh yeah thanks for listening thank you yes indeed audits always very important giving people a sense of confidence any questions about audits uh please raise your hand we'll pass you a microphone any questions you want to ask about Fin uh not Financial audits but rather protocol audits in web 3 raise your hand speak now or forever hold your peace all good or everybody's expert audits okay maybe you can ask them a question and then no I'm kidding uh ladies and Gentlemen let's give him a big round of applause thank you very much Mar appreciate it