Panel: Security

a security panel so we're about to start and uh just right before we start uh we really well want to thank the one of our sponsors our internet sponsors uh for helping out you know connecting uh so far this five days there was no issue whatsoever so here we go uh in Mandarin though uh take a take a pan on that oh cheers anyways the internet is good that's that's a really good thing like during the hackathon so many people are hacking here that's the best I would say that actually the best thing so uh well back to our security panel so um hello everyone and I'm Martin Lee I'm one of the uh co-organizers of E-Type Apes um and also a security auditor by trade are currently serving at quantstamp and today we actually have a really really great lineup uh with us today uh you know so um maybe well joining with me we have Natalie Chan uh Neville and Richard uh and Hiccup and um so actually uh let's just do a self-introduction and how you got into the web free security space um you know to kick us off hey everyone I'm Nat um I am a security engineer at chair lipids I got into blockchain back in 2017 at 8th Waterloo I started going to hackathons when I was in school um just kind of stuck with like blockchain development and I switched into like the security side in 2020 when I joined trailer bits hello I'm Neville I'm a co-founder of ddub I used to be an academic researcher theoretical computer science program analysis and compilers in 2018 I shifted my research onto smart contract security whereas applying my techniques there so that got me into this space and then I was more and more interested in you know the security space as we went along created the products and then started an audit company called Tidal um all right oh wait there we go all right hey everyone I'm Richard Ma I miss you and founder of quantstamp uh started the company five and a half years ago and uh before I started Quant stamp I was a Quant at a hedge funds um and uh I invested ether pretty early and I was a investor in the original Dao and so after the Dow hack decided to start constant test hi everyone I'm hiccup I started off as a developer with kyber in 2018 then I swapped to webpage securely in middle of 2021 so I currently am a C4 Warden slash judge I'm also with Sherlock and uh hi I'm Trey so I lead bookings at code Arena I spent about 10 years in web 2 before I sort of fell into the crypto rabbit hole and then my story accomplishment in the first two years include getting rug buying High selling law all of these mistakes click on phishing links and then at some point I got really interested into the security behind all this a lot of scam projects and I kind of stumbled my way into code Arena and so I'm relatively kind of new student Security in this security space in web 3 about two years yeah um yeah so you know since the dial and then like D5 boom like millions and billions has been hacked and so um you know obviously like auditor other thing is quite important in the space um I guess um but there's like one thing I think that is like developers might really want to know and get more clarity on and so as Auditors we're also pretty just familiar with our own audited processes but you know um maybe we can like talk about that uh how our auditing Works in general uh and then I've also uh sure that like the traditional auditing is a bit different from the currently like Community auditing so uh yeah yeah for sure um so in terms of process um at trailer bits we generally start with usually static analysis and kind of running tools like Slither on contracts just to get an idea of kind of low hanging fruit in um just like code and so we can get a chance to basically focus on kind of the fun bugs and and focus on like the ones that are like more complicated um so with static analysis we can pretty much get kind of that like low hanging fruit and then usually I go into like manual analysis um with kind of a focus on like looking for invariants and then there's not really like a clear time in which I stopped looking for invariance and start implementing them sometimes I just kind of bounce between the two and kind of knowledge that you gain from like doing manual and manual analysis on your code base and then just time to like set up something like Echidna on your code base can also kind of provide more context um as to how your invariants are expected to work and then it's just an iterative process I think from from then on well uh at Dita the process is actually very similar to what Natalie explained so we we also use static program analysis tools we don't use Slither we use Watchdog which is our own two um and the process actually starts I believe before the office so we try to make sure that the the project is actually ready to be audited sometimes you have to delay the audit process because the you know the business development guide that we're speaking to is very keen on getting the audit done and dusted and sometimes developers haven't had the courage to communicate that they need you know a couple of weeks more to test the protocol so yeah we we start off by you know understanding the the protocol itself we read the documentations very for the first day or so and then the first phase of the audit mostly consists of getting a deep understanding of the protocol such that you know if we had to rewrite it from scratch can we actually dump all the documentation and and do it ourselves you know so that you need that level of understanding and the second phase of the audit is to actually work like a hacker and actually you know understand what assumptions the project has what the invariance that the project has and whether you can break them in any possible way you've got a lot of tricks so sorry I actually want to like kind of follow up on that so you touched on another perspective so Natalie touched on like the technical perspective on the review and he likely touched on the uh like you know like making sure that that documentation like a pre-process I think like uh maybe that's something that we can you know talk like you spend a bit more sure sure I mean this process I mean evolved naturally as you know as we had experience with auditing more and more projects um so you know one of the things we do we kind of understand whether the project whether this is the first audit of the project because if it's the first automobile project and we worked with others before then you know you gotta explain the process you know more clearly we send a document before the audit saying you know this is you know the level that your project should be you know when starting the audit um so yeah things like that foreign just to follow up on that um I think like for auditing uh we try to be very collaborative so um you know usually like for crypto projects they don't have the best documentation because you're like everybody is trying to you know deploy things pretty quick and so um often like when the projects come to us we'll work with them to try to figure out okay like exactly you know how um how is the project meant to work and like you know do some calls with them and then also like internally um for our own auditing team we try to just focus on uh learning for the team and collaborating during the audit to like find more complex uh vulnerabilities so I think that's sort of like our main style in it it really works well over the longer time period because like you know every like three months or four months there's something new that um people invent in the web three space and so like for Auditors like we're constantly learning about new projects and learning like how new things work and it's like easier to learn as a team with like other smart people and you can like ramp up faster so I think that's sort of the approach yeah I can attest to that so I've worked both individually uh as a warden in code arena in Sherlock so uh I've also worked as a team with spear bit so I've seen how both like working solo and as a team both have their pros and cons so let me talk about me individually as an auditor what my process is like um usually I start off by reading any documentation that is provided by the client so it's for me to get a good understanding of a rough understanding of how the project is supposed to work and then after that I would start with a shallow dive into the code base um to understand the business logic but of course not doing a deep dive yet so it's mostly to try to understand the business model um then after that I do like a deep dive into the code base so uh with that then as for spear bit I think it's something similar as well each team member tends to provide different bring something different to the table so like the one that I recently worked on somebody was doing invariant testing right so others could just be looking doing manual reviews on the code base and others could maybe try uh writing like uh Slither running automated tooling on the code base so I think with Spirit was a little bit more collaborative and we get to explore different paths uh I would say the process that kodorino is perhaps a little bit simpler up front um basically during the scoping process we try to understand of course the scope of the audit any complexity factors that might be at play here then from that point onwards it's really getting the repo up into shape for the for the success of the competition we do lean quite a bit on the incentive design because with code Arena you basically sort of eat what you kill so there's really no limit as to how much you can earn if you're good I mean the limit to the port per competition basically so um with the given context many of these Auditors who are either full-time Auditors full-time independent Auditors lead devs they basically can take what's given to them in this initial phase and then basically go on from there um okay I mean more increasingly we're getting more teams of wardens uh competing as teams precisely what hiccup said and they would basically develop their responsibilities or maybe vectors of attack between themselves in terms of competing and then they will split up the rewards accordingly says uh in in code Arena like who doesn't so I I would say like probably in the traditional other firm that I know uh it's like well we have the try to have an auditor like go on and scope and so how how does the code Arena like the community effort does that so the question is how does the community partake in the scoping process that's correct um so it's largely I would say maybe do 70 80 or internally decided and we basically have a system where there are complexity strikes so we have a scoping form that has about maybe 20 22 questions some things like that and depending on the answers they're given in each parameters that will be considered the complexity strikes that's added to it and based on that how we typically scope and advise uh Bounty sizes or work posts or projects is for this scope historically this is the whirlpool that we set if there are certain complexity strikes that are occur occurring above a certain number we would advise that hey you probably should think about boosting the pool a little bit um to foreigns in terms of involving the community members we do have like a team of let's say pretty active community-based of members that I or some other team members might personally talk to them when some when some questions about scoping arises then I might get let's say five different opinions to kind of triage on something as a Baseline thank you so um then I guess follow up on that the um so so hiccup you have been involved in like both right and so how would you uh say like the difference between these uh Community effort and then like individual ones yeah okay first off uh I think one key difference is the incentive structure right with traditional audit firms you get a fixed pay but with audit contests your payout is determined by the number of bucks and the severity of the bugs you find so with that I think there is in a way different Behavior with a fixed pay of course you would still try to do your best right but um there is a bit more of a level of a competitor of competitiveness when it comes to audit contest so um I think it does there is sort of like a game theory to it uh with uh the audit contests or the competitions uh that you probably don't see as much with traditional audits um I think the other uh part of it is uh with audit contests you have basically anybody can jump in right because for a code Arena on the anybody you could have no prior experience at all and you could just like take a look at the repo if it's a public contest and from there then you could just start auditing and it's flexible enough for you to not even have to spend it's up to you uh to decide how long you want to spend on the code base it could be say 15 minutes half an hour one one full day right and I mean you don't really have to find all the bugs but uh it's you can tailor it to your schedule based on your own uh like lifestyle right and with that uh with it being so open then it allows for it caters to a wide variety of Innocence Auditors right you have people who will be like hardcore going going hard at it like it to them it's like a full-time job and on the other Spectrum you have people who kind of like do it more as a hobby to be exposed to new code bases to new features that they might not be aware of or to keep in touch with the latest uh developing practices so uh yeah I would say like these are the two main differences that I see between Community style Audits and traditional audits yeah I guess uh to add on to that uh you know I have a bunch of the Arena tokens and um I think code Arena and like things like code Arena it's a really good way to onboard more people into auditing uh because it's you know it's like has very high barrier to entry as a like a profession um and you know most people like you know they don't teach security auditing in University so um the like the funnel into being a great auditor is like very tight um and then like the great thing I think also about um code arena is that like you can get exposed to a whole bunch of different projects right because there's uh you know various contacts happening um so yeah that's what I would say for it and then I think for traditional um like you know audits it's more collaborative like in the longer sense where you have like people that you know and that are all like very smart and nice and that have worked together for longer um and so it's it's more just like a you know like a team exercise where you know everybody's skill level and uh there's like in some sense I think some level of competition is unhealthy but like you don't want too much competition because uh especially for like the really complicated projects um it's kind of a team effort to find those uh more complex bugs right so you don't want people to like withhold information or like withhold like they might think they have suspicion about some part where it like feels weird and they want to look more into it um so that's sort of where I think is kind of like the trade-off there thank you well if I could add something like the uh the community approach obviously has a lot of Pros as well also cons so the you know another yeah sorry so another con of that is that um the the process is a little bit inefficient so you do get a lot of information a lot of duplicates whereas in a in an audit which is performed by say one or two people it's going to be pretty concise um so yeah yeah I would just add that um I think when you consider Community audits a lot of the focus is on bug finding whereas I think um there's there's another area that needs a lot of attention which is usually related to like system design how just recommendations around like simplification of your system or just high level like architecture which I think is is one um one aspect that may not be covered as much with Community audits cool um and so with that um so so we have the uh developer perspective I think I think uh pick up the touch on one thing which is uh bug we do sometimes less bugs as Auditors uh and there's like a very common questions uh which is well when when bugs are when bugs are being missed um like who are responsible for it so you know would like to take it the user for sure yeah I can I can go yeah I mean the government this is my personal opinion I think generally as as as our industry and the domain matures even as an end user on both sides of fans I would like to see it trending and leaning towards more accountability right now I feel like yeah I mean it's really hard to say right because now that I'm involved in security for sure when I'm putting money in a protocol I want to understand like how they push Security in general and also I think many times sometimes some questionable protocols which I should not be named they would just do an audit to get their brand to get a brand out there but actually the scope of the audit is much smaller than what is that but most of us who are not technical have no capability of understanding that so I think greater transparency leaning towards that on all the parties involved definitely helps as well yeah um I would say I we wouldn't really want to do finger pointing um I think sometimes as an auditor like I I sometimes ask myself right like for the projects that I've audited are there any bugs that I have missed right and there's always that possibility um so I I think it did happen uh to a notable like modern uh with the Euler uh Finance hack right so they were auditing like an update to the code base but then sometimes you sometimes it's because you're only given that scope right um and you're you don't have a full picture of the code base so then it could be probable that you miss something that is that is a system wide level uh attack but it's because it will only like us to look at specific contracts right um but then so that it does so I feel like if I were in that situation I I would it would eat at me I might not be able to sleep well so in some sense I do feel that the Auditors who were involved in the uh in auditing that project would feel like they are responsible for it at the same time it's also the projects uh themselves I think like I've been in the shoes of a developer right so if I were to write code that was hacked it would be like oh um and I I would also feel responsible right so in general it's not I think it's not just one party it will be whoever was involved in that process would feel would bear some kind of responsibility yeah yeah I guess just to add to that um probably like every two weeks for the last four years I've been having some calls with the founder or CTO of projects to try to get them to slow down on deploying stuff and it just uh it's it happens all the time it happens especially during Christmas and New Year's because like I don't know what the reason is but people really love to deploy stuff during Christmas and New Year's and like January 2nd um and January 1st for some reason so um you know like when we do audits you know on the one hand we we serve the companies and so um you know ultimately it's up to the company and how much budget they have and like when they decide they want to get an audit and what the scope is and then on the other hand like when we see um after we do the audit we generally have a really good idea of what is the chance that this thing could get hacked in you know the next um year and like often we might have you know found all the significant bugs but from looking at the way that they're doing development we can tell like okay well you know if you're not going to come back for another audit in the next six months and based on the pace that you're doing stuff looks pretty risky and so often we have those calls and I'll say about 70 of the time they don't really listen because nobody wants to slow down you know they raised all this money from investors and they got all these developers like pumping out stuff and um it's usually after the first time they get hacked that they really take it seriously and unfortunately it's for most projects it's their first project and so as as a repeat player I think um one of the main responsibilities is to like use as much of our soft skills as possible to try to convince people to like you know slow down a bit thank you answer the question who's responsible um I mean generally it's more more the project than than the auditor I mean an auditor like us for instance I mean we're very specialized on security we're not an insurance company we don't have actuarians we don't model risk of of hacks and things like that and in addition to that you know I would argue that there is you know there's no quantitative way to to you know find out the the risk of these hacks I mean if you look at Euler for instance they've gotten quite quite a few audits um you know six Auditors formal verification uh but yet you know there was one you know one missing detail you know one function and you know in combination with a lot of complex Machinery that cause this to be you know hacked so yep yeah I think uh just to Echo the sentiment I think audits in general are not really meant to be okay you've like passed the audit now you can deploy that's not really the purpose of the audit in the first place the purpose of the audit is basically to tell you um within the amount of time that we had these are the bugs that we found um and I think in terms of like even just reading audit reports the number of findings is definitely going to help with kind of providing context as to what the system looks like right now but another area that is really valuable is what are things that this like in terms of like development life cycle that a protocol can take on to improve their security process and more specifically how can projects like integrate something very static analysis or fuzzing into their CI so that when they get to the audit stage we're not looking for okay here's output from static analysis and not necessarily having to report that and we can focus on kind of more complex system interactions the perspective from the Auditors but still from the project side I would say now if audits are not to secure the project uh how do we determine or assess the quality of another because oftentimes like when when developers engage with other firms sometimes they just get a report uh and you know like a lot of questions like in private I also get a lot of these kind of questions like how do we know is it a good audit or not so anyway takers yeah um so I I oftentimes read audit reports by by other Auditors as well to you know learn about the other process and also the quality of other Auditors and what they're doing and you know the protocols they've audited um especially if we kind of like have um you know our auditing a protocol which is building on some other protocol so the way my Approach basically is to just first of all eliminate all the boil a bit right so go directly to the you know critical vulnerabilities and high severity vulnerabilities now what I do is I I look at you know take a sample of these high severity and critical severity vulnerabilities and you know sometimes I find something which is marked as high and it's like what is that really a vulnerability um you know so that's means that you know probably the auditor is not like trying to find you know heavy stuff in the oven they're marking you know some vulnerability as a higher severity than they actually are so you know there's some telltale signs like this that you know that that I use in order to uh you know estimate the um the quality of the audit well the other thing is that uh there's some audit firms that have like uh you know different skill sets in in the auditing team and sometimes you find you know extremely good audit reports and then you know less you know high quality audit reports you know for the same brand so that's because you know they've got many people with different skill sets I think in addition to just looking at I mean there's a lot of information that you can find when you when you're reading like uh finding descriptions and kind of gathering like uh like how things could be exploited um another thing I generally look out for is just um again system level recommendations around um kind of whether the reviewer has kind of thought about the system is a whole water potential exploits that might occur when like the protocol interacts with another external third party and kind of um looking being able to kind of holistically look at the system um in addition to the findings themselves yeah and I think like another um just like follow-on point is uh you know if you're a developer and let's say you're getting audit and you don't know if it's good audit or not um a really good way to tell is like what questions are the auditing firm asking you and are they good questions because I think um you can tell a lot if you you know like let's say you give your code base to auditing firm and then they like disappear for two weeks and then they come back and they give you a report most likely that's not going to be a great reporter and the reason is you're always going to understand your project better than the auditing firm and so if the people who work at the auditing firm really care about doing this right they're going to be asking you questions like you know every day every other day and they're going to be like good questions that are going to make you think a little bit before you answer uh you know and so that's like a really good sign and then you know having a very well written report that's like thoughtful and not just Miss ranking some things as more serious right so yeah I guess to add on it's really about communication so um how much has the auditor communicated with you uh during the audit but of course that's doing the audit right so if you want to find out like you've never engaged that auditor before um you can number one read past reports that they've written and try to get a sense of the vulnerabilities found that were found right is it something that could be picked up with static analysis only or do they like really try the best to explain maybe they try to explain the system uh the business logic in the report or maybe they could say explain like what the centralization risks are right so um I think that there's wait it could one way to measure is the comprehensiveness of the report of past reports that they've written yeah so there are many ways to measure the uh an Auditor's like ability in that sense uh what hiccup said uh I think on a high level as a non-technical person um the breadth and depth is what I look for so for sure I think traditional Audits and all these other folks on on the stage has a complementary approach towards C4 as much as I like C4 I don't think it should be the non b or when it comes to security and I think having objectivity with working with different firms different approaches to make sure that you basically cover a depth of scope that would be a signal of like a company taking security seriously rather than just going for one option yeah yeah so on this line I think like Community style audits are a bit different so I like to ask this like have you noticed like different sort of quality or do you measure quality within like let's say code Arena audits does that make sense like they um during different audits of code Arena because it's always open to different people uh have you noticed like difference uh audit quality as a result uh like the final report yeah so we have maybe over 3 000 registered wardens which are what we call Auditors now and what I know so by speaking to wardens like hiccup is that different people show up for different competitions and this can be both based on let's say availability they have full-time jobs perhaps also based on interest so some of the Auditors the most senior ones they are like I don't want to do nft stuff for example they only want to do novel map stuff so you have different sets of people showing up for different competition and that problem kind of takes care of itself partly because of the incentive design as well as when we right now we have about 80 to 100 Auditors per competition so when you spread out against across that kind of number that quality itself has been taken care of in like in so far cool and um an earlier threat that we sort of touch upon is the difficulty of fixed reviews right like fixed reviews uh like pickup that said like it was extremely uh difficult because if you're only looking at like a specific code at one scope you might miss like systematic bugs so uh I was wondering like how everyone uh like you know deals with uh diff audits or like yeah these kind of stuff so any takers thank you go for it okay yeah sure so with div uh audits I think there is definitely an advantage if you've audited that particular code base before so like maybe a V1 right and they're adding something on top of V1 so having that prior knowledge definitely gives you an advantage and it kind of allows you already to have like a good understanding of the whole system so then that the chances of you missing like uh with more context it allows you to find like that specific business uh specific bar right um as opposed to somebody like a fresh pair of eyes that has never seen your code list before and they're just looking solely at that uh uh particular part of your code base right um so there's definitely an advantage if you have uh work at least work with uh the entire code base in that sense yeah I think specifically for uh fixed reviews another thing that the the problem that I think arises from um this like very small scope um of code that you're looking at in terms of like um fixes is similar to a problem that arises with um like very limited coverage audits as well um because with both in both cases you essentially need to isolate a very small subset of your system and be able to test it or check it based on those assumptions and I think with fixed reviews um as like a security researcher you also have to keep in mind um that you're not necessarily like you're reviewing the changes with respect to like that specific scope and not necessarily potential Upstream changes or Downstream changes so I think being very specific about what conditions um you've actually like tested and checked those fixes is really important um as well as just making sure that like those any assumptions that you're making um are as explicit as possible so on a technical level one of the things that helps other thing you know fixed reviews is ASD diffing tools so if there's like you know changes to the you know spacing or you know the formatting of the code that that sort of stuff it doesn't get picked out so that's one thing A variation of fixed reviews sometimes a lot of people you know come to us and say well we're forking this protocol and we're working in this way right now the way that we address you know audits for those is you know sometimes or you know half of the times we avoid them right unless we've actually audited the original protocol and in that case you know we would go to the same try to get the same team of Auditors of that protocol throughout the the fork now there's exceptions to that you know there's some other protocols that we're you know intimately have knowledge of you know like things like curve up and things like that so for those kinds of things we also accept to to do cool um so kind of moving forward um I think like in the beginning we said like oh you know when a project has like oh who's responsible is that user user well that's actually something you know maybe an interesting topic because when this is happening like we never talk about like the user responsibility and all that and so like one aspect maybe is um what does like uh what what can the user do uh to kind of like improve the security of the space not necessarily the project or uh you know like the auditing process but like just a normal user how how can they uh contribute to security revoke approvals probably like don't open to so many things because uh you know start starting from D5 summer I think basically all the like animals that exist like in the world have been used up and most of the food groups food things um and so like a lot of those I feel like you know if you're not a security expert and um you just you see uh you know uh you know D5 protocol like probably don't put your money in there even if they're promising like 100 apy um and uh that would actually like help a lot with the Dynamics I think in the current bear Market that sort of like happened in a very like natural selection manner because um you know recently I think most of the food group uh type of projects have kind of gone to zero so you know just like next bull market like don't don't do the same thing even if now they've switched from animals and food to some other naming convention um I have to agree so use a bit of common sense don't put 100 into one project and especially if it's a very new project so I do recommend spreading your funds uh whether it's across multiple wallets or into multi or putting into multi-pro multiple projects right um and also I would say if you are able to for the more technical people you can take a look at the audit report to see what kind of vulnerabilities are found so then you have like a sort of like a slightly better understanding of uh the development process that was like spread your funds across different food projects right yeah um yeah I mean I don't know like I I hope one day this when token meme can become when audit or when security I doubt we will see that um but let's see I think uh beyond the whole private key stuff uh for sure getting a basic understanding of like Audits and what what they do how they relate um in a web tree space when the table sticks are so high is critical um as an end user again I think these days if I'm interested in any project I would definitely be asking questions about how they are approaching security and what what measures they have taken um that's for me is a very clear sign of them putting putting their money where their mouth is verified before you sign um I think that's well one one aspect um the other one I would actually argue that when you for audit reports you don't necessarily need to be technical to actually read them I think if you just go into like for example like a detailed finding you can actually skip right to like an exploit scenario so that you can actually see how some how can something be exploited without actually understanding what your source code does um I think is is one thing that is useful as well cool and um on the perspective of moving forward what kind of changes would you like to foresee in the auditing industry like let's say the next two years or so better tooling um putting more time into like reviewing kind of pausing or not necessarily developing as quickly putting more time into making sure that static analysis is integrated in all of your code so that you're actually checking and that you can essentially find bugs while you're developing instead of when you wait instead of waiting for an audit and better unit testing and fuzzing as well so as an auditor I'd argue like to understand to put it back into the development process you know when you're developing the protocol you're like well if I develop the protocol you know in this way if I develop this algorithm in this way will it be auditable right so there's a few tricks you can you can do to make the protocol more audit depend you know like for instance you know you know naming the variables in appropriate ways removing certain features which are going to be extremely hard to wallet um you know comments invariants like adding for instance checks that all right they might waste a little bit of gas but uh you know with then the auditor is assured that you know certain execution will not happen so then the complexity in your mind when you're looking at the Smart contract is reduced and a host of other things like that so um yeah I think just to add to that like with the rise of l2s one thing I hope to see a bit less is uh these like crazy gas optimizations so for some of the projects we've done it's like gotten maybe like four times harder because uh the developer just happens to be extremely good at optimizing for gas and writing an assembly and so um you know I think like with uh you know more l2s taken off that's gonna have less of a priority for the projects and like focusing on understanding and like often we are all we also see like the internal team not everyone on the like the projects team really like fully fully understands like the project of their architectural decisions and like um they might have somebody that's just like trying to optimize for gas but other people don't understand what they're doing um so and that's like really hard when they bring that to an auditor um so I I hope like we'll see I think we'll see less of that as like the you know l2's help with scaling yeah I would like to add with the today so one uh contest like opens open sea right they they were working on C plot uh with regards to writing assembly code I think one nice practice that they did was to provide a reference contract that wasn't written in uh assembly so it's like a side-by-side comparison one version was in assembly the other was in normal like non-assembe code so you could do like a line by line comparison and from there you kind of have an expectation versus reality comparison right so I hope that's something that uh developers who are writing in assembly would adult I think the other thing that another security practice that I would like to see more adoption of is the hiring of say Security Consultants so the idea is to get a security-minded person uh onboarded uh be just even before you start when you are from from the very start of the project so for example like these technical people like maybe you're new to solidity right so you are unaware you unaware of the best practices you aren't aware of what libraries could be used or you aren't sure what existing code base you could say you know in a sense copy from right um so it's good to have somebody who is already familiar with these libraries uh to kind of advise you uh to the best path forward so like you don't really want to write your own erc20 from scratch right you could just use um Soul me or ERC or open sequence version for instance um to add on what that said so probably better tooling and standardization um also I guess everyone's talking about AI so it remains to be seen what the impact but I'm sure it'll be more pronounced over time yeah yeah just one thing to add around having um with like gas optimizations and having the reference implementation um gives developers just this amazing breadth of opportunity where you can go through with manual review and potentially compare two different functional implementations um then there's also the possibility of using Code analysis and and something like program analysis in order to compare output of like a high level reference in solidity versus um whether the gas optimized version actually behaves in the same way um and kind of exploring that area in more depth is an interesting area as well because it becomes very clear where functions might deviate awesome um so now I'm going to select two questions from Discord let me see one sec cool um so this is from Anton can you share some projects you remember specifically that have good pattern test coding Styles so you we can learn from what it's called like clean and secure yeah can you share some projects that you remember specifically that has good pattern test or coding style so that you know the devs can learn and what is clean and secure oh I can I think compound is great uh so um it's yeah it's you know um really clean like battle tested there's not really any extraneous stuff in there um well commented uh every single update they do it also goes through governance and there's a whole bunch of reviews which is probably as good as you can get that um and then also like because it's a mature project the pace of updates is very slow and so um as as far as like kind of a security person goes that's about as good as you can get it you know apart from like you know swap so so one project that impressed me in that aspect you know some of the chain link products so you know sometimes we get comments like you know this you might that this is secure because of that or that you know like they really make sure that the project is well auditable and that the Auditors don't waste their time to um you know like figure out what's going on um as mentioned C Port because of how they provided their reference implementation together with the guest optimized version so that's one project that stood out I think the other is actually unisoft V3 uh I felt that their code base was rather clean they had a very comprehensive test Suite uh then there was one more okay so recently I heard of primitive Finance I think they had some kind of sick tooling that was used I saw a tweets tell me I don't really remember the specifics it's something I have to look into yeah yeah I would just say primitive Finance as well I think their first version was um one of those they they integrated Echidna I mean that integration was incredibly smooth because of the way that because of the fact that they had thought about all of their invariants beforehand um so that by the time um and by the time that they actually got into the opportunity to implement them it was very clear what expected Behavior was and what wasn't and so that's one of the most important things when you're writing in variants and having that list beforehand of just what's expected and what's not is going to be essentially the one of the most important things to add something about primitive Finance so you know those you know the Developers evolved quite a bit so we look at the latest code it is you know exactly as you say we can understand that they actually got a very big shock at some point you know in primitive V1 when we had to find a you know exploitable vulnerability as white hat hackers and you know those you know three to four days in a war room you know figuring out how to you know get out of the situation you know made a difference in their you know security uh the following question is from Shanti for a newly established projects maybe from hackathon without a lot of funding to engage audit from what methods would you recommend to increase its security use open source tooling to secure your software trailer bits has a lot of external and open source tools that are available out of the box slithers provide static analysis and that's something that you can run on pretty much any smart contract as long as it's evm based and just makes making sure that you're continuing to use these tools that are readily available um as long as you're updating your code and even afterwards like uh try to not over engineer your system you know keep that as simple as possible uh and that'll help tremendously and then um you know uh yeah I think that's my main tip um I think keeping document having documentation early it doesn't have to be in the form of like a documentation portal immediately right but uh I think as long as you have inline comments that are updated uh that really goes a long way to people who will be reading your code so if you use battle tested code already you know like standard libraries no open Zeppelin libraries they're pretty good for instance and things like that um you know don't over optimize don't try to be you know super smart with your first implementation reduce the number of features and then of course apart from the um the open source you know static analyzers there's also like uh you know cheap versions as well like not open source yeah um I think working with good small independent Auditors could be a good start because they tend to be in affordable range uh also possibly considered a route of getting grants because I know ecosystems are also offering grants to small projects especially when it comes to I mean most ecosystems are interested in securing their the security of the ecosystem they don't want to get hacked right so um that could be a way to go uh I personally just want to add tests uh like also understanding the quality of your test uh because test coverage is one thing recently there are some other tools called like mutation testing that one is pretty cool like lets you basically it changes uh like your code a little bit and to see whether your test kills uh the the mutate the mutant and that actually tells you like how good your test is because in theory it should kill every mutant so yeah cool um let's uh perhaps go through one last questions um when you're about to start auditing contracts uh what do you most appreciate the project team doing and what do you least appreciate foreign a little bit things that we that were said earlier um so mostly not appreciative the if the project is really not ready to be audited um so one thing to keep in mind is well it's like if I had to point a gun to your head and say well is it ready to be deployed tomorrow are you sure it's going to break you know the first time it's going to be used so you know functionally that it works you know it's well documented when we see projects like that we appreciate it when we see projects that don't build for instance that's something that is very problematic uh one thing I least appreciate is projects just giving me the contracts only no tests nothing just contracts for the you know you're supposed to do the setup yourself and you know there's nothing to run with now so yeah that's not the way to go I think um one of the the misconceptions with audits is again that it is just a green check mark that you can just deploy right afterwards I think in when you're going through and on it and especially when you're kind of interfacing with reviewers the most important mindset is to be able to talk about your code openly and be able to take in that feedback uh because the feedback is meant for you to essentially improve your code if you in if your project or if your your development team is in a state where you're not ready for that feedback um then the audit it may not be as productive as it could have been um and in terms of what helps and and kind of what makes auditing kind of more efficient is really I think documentation and kind of as much context about the system as um you would give like a developer that you're onboarding into your pla into your ecosystem um so that from an auditing perspective at least we know what is how the protocol is expected to work in the first place um yeah and just to add to that like yeah I think uh communication like um you know well I think you know the auditor is like where we care about your project and so like uh if we call you up and we're like hey probably don't deploy that yet like maybe take that a bit seriously and you know whether that's like myself or like that you know the security researcher that you're working with um and then I would say yeah like during the audits um you know if we say hey look like you could use a few more tests here or like inline comments uh probably is not a good thing to just respond with well like the code is the documentation or whatever it's like endless times over the last like five and a half years um so yeah you know like often I think if it's your first time doing a project um you might feel like you got you got everything like pretty like all got all your ducks in a row um but the the bar for having a project that is not hacked is actually really high and if you look at most D5 projects that have been around for a couple of years there's very few that have not had an incident and so if you just think about like probabilistically um you know when you start a project like you really want to try to like reduce the chance of that as as much as possible right so um I'm actually surprised at the consensus or the answers I've given so communication is definitely a big part um what uh what Richard and that said I think openness to receiving feedback and generally responsiveness um for me is a big signal as to how interested the protocol is in the security least appreciative fact is probably someone who just wants to come in get something done the quickest for the least amount of money and they don't ask any questions like Richard said generally the people who are super responsive they also tend to be the one who are really on point during onboarding they do the right documentation they're responding to us responding to the wardens to the community so yeah that would be it all right uh with that I'd like to conclude our security panel and so uh thank you panelists thank you Natalie Neville Richard uh Hiccup and trivia thank you [Applause] [Music] thank you [Music]