Rethinking Security Auditing for Web3 - Spencer Macdonald

hello hello hey everyone i'm i'm spencer mcdonald i'm going to be doing a quick presentation on the current state of auditing in web 3 of security auditing and then some potential future trends that we're going to explore i'm not an expert i'm a generalist just going to be kind of going over the the overall business model evolution that we're seeing so hopefully you find this talk useful okay so we're going to talk about uh you know a couple different things first we're going to start off with some some common definitions of security auditing in web 3. then we're going to get into some trends uh that are emerging in the ecosystem uh and then we're gonna introduce uh the project that myself and some friends are working on uh spear bit okay so about myself again i'm spencer co-founder of spearbit previously was a devsecops product manager at a government cloud agency now spearbit is a freelance marketplace of vetted security researchers who pair on collaborative teams to conduct security reviews with clients and we'll we'll get into that in a little bit okay but first let's talk about web3 security auditing so so what is it um well the term itself is actually pretty controversial okay um some folks in the industry have shifted to using the the term security review actually instead of security audit because audit sometimes can can give the implication of a one-time stamp of approval whereas review you know kind of implies a more collaborative and continuous process for the purposes of this talk though we're going to use the term audit just because it's still commonly used in the industry so what is an audit well rajiv from securiamg goes a really good definition it's an external security assessment of a project's code base typically requested and paid for by the project team so it's not usually paid for by the users of a project so the incentives are you know normally aligned between auditor and project but you know maybe that'll change in the future um so how a security audit works you know a project uh signs with an auditor the auditor you know understands the architecture of the project then goes through does you know automated code scans maybe manual code review maybe formal verification and then normally across the audit industry the end result is a is a formal audit report and and this isn't going away but what we're seeing is is really a shift towards a more collaborative model where you know the report still exists really for marketing purposes and to give confidence to the project's end users but really what we're trying to change is two things so number one what matters isn't necessarily the report it's whether the vulnerabilities that are identified in the audit are actually mitigated by the dev team and then number two which is even more important over a longer term time scale is actually you know changing the developer team's security software development process into a more collaborative and continuous approach with with external security researchers okay so um we've defined you know what a security audit is now let's talk about how web 3 potentially compares to to other industries so it's interesting rex from d5 safety has brought this up before and i think it's an apt metaphor um you know smart contracts have a similar threat model really to to aerospace engineering uh in terms of you know both with a plane and with a smart contract uh sometimes it can be harder to upgrade it in flight however the difference between aerospace and and crypto is that the economic incentive to ship uh is much more rapid and iterative and crypto so it's more similar to you know what we see in a web 2 product team whether it's at amazon or google or you know a large devsecops product organization um so specifically upgradeability you know can add additional complexibility complexity since upgrade control is a potential threat vector meaning like who controls the upgrade uh is a potential risk um but again this pressure to ship fast and iterate quickly is similar to this devsec ops model we see in web 2. um so devsecops can be defined at least in the web 2 concept kind of way as the ability for a software developer to push code through an automated cicd pipeline into a common deployment environment and then at every aspect of that process you know being integrated with with organic security researchers so this is something that we need to get to in web 3. the difference though is that we may not be able to get to this continuous delivery of you know dev teams being able to ship every single day as is common at like a google or netflix uh but we definitely can can get to you know a sustainable and safe uh development cadence and then finally incident response is possible in web3 it's a lot easier in in webtoon you know depending on you know how upgradable the smart contract is but in any case uh you know web3 teams need to to front load security confidence early in the process okay so speaking of that um you know we defined what a security audit is how it compares to web 2 in terms of the threat model um when do you want to get an audit well hopefully as early in your development lifecycle as possible um and and really this is best achieved by by having a retainer deal with a with an auditing firm so speaking of auditing firms the top firms are excellent and they they offer great opportunities both for their clients as well as the security researchers that that work for them however you know talent and audit availability at scale is a systematic bottleneck that we're seeing in our industry so in talking to people around the ecosystem it seems like the the average wait time is around six to eight months to get an audit so again we may not be you know netflix we're not shipping every day but we definitely in web 3 want to ship new features uh faster than every six months right so that's that's a bottleneck that's kind of holding up uh the security and also scalability of our ecosystem and this is because as is in traditional infosec uh the barrier of entry to crate value is much lower than the barrier of entry to to protect value okay so basic definitions uh on security auditing um talked about you know how to get an audit now let's talk about where the market's going to to kind of fix these bottlenecks um so like the rest of crypto um security auditing is trending towards these community driven approaches to fix these bottlenecks so three trends uh in particular so number one uh talent upskilling so growing the pie of folks that are qualified and trained to to help teams out with their security uh trend number two is firms servicing dows so uh security firm is cutting continuous retainer deals in an open form uh with with dowse and then trend three is is dao's servicing daos uh these community driven uh talent marketplaces that we'll talk about okay so trend number one uh honestly probably the most important trend which is talent right at the end of the day all web threes is people uh and we need more trained and people uh to help ensure the security of web3 so right now like an estimate there's probably only a couple hundred at best security auditors on the planet that work for the centralized firms and then maybe there's a couple hundred other freelancers that have the requisite skill set of both uh you know software development knowledge as well as infosec as well as blockchain knowledge uh to freelance on on audits so we need to grow the pie and rajiv at securium is doing just that so rajiv founded securium it's a it's a web 3 security engineering boot camp it puts people through a rigorous curriculum and then after people graduate then they can get opportunities either at traditional security firms or at other web3 security organizations and then another uh community driven resource which is pretty cool uh is the east security telegram chat uh so that has a lot of people in it uh and it's a place where juniors you know people just getting into the security industry can really ping uh some of the ogs uh in the blockchain security space um so really cool place you should check it out okay so first we talked about you know growing the pie you know getting more talent in the space now we're going to talk about growing the pie in terms of you know over a time scale for for dows to get a continuous security deal and this is happening uh in open governance forms where security firms are bidding on these huge multi-million dollar contracts uh to get continuous security contracts with dao so an example is a munify uh with makerdale where aminify and d5 security uh want a a contract to give you know continuous security services as well as incident response uh to uh to maker and then the second one which was uh reported on in the media uh was was compounds audit rfp uh so uh long story short uh compound uh launched an rfp late last year uh for for security audits that open zeppelin one uh open zeppelin is being paid a million dollars worth of of compound tokens per quarter so four million dollars worth of comp uh yearly um and this was a very nascent development right um like the compound rfp was only opened up uh after an initial you know governance vote with one firm was uh was put on on the forms you know most people understand that you know selling the dows especially in an open governance form is complex you know there's limited accountability in the counterparty um it's complex it's a complex agreement and it's a tough sales challenge that we expect firms are going to have to adapt and grow into dealing with okay so we talked about firms servicing dows now let's talk about the next logical evolution of that which is obviously dao's servicing dows so specifically community driven market marketplaces are supplementing the life cycle of existing centralized firms okay so code arena is a great example uh it's an open audit marketplace where projects post their code publicly and then freelance security researchers are able to uh you know win bounties when they find vulnerabilities in the code and then code arena recently became a dow and launched its its governance process okay so we have these public uh competitive dows uh in these these community marketplaces that are public in addition to that these private service dows are popping up um so some of them are like raid guild do an amazing job radio is you know doing great work in web 3 web development another example is is brain trust which is a dow that provides web 2 freelancers to to web 2 corporations like adidas and even nasa and then another example is spearbit which we started to extend this private services marketplace model to web three security audits we'll talk about that in a second okay so hopefully you guys got something out of the the three trends we've identified um now we'll talk about you know what specifically spear bits trying to solve uh and and you know looking forward to your feedback uh at the end um so as we discussed there is this bottleneck it takes six to eight months uh to get a security audit and we feel that the best way to scale this is through uh a progressively decentralized security audit dow uh so from the security researcher perspective there's a lot of freelancers out there who could potentially get into you know helping projects out and improve the the project security uh but they need access to deals they don't necessarily want to spend their time you know trying to get a a deal they need liability protection um so no one's been sued yet uh in this industry but um you know given that there were a billion dollars worth of hacks in 2021 um maybe in the future that'll change so you know security researchers need liability protection and they also want the ability to to collaborate uh with other other researchers so we've talked to some of the best blockchain security folks on the planet um and even the best people you know want a secondary look at at their work um and then on the project side obviously uh projects want to get a high quality audit they want to get it in a in a quicker time scale than what currently exists in the industry okay so um the problems we're trying to solve so one is is freelance talent management right so in these dows um you know growing talent is really important right so in the spirit model we accomplished that through pairing so we pair again some of the best blockchain security freelancers on the planet with folks more at the intermediate level or the uh or the apprentice level and the goal is for you know new people to to learn from from the experts collaboratively a second thing that we're trying to solve for is how to ensure that black hats don't penetrate the dow right so you know a failure mode could be uh you know a black hat you know gets on an audit uh you know keeps the vulnerability for themself and then exploits a client that's something that is a no-go situation for us so we're trying to avoid that uh and our mitigation for that is a is one it's a three we have a three-step onboarding process where we you know do a couple interviews with folks and then we currently we're kyc and our security researchers in the future we're looking into forms of decentralized identity like violet that allow us to know whether someone's oh fat compliant or not without actually knowing their identity so we'll be able to onboard and on auditors while still being you know legally compliant and the third thing uh that's an important challenge for us to solve is auditor retention so you know security researchers are in high demand right a lot of them burn out um which is a bad thing obviously for the ecosystem so what we're trying to do is offer folks you know flexibility um they can you know hop on an audit when they want you know where they want and if they don't want to do anything for another six months that's okay right so come as you go model okay um a couple more challenges so you know as we decentralize and and kind of open the community you know how do we maintain quality control well interestingly brain trust has had some initial efforts at this where they have a kind of service node model where they have service providers to their dow that accomplish certain functions as an example project management they have you know legal entities that are contracted to make sure that the service contracts run on time uh and and same thing for you know service quality so spirit's taking on a a similar model uh and um you know again trying to get the same level of quality that you would get from a centralized firm but from from a freelance team and then speaking of centralized firms uh we see this this ecosystem emerging in a complementary manner uh centralized firms are going away they do they do a they do a great job so a talent marketplace whether it's code arena whether it's spearbit can complement these firms because it can give them access to freelancers that maybe they wouldn't have access to in the first place and then finally um we're thinking through how we do governance in a way that really incentivizes security expertise um that's that's really important for this particular use case of the the dow model and then we're also trying to incentivize auditors uh for audit completion for mentoring you know folks in our community and that's contributing overall uh so yeah um thanks a lot for listening hopefully everyone got something out of this um if you have any questions comments uh shoot us an email hop on our discord and then we're also hiring some folks so yeah let me know if you have any questions thanks on [Applause] and then the final thing i wanted to say is um the ethereum security community may be small but it's really uh tight-knit uh and i just wanted to thank the following folks on this uh on this slide that gave me feedback um i really appreciate it and we're all growing together so it means a lot thanks any questions it's a great question so my question so my question was how does spear bit think about uh methodology in the in the space so like is it would you require auditors that use your platform to all do fuzzing to look for a certain bug class uh how do you think about those types of issues or is it more like laissez-faire yeah it's a great question so our current approach is you know we have you know a small community thus far of freelance experts that are kind of defining that right so with the audits we've done thus far you know every audit's included uh you know automated code scanning static analysis and the like it's included a manual code review um and i think you know in terms of the standard it's really something that our community is going to define specifically our community of security experts as we move forward um and that's that's kind of potentially the the upside of this approach is that you know we could we could help define for the entire industry what that standard of an audit looks like so rajiv from securium's uh done this you know in an initial way so far he's you know in terms of his course where he's listed out you know the specifications for an audit and i think we have the opportunity to to build on his work there other questions go for it how do you see roles a role of bounties instead of audits as a way of ensuring uh security of protocol i think they're complementary um i think immunify's doing great things with with bounties it's it's all about audits and bounties are just different parts of the software development life cycle right so audits are bounty is our post release and yeah i think good projects want both right you want to really engage in a continuous way with an auditor but then you also want to do a bounty after you're live on mainnet thanks everyone