Into the Dark Forest A Discussion on the State of Security in Ethereum Liz Daldalian mod; Goncalo S

welcome everyone to a discussion on the state of security and ethereum everyone's favorite topic welcome to our panelists um so before we get started with the questions i'm going to give everybody on the panel the opportunity to introduce themselves and tell you what's more the most important thing about the work that they do and then we'll dive right in so gonzalo do you want to get us started sure absolutely hi guys uh i'm one of the co-founders of the consensus diligence team being a hacker since i can remember stuff and i'm still hacking and still securing the ethereum ecosystem hi i'm rex highgate from d5 safety we are an independent organization that rates d5 products both contracts and um protocols and you can check out our stuff on d50.com and my background is in aerospace aerospace software where they've got uh quality is a big thing because uh it's bad if it doesn't work so yeah that and i tried to contribute that to ethereum hi i'm liz steininger uh the ceo of leased authority we build more secure products ourselves but mostly what we do within this space is security audits so we help with security consulting and supporting projects to become more secure and yeah basically improve their improve their products hi i'm mike calvinis uh co-founder of brink uh we build an automation protocol um basically enables uh users to execute uh things like limit orders and stop loss orders uh in a completely defined native way um and my background is in smart contract development uh smart contract security auditing uh prior to that i was a web 2 developer for a long time um yeah awesome welcome everyone and thanks everyone for being here so the first question is for everyone on the stage and um it's really about security in general within this ecosystem so we all know that security is a very hot topic and a controversial topic within the space and within the ecosystem so it'd be nice to hear from each of you about the role that you individually or your organization or both play in helping to secure the ethereum ecosystem so let's start with that yeah absolutely so diligence itself started very organically within consensus we were doing pro bono to begin with um the first example of which might have been an ll smart contract vns and our mission has always been to do work that is important for the ecosystem so you might have seen our extensions for vs code around so if you develop solidity you you might have used one of our extensions so visual developer or something like that and we try to align um our mission with the work that we do with clients so we select our clients so that uh um we we try to get as much out of out of the work that we do to raise ethereum as a whole as a community so um with d5 safety our customers are the users of d5 not the protocols which is a big difference between us and auditors and we work to give a clear transparent and objective rating for the d5 products sometimes it's the protocol as a whole sometimes it's a specific contract like a curve pool that you're investing in so that's our main area and that's how we want to contribute um yeah with uh with least authority yeah we do security audits and for us it's really important to see the success happen in the general you know the broader web 3 space and ethereum especially to to help improve the security of the tools that are out in the space so you know it's important to get independent reviews so um like consensus diligence this is something our kind of service to the space which is we're here uh we can help you become improve the security and also we think that this is very important for the users to be aware that security audits are happening to the protocols and to the projects smart contracts everything involved and one of the things that we also try to encourage to give to the space is this idea that the knowledge should be open and transparent and so if a project gets an audit we really encourage them to publish the report and we have pretty good rates on that i think it's something about like two-thirds and so that's really important and i know they're not the easiest thing to read security auto reports and so that's why it is nice to know that you know there's other services within the space that are helping to make the information about security a little bit more digestible but um yeah we hope that that also helps the space by just generally educating people about security and facilitating the conversations about vulnerabilities so that the information isn't just behind closed doors and everybody benefits so brink as as protocol developers we obviously have like a responsibility to our users and um we uh go through like certain processes kind of balancing like a speed of iteration and deployment with proper security practices um we are we are almost deployed on mainnet now so we're we're pretty uh pretty new protocol um we've been very thorough about security you know we're the expectation when we do deploy is that uh users will be trading at pretty high volumes um obviously these protocols scale to like any any volume so um we we have a responsibility saying here's a product to use with your your value and your money um and we try to follow best security practices of course multiple audits we release our audit reports publicly too of course i might be killing time in the panel but is that subjective or is that objective how do you guys like balance the the how fast we ship versus how much we test and secure code like i said do you have like protocols or is that like yeah i mean for a small team like ours like it is somewhat subjective like we don't have like documentation that says this is exactly what we're doing or any formal processes um so it's you know somewhat of like a gut feel from us uh but we definitely prioritize like the security side of things but of course we want to ship product as well so yeah i think we strike a good balance i think that's actually an interesting question in general about security which is yeah how do products know that they've done enough and um there's this there's a stain and i'm going to probably mess up the quote and i don't remember who said it but about security in general that uh you know you don't have to be the most secure on the house on the block or the most secure car on the block but you just have to be more secure than the ones around you um and so yeah i don't know there's no such thing as 100 secure and so it's really about like just making that you know judgment and risk management and stuff like that so yeah it is a hard problem for projects disincentivized yes yeah good point um okay this question is for liz of lease authority so leased authority works on a wide diversity of projects and code bases is there anything you'd like all projects that lease authority has been exposed to through their work to prepare prior to an audit to help bring the most value to those projects the product itself the development team but also to the users yeah there's a lot you can do to prepare yeah some of it's going to be you know boring advice like create tests for everything have good documentation put comments in your code you know have a solid roadmap for planning that you you know you can work against and that you can communicate with auditors about new features and stuff like that but just this idea of like really even thinking about security up front too can really can really help projects prepare but um it's helpful to think of it as like we're yeah we're a service provider and we're here to help you but you also want to make the best use of our time so if there's something that you can do yourselves then you should do it and then like the code comments and tests like that and just have good high quality you know work and sometimes it does take a little bit of extra time and effort and i know that that's really challenging in the space because things are moving fast but it's that you know but it's also a space where if you aren't secure and something bad happens there's a high cost to it too so um it's it's difficult to find that balance but it's really important to to do as much as you can to prepare and also in thinking about security up front there's like cost savings in that regard as well right yeah the last thing you want to hear from us which is never fun is for us to say that you need to fundamentally change something in your design because um security wasn't like thought about at that stage so yeah the whole concept of security by design and and keeping that in your mind while you're working on your on your project is really important so that you don't have some bad news later that it needs to be fixed differently awesome thank you um the next question is for rex a couple of questions i'm going to start with one and then i'll go to the other so could you give us a brief overview of defy safety's products so we have two products that are out now the main product that we've been doing since the start of d5 summer are process quality reviews and this is where we read the audit we check is it is the code documented uh did they run their tests did they do a test report we would love people to do test reports you do all the work on the test have a nice report to show it off afterwards please but um and uh that comes down to a simple 0-100 score and you can dig down to the details of that and then you can dig down to the detailed report if you want to so always transparent we show our process we show our work the second new product that we have is contract scores so this is an actual score on the smart contract address so we'll have uni swap pools we'll have curve pools urine pools any address that you would interface with we'll interface with that and we've got we're still starting we've got about 2 400 addresses on five chains for that thank you based on your experience what can protocols do to improve security so when we look at some of the hacks that happen we look at hacks differently we look at them from a process quality point of view what did they not do on the process and two in the past year sort of shown out the the cream amp hack and the compound hack where they had the greater than rather than greater than equal in both of them and these are big organizations they've got lots of money they've got a healthy treasury but they only had like uh they didn't have enough eyes on the changes so comp should have better reviewed the the token contract to see oh wait this is different um but they didn't they added a different token contract into a contract that worked perfectly and they created a hole and compound they did a they had a really good software engineer who did a good job but he he did everything he wrote the code he wrote the tests he deployed it to testnet and not enough eyes on the thing and they missed uh a minor detail and they lost a bunch of money so the big thing i would say is if you especially after you've got some money get the standard software process people in quality people testing people and jesse you've got more eyes on your code changes thank you very much gonzalo the next questions are for you diligence builds a lot of automated vulnerability tools do you think that these tools are going to play a significant role in the protection of the ecosystem in the long run yeah absolutely yeah who of you are writing code right now you are okay who of you is actually using any static analyzer or dynamic analysis tools you are okay great not okay that's great actually yeah this is this is a surprising response um so we're not at the stage yet where tools can can replace a human brain for sure i don't know if we'll ever get there i'm sure that we we will improve our methods enough that they'll replace part of the human effort of auditing uh but they are a very very nice compliment and they they should come first as well right so picking up on what liz was saying before um within the realm of things that you can do to prepare for an audit part of it is to actually test your security before you hand it over to people that are that you're paying right to to look for bugs um i again i do think that we're still not in a sweet spot of like what these things can do we're trying really really hard both on the implementation side of diligence and on the research side so like we're specifically on the fuzzing side where we're doing a lot of like important research to to better the tools but they're not a panacea is that how you spell it panacea thank you where do all these tools belong the development life cycle of a product um yeah so again picking up on the on on liz's remarks they should come so the whole idea that i've been pitching uh to our clients at diligence is that you should shift security left right shifting security left meaning in the in the timeline of things you should start using tools thinking about security way earlier in the development life cycle of your product right so if you're building something and you think i should just like start writing some code and see how it works maybe not you're just like just use the tools it's not that hard so uh whenever you think that your code is not yet ready for the usage of security tools it is so yeah is it way earlier so it's shifting our mindset about when to integrate security into a project rather than it be a waterfall thing it should be integrated into the life cycle of development in general work in progress okay awesome mike your turn okay leading a product company in the ethereum space what are the measures you're taking to protect your code base from being exploited in a live setting um so of course getting external audits super important as you guys know um so we went through uh two audits we uh with abdk and spear bit um we actually did two separate audits uh with spear bit um and they uh interestingly also did a lot of like gas optimization in addition to security which is just a cool side note but yeah security audits important obviously but just code quality in general um for us like you know mentioning like where in the life cycle this has to happen like right from the start from the first line of code that we we wrote we're thinking about security we started with smart contract core and we build out from there um but every step of the way every code change we make we're doing kind of internal reviews um on the security of that we're making sure that like the code is super clean right from the start i think unit testing is super important not just like having tests but having very extensive easy to read um not necessarily like line coverage but just like use case coverage is important that's amazing yeah then when you hand it over to the auditors they they can uh they don't say bad things exactly um but yeah it's just it's an ongoing process as well uh every every step of the development life cycle we are we are thinking about security i saw that you guys are also on immunify yeah so yeah i should mention that as well we have a like pub public bug bounty program on immunify um and uh yeah in addition in addition to the audits yeah one of the things you just mentioned that maybe people who are less involved in the security space might not catch on to is that you're getting feedback on performance optimization like gas costs and that sort of thing and this is something i think auditing teams struggle with because our role is to comment on security but you're actually talking to very highly skilled engineers who just want to talk about improvement including security and sometimes it's a hard line to draw where to be like but we have something to say but maybe it's out of scope so these are sort of like the drawing the lines between security and optimization and sometimes those lines can be blurry and it's a it's a ongoing discussion always within security teams thank you um okay this last question is for everyone um and it's it's the final question so um to close it out can you tell us about some of your lessons learned and opportunities for improvement as it relates to security in the work that you you do or within the ecosystem um okay i can get started with this one uh there was one very big learning uh that we had at delhi so we were auditing 0x v2 and there was a glaring bug related to we see recover and and the way they were using signatures and how it would return like a a zero value in case it was it was inadequate the the parameters and i think that has shown us how important it is for us even like the people that are doing security to have some sort of meta perspective into what we do so like we went uh back to the drawing board we had a bunch of retros and we implemented those protocols in the way we tackle the audits themselves so after each audit now we do a retrospective we try to better the protocols of auditing code itself in at the very beginning i thought this would be very static that was i think that was our biggest learning i i thought auditing code and like hacking was a very static kind of thing that it would never change just hack away that is not exactly true so even us even the auditors are we need to audit ourselves um for for us one of the lessons that we've uh it's basically an ongoing lesson that we're learning over the years which is um that when we work with projects we go in with the attitude that we genuinely want to help and so we are a bit biased in that way and we try to see the best and like improving security in these projects and stuff like that but you know security isn't always easy like we were talking about like sometimes you can find out later in the project that um there was something about your design that you needed to fundamentally change or you know optimizing or whatever the quality there's a lot of work that can be involved and sometimes it's not realistic for for teams to be able to do that immediately and so we've run into some challenges where um we work with teams and we find you know these issues and we're all excited because we're like yeah we got you some information about how to make it more secure to fix it and then they're like we don't have time or that's too big we'll do that next year like that doesn't work on our roadmap and and then we run into this challenge where you know we're at this like odds where we want to even help and we can you know oftentimes consult and work through things with them and help them figure out you know a good mitigation path in the meantime or something but you know sometimes it doesn't work out and it's it's been something that we've learned throughout the years is how do we better incentivize the projects that we're working with and how do we support better um and yeah and it comes to like a lot of communication a lot of just like details and talking about the risk management and stuff like that and working with the projects and and then you know how do we keep ourselves in a position where we're constantly supporting the improvement of the security of the community also and not just these projects and not keep these you know vulnerabilities and stuff to ourselves and um yeah i you know last week there was some news that we we came out and we did a responsible disclosure of about um about a wallet and this was a really internally challenging thing for us as an auditor because you know we again we genuinely care to help things and to have to come out and say um you know a warning that something is not secure because you know we tried for months and we tried to like help and and then this is just a challenge and i think that you know i would really appreciate too that the larger community discusses problems like this about you know how do we educate each other about the vulnerabilities that we're finding along the way and how do we talk openly about like yeah what responsibility do different parties within the ecosystem have to each other and to the users and stuff and it was great backstage to you know before the panel to talk a little bit about like how we all can work together and you know how we can um and i just i want to see more of that but that's been a hard hard lesson um just over the time is that yeah while we might be super passionate about security and improving things that it's not always everybody else's priority and that and you know and i sympathize too i empathize that like projects struggle um to you know balance the the need of security by design and stuff like that along with yeah like delivering and and all that stuff and and yeah there's no easy answer to all this because it's just risk mitigation and stuff like that and risk management so it's just good for all of us to talk about it um yeah i would say um uh keep things as as simple as possible like from a developer protocol developer perspective try to keep things simple and when you be ready for the audit when you go into the audit um i think that's like yeah we sometimes have like gotten ahead of ourselves like development has moved a little faster than then security or you know we're used to like iterative cycles so we've we've had cases where we've built some tooling around um like smart contract code that we built and then realized later oh we have to change that that core because of uh various reasons not always security reasons but other reasons too and then that means that everything up upstream from the stack also has to be changed so we've we've run into that a few times in our cycle uh i think yeah just being like very cognizant of like the core that you're building and making sure that that is that is solid before you kind of move move beyond that um and then just like yeah writing as clean code as possible and being ready when you when you go into an audit you have something scheduled and you've got a finalized commit ready to go and uh yeah um or for us uh like we've iterated we're up to the eighth iteration of our process for the pqr reviews the big thing that we added the lesson learned was um access controls people should say like a lot of the code now is upgradeable and a lot of the people when they go in they think it's immutable it's a blockchain but it isn't really and the protocol people should be very honest in saying this is the control we have over the code this is how quickly we can change it so that was a big lesson learned that came in and it's a big part of our score now thank you thank you so much to everyone for sharing your insights into this uh never a dull day space and thank you everyone for attending so uh applause for the panelists [Applause]