Current State of Security Panel (Devcon4)

alright hey everybody welcome to the end of day 2 now for DEFCON thanks for sticking it out with us we're just gonna have a conversation about current state of security in aetherium we're gonna do about 25 20 25 minutes of questions here and then we'll take some questions from the crowd so first I guess maybe we'll introduce ourselves you guys want to go down the line I'm just saying who you are and what's your what you're doing alright I'm Martin Olsen spandy and I work as the security lead for the Theorem Foundation and also work at gas developments I'm Phil Diane I am at Cornell University and I see three I'm a PhD student working on smart contract security I'm Matthew D Ferrante I am the founder of Zika labs he theorem focused auditing firm and also cartographer I'd you know by implementation I do if he cryptography related things on the side I'm brilliant a founding member of consensus diligence and a smart contract auditor hey I'm Dan Guido the co-founder and CEO of trilobites software security research development firm awesome thanks guys so we're gonna start it off with a softball and what's the biggest change in aetherium security in the last year so I'll take it first yeah I think there's been a couple of things I've noticed the first is that the soluti compiler itself is a little bit less on fire today than it was a year ago there are certain patterns that we had implemented in our own static analysis tools that have been deprecated because the compiler now checks for those issues things like the constructor being the same name as the contract and having divergences of those two allowing you to exploit you know on an uninitialized contracts and variables within them another thing that I think is pretty unique is that a lot of the less serious people have dropped out of the community so the post ICO kind of field of people that are left now are a lot more serious about developing applications that work and I think that shift in culture Thanks that shift in culture has led to much better code and the last thing is I think that today we have substantially better tools so that if you care to write good code you can things from you know my company slither Manticore echidna things from everyone else on this panel of experts here have demonstrably improved the code that I've seen from our clients yes I was gonna say that a year ago when we were in Mexico there wasn't much of a security community and that's something which I feel is really taking off in the last maybe half a year and most of the people who were interested in in security came from the ethereum space and and really were ethereum and through the enthusiasts who were also interested in security and now I think we've seen a kind of large influx from traditional security into this space and that really changes things yeah to speak to what what Martin just said sort of recently I attended an academic conference USENIX 18 and they had actually an entire session on smart contract security which had I think around four talks that were aetherium related and that's a community that's completely outside of this space those are people who normally would not be working on these tools so definitely a lot of outside people have come in and we've also seen sort of the people who are already in the space start producing artifacts so we've seen the first set of formally verified contracts we've seen smoother processes for like engaging that and just a lot of improvement across the board I think kind of everything is a little bit less on fire than it was a year or two ago so yeah I think sort of echo what everyone else has said I think really like not only has the security community grown there's been massive increase awareness and security issues I think you know like most exchanges now require a security audit of some sort before they you know interact with products which is you know surprisingly shockingly that wasn't the case always right the like there's more awareness and generally that you know before you go live we do need a security audit there's like that there's your best best practices that need to be followed so like you know we're playing a bit less fast and loose and and more that more realizing that you know if you write a smart contract you know that may hold even if you don't intend it to many millions of dollars and you know that you don't want to be directly or indirectly responsible for a loss of that it's hard to go after all these guys and have something original to say so tools this it's been an explosion and I just wanna like call it like Zeppelin change security trail if it's really like coming on hardened last year is really like stepped up everybody's game up and then like Bernard and mithril within consensus are kind of going crazy you know good fun way and then I think like maybe the thing that I've heard more conversation about recently is like the awareness of incentives and how crypto economic mechanisms themself can actually contribute to security in like one way is just like automated deterministic non-subjective bounties that payout when you break the thing I think that's amazing cool so next one compared to securing traditional systems how do the economic properties of smart contracts change how you approach their security okay well so I mean like what I think is one way that we're tackling that is actually through actually creating like incentives for coordination within the security community so our Pam ballot project is a set of incentives that bring people together developers auditors to try to establish clarity around what is a safe contract to use and if you're interested in that we recently put a video of the process for for seeking consensus around that I'll tweet it after and we'd love to get your feedback on it and I we have got some already and I'm looking forward to more okay I think like one of the big distinctions is the irreversibility and the transparency of the smart contract environment versus the you know like traditional security domain kind of you can can't usually just read traditional security almost like insurance you know it's like often less costly to just fix the issue after the fact then to prepare for it or you know you have like safeguards or abilities that you just don't have in smart contracts where you know if somebody drains your contract I mean it's over there's there's no recourse so and I think like it's it's a lot of people say hey look you know smart contracts just never gonna work because look at all the exploits that have happened but I don't think it's a smart contract thing I think it's well of course it's it's a new space and it's a bit hard to have a good security framework in such an ass in space but I think this is a problem that's exists in all software and it's just far more clear in the smart contract space and and damages are reversible so it's more it's not that you know smart contracts are necessarily hard they're hard in a different way but I would say that the level of difficulty is similar to other domains it's just that it if you're bad if you don't do your due diligence you don't follow the right process there is no hiding behind you know like a firewall or closed guard and there's no I'll report it if I have to it's just there for the world and so all mistakes are very clear so it seems like there's more but you know like there's the banks have lost far more than some are contracts to the date to date yeah so I think for me economic sort of has a dual nature here in one way it's very helpful in securing these systems so something like just knowing the value of the system you're securing is not something that you can do in sort of classical systems like what is the value of a power plants Ethernet system or what is the value of an election system for a county or something like that so it makes some kinds of security analyses much easier on the other hand because a lot of the security assumptions we have to make to show that these systems are secure are themselves economic it also sort of brings security into this realm of economics which is much more subjective and nebulous we're like even to economists might not agree on basic definitions or basic principles so I think we're gonna see a lot of subjective security sort of develop as this space starts to differentiate itself from traditional systems and I kind of look forward to that also on a personal note the economic nature of these systems kind of keeps me up at night so I don't know about you guys but when I'm doing an audit I'm you know I'm worried about that a lot more than yeah it motivating exactly yeah that's kind of really what you should you should be awakened that because of that because I mean if you come from from traditional IT security you try and find vulnerabilities and you might find I know like cross-site scripting some SQL injection some data leakage you can take money even maybe sometimes from one account and do something but the stakes are so much higher here and it's like someone can randomly make a contract which all of a sudden has millions of dollars and we need to kind of realize that the stakes are so much higher so so easily in this system which is just negativities with currencies stuff oh keep my answer brief I think the one side of all anecdote back up Phil's point is this bank chain bug that came out just a few weeks ago where the blog post described hey we didn't do an audit of this because it would have cost more than the value held in the contract itself that's not something you can really do with a classical system and I totally agree with Phil's point there that this is something completely unique about the field that we're in so I think that's the biggest difference I could point to yeah cool and so actually Dan we were having this conversation the other day about bug bounties and whether they're a good practice or using tools to find repeatable bugs and you want to launch into that yeah so I think the concept of bug bounties makes sense if all you get if all you want to get from an auditor is list of bugs if that's all you're expecting to receive from hiring anybody on stage here to review your project as a list of bugs and sure a bug battery makes sense I wouldn't think that any of us expect to only provide that as a service second really big issue here and really you should be upset if you've hired someone like one of us up here and that's all you get that's that's not the service that you should get in order to end up eventually getting secure code the second thing is you don't know how to measure the output of having that bug bounty so what did you receive like did enough people look at it how much code coverage was gained did they inspect it for the issues you actually care about you have no idea a lot of this has to do with just a lack of talent in the industry like people at trailer bits have done bug bounties before and the way that we do it is we spend about 10 minutes to just scan over the code by hand run our slither static analyzer on it and if it doesn't find anything we move on the smartest people in the space are not doing bug bounties because we have better things more impactful things to do to improve the ecosystem itself like build tools and do research than to spend it all on bug bounties so I think that a lot of people in the community have been misled about the efficacy of what bug bounties actually can accomplish so if you really need security guidance you really should be talking to a smart human that can reason about security properties of your system and the bug bounty thing is just a secondary kind of cover your ass type type deal which sure you could have you could not have but it's not the first place you start and it it doesn't cover nearly as many bases as people think it does I agree I don't have anything that I'll go and yeah I know I agree that the efficacy of background T's is dubious in certain some cats circumstances I think like if you have like the contract is the bug bounty in a lot of cases right if you have a monetary compensation for being ethical that's like you know an auxiliary incentive like if really the people who have who go around looking at contracts and testing if they're vulnerable you know randomly generally those are all so unless you're like you know unless you're one of those hacker one types those are not the people who are going to report the bug bounty to you there's like many there's my other ways through the profit besides reporting it to the arena responsibly to the to the company and you know and like I'd echoed a statement that you shouldn't really just be looking for lists of bugs or vulnerabilities like how many vulnerabilities did this guy find shouldn't be the metric for quality security audit it should be how are we informed now to make better decisions in the future how have we gotten comments on improving our process such that you know we we don't make fragile code we don't get into that situations in the future I think realistically both in this industry and outside if possible it's always best to hire a security person internally right like someone should be with you every step of the way security is not like a an afterthought it's not like a thing at the end of the process it's part of the process of creating software and really in the blockchain space you know it's this is very evident but I hope that you know like even the traditional industry starts to think more this way so I agree with most of what's been said so far I do think dan was maybe a little too too negative on bug bounties so the way I see bug bounties is that it's obviously not a sort of substitute for doing your homework so it's your job as as a company to ensure that you hire the right people to have them put into your code that you hire external people as well as yourselves that you run security processes and engage in critical review of your software that you understand the limitations of the people you're hiring because while all of us are a competent security professionals were humans who have like a certain amount of hours in the day and we all focus on different aspects of security so it's your job to ensure you hire a diversity of opinions and skillsets to sort of inform you about how to proceed with your code but at the same time if you don't have a bug bounty you're sort of skewing the incentives for anyone who eventually finds a vulnerability in your program so you want to have something to sort of offer somebody who does find that vulnerability to stop them from just taking it to the chain or something like that so I think from an economic security perspective bug bounties are sort of necessary but not sufficient just to respond to that I agree with everything you said and I think that if you don't have an easy way to report bugs that are discovered in your application after it's been deployed you're making a huge mistake we have a little Wiki that like talks about some easy things you can do to make it easier to communicate with your project I believe it's called blockchain security contacts so that if any security researcher does find a bug in your code they can actually communicate it to you so don't neglect that but yes you're completely correct yeah I just echo what Phil said I'm all for bug bounties but not exclusive or security work yes Amy I don't think it should be the only security that's done I mean there should be like a process that you know makes your development more more secure like there should be someone in-house there should be a formal audit or three and then there should probably be a bug bounty like a cross at all I just think we've been sort of like a false equivalency there's like to an audit don't do a bounty instead of an audit and then run a bounty so that somebody has a reason to report it to you there there's separate things right all right next one is what advice would you give someone who's interested in becoming more involved in smart contract security wherever they start so my answer to that would be because that's the way I came into smart contract security to learn the intricate details and integrate the mechanics of the EVM and execution of transaction from the bottom up that would be my first advice kind of yeah that's definitely great advice I would also say look at past security failures and replicate them yourselves so you should be able to pull off something like the DAO hack you should be able to understand how to exploit various peoples coding error and you should try it by example and then once you're sort of done doing that and you feel comfortable that you're at the state-of-the-art maybe try finding something new even if it's not a very high severity thing find something that people haven't thought of yet in some edge case and whether that takes you three months or six months sort of at the end of that process you'll probably know a lot more about the EVM and smart contracts yeah I know I agree with both of those statements especially the intricate details part like most security issues come from misplaced assumptions right so you have this black box and has a set of properties that youth you feel comfortable with award that or someone else has told you those are the properties and like it's that telephone game right like depending on how far away the person that originally actually you know made the thing how many levels around between you and that person like it's likely that the your understanding of that black box is twisted and then you write code that has a better understanding and misleads the next person and then it just you know builds up over and over right so the more you can open black boxes and say this is how that really works this is where that interacts the less prone you are the security mistakes and also just try just write a big a developer and write anti fragile code you know if the the big one of the one of the recent bingo bugs is a perfect example of that is when you write a function that seems like it makes a check you know like the the name misleads it into the into the making you think that it makes the check like the the file it's in whatever and so somebody will see like Oh fine that makes that check and I don't have to check it in the container function and then you get you know infinite money being printed so we'd be really careful about your assumptions just cultivate that - mindset of awareness that's how you become good at security it takes I guess taking a bit different direction I'd say contribute like learn by doing so like participate in bounties which are like not that bad we agreed and like run the tools on the contracts before we do so that maybe you'll like get a list of things and make some money and then you'll feel great and then you'll do more and then you can start contributing to you know very like documentation our best practices would be great or like Dan's not so smart contracts and as well as our smart contract weakness classification registry like you you know and then teach other people like that's that's how I would go about doing it and then show up at these things and ask good questions so I think it kind of depends on what you want to get out of working with the community like are you someone that wants to go like stun hack and flex on Twitter about it are you a researcher that wants to build tools or are you a developer that needs to secure your code and with each of those different constraints there's probably a different pathway to learn what you need I think that each of those starts with two things and first to Martin's point you have to learn the lower level details of all the languages and tools that you're working with and the second is that you should review what's already been done we tried to do our best to make a compendium of that with the awesome aetherium security list just a few links and places where we think people can start off to learn but it's really important to keep in mind what your goals are because you know you want to spend the least time to get the most effort cool so I think we might actually go to the crowd now if anybody has any questions we have mics down here on either side of the stage and if you want to come down I've got more questions to ask but thought I might turn it to you you want to ask you at the mic here hi I was making some notes my name is Dimitri CEO and co-founder of cyber security consultant company how can I've made about 70 woods for the last nine months and we are all based in Ukraine so what woods I made notice that you like were saying that people from traditional cyber security companies are entering the industry yes and I totally agree that the bug bounty is not enough but my comment is that it's like the traditional people like I'm eight years delue it like what we want we want to have a methodology and guideline like what needs to be done in order to be sure that's the word it was done correctly and I think that this is what is missing so far in this industry yes the weakness classification work is great but there are still like so many things needs to be done like there are two ways how you can do the methodology is a principle based approach like what are the risks and how to cover them or the second is the checklists what what tests needs to be done so my question is like how do you see what is the right roadmap to create this methodology who is leading this process and like like I want to underline I think this is one of the most important thing right now that needs to be done in this industry thank you I feel that one yeah so we do have a a community called East security which is a community that I lead and then out of that we have a number of working groups so there's actually a working group for developer guidelines and as well as auditor guidelines or assessment guidelines so the smart contract Alliance is one of the groups in SecureAuth is making the developer guidelines so I encourage you to reach out and get involved in those groups because they are open if you want to hit me up on telegram I can add you to the correct channels you're in the group cool awesome yes I'm you having elseis anything on that yeah so I would caution against trusting any formal like checklist or process too much because the entire sort of point of an attacker in in in the security world is that they understand your assumptions and try to violate them so that checklist could actually lead to sort of blind spots in your process as well so what I would say is that any sort of halfway competent person you hire to take a look at your contract should sort of know subjectively like as a human being like where are my blind spots what have I looked at what have I not what state-of-the-art stuff have I done what state-of-the-art stuff have I not done and they should actually inform you about that as part of the process and and having someone internal who also understands the state-of-the-art could help you with that - yeah I just like to add that I think checklists are great but they're not for auditors they're for developers to kind of check while they write in the code but the auditors shouldn't have to you know rely on those who use those extensively yeah I will add those the developer and assessment guidelines are more framework for like as I enter into the product like as a developer what are the steps I'm going to go through so that I have a consistent product to then hand over to an auditor when they receive it so it's not just like a bunch of like files and only only assessment sighs like okay what are we going to be delivering back to the client and so that the process is a little bit more there's some expectations on what you give and receive somebody else over here I would like to ask you about security in future proof of stake actually security of validators and because basically issue is that what you mentioned anion validator I will put like 1,000 of ether in the my wallet and I will register the wallet for force taking right I will register with my private key and what happen if I'm signing blog with the same key in the internet work and let's say that someone will hack my note can he actually use the same private key to the registered my note and actually take mistake or my question is how can I secure my stake in the future proposal I'm trying to find it but did you get it what I can i rephrase it because i'm not sure and i'm no probe a stack expert but what i think you're asking is the same as for proof of authority chains where the actual sealing requires the presence of the of the private key in the node yes and yes that is the case so you will actually have a network node which contains the private key or private secret alternative they can talk to back-end to get something sealed is there actually any solution actually there is a solution that there are different ways to do it one thing is the this i've been working like the last year building something called clay clef it's in the go theorem repo it's an account management tool which can also do signing for click blocks which is a proof of authority chain type so there are solutions but it's not production-ready i guess yet so so from my conversations I've had with caspere people and it's kind of hard to nail down because the spec is so constantly evolving it would certainly be possible to have a separate key for signing proof of stake messages than for moving funds and then essentially what you would do is you'd sort of delegate to that key and that key would put you at risk in the way that it could get your funds slash if someone misbehaved on your behalf but maybe the protocol only allows funds to be slashed so quickly so you can still sort of get some guarantees by maintaining some sort of cold storage and I'm not sure if that will make it into the final protocol I think preliminary discussions I've had suggested that it might it's kind of an engineering detail but in theory that's sort of one way to mitigate against those risks also I'll just say that if people have detailed technical questions they'd like to get answered by security experts they should join some of the communities that Kevin mentioned before and then I personally trail Abbot's has open office hours we run every other week where people can just engage with us and pepper us with questions like that one and get sophisticated answers back khulus we got about two minutes left let's come back over to the said hey what should be added or changed to the standard development process to enhance security so it's built in and and enhanced as much as possible during development not including the external audit something like that yeah I feel like this is a chance for all of us to plug our tools here yeah in general I think trying to work out like what are the properties that your code is supposed to have and the invariants that you don't think should ever be violated about it are kind of a good first step you know we have a wealth of tools available now we mentioned in 2018 that we didn't have last year we have static analyzers that pick out examples of incorrect code but we also have tools that allow you to prove properties about your own code so like at a basic level you should be running static analysis tools and bug finding tools like slither or mythril early on in the development cycle you should be thinking about what kinds of properties the spec or invariants that you'd like your code to hold and then using tools like a kid 'no Manticore or k to actually prove that those properties hold for all the cases you think they do and then you should be engaging with experts to think like you know in an adversarial fashion about how the environment that you're executing in can affect that contract so like whether you can be front run or not and then also whether there are ways for people to work within your system to collect you know tokens or incentives and produce suboptimal results and eventually just you know degrade the effectiveness of whatever the decentralized system it is that you've created those are some things that I think I would think about if you want to end up with a good product at the end right a damn spec please because otherwise we can have no way to understand what you think you're building so write a spec and make it as formal as possible yeah yeah I also again write a spec write a lot of specs write specs every time your code changes and also code review with each other make sure everyone's on the same page about what your code does the like huge amounts of security issues come from the fact that just people are not on the same page yeah so thrill me throw me through mithril you should run it you should also run his run a linter I really like solium and like if you if it's enforcing like some reasonable code style that goes much further than you might think right a lot of tests ideally write write a spec and then write tests and then write the code that does the thing yeah that's I don't have anything to add at this time awesome and that is all the time we have so thank you all for being here and I think we're gonna have Hudson come back out and close this out these guys you