Krum Pashov, Pashov Audit Group: Finding the Best Security Audit | ETHSofia 2024

[Music] hello everybody good to be speaking here at tth Sofia um yeah we'll be speaking about uh finding the best security audit today because there is a big problem in the space that uh most project Founders can't seem to uh to nail this uh very often serious protocols get AUD that are not high quality that are not their best fit so there is a problem to solve here first just an introduction about myself currently the founder of POF audit Group which is a security company we do team audits for all kinds of defi infrastructural protocols mostly smart contracts um I also have some experience as web 2 and web 3 developer for four or five years at various companies and I know also do some Angel Investing on the site so as I said the big problem is that uh project Founders can't seem to just really don't know how to get a high quality audit a good one that fits all their needs uh let me ask you something how many people here are actually security Auditors can you raise your hand okay lots of security Auditors and how many people here are decision makers in their company and have been looking for audits in the past please raise your hand okay a few some okay so yeah um you have been through this process of uh choosing U an audit provider you have probably asked yourself many questions like how to get a good one currently uh the most common approach is just to ask around ask some friends ask other protocol Founders uh who they have worked with and um how good of a service they have gotten uh which is which is good usually this is the best way to put as little effort as possible and get uh good because um if some company did a good audit last time with your friends with your um fellow Founders probably next audit they do will also be good this is a good bet but it's not the best one you can do much better than this so here I list some um qualities of a good audit first of of course it needs to fit your timeline because there are many great companies that uh they can give you a quote they can tell you uh our audit costs this much uh we we are the best at Alli thing but um if they cannot start when you need them let's say a week from now or tomorrow it wouldn't be a fit so of course they need to fit your timeline this is very important then you continue with uh with the budget of course they need to to fit the budget that you have they uh if they want if they want you to pay $100,000 but you only have uh $20,000 to spend this wouldn't be possible of course you would like good communication in your audit because um this is a good thing this is not the the absolute must have but it's great when you have good communication you know Auditors are um are into the audit they're actually doing the work they are starting to understand the protocol you can even see some Auditors get very very deep into the the protocol itself and even start understanding it better than the developers themselves so this is a very good sign that you are getting um a good understanding of the Auditors and usually a good audit and of course uh I wrote here high quality because the overall quality of the AIT uh which means uh the actual findings how good they are how many findings uh were uh discovered through the audit and how many were missed potentially this is the most most important thing in an audit so um for for this specific thing the quality itself this is my solution this is the thing I uh the things I believe that are the most important ones and uh we should start with team transparency in my opinion Mo very often and in in the example I gave where most uh people get get their another fellow founder to recommend a security auditing company to them they just ask uh for example is company X good they say yes they did a great audit for us this is this is good this is good for a start but you never know if the Auditors the people who are actually doing the work will be the same people and if they have experience exactly in your own in the type of protocol that you are building which is much much more important than uh another team from this company doing a good job so it's very important that uh you you have Team transparency that you know the size of the team this is uh crucial because a team size of two is very different than a team size of four of course it makes a difference and uh you also need to know who are the actual Auditors when you know who are the actual Auditors you get to the next point which is the Auditors Auditor's experience Auditors experience makes a very big difference uh a person who has been auditing for uh 6 months will do a much different job than a person who has been auditing for two years or three years uh especially if especially the the background with the protocols that the person has audited this makes a huge difference if you are building a new amm and uh the person has been auditing only uh lending protocols he's an expert in bridges and Landing protocols this helps a lot he's still a defy smart contract security expert but he wouldn't be so experienced in amm which is what you're building and which is the person you need so this is very important as I said you need to know the the team size you need to know who the Auditors are and you need to know their experience uh nowadays in 2024 you can a lot of Auditors especially uh people who are doing auditing independently or part of of teams as Freelancers they have profiles on different security contest platforms like codena sherock Cantina cold HX and some others heads Finance so their profiles there are very uh are a great thing to to check their experience to see how many findings they have been finding on each audit to see how long have they been auditing auditing uh for and of course you can see if they have one security contest if they have been doing much better than other Auditors if they're just special Auditors because you need the best ones on your team that your budget can uh provide of course and the third thing that matters so much is incentives so even if you get uh a great great auditor who has uh let's say six years of experience in security auditing it is very likely that uh this person has been doing it for so long that he's not crazy motivated to do it uh the the work is not that interesting to him so you really need to know that he's incentivized well because uh crypto runs on economic incentives right now this is the most important thing and it has proven to to be working so you need to know that uh the Auditors are incentivized well uh as you know in um in security contest platforms uh Auditors are not getting paid with a base pay uh which is predefined before the audit uh so not all of them are um fully incentivized to work full-time every day on your protocol but still there is the incentive that uh if not many people actually work on this audit if you are a guy who does fulltime on the audit and you put all your effort effort into it you can win a big uh part of the contest spot you can win a big prize so you can make $20,000 or $30,000 doing maybe a week of work which is a great incentive um with Team audits for example or with company audits you know that people are incentivized so that um they will be getting paid in in a base pay let let's say a PR predefined a few thousand do per week let's say but sometimes they get bonuses or they will get uh to do more work in this company which is very important for Auditors so you really need to understand the the incentives behind it and to be certain that the the people are motivated enough to do a great job on the audit so here I listed a few options basically that you have as a protocol right now to do uh when it comes to audits which is the predeployment part of your security process you can start with soul audits which uh were very trending last year this year not so much but I still believe it's a great service because you have Team transparency you it's it's just a single person which is the the least attractive part it's only one person but you know who it is you can choose him yourself or her you can choose the auditor yourself and uh you can check uh the experience of this person which is very very important as I said you know the incentive is good because uh if they do a good job they will continue doing more audits you'll recommend them to other people and you'll probably return for another audit so they have a good incentive to do a great job especially if they have been doing it for a while while and have a brand to protect this is very important so um I feel like so audits are great option for lower budget protocols but still it wouldn't be the best because team audits can be the same thing just without the downside of a single person auditing you you can have three four two or even more Auditors whichever number uh you have a lot of options with this um there are so many teams that have started doing aits just if you go on Twitter you'll certainly see some Twitter spam uh with regards to people doing promotion promotions on their uh company or whatever but you have a lot of options and uh a lot of great Auditors are doing this so this is great team audits are not only the smaller companies from people who themselves are auditing but also the big companies the traditional ones so they are also great options but again you have to do uh the extra to walk the extra mile with your um due diligence to check the team size and to check people who are auditing you and their experience this makes all of the difference in an audit and of course you have a security contests where it's a little bit different basically you have no guarantee that great people will be auditing you uh unless it's a platform that guarantees some people being pre-booked which some do but um yeah here it's more risky you can have no participation in a security contest but if you look at the history uh all security contests have done pretty well not many hacks after uh security audits from let's say C sherock or um the other ones so basically it's a good bet but still you cannot be certain about the team transparency because there is usually not a book team with this and here is our approach to this this is my company POF audit group we have done um I think 100 audits uh for the past 11 12 months so we have been doing a lot of work we have been deep in the trenches with this and we want to do our best uh quality service for all of our clients for the timeline we have a lot of elasticity with our team because uh we have we don't have really full-time employees when it comes to Auditors we only work with Freelancers and we can hire on the on the goal we can do this very fast we work with around 30 Auditors and are hiring even more so um when it comes to timeline usually we can start very fast we can start in 24 hours 8 48 hours or if you want in a month we are usually very flexible with this this is what uh a company should strive for if they want to be um flexible enough to work with any client because a lot very often clients come and ask for um for an audit to start today or tomorrow so this is very important when it comes to budgeting this is uh of course usually the most important question companies ask which is uh uh it's a Pity because uh a lot of projects do not really put uh time and effort and due diligence into choosing their auditing team they're just comparing uh when it comes to budget which is kind of fine of course if you have as I said 20,000,000 budget you cannot do a $100,000 audit but still you can do the best for for your money so you have to do the due diligence when it comes to the team uh yeah when it comes to budget we're doing uh the normal prices of uh high quality uh auditing companies uh it's it's not too special but uh of course anybody can ask and compare this is a good good approach when it comes to the team we usually uh because we know good good protocols we uh they will do their due diligence they will follow the advice from this St and they will check the team so we have gathered a lot of great Auditors from Cod from sherro from all the contest platforms or from big security companies that that want to do some auditing on the site uh and we have hired them basically looking through their past experience looking through their um the protocols they have audited this way we can make a great fit when there is a new client as I said let's say a lending protocols come comes in and ask for an audit we can get people with great experience with in Ling protocols and and this does make a great great difference because a lot of teams do not put much effort into this they just if you want an audit to start tomorrow they will figure it out they will start really tomorrow but they will get just the people who are available which is something that you shouldn't be okay with you should get the the best fit Auditors for your for your uh protocols audit for your security this is why not uh it's not just the brand name that matters but actually the the people who do the actual audit this is what truly matters of course the brand name matters but this should be secondary and when it comes to incentives the biggest incentive we do is that uh um the Auditors usually when we hire the Auditors are already experienced they have proven they always do a great job and um biggest biggest incentive is that uh we have an internal leaderboard uh where we follow every Auditor's uh performance on our Audits and if they do a good good enough job they will get more work from us so they will be making more money and this can be very long process we have Auditors that have been uh almost uh they have been doing two three or more audits every month for the past 11 months so this is pretty good when you do good work you will get more work so this is a good incentive and sometimes we do some bonuses as well so this this is also important this makes a difference incentives are very important you need to know that your team is well experienced in the protocol that you are building in securing such protocols and that they are motivated enough because if you get the best person on an audit and he's not motivated it wouldn't work this was it this was basically my advice make sure to know who is auditing you and that they are very motivated thank you I think we have uh just uh two or three minutes for maybe some questions so if you have feel free to ask them right now yes I can hear you yes uh no I cannot hear you sir now I can what do you think of clients which want C audit of their project but they come to you purely for marketing purposes because if they out like the smart contract their whole infrastructure not audited completely but they receive this Mark audited by Pas what's your attitude you convince them to do full audit or you doesn't matter and you go for what the client requests or you refuse to do it what's your okay so the question is basically what do we do as a company POF audit group when a client comes and uh they just want a security audit of their smart contracts mostly for marketing purposes and they're not really auditing their infrastructure or their their um website or the other parts of their project uh so yeah this happens very often a lot of protocols do audit just for marketing purposes which is kind of fine we always tell them to audit everything they can but uh not always they have the budget for it so um we're we are working as consultants consultants and advisers we do our best to um to tell them the best way to do things so we will say what about your infrastructure what about sometimes protocol protocols even come and they have let's say 10 contracts in their code base but they only want us to audit two of them and I asked why why don't you audit the full scope and they say uh this is this is the budget we have we cannot do more I usually advise them to maybe raise some money or figure out some other way uh and it's it's up to them it's their opinion they will still get the PDF report they will still promote it on their website and on Twitter which is fine but uh what can I say you users usually don't get don't put much time into um reading the audit reports anyway so even if it's uh if they just know a project has an audit report they don't even check if it's what is it covering so this is not good you you can do you can do better as users you can check and you can ask the project to uh audit more but it's up to their decision we just we just advise them to to do a better job and to audit their full scope but it doesn't always work okay I uh we can take one more question but uh not sure we have I think they should be good okay guys thank you very much again [Music]