[Music] Thank you so much, Harsh. We are ready for our next talk. And for this one, we'd like to uh go back to the the overarching theme in tech right now, which is how do we actually use AI and integrate this with blockchains. And to talk all about that is Shashank, and we're going to talk about how to make smart contracts safer. So, please give him a big round of applause.
Hello everyone, I'm Shashank. U I'm talking about you know scaling smart contract security with AI and automation and this is something what we are building and uh I I I hope you like the talk. So um so I'll introduce myself. I've been in security for almost 14 years now. Starting with web application security.
Uh I found bugs in Google, Facebook, Microsoft. Uh some of finest work was uh 30 days in Apple iOS 10.3 which allowed me to remotely shut down anyone's iPhone and I've been acknowledged for that. uh my web3 journey started with avalanche uh where I was a security engineer and uh that's where you know I realized there is a problem with you know uh web3 security the the world of smart contracts where so many hacks are happening and how do we scale security and that's where you know we started building solidity scan there's an AI powered smart contract security scanner and uh I'm not talking about the entire product I'll I'll I'll just talk about one problem which we tried to solve with AI with a real example of a hack So why is smart contract security automation? Uh it is quite evident that you know like hacks have been a big problem in the web 3 world unlike the web two world where uh the the biggest compromise just leads into you know it's like maybe uh data leak or sometimes financial frauds here it's always financial fraud millions of dollars if if you see the numbers itself uh last year it was 1.
5 or 1.8 8 billion. This year we are breaking records. We are already at $5 billion. So uh and out of these you know like billion dollars of hack 55% of them are due to smart contract bugs like the vulnerabilities in the smart contracts and uh that's that's what we are trying to solve.
So you know it's like we have discussed about the problem you know billions of dollars in hacks and the way people resort to it is through manual audits which is great but uh that is not a scalable solution. It is required but it is not a scalable solution. For example, you go for an audit right now and then you make some changes then you have to resort back to audits again. What if one of your packages for example you know it's like open zeppelin package if you're using that it has a vulnerability how do you get realtime insights about those zero days and that's where you know automation comes into place which the web 2 industry has already done. It's like they have security tools in their CI/CD pipeline which uh which sits in their software development life cycle and then they you know it's like they keep on finding vulnerabilities.
So this was a solution you know it's like you automate smart contract vulnerabilities right you know it's like algorithmic uh patterns to figure out smart contract bugs but then it has certain limitations which can be solved by AI and this is what we did and I wanted to show you one example. So for example on 2nd September uh uh 2022 Shadowfight tokens were hacked and that resulted into you know 316K in loss and let's you know just just try to reverse engineer the hack and how we can use AI and and you know like automation to detect those vulnerabilities way before you know it's like it goes into production or mainet as you say. So let's let's see the transactions. So the hacker what the first step he did was you know like got some BNB of course it was on BNB chain so you need gas fees. So that makes it very clear.
Second step the hacker tried to you know buy some SDF tokens which is the shadowy tokens. Uh I think the first was a trust transaction and then the hacker bought like nine shadowy tokens. Now the interesting part is the fourth transaction. Uh anyone can guess what happened there like by looking at the transaction anyone can guess what happened there. Yeah, correct.
So, null address, you know, almost 10 million shadow fight tokens were sent to null address, which gives us an insight what the hacker was capable of. He or she whatever uh found a way to burn the tokens means uh they either had access to the burn function or something like that. Whatever. So, it is pretty evident the shadowy tokens were burnt and now what's the result of that? The result is that the supply is less.
So his or her token 10 tokens are more valuable which was bought for like some cents like 29 cents. Uh but now it has more value. So uh if you see the next steps five again a test transaction maybe and then swapped everything out made $316,000. So this is one classic hack, a very simple hack and we have multiple variations of these hacks in last year as well and access control has been you know number one uh reasons for smart contract hacks. So this is one simple example.
So now we know you know it's like uh we can verify from the code itself that you know the burn token was public. So anyone can call the uh burn function and you know it's like uh burn the token uh deflate the supply sell make money get out. Now we decided you know it's like uh two years back that okay let's automate these kinds of things. Uh we managed to automate almost 300 or I think 400 different kind of vulnerabilities but there was a limitation on top of that especially with this main bug uh you know uh access control. Now access control can be implemented in various ways.
For example, here the situation was they did import ownable module but they never you know it's like called the modifier there only owner modifier. Okay. So this is one way but each developer has their own way of writing codes. Maybe they can have a require function or maybe they can build their own rbacks. So how do you you know like write all these edge cases?
It's it's like if you go for an algorithmic coding, you have to write so many edge cases and still end up missing vulnerabilities. So I'm going to show you a simple example. You know, we scanned this on our platform and uh we did find the bug. The bug was getting spotted. But the problem was this the the our automation program was saying there are 24 additional vulnerabilities as well which is not valid because they had some different kind of access control in place and now writing an edge case for everything is a very difficult process.
So what do we do? It's very simple that you know it's like uh this is a problem of business logic which can vary from person to person developer to developer. So what do you do? you use AI because AI can understand context. So what we started doing was uh initially what we did uh we started you know like feeding the smart contract function by function so that it can understand you know like what kind of arbbacks is in place or what kind of you know like access control is in place and then let's see you know like uh uh how you know like how it performs.
So when we did that uh okay when we did that you can see uh the the cross mark with that robotic feature is basically the AI is saying that okay this is not valid and then certain reasons have been mentioned that okay this is the reason why this doesn't seem to be a valid issue and it correctly says that okay you know uh the burn function was was a valid one and this is the result of the matrix uh okay sorry this is the incorrect one and it's saying that why it is not it is incorrect is because there is already a you know like a different kind of rback in place and this is the matrix. So initially when we were using it without AI uh there was 24 detections one valid detections and 23 false positive. So this is huge amount of false positives right like if you are you know the whole idea of automating and scaling things is to reduce human work and if you are giving you know it's like 23 false positives it increases the work of an auditor or maybe developer to validate okay this is false positive or not but then with the AI it 19 false positives were reduced which means an 82% you know like reduction in false positive so this was our first step unfortunately I don't have slides after that because uh The recent deployment happened last week. So I'll I'll just speak about it. So what was the next step?
What do we do? You know, it's like we have solved the problem of reducing false positives. But then even vulnerabilities can have you know like a different ranges, different variations etc. So we came up with the multi- aent model where what we do is there is a simple a master slave concept. So uh there is one you know AI which distributes the task to multiple AI agents and we scraped you know it's like uh the audit reports of all companies you know solo date or audit contest and everything we I think we collected over 100,000 you know audit reports uh with you know like you know maybe a million bugs or something like that and we started training our own AI model so that next time when we see a similar context uh in the codebase we can find vulnerabilities in it.
So this is something we deployed last week and it has been working very well for us and uh that's that's what you know it's like this talk was about that you know how we uh use leverage AI so that you have kind of a you know it's like an AI powered auditor sitting in your pipeline to reduce false positives uh sorry to reduce you know vulnerabilities in your code so that you are more ready when you go for a manual audit or you know you go for a bug bounty program or something like that. That's my talk. Uh it's it's very short because I tend to keep my talk short. I would be happy to take any questions if you have. Uh anyone?
Yeah.
It just gets the contract and uh reviews it.
What? Sorry.
Reviews it like
oh no it not it doesn't review it. So I'll tell you the problem with AI right now. If you give the entire codebase for example you know you have a maybe even a midsize contract would be like 200 to 2500 to 5,000 lines of code right now if you give that completely to the AI it won't work because first the problem is the context window cannot be that long second it will start hallucinating so uh you know consider how we are solving this problem is how you use chat GPT chat GP doesn't you know like tell you everything you ask questions and then based on that you know gives you response and then to change your prompt to uh to get uh answers. So what we did was we already had you know like uh algorithms written for for us for certain vulnerabilities. Now we guide that with AI you know the multi- aent mode it will have you know like certain prompts and rags set and what it will do is like you know it will ask appropriate questions for validations and then it will pass on into the back end and then you know like we show the results.
So it's it's a complimentary mode with uh with our own algorithms. Yeah. Thank you. We'll just do one more question and uh then we'll uh end.
Thanks for presentation. The question was uh regarding the security guarantees. Do you provide also formal verification? Are you able to define uh for certain uh pre and post condition that function works or is there invalid condition?
No. Uh we we are on the process of doing that. Uh so actually it was being very resource exhaustive what you are saying formal verification or you know even fuzzing. So we have been doing that you know like uh this is again under experimental mode right now where what we do is like we give smart contract as a context and then it tries to generate first test cases using AI. Uh but yes right now what we do is like we read the code and we do static analysis of uh the code to find vulnerabilities.
Uh that would be something we'll be doing in the future. Uh I guess that's that's it. Thank you guys. [Music]
Automatic transcript — names and jargon may be misspelled.