Where are Your Keys I Patrick Collins

All right, two more talks to go before we go into our happy hour. For the next speaker, I'd like to bring on Patrick, who all of you know likely because you probably got started with building smart contracts and uh he's going to talk about how do you actually manage your offspec and securities. Please give Patrick a big round of applause. [applause] Hello. Hello.

All right, I got 20 minutes 20 minutes on the board. We got a lot to go through. So, uh, let's just jump into it. But before we jump into it, let me see a show of hands. Who here considers themselves a developer, software engineer?

Oh, hell yeah. You are in the right place. Who considers themselves a security researcher? Okay. Excellent.

Excellent. And then who considers themselves neither of those in some other category? Okay, all technical folks for the most part in a couple couple strikers. This is good for you as well. Um so we've all heard the mantra not your keys not your cry and in our bones we know this is what we should think about when we think about private key safety.

And yet when we go to build stuff whether it's you know deploy something to a a blockchain whether it's you know build some automated thing in the cloud we don't do this as an industry and that's very bad and it results in a lot of hacks and we need to fix this uh otherwise we will never get to where we want to be and we'll never get the amount of money in this industry that we need. So I want to go through some hacks to illustrate some of these issues that we in the industry have been doing some of the ants and patterns that we've been doing. What happened in these hacks and what we should have learned from them so that we can be better. So this is all of you software engineers what's going to happen and I I promise you this is what's going to happen. you're going to, you know, maybe leave from listen to siphon updraft.

Okay, I'm never going to expose my property key plain text and then you're going to get the job and your boss or somebody else or an institution is going to say, "Oh, you need to expose your key because this is just how we do it." And you might go along with it, but this is me going to encourage you to push back and say, "Hey, if I do that, we're going to get wrecked." And then you can point to these as an example of what happens if you don't push back and you let your boss or your manager cycle with your private keys. And many of these has happened in the last 2 or 3 months. So these are not like things from 1015 years ago.

These are things that are happening today. These are very relevant things. And so I want to embed in you that these are still happening today. Even though we all know not your not your keys, not your crypto. Who here has heard that before?

Not your keys, not your crypto. Great. We've all heard that before. So, you all know this is true. And yet, we still see issues.

And this presentation is for everybody, everyday crypto holders, you as well. A lot of this is going to be more focused on, you know, working with enterprise or DAO amounts of money. uh so especially for developers and security researchers because you will be in these situations or hopefully you will grow to a point where you are in a situation where you are managing a lot of money because you know uh that is very good. So uh let's get started bottom line. So there was an npm hack recently in the last 3 years.

Who heard about the massive npm hack? Okay, about half you. That's about three. uh 2 million projects were hit with this exploit where um and this is um the CTO of the Ledger Wallet team talking about it where there was a very popular MPM package that had some malware in it that was malicious and if you ran like classic MPMI or yarn or just updated your node modules if you're using JavaScript you could have downloaded this you could have ran this and you could have gotten correct um we got kind of lucky in this one that this hacker dropped the ball. They basically infected billions of projects and they didn't really exploit them as best they could.

However, earlier this year, we or excuse me, earlier um December of last year, there was a um there was a piece of malware in a Salana project that actually did not drop the ball and literally all it did was go through all of your projects, all of your files, and looked for private keys in plain text. Uh-huh. We don't want to do that. This is literally what the bug did. You download it.

You ran up install. Boom. you have a bug that you have a little worm that is literally looking through your files looking for these plain text private keys. Who here since you're all developers has seen some documentation or tutorial that says, "Okay, great. Now just expose your private key in plain text and put it into a new file.

Who has seen that before? Who has done it and followed along with a production private key?" Uh, I'm glad you're being honest. I'm glad you're being honest. uh stop doing that because you're setting yourself up to get wrecked by this.

And we've seen um so one person in this specific specific scenario lost half a million dollars. So if you want to keep your half a million dollars, don't do that anymore. So the warning to this is never store your private king plain text. We know this and yeah all of you raise your hand and you saw and you saw everyone in this room sees documentation that says yeah it's cool to do this. It is not.

Never do that because this is what you're setting yourself up for. Does that make sense? Raise your hand if this makes sense. Okay, cool. All right, move to Ben.

Kick him out. He's out. Uh, next. Oh, and and see the other frustrating thing is this is where the vibe coding trend is setting up a lot of junior uh developers for failure. I asked Quad, hey, how do I deploy my smart contract to blockchain?

And it says, oh, just expose your private key, put it in plain text. Why does it do that? is trained on all the documentation that all of you have read that tells you to do that. And I'm here to say don't do that. So, let's go on to the next.

Now, we're dealing with some real uh some large amounts of money.

Uh there's a $3 million hack in the last 3 months uh with this project, Griffin AI, and some of the details around it were a little bit hazy. So, I'm going to take some artistic liberties to assume what happened here. Um, now even though we just learned, hey, don't expose your private keys, don't put them in plain text because there are a bots trying to steal your money. For whatever reason, automated processes and and things that you, you know, in um tools that you run in the cloud, a lot of developers go, "Ah, but it's easier if I just do it." And then they just expose the private key and they have it running some VM or some Lambda function that they have in the cloud.

And the exact same thing happens, right? somebody gets access to your your AWS account, your GCP account, uh or you have some bug that allows people to, you know, reverse shell or or something silly. That that might be a greedous example, but you know what I mean. And same thing, private keys in automation should still be encrypted. And I'm telling you this because you're going to get to a scenario at the job where your boss goes, "Okay, we're going to run this bot.

We're going to run it in the cloud. Just put the private key in like a little plain text file and that'll be good enough. And that is how you could lose $3 million because again the same exact issue from last time. Run mpm install. Boom private keys are back.

Somebody gets access to your shell. Boom. Private keys are back. Never. This is me doubling down.

Never ever ever have your private keys in plain text. If you're using the cloud, uh it is a a little bit trickier. Like if you're just doing running a one-off script, great. You connect your your scripts to your wallet um and deploy like that. In the cloud, obviously your tools, your scripts need access to your private keys to do stuff.

This is where you want to use a secrets manager. Uh if you're using AWS, you're already trusting AWS. AWS has a secrets manager. GCP has a secrets manager. Azure has a secrets manager.

They all have secrets managers. The thing you want to embed is your private keys at rest must always be encrypted. And it's very tempting when you go to the cloud to not do this. But if you don't, but if you leave them in plain text, you will get what happened here. You'll lose $3 million.

We've seen many bridges also fall to the same um same types of tax because for bridges, they need to automate bridging the money over. So they go, "Ah, it'll just be easier. We'll just keep it in plain text. and they get hacked for Linux dollars. So, don't do that.

All right, let's keep going. Fishing. This is for you developers and for the few few people in here who are not developers and not technical. This is the hack that your boss is going to fall for. How do I know this?

Because [snorts] I have been on calls where a very high netw worth individual says, "Hey, I clicked this link and I ran this script on my terminal and now my account's affed." And I go, "Shit." Um, they are gonna fall for this. Very, very powerful people fall for this and it sucks and it doesn't feel good. Uh, don't do this.

Like, obviously, don't do this. Um but but kind of like more um better [clears throat] better advice than just oh don't don't click on malicious links is have more security trainings for yourself and the teams on clicking malicious links and running scripts especially if someone in your organization has access to a large amount of money because that's often the norm. Uh, you know, I've been on on way too many calls where somebody jumps in the call. We're on like a multistick or something and they go, "Hey, uh, yep. I'm looking to uh, you know, we want to do this routine transfer.

Uh, I don't really know what I'm doing, but I just signed it and like whatever, or I just click the link that you sent, whatever." You need to push them and say, "Hey, if you do that, you're going to wreck it. If you click this link, you're going to wreck it." You need to be incredibly uh, vigilant at these. Um, but even more so, the learning that I want to to push here is when you download these scripts and when you run these scripts, when if somebody on your team does this, you can actually mitigate it in a number of ways.

In this specific scenario, oh, excuse me. In this uh specific hack um that happened recently for $13 million, the user had all of their money in a software wallet like a MetaMask, like a Phantom, uh like a Rabbit, and they did something that's actually good here. Their private keys were encrypted at rest. So that's very good. However, they clicked the link, they ran some malicious software on their terminal, and what do you think happened?

they had a bug rip through their code, looks for any um any caching of passwords or well or in other cases it could have installed a key logger where it just waits for you to type in your password. Once it gets your password, it doesn't really matter that your code private key is encrypted. It can decrypt your private key. Boom, get hacked. So, we're kind of at the next phase where it's like, okay, you did something better, but you're actually still at risk here.

And this is kind of showing where software wallets kind of fall apart. Well, fall apart is the wrong word, but where software wallets are actually less secure than hardware wallets. If you run some malicious software and you have a software wallet, software is much much easier to hack than uh hardware wallets. So for any critical amounts of money, for any money that you actually care to to save, you want to use a hardware wallet and multistakes. And the reason is exactly this is because hardware wallets are much harder to hack.

They are isolated from your environment. If you have all your money on a hardware wallet, even if you do download that malicious Zoom link and run that bad script, your money is actually still safe. So when we're talking about security, we're talking about our private keys. We want to make as many barriers as possible. And this is why people say, "Oh, use a hardware wallet.

Use a hardware wallet." Yes, they're less convenient, but they give this level of security that software wants just over. Moving on. I got 8 minutes left. A $3 trillion fat finger.

So, uh, this was this was, uh, great to see, uh, the classic fintech world kind of taking their first couple steps into our crypto world and for some reason not asking us, you know, how to not, you know, print the US debt worth of money. Um, they did a giant poopsy daisy where they minted $300 trillion worth of stable coins. Obviously, the price uh should have tanked to zero. Luckily, they fixed it within 22 minutes, but for 22 minutes, the total market cap of the PayPal stable coin was over $300 trillion, which is hilariously wrong. Uh, the issue that they made here was a dev sec ops.

So, now we're kind of moving away from the private keys to dev sec ops. They had a single uh ownable person on their contract be a single address, single EOA, and not a multi-stick. So this is something that I know for a fact all of you probably have done and probably will continue to do in your deploy scripts or in your automation scripts when you deploy smart contracts. Who here has worked with an ownable an ownable smart contract with only owner functions? Okay, great.

Many of you that ownable smart contract that only owner the the admin of that contract should never be a single EOA. Why? Because you can get this to happen. single EOA makes a big loop aid. He doesn't they don't have the oversight of multiple people um you know watching that transaction boom 300 trillion dollars is printed.

Having a setup where multiple people need to sign things is very good option. So when you do your deploy scripts um and this is something that we at Cipher uh will even look for in people's deploy scripts when they come to us for audits. We will say hey I noticed that you have an ownable um contract here. you're making it your deployer key. In your deploy script, you should actually change ownership to the actual multisig that you plan on being the owner.

And this will save issues like this uh from happening. We've seen it many times where, excuse me, [clears throat] um we've been on, you know, rescue attempts as well where somebody goes, "Oops, like I forgot to switch my admin from my deployer key to my, you know, multisig. my deployer key got compromised and now my app has to act. So the the learning here uh and amongst many other reasons is to use a multistig especially for contracts with sensitive functions such as printing $300 trillion. Even if you have a small number of users people on the wallet, it's still usually best to have a multistick because you can get that actual uh step of validation.

If one key does get compromised, you have multiple other keys and you can be uh safe. And now the final one, uh my favorite one, the buy the hack. So, who here heard about the $1.4 billion? Okay, we've all heard about it.

Uh largest heist in history. Uh just absolutely insane feat. $1.4 billion is a lot of money. And the thing that was the thing that the reason that this kind of act took a lot of the security industry by storm is because they actually did so many things correctly.

So they used a hardware wallet. Well, I I think they did. I they didn't really say too much about that, but I'm pretty sure they did. They used a multi-IG. They had a process of signing transactions and yet they still got hacked.

Still got hacked for $1.4 billion. So, what can we learn here? Patrick, you just you just gave all these tips. They did all the things that you told them to do.

Like, what what can we learn here? And this is where we get into the area where I am most frustrated with signing rooms right now. And I know a lot of wallet companies are working on uh improving this. At the moment, who here has seen something like this on their hardware wallet, had no idea what it meant? Some of you already raised your hands.

Had no idea what it meant, and signed it. Anyways, okay. Uh, who here has heard me rant about this before? Okay, great. A couple people, a couple people have.

Okay, cool. This is a big issue because this is what showed up on the Biden wallet. They obviously went, "I don't know what the hell this means, but like I do this all the time." And they signed it and they sent it. And what had happened was North Korean hackers were in the safe UI for maybe like a year, something like that, and they would choose certain transactions to send malicious payloads to their hardware.

So it doesn't matter that the private keys were safe. They were given malicious data, but the UI said, "Oh yeah, this is just like a routine small little transfer to the normal addresses and stuff, but the data that was actually sent to the wallet was send over Korean hackers all your money." So the issue here was that they didn't verify their call data. Now, this is where it gets a little bit nuanced, a little bit trickier for so I'm on a number of security councils for different layer 2 and different protocols, and it is a pain in the ass for me. Oh, sorry.

I didn't I didn't ask for clear. Sorry. It's a pain in the butt for me because every single time I see one of these on my hardware wallet, I have to verify every single character. And I do that because I'm insecurity, but I know retails to do that. And that's the issue that we face today.

So, yeah, they send malicious data over here. And uh on my on my YouTube channel and on the different blogs that I've written, I've tested this kind of uh signature intent verification is what I'm calling it. Um on all of these wallets here and some of them are better than others, but they all still have the same issue where if you have a big enough weird enough transaction, it's going to be incredibly difficult uh for you to uh verify this. And as of today, not one hardware wallet will tell you in plain English what your transactions is going to do. There's a little caveat to this.

A couple of wallets have some beta, new features that work in some specific scenarios. Uh I'll get to that in a second. I have two minutes to do so. Um so for all of you in this room, what can you do? Well, so first of all, when you're on a multistick or if you're in an organization where you have to sign and send critical transactions, you need to know how to decode call data because you can do it.

It's a little bit extra work, but you can do it. And you can make sure every single transaction that you send is correct. And you have to because what's going to happen is the one time you don't check is the one time that you get hacked. because we've seen this type of tag uh many times actually with many many different websites as well where a website will get compromised. you [snorts] know, you go to, you know, let's say you go to the unis swap website.

This doesn't happen at my own thing, but let's say you go to the unis swap website, you click swap, you get the data to your wallet, you hit confirm, and oh, oh my god, I actually sent all my money to someplace else, right? If a hacker takes over the unis swap website, they do some domain uh name swapping, boom, they can steal all your money. Um, so for yourselves, for the people around you, you can learn how to do code call data. This is something we teach on our education curriculum, sniper updraft. We have some tools and games that um help to call data.

I've also created some different like MetaMask plugins that help you do this uh as well. Uh and then we can all yell at the hardware wallets to be better as well. Now, I will say I just said none of them tell you to plain English. I will say a lot of these wallet teams are making progress, which is fantastic to see. So like the Keystone wallet decodes your call data.

It doesn't do nested decoding and so that's why I'm like not super happy. Grid Plus is the same thing. They do decode your call data. They are trusting the APIs in the back end. That's a whole other story.

The ledger team just came out with their Ledger Multisync product where in very very very specific scenarios we actually will see a transaction in plain English under our wallet but it's for like two D5 protocols which makes it a little bit unreliable. But we are seeing progress here which is great. But until every single transaction is in plain English, you as developers, all you security people must be uh checking this call data. Uh also kind of a side note, if you want to work with a safe UI um that is local that doesn't have men in the middle of the tanks, uh check out local safe. Um and the final thing, and I know I'm over time, sorry.

Final thing here. Uh the one thing I didn't talk about was centralized exchanges and I knew I wouldn't really have time to talk about it. Um cuz another uh another question people might ask is okay well if there's so many risks here decoding call data is so hard why don't I just leave it on a centralized exchange and the unfortunate thing is for a lot of your retail friends the answer might be you know what you should keep it on a centralized exchange because I don't trust you to be able to decode health data and as of today if you do a transaction and you don't know what it does basically every single transaction you send you are at risk of being hacked. Unless you dispose of quality. The downside of telling your friends to put their money on central exchanges is this right here.

Everyone in the room knows about FTX. Uh so hopefully Coinbase or Kraken or any of the current exchanges don't do that to us. But not your keys, not crypto. So put it there at your own risk at the same time. Thank you.

[applause]