Protocol Berg v2: Daniel Norman - Don’t Trust, Verify: IPFS for (D)App Distribution on the Web

So thank you first to the organizers of protocolberg and thank you all for attending today. Our topic for today is trust and verification on the web and the inherent tension between the two. This talk is quite technical but I want to preface it by saying that IPFS and verification are ultimately about elevating user agency and a lot of this talk is about actually executing on that. So this talk is going to be split into three parts. In the first part I want to talk a little bit about what we learned from the bybit hack and then I want to move on to verification on the web and finally to IPFS and verifiable DAP distribution.

So what did we learn from the bybit hack? I think we learned a bunch of things, but the main thing is that this hack didn't happen because of a blockchain bug. It didn't happen because um it happened because someone trusted malicious code and uh this has been a big bug and and the investigation revealed that the front-end deployment was compromised and there was no mechanism by which to verify that front end. Security on the web is layered. And the idea is that when one layer of security is breached, you have another to protect users.

So a great example of this that we already have widely deployed, probably your parents are already using this too is second factor authentication. So the idea is if your password is leaked, you have uh your device that does the second factor and you protect users. And of course there are no magic bullets when it comes to security. But in the case of the bybit hack, we've identified at least three important bugs and failures of security. The f the first one as I mentioned was that the front end was not verified.

The second is that the multic owners signed a malicious transaction uh using blind signing and so they didn't really know what they were signing and it was a malicious transaction. And finally that there was no additional approval layer by which to uh verify and prevent that uh transaction from going through. And of course anyone who's been uh building frontends, wallets or anything that is sort of userfacing in this space knows that it's a pretty hard balancing act between user experience and security. And in the context of IPFS, of course, we we think a lot about hashes and using those as uh identifiers uh for content. And hashes are just not human friendly or memorable.

And so we end up with this classic uh statement, you know, about how cryptography makes everything a uh key management problem. This sort of brings us to the second part uh of this talk, which is verification on the web. The TLDDR is that it's not great because the web is not self-certifying and it lacks the primitives for verifiable content distribution. And so self-certifying is the defining characteristic of web 3. And we usually do this with two building blocks, integrity and authenticity using hashes and cryptographic signatures.

And I think a lot of uh the innovation that we're seeing in the blockchain space and you know really this event is a is a a perfect testament to that shows how far you can go with just these two basic building blocks which unfortunately the web does not have by default. And this is because the web is built on hierarchal trust. So yes we have cryptographic signatures but they're basically signed by authorities which serve as like the gatekeepers. And of course these are certificate authorities who uh sign these uh certificates and uh these certificate authorities are pre-selected and and so we lack tamper proofing and uh a good way to sort of emphasize this is sorry a good way to sort of uh contextualize this is how do you know when you log onto your online banking that it hasn't been tampered with the typical way to do this is to just trust that green check mark that your browser gives you. But suddenly you're basically delegating all of the verification and all of that trust aspect to your browser vendor, maybe your operating system that uh ships with a bunch of pre-selected certificate authorities.

Now package managers like npm and brew and all of the other ones rely on hashes and signatures whereas security on the web is built on trust and we have the same origin policy which delineates these trust boundaries around the protocol port and host and we have one exception for this that we got recently on the web platform which is uh subresource integrity. So the idea that you can load scripts or CSS from other origins and you can attach a uh a hash to the uh script tag and the browser will verify it and if it fails to match um the hash the script will just not load. But this is of course only for sub resources and a lot of this has to do with the fact that the web is location addressed. Location addressing is foundational to the web and how the internet works and the basic idea is that every network device has an IP and DNS is used to map human friendly names to IPs. Now this works extremely well in a world where we have packet switching because it makes you it possible to find multiple paths between any two IPs.

But of course there's a lot of drawbacks to this approach which are that you have a single source and uh IPs and DNS are very very easy to uh block by state actors and we've seen this happen again and again. Moreover, if a copy disappears, there's no sort of fundamental way for you to know whether where other copies for that content may be. And so really location addressing is so fundamental, but it's also a double-edged sword. And this brings us back to the web platform. So the web platform, despite all of its flaws, always wins because it doesn't require you to ship another binary to your users.

The web platform has the widest reach with over 5 billion users across all continents either via browsers or web views and and other devices that have um uh browsers. And it's also the most ubiquitous. You practically have browsers on every possible device spanning from your smartphone all the way to your uh smart fridges and it's an open ecosystem where anyone can publish content um or build sites with minimal gatekeeping. So this brings us now to the third and final part of this talk which is IPFS for DAP distribution. There are three core ideas behind IPFS.

The first is content addressing. the idea that you address content by its uh uh hash and uh the second one is peer-to-peer networking. So the moment that you decouple from a canonical location that serves your data, you can uh now fetch it from anyone who claims to have it. And this is why you need sort of peer-to-peer networking. And of course, this is a key part of this, the third part, which is web compatibility.

the idea that all of this should be able to function on the web. Now the benefits of IPFS are manifold but you get censorship resistance and resilience and you prevent link uh link rot and uh you get a lot of these really really nice features like immutability which makes it cachable which plays really really well with existing sort of commodity infrastructure that you can either self-host or rely on cloud providers for. And of course we have these content identifiers which are a bit of sugar around hashes and allow us to encode some additional metadata like the codec and the hash function. And you can encode this either in string or binary form. And we also have a URI scheme that is used to address and also encode paths on this content addressed data.

And so you can do things like um pathing and directories and encode them all. And of course I mentioned peer-to-peer networking. I think it's worth just pausing here that like the like we're all pretty familiar with this concept but the key idea is that every client is also a server and this really complements content addressing because we you know do away with these canonical and trusted servers and I I guess the key point that I want to stress here is that peer-to-peer networking is really really hard just because of how networks are. um you have a knats and firewalls that prevent direct connectivity and so you have to do a lot of tricks in order to establish direct connectivity without going through relays and of course we rely on this cademia DHD in order to find providers with you know login complexity but there's still a lot of network hop hops in order to do that and so IPFS essentially tries to answer the question of what if we could take that same origin policy and apply it also to hashes and essentially allow IPFS URIs in browsers. Now the reality is that this is almost possible today using standard web APIs.

So let's take a look at how that's possible. But I should say that there's a lot of nuance around this because the origin model of the web prevents you from doing that. So almost in any case you have to basically link to an actual origin that is tied to a DNS name. So in the context of DAPs we have this um we have a format in order to encode these builds. Um so you want to be able to encode files and directories like a DAP build and this is where Unix fs comes in.

Unix fs is a binary format for encoding these files and directories. And the idea here is that every block is addressed by a SID and large files are chunked into these smaller chunks. And all of these are stored typically in a key value store. And they essentially make up what is a Merkel DAG. And uh the beautiful thing about this is that you get, you know, this single seed that represents a whole build and allows you also to verify just starting from that SID.

And as I mentioned, this is like perfect for publishing static sites and apps with IPFS because you just take your build output using whatever web technology or framework you're using and you just encode it with Unix FS. And a good example of this is Vitalik's blog that is also, you know, getting published regularly to IPFS. We also have this format um to just allow for a lot more interoperability uh called car files. And car files have just a transport agnostic file format that is very similar to like tarballs but it's for content addressed data. And so to not make assumptions about how you actually provide content and to decouple the process of mercalizing your content from actually providing it to the network, you can actually use this format very easily and it allows for a lot of composability.

And so we'll take a look and again this is just a file so you can store it anywhere. And this is also the it happens to be the same format that Blue Sky also uses for their user exports. So how do we sort of take all of these ideas and sort of deliver to them deliver them to you as as a developer who's working and and sort of wants to ship this stuff on the web? There were a bunch of talks that hinted at this. But really where all of this comes together is this library called Helia Verified Fetch, which is essentially an implementation of IPFS that was built for the web.

It's built in Typescript and it's just built completely using web APIs. And the key idea is that it takes it's it's modeled after the fetch API. So it takes a SID as input and basically returns you a response object. And this turns out to be really really useful if you want to do things like load a whole website um or a DAP for that matter uh and do that in a verified fashion. So this is why we built the service worker gateway.

So the service worker gateway is an app on its own that essentially takes a it installs a service worker and the service worker works like it's a web API that allows you to uh install a script that serves as a proxy for all requests that go out of the browser and initially it was sort of conceived in order to be able to do things like caching caching and offline experiences but it turns out that this is really great for doing uh content addressing because all data is immutable and so it's cachable which means that you can take now any website including Vitalik's blog uh blog and you can load it using in browser.link link which is an instance of this service worker gateway and uh everything that is loaded is now offline ready. You can read it without an internet connection. And we do all of this by installing a service worker giving you a unique origin and then doing all of the verification you inside a service worker with verified fetch. So this covered the retrieval side of uh things, the verified retrieval side of things inside of web browsers.

What about the publishing side? We realize that the IPFS uh ecosystem is uh quite dynamic. There's a lot of providers and there's a lot of different ways to do things. You can provide content by running your own node um and you have multiple implementations or you can use one of the existing uh content providers. The idea with this GitHub action was to actually bring the mercalization of artifacts of of builds of front ends as close as possible to where the development happens.

And I think that there's a there's also a convergence in the broader ecosystem just speaking to the radical folks um who are doing some great work in this space to really basically go all the way from where the code development happens have the verification start there and have full supply chain provenence. So with this part we actually just bring the merkealization and the uh onboarding of this content address data to the network uh very easily and the idea is that it gives you a similar experience to what you get with these uh platforms like netlifi and vil where essentially just takes your build and it uploads it uh to your nodes or providers for you. This brings us to another problem. So assuming that you've embraced all of this and you're using the service worker gateway and you're getting verification and all of your builds are, you know, getting pushed and you have SIDS and it's all, you know, you're in that dandy land and there it's all unicorns. You still have the problem of how your users discover SIDs and there's a bunch of approaches that you see commonly adopted inside around the ecosystem.

And the first is uh a DNS link which just allows you to use DNS to link uh DNS names to to SIDS. And the problem with this approach is that it's not verifiable. And so you really just rely on whichever DNS server you went to to give you the honest response of what that record was supposed to be, which is why a lot of folks are embracing this idea of like ENS. Um IPNS again doesn't really solve this problem because the names are not human friendly. And finally, GitHub releases are pretty widely adopted just because developers are familiar with this and there's just a natural tendency to trust where the code development happens as the place that you can actually trust the artifact identifiers from or the SIDS.

This brings us uh to trustless gateways. So one of the big shifts that has been happening in order to ensure that direct retrieval can happen so we don't rely on centralized gateways is the move towards trustless gateways and there's two sides to this. The first one is to shift away from centralized gateways toward more peer-to-peer retrieval and the second is to just use HTTP. And so initially IPFS gateways would ret return return you deserialized responses which you can't verify. And so the trustless gateway API was a way to still use the same HTTP semantics but return you responses that you can verify locally in your clients.

And of course this is just HTTP so it's fully cachable. The second part was how do we move away from this model of having to go to a central gateway that does all of this peer-to-peer magic for you. And the way that we're tackling this is using this service called autoTLS which essentially issues every li sorry every lip host which is also an IPFS node um sorry every IPFS node is also a lip P2P host uh a CA signed TLS certificate uh which makes it dialable from browsers because when you dial from a browser you need to have um a TL a CA signed TLS certificate and IPFS on the web is hard because the web platform form is very very constrained. You're limited in the number in terms of the number of connections you can open and you're also limited with transport. So you can't open just arbitrary UDP or TCP connections.

And so a lot of the work that we're doing at uh shipyard is really to enable all of this happen inside a bro inside a browser with autoTLS and verification with these service workers. And so as we come to an end, I just uh want to finish off by saying that you shouldn't just trust the front end, you should verify it. and IPFS gives you the tools to do that today. So, thank you. Another quick one here.

We have a couple of uh questions um we can go through. I'll read them out. Um so, many clients use gateways to access P2P. What impact does it have what uh on the security assumptions especially when it comes to hash discovery? So finding a hash to something human rememberable.

Yeah. I mean you ultimately so I think there's a point here to be made about the fact that you can't fully eliminate trust. You can only move it. But if you can trust the SID that you got um by whichever means you got it then uh you have all the means to do the verification either with centralized gateways or even with direct retrieval and this has been a big part of the effort um to first make sure that everyone is verifying and you can do that in a centralized way or in a more decentralized way. Um, and the trade-off there is just, uh, it tends to be performance because, you know, if you're going to start, you know, a DHT lookup, it's going to take time or if you're going to go to a delegated routting endpoint, it's going to add a bit of overhead to uh to the point that you get the content, but the content is always verifiable.

Thank you. Um, Swarm already provides in provides integrity protected addressing addressing. Uh, you can also host websites on it in an integrity protected way. Have you considered using Swarm? So as far as I understand, swarm is like in an incentivized network.

So there's uh some building blocks that it gives you to actually uh ensure that someone is uh providing your content. Um IPFS is just a set of building blocks and protocols that you can mess with and you know use. Uh so if swarm does that for you, great. I think the most important thing that we can get in the ecosystem is just interoperability and and standard ways of doing things. Cool.

Uh, what is the recommended way to get the sit for some content? Where should I where should it be published and how do we verify that? Yeah. So, I I think the slide about uh I had a slide about this, right? Um, I think ENS is a pretty decent way to do this.

It's it's verifiable. I mean, of course, it like also depends. Yeah. This was the one. Yeah.

Yeah. So, ENS, both ENS and IPNS are verifiable. They're trustless. Um, I think that's a I think we should be we shouldn't see it as just a one solution fits all, but ENS works pretty well, but I in general, I think as close as we can get it to the uh uh where the code development happens to the source control, the better. Uh and so we want signed commits and we want an ability to actually sign artifacts uh using the same signatures that we use to sign the commits.

Um because that's where the developers are and you know the trust is essentially it always goes back to who's building the software. Um the teleosters blog on IPFS. Do you have any thoughts on why that is and why he doesn't use swarm for this? Pik would be a great guy to ask about this, but I I'm pretty biased. I think as I mentioned like uh IPFS is a is a set of open uh source building blocks to do a lot of this and um you know if you want to build an incentivized network around this great I think you should still embrace all of these uh tools and I I don't see any reason not to.

I mean we can have discussions about the actual implementation of things about whether Unix FS is the right format for this but even as a as a community IPFS is not uh is not monolithic. We we we take a big tent approach and we have other ideas about how to do things. So just to just give a shout out to another example in another project is Dazzle which uh is trying to you know create a set of building blocks um that take a bit of a leaner approach to doing um content addressing while keeping the implementations pretty tight and uh not too open-ended. Yeah. And actually I think uh it might also be the fact that IPFS was ready and mature way before Swarm.

That would be my guess for why he chose that. And and as far as I know IPFS is the only technology that really works on the web today probably with verification I would say. I think we have time for one more. Why don't we build together the next generation of the internet by moving to decentralized hosting and removing DNS resolution from the equation? Yes, let's do it.

That's why we're here, I think. True. Then we can do one more. That was a very short answer. Um, a lot of users use extensions, Chrome or VS Code, which allow change uh of the UI at any point.

Should we stop them and reduce flexibility to maximize security? I think it's a little bit in this. I don't know if you heard about like some weeks ago there was the VS code like the solidity extension for VS Code. I think was like cloned or something like that. And this Juan Blanco uh who developed it, there was a new one and it showed like five stars and a lot of ratings.

So like someone just hijacked it and had malicious code in it. So I mean in this case it's a supply chain problem in the case of the VS code extension. But I think on the web I think you can prevent this today with content security policy. Uh I mean like the web is is like a mess a bit of a mess. It's a very like it's a hairy complex set of APIs and technologies.

Um, and security is a bit of a moving target. Um, should we stop them? I think we should give I I think the first thing we want to do is to be able to create origins that are not tied to domains. We want to be able to create origins on the web that are just hashes and allow using extension APIs in order to actually handle those um those origins that we can create on the fly using hashes. Um there's there's a lot of work that we're doing behind the scenes to make this happen and there's like a lot of really arduous standards work.

I would say that uh we're we're making progress. We're making progress but we're not quite there yet. Amazing. Thank you so much for your presentation and your time.