Modern Blockchain Indexing I Denham Preen

[Music] We are ready to resume the afternoon sessions and the next two hours are going to be backtoback on all the interesting things happening in the world of developers and how do we actually get things in the hands of more and more people. And to kick this off, I want to bring on Tenum to tell you all about indexing blockchains at scale. Please give him a big round of applause.

Hi everyone. How's everyone doing? I hope you're all feeling quite full after lunch. It was very a good one. ETH global events always have delicious food.

Um as well as an awesome lineup. So, I'm super pumped to be able to talk here because I truly think the Pragma talks are really, really well cured and often super interesting. All right, I'm going to tell you a little bit about u modern blockchain indexing. All right, we are in a future era of uh blockchains. Blockchains aren't uh only 15 transactions per second anymore.

And so we the tooling that comes along with with this needs to be upgraded too. So super pumped to tell you about this. All right, quickly a little bit about myself. My name is Denim, pronounced like the jeans. I'm one of the co-founders of Envo.

I uh been playing around since gas prices allowed it on mainet. So it was very easy for experimentation and I think uh notably I'm a safeguardian pulley holder grounds member. Okay, enough about me. Little bit about Envia. We're a team of 10 engineers.

We've been focusing on this project for about two years now. Really focusing on um trying to bring it as fast as humanly possible. Um, and yeah, I'm going to tell you a little bit more about what modern blockchain indexing looks like. All right, first off, what is indexing? If you're not familiar with it, it's uh the invisible engine behind every DAP you use.

All right, so um sorry, the speaker notes are slightly off. Uh another way of explaining it which is really quite simple is it's the layer between your smart contracts and your front end. So usually what happens is you you get some raw event data, you process this in in an indexer and then you serve it over an API to a front end. Now there's many more applications but this is probably the most simple flow of what an indexer looks like. Cool.

Um let's take a look at a as an example. um it's a little bit more practical to see, but essentially a if you're not familiar with it, it's like an online bank and users deposit funds into their bank and it earns interest. Now the deposit action emits this raw event of deposits and in an indexer we usually aggregate this. We might assign it to a a person. So we saw this in a database and then we can serve that to them over a front end.

All right, let's start with Ethereum. the OG the start of EVM um in 2015 it was it supports about 15 transactions per second all right which is perfectly fine and uh this is very manageable from the kind of tooling that was created around the time however fast forward 10 years and we have the monads the megs the sonic and this is quite a substantial jump so if we take a look here we can see that we have um 10,000 transactions per second. It's a big jump from from 15 and Sonic boasts a theor theoretical max of 400,000 transactions per second. Now, this is a huge jump and progression in uh blockchain technology. And for context, the existing tools, you can't just double the speed, triple the speed.

We're talking about a 25,000x increase in the speed that blockchains can now support. And so of course the tooling needs to follow too. All right. So what I want to chat about is what is modern indexing. All right.

And there's three pillars to what we see as modern blockchain indexing. First off, it's performance. Can the indexer keep up with the transaction throughput of that new chain? And not only just new chains that are faster, we've got smart contracts that have been in production for 10 years now that people are still interacting with. There's a lot more history.

There's a lot more data to to look at. The second aspect is multi-chain. If you are not worried about fragmented liquidity, why not deploy your application on many different chains, many different EVM chains, you know, go to where the users are. Makes complete sense. And the final one which I think is the most underrated of modern indexing is flexibility.

Right? So we'll go into that a little bit more in detail but it's really around uh using an index as a full end toend backend solution and not just for like maybe one part of your stack. All right let's go into performance. So first off who here knows metry? If you do, let's definitely trust each other later.

I'd love to connect. Um, they are the gateway to circles. Um, it's a really cool uh UBI like uh token. Best UX out there. You don't need to um you don't need to have any gas tokens.

Works out the box. Super friendly. It's literally just minting money. Um, but they have quite a unique case in that they're indexing 80 million events and 200 over 250,000 contracts. All right.

And performance becomes incredibly important. So here's their indexer deployed and you can see it's just under 80 million events and it took about nine hours to sync. All right. Now, with other solutions, this could take in the order of weeks, if not months. And it makes it really difficult to to iterate and develop features when you've got to wait so long.

So, performance is really critical for these big data applications. Um, scale isn't scary anymore. It's a feature. All right, let's chat. Multi-chain.

Um, who here knows Sablier? Woo! Lots of hands up. Cool. Um, so Sublia is kind of onchain token distribution.

They have about 344 million in TVL at least when I made the screenshot. Quite impressive. But I think what I would really like to highlight here is they have they support over 20 networks. All right. And Sablia specifically supports has four different indexes for different parts of their application.

That's 80 deployments to manage. It's a crazy amount of deployments to manage. So this is their this is one of their indexes on our platform. And on the left you'll see a bit of a zoomed in view. And then on the right you can see all those different networks um coming up there.

And it's just a huge uh devops experience improvement that you're able to deploy a single index and index multiple networks. Cool. Another special mention in terms of multi-chain PIML account abstraction super hot right now. um they're dealing with over 100 chains and account abstraction transactions are extremely high in uh the amount of transactions that are created and so they really need like this performant multi-chain tooling for for indexing their stack and I think uh flexibility I touched on arguably the most exciting pillar of modern indexing is the ability to integrate your indexer so that it's more of an end to-end solution. Okay.

Uh one example I want to just demonstrate here is Monad clip. It's a Monad test net uh campaign around gaming on Monad. Super nice UX as well. Um check it out. I spoke with Haral, one of the dev rolls at Bonad, and he told me he used NV to index this because it was just flexible.

They've got a single tool for logging to all the different games, which makes it really nice. And this plug, you set a username and this username, it works across all the different DAPs. And because within the handler, within your event indexer, these functions are asynchronous, it means you can integrate with other tooling. you can fetch that data from another tool and bring it into your index and have more of a complete uh indexing framework tool. All right.

So I think a little bit more on the flexibility side of things you can so this is just the indexer deployed. Um you can also do some interesting use cases like you can fire events off directly. You can trigger notifications directly from your indexer. You can send telegram messages. You can create uh keeper services that actually call another event on chain.

Um really you can plug it into your existing you know backend solution. You can plug the data into your own database format you like. It's just really powerful because you have the world of let's say TypeScript or JavaScript at your fingertips. So flexibility is one of the major quality of life improvements with um uh modern indexes. All right.

And just to touch on it a little bit, um, we've got two parts of the stack. We have Hypersync and Hyper Index. Now, Hypersync is kind of like this massive performance engine that sits beneath. And Hyperindex is that more developer friendly framework that handles the multi-chain aspect for you and the flexibility side of things. So these are kind of the two pieces of the puzzle together that we've uh created which um helps to develop these things.

So here's a nice example too. It's it's a little bit of a Bose slide if I'm being honest mostly because it indexed 45 million events in about an hour. Um but this indexer is indexing every single uh 50 million safe events. So safe sorry 50 million safe contracts. So really just designed around scale.

Cool. Um here's some benchmarks in comparison. Um to touch on hypersync a little bit more. I won't go into too much detail on it, but it is kind of like the engine beneath. It's our version of a blockchain node written in Rust designed for getting data.

All right. So if we look at Ethereum 15 transactions per second we the way to currently access blockchain data is via RPC. All right this was very functional with Ethereum but when you start getting to big data problems it becomes quite limiting and so we didn't necessarily reinvent the wheel. We took a lot of cutting edge modern database technologies from the web two space and bringing it into to web 3. So I won't go into the secret source too much of it but um written in rust purpose-built design for getting data and searching through you know the specific blockchain schema.

All right, this is a bit of a copout slide, but uh I really just want to highlight that indexes are a tool to building your application. They're not the be end or end all. It's a little bit like Chatty PT is a tool to help you develop your application. And so I think the future DAPs that we see is really going to shape what the future of uh indexing looks like. Um really excited to see what comes from this hackathon this weekend.

It's going to be very very cool. All right. And I've got a little bit of time still, so I just want to run through a couple practical examples of things you can build uh using kind of a more modern cutting edge indexing solution. So on the left there is a headless block explorer in the sense that it's a front-end codebase that's using this hypersync engine as its kind of back-end database. So, it's just a front end and uh this developer was able to build out an entire block explorer.

Um, who here knows revoke.cash. Um, I hope for only good reasons and never having been hacked, but revoke.cash is a very good security tool. Really cool by Rosco and it allows you to uh uh revoke unlimited approvals.

So, it's a bit of a security thing. Uh, and this tool snub is a a tool that searches through all these different chains and finds where you've done unlimited approvals and it'll link to revoke.cash so you can undo those transactions. Cool. Another two more hypersync focused uh tooling.

Um the one on the left there is a kind of a streaming API and you can see uh the speed there is about 490,000 events per second. So it really does allow you to to deal with big data problems. And the tool on the right there is called chain density and it's real time analytics on specific contracts. It'll it'll perform the analytics without it being pre-indexed. So you can say this is my contract.

I want to get an idea of when people have been interacting with it. And you paste it in, select the network, and in real time it'll search through the entire chain for every transaction. Um, here's another cool uh demo. This is a UI for UNISO v4 analytics. And you can see uh that feed coming down.

It's a bit light but those are real time swaps happening and you can see them across all the different networks polygon world chain BSC um really nice just to see some highle kind of insights into into that data and unisoft v4 hasn't been around long already has 400 million events emitted cool another two cool pieces of tooling. Um, a price arbitrage uh dashboard. Obviously, arbitrage is kind of like risk-free money. So, if you can visualize it, you could potentially take advantage of these price differences and benefit from the arbitrage. So, nice just to see that visually.

And then uh that project on the right there is Oracle Wars. It is Redstone's uh very very fast oracle and you can see that being indexed in real time super fast. I mean just look how fast it's moving along. The the amount of transactions that Mega Es can support and the latency is is quite exciting. I think we're going to see some really exciting apps built in the future.

I think it's Visa that supports about 65,000 transactions per second. um which obviously we wouldn't be able to move every Visa transaction onto Ethereum at 15 transactions per second, but these new cutting edge blockchain blockchains will essentially allow for this specific use case and they need the tooling alongside it. Cool. Just some really nice feedback points from different people in the community. Um if you want to hear more about it, definitely check out our our landing site.

Um, mo modern indexing, just to recap, it's performance, it's multi-chain, and it's flexibility. It's ability to do more in your indexer than, you know, just get that that raw data. Thanks so much for listening to my talk. Again, my name is Denim. Um, if anyone's around, I'll be here the rest of the day as well as, uh, over the weekend if anyone wants to chat.

Uh, I'd love to hear what you're building. Uh, it's always very very exciting. Um, and yeah, take a look at us on Twitter. Thanks so much. [Music]