Bridges: how not to hack up everything with Alex Shevchenko

All right. Uh, apologies for the delay. We had some technical issues. Thank you for sticking around. Uh, my name is Alex Nathan.

I'm a co-founder and head of analytics at Metrica. And today I'm going to be talking to you about uh, MEV relays. If you're not familiar with Metrica, uh, you can visit our platform. Uh, we have free dashboards and alerts for the community at e.metricka.

co. Uh if you like what you hear today, you can follow us on Twitter, metric co. And if you scan this uh QR code, you'll be able to join our Discord. So uh before I get the meat of the presentation, just want to say a couple of words about Oh, I saw you were trying. Do you want to try again?

Um yeah, before we get into the uh meat of the presentation, I want to say a couple of words about Metrica. Uh so Metrica is an operational intelligence uh platform for blockchain networks and uh we work with a number of leading protocols like Ethereum, Solana um as well as uh key players in ecosystems like Lido uh block demons right there. Um and uh what we do is we help all these different organizations and entities uh monitor blockchain networks um with uh our analytical tools. So we provide them with uh alerts, APIs, dashboards so they can monitor the performance of the network real time and uh be able to troubleshoot issues uh uh as fast as possible. So uh the agenda for this uh presentation is the following.

So we'll start with a quick overview of the MEV supply chain as it is today and talk about the role of relays um that the relays play. Uh then uh I'll show you something that we did with a GL with a grant from LIDO on monitoring uh the performance of uh ME relays. And finally, we're going to touch on a somewhat controversial topic which is uh OPA compliance and ME relays. So I imagine most of you have seen some version of this slide uh over the last few months. Uh this is what the ME supply chain looks uh in the postmerge world.

Uh and compared to uh premerge there's a few different uh new players that emerged. Uh so on the left on the left we have the searchers. Uh there's about a hund of them but it's kind of hard to tell uh precisely. Uh and basically what the searchers do is they scan uh memp pools and they try to find me opportunities like um front running sandwich attacks and so on and so forth. Uh so searchers are very specialized bots and that's all they do.

Once they identify uh the me opportunities then they submit bundles of these opportunities to block builders. Uh so block builders as the name suggests all they do is they create blocks that are ready to be committed to the ledger. Uh there's about 120 block builders uh right now. And uh well as you can see there's something in between the block builders and the validators and that is the relays. Um there's about no actually there's exactly 11 relays right now.

And uh as you can see relays are just an intermediary between uh block builders and validators. Uh but as you can imagine the devil is in the details and relays have to do a lot more than just simply relay blocks alone. So how do relays exactly work? So relays constantly receive blocks from uh the builders that submit blocks to them. And what they have to do is go through all of these blocks and find the one with a uh with the highest me reward for the validator.

Uh in addition to that they have to ensure the validity of the block uh that it uh you know uh the rewards check out that the validator preferences are uh respected and then uh what it has to do is it has to send just the headers of the block to the proposer the validator. The reason why it doesn't send the whole block is because otherwise we wouldn't be having proposer builder separation. And uh once the proposer which receives a bunch of uh bids from all the relays it's connected to picks the the final block the one with the highest reward then the relay is responsible to uh propose a block on the validator's behalf. Now there's also something that I left uh out of what relays do which is they also well some relays they censor transactions based on who the sender and the receiver is. Uh but we'll get to that in a minute.

So uh relays essentially operate on trust right they're trusted by various uh participant in this supply chain. So validators trust that relays will get them the you know as fast as possible valid blocks uh that are profitable and also validators expect that relays will propose the blocks on their behalf. Uh we have seen a few cases where relays actually fail to do that. Um, number two, uh, builders trust that relays will transfer the their bids, uh, as fast as possible to the validators so they can be re be rewarded for their work. Uh, now this is especially important when builders and relays are not operated by the same entity.

Um and finally there is validators uh in Ethereum with uh regulator regulatory obligations uh and they trust that relays that claim to be compliant are actually compliant. So there's a lot of trust going around and our motto is trust but monitor at the same time. We cannot blindly trust that relays are doing what they're supposed to do. So we came up uh quite recently as I said with this slide ground with a solution to monitor relays and um uh we went live with these dashboards and alerts last week. So uh you can check them out and uh give us your feedback.

So there's two sets of the solution. one is the the dashboards that allow you to monitor the performance of relays in real time and the second one is the uh set of alerts that will notify you if relays are not doing what they're supposed to be doing. So how does this solution work? Um we uh use something called a relay monitor. Uh this is an open source project by Alex Stokes from the Ethereum Foundation.

And what a relay monitor does is the following. So in every slot it impersonates the proposer of that slot and then connects to all relays and collects bits from them. Um in addition to that we've slightly modified the the relay monitor to collect other metrics like the roundtrip latency for getting the headers on the relays. Um and in addition to that because latency plays a key role in uh when you're collecting bids we decided to deploy five instances of the relay monitors across the globe. So right now we have five relay monitors two in the US uh one in South America one in Europe and one in the APAC region and we keep pinging all 11 relays collecting bids and other metrics.

Uh so all of this data ends up in our data lake which along with our analytical capabilities is powering our alerts and dashboards. So in terms of dashboards, we have three dashboards right now on a metric.co for uh me relays. The first one is a relay overview dashboard. So gives you some key information like which relays are online versus offline.

uh the total me blocks uh in real time as well as uh a time series uh bits we collect from relays the total numbers and so on and so forth. The second dashboard uh focuses exclusively on the operational performance of relays and here some highlights are uh the bids missed. So this is a very good indicator of whether a relay is actually online or offline. Another one very another very important one is the bit faults. So uh as I mentioned earlier relays are tasked with actually ensuring the validity of blocks.

So with the relay monitors we simulate what relays should be doing and if we find that something is off then we trigger uh then we uh log a uh bitfold um message. Uh and then as I mentioned latency is another very interesting metric. So because we have five um relay monitors deployed uh we can give you an uh median latency across all the regions but we also give you the ability to doubleclick on a particular region and see how relays uh perform in different regions. And the last dashboard is on economic performance. So we look at uh the reward distribution uh for different relays just to know what's uh what's happening at all times.

In terms of alerts, uh we have currently four alerts and all these alerts have uh integrations with a variety of uh channels. So we do pedagra duty, ops genie, telegram, discord, slack. Um and the four relays that we went with were the following. So consecutive misbe alerts. So again, this is a very good indicator of whether a relay is online or offline.

Uh the second one is a bit fault. Uh again checking to see that the the relay is validating blocks properly and actually yesterday we this alert triggered for the first time uh in a while and uh we found that a couple of uh relays were having trouble processing registrations. Uh the third one is relay latency. Uh so we've analyzed the data and we've seen that uh a relay well 99% of bids are sent within 240 milliseconds. So if a relay is consistent cons consistently taking longer than that then we trigger a relay latency alert.

And finally we have the intended fee recipient alert. Again not a very common alert but basically what it does is if the relay uh sends a block where the recipient of the me rewards doesn't match the preferences of the validator then we trigger this alert. And again, this one went off for the first time uh last week with a couple of blocks route um relays. Um so this is the first part of the presentation. Again, uh would love your feedback.

Uh go on etopmetka.co, check out our dashboard, sign up for alerts, and either talk to us offline or uh ask us any questions on Discord. Now for the second part of the presentation, uh I want to talk about alpha compliance and relays. So one of the core premises of Ethereum is that uh Ethereum is credibly neutral which means that Ethereum is fair, transparent, uh impartial, so no single entity uh has control over how the network operates. And after the merge, well I guess with the merge, uh this changed for the first time because uh Flashbots launched their first Alpha compliant relay.

And again this means that the relay was censoring transactions based on who the sender and the recipients were. And uh today there is more uh compliant relays. So blocks blocks route regulated Eden and block native are all uh compliant relays. They're they are censoring transactions. Uh but first let's take a step back and talk about what OFAC is.

So, OFAC stands for the Office of Foreign Assets Control and basically they are tasked with putting together the SDN list, the block person's list and once a person or an entity ends up in that list that means that US persons and businesses cannot be facilitating transactions with these businesses or individuals otherwise they're going to be facing uh fines. And the big news here was last August, OFAC placed Tornado Cache, the famous cryptocurrency mixer on the SDN list. So that immediately meant that validators that wanted to be compliant, they had to be very careful with including tornado cast transactions in the blocks they were proposing. So the the big concern was that suddenly ME boost was getting a lot of traction. uh I think around December we're getting 90% of blocks coming from MEV boost and uh it's not just that me boost was gaining a lot of momentum is that flashboards was extremely dominant uh the flashbots relay at some point was responsible for 80 plus% of me blocks and uh since then the community has really responded there's been a number of uh uh initiatives to try and mitigate this problem uh so flash open source their relay and their and their builder.

Uh now validators have control over which blocks they accept with the mean bit settings and today we have more non-ensoring relays. So uh there is seven out of the 11 relays are non-ensoring. Uh so in light of all that that big controversy uh we decided to look under the hood and uh ask the question well are the relays that claim to be compliant are they actually compliant? And what we found was quite interesting. So we analyzed uh some data from uh November 8th, 2022 up until uh February 1st, 2023.

So what you can see here is the blue line is all the blocks containing tornado cast transactions from that period of time. Now the green line is blocks containing tornado cast transactions that have been uh relayed by me relays. And then finally the red line is blocks containing tornado cast transactions that have come through supposedly compliant relays. And you can see that from November 8th up until the end of the year, the end of 2022, well, all tornado cost transactions were coming through just fine. Uh, and I think at that point, the supposedly compliant relays figured out what was happening and they fixed it.

Uh, but even now, you can see in January there's some spikes every now and then with transactions coming through. Uh if you ask why this happened, our hypothesis is that uh on November 8th, it happened that uh some more tornado cache addresses were blacklisted and the supposedly compliant relays never picked it up. Uh and it took them a month and a half to fix it. So uh what are the key takeaways here? Uh relays have been called dump pipes just because they are moving blocks around but clearly they effect they affect many participants in the ecosystem.

Uh and we really think that uh builders and validators should uh monitor uh the performance of me relays because uh the rewards depend on uh on them. But I also want to make a comment about uh me boost and censorship in general. So we believe that MEV boost is not synonymous to uh censorship and there's a blog post that uh my team has been working on and it's going to come out next week uh where we actually looked at mele data to see you know what is can we quantify this censorship and what we found out is that 99.5% of tornado cast transactions that are in the meool end up on chain so there's not a lot of censorship ship in terms of uh whether a transaction is successful. Uh where there is some censorship seems to be in the waiting time.

How long does the transaction a censored transaction stay in the meool and what we found out was that the median uh wait time for regular transactions was 8 seconds but for non-offa compliant transactions it was 22 seconds. So it is a big difference. It is three times more but still the success of a off uh non uh non-compliant OFAC transaction is not affected. So uh next steps if you haven't visited our dashboards already in the past what 20 minutes uh please do. Um you can go to e.

metka.co/me comev. Um, join our discord. You know, would love to hear your feedback uh and improve our dashboards and alerts as much as we can. Uh, as I mentioned, we have, uh, a number of blog posts coming up coming out this week and also next week.

So, all our blogs are published at blog.metka.co. And, uh, we also have a booth uh, in Devtopia. It's uh, further behind us.

Uh, so please stop by and say hi. Thank you very much. [Music] Heat. Heat. N.

[Music]