Amir Bandeali I DEX Aggregation Is Harder Than You Think I 0x I Pragma San Francisco 2024

[Music] we're actually going to continue this theme of of Defi and kind of how do you actually make sense of all the stuff that we're seeing especially the proliferation from all the different chains and rollups out there so we'll continue this with our next talk and for this talk I'd like to invite Amir on stage from zerox he's the co-founder of ZX you may know them mostly from the product called matcha and he's going to talk about how simple Dex aggregation is with an asteris so please give Amir a big round of [Applause] applause thank you thank you uh quick show of hand so I could calibrate the presentation a little bit uh who here knows what Dex aggregation is okay uh who here knows what me is okay who here knows what a solver is okay okay pretty good pretty good cool well yeah today uh as as cartic shared I will be doing a presentation on why Dex aggregation is harder than you think um so for those of you who aren't familiar with ZX um we started building liquidity aggregation technology in uh around 2020 we've been around since uh 2016 uh working on dex's before that um it with a very different model um in 2020 uh it was a very different space ethereum was pretty much the main blockchain that was being used at the time um and we we built the first version of our liquidity aggregator um and uh we finally upgraded uh to a brand new architecture uh just last month and I wanted to talk about some of the design decisions that went into that and some of the the learnings uh that we had from the first model uh so yeah first all I'll just give a brief overview of zxp 2 uh and then uh everything else cool so yeah as I mentioned uh XXV 2 is our latest liquidity aggregation API uh we worked on it for over a year uh it was pretty much an all company effort all all hands on deck it's uh difficult to you know prioritize such a large change uh within the company we decided that uh this would be the most impactful thing that that we could possibly do um given that we were on this very functional but Legacy architecture uh from before um our smart contracts our pricing engine uh and our apis were completely redesigned to address the biggest pain points of our customers and our own internal pain points which ultimately end up impacting the customers uh so yeah I'll get into some of the features and the metrics that we designed to ZX V2 around first things first is the pricing engine uh pricing is well uh argon is the name of the router that we use internally and and externally um it's a very fancy uh pricing engine um pricing is King everyone knows that generally if you're using a DEX aggregator you're doing it to get a better price it's the first thing that comes to mind it's table Stakes to have very very good pring um and it's a very difficult problem uh routing is not a solved problem there are a lot of academic papers out there uh many of which uh we used in uh in recreating our our routing um but they're generally not adapted for blockchain environments um and it actually takes a lot of effort to um build them out in the right way um routing matters a lot for large trades uh especially uh on ethereum where gas costs are pretty high and it matters for basically all trades on the cheaper chains uh you know you could do like a $20 trade on some really cheap chains and you'll just have like an insane looking route uh there's sometimes a narrative that it's not worth using an aggregator because like it's just going go to Unis swap or whatever and gas costs are higher um that's like a 2020 narrative like that is just incorrect today there are hundreds of different decentralized exchanges out there and you will not get the best price unless you're using an aggregator and today they're also very optimized uh for gas costs and such um so yeah argon uses a combination of published and unpublished techniques I am not going to get into those today if you are interested in a more technical Deep dive uh of different routing methodologies uh you could check out this other presentation that one of my colleagues Duncan gave uh at uh I believe it was eth Denver earlier this year and also follow this Twitter account sorry X account uh machra routes for some beautiful routing art uh which I will show you in a second uh these are just some pretty crazy uh routes that we see in the wild today these are trades that actually happened um you could also go to matcha XYZ our consumer facing Dex aggregator and you could just see the routes for every single uh quote that that you get um it's these exact same Soni diagrams uh so here's an example of a $50,000 madic trade not a huge crazy trade by any means but you could see it's just like a pretty insane route uh around $400,000 Pepe trade on mainnet uh as far as mainnet goes that is not a huge trade believe it or not but again very crazy route uh and then finally when $ 1.8 million uh we trade on arbitrum again pretty crazy route on those cheaper chains you will see some pretty interesting things this one I feel like I could hang up on my wall all right well with good pricing uh also comes a lot of challenges uh routing introduces new complexities um the first one being latency when you are trying to find really really crazy routes and utilize as much liquidity as possible you're scanning thousands of different liquidity pools across hundreds of different dexes and trying to find you know the exact right combinations to to get you the the optimal route um that is just a huge search space of of possible routes and uh generally as routing complexity goes up so will latency um and latency is really important for Dex aggregators and I I'll look get into uh why that is exactly um also uh kind of a lesser known issue is uh quote accuracy so when you have really crazy routes uh there is a possibility that your quotes will actually just be incorrect and you're going to be showing users a a price that's often too good to be true uh so why does latency matter as routing complexity increases you need to get latency down uh it's a constant battle um it's really important for Consumer facing apps um you know there are a lot of studies just like in consumer apps generally like not specific to the the web three space at all that the lower latency you have the the better your user conversion is going to be um but just from a trading perspective as well there are um a lot of reasons why you need low latency um so there are a lot of solvers and trading Bots that use Dex aggregation apis uh in these auctions these you know intent based auctions um and you know if you are participating in an auction that is only a few seconds long like obviously latency needs to be quite a bit lower than that also a lot of these newer chains cheaper chains um have pretty fast block times if your latency gets out of control then uh you might just end up with stale quotes right you generated a quote at block X it's already block x + 10 and that that quote can can no longer be executed I don't think this is something that people generally think about when they think about Dex aggregation but there is trust me a lot of work going on behind the scenes to make sure that latency is under control quote accuracy this is more of a hidden problem I feel like this is uh I don't know kind of the the Dark Forest of uh Dex aggregation um because often users are getting I would say un intentionally uh deceived um so there kind of two categories of quote inaccuracies um one is just you produce quotes that are too good to be true they cannot possibly be executed uh and the second class is you produce a quote that could hypothetically be executed if you did it at this exact second and like there were no other transactions before after um but in actuality there's a very high probability that that quote will not be executed for a variety of reasons um so I'll explain the two good to be true quotes a bit um first of all depending on the routing algorithm you use uh it may or may not be possible for it to just simply find routes that are too good to be true um yeah they're just kind of false positives um argon is built with a constraint that it cannot do this um other routers uh that we've seen some like pretty big names um actually don't check to see if their quotes are valid and they generally do this to get latency lower um so they're just kind of taking shortcuts um and you know in a way it's it's a win for them because they show users prices that are too good to be true and the users might end up going with that um but for us trust and accuracy are really really important um and we are making sure that we are validating all quotes to the best of our ability um and then finally if you're using any intent-based decentralized exchanges they are notorious for this problem um because uh generally they are only showing you an indicative quote of what you can get you can't actually just like take that quote and execute on the blockchain um you just need to see a price say you're okay with it sign that intent and then an actual auction begins and a solver or a filler will will try to get you that price um but at times that indicative quote could just be wildly off um I've definitely had the experience of seeing a quote that's just like look crazy and I'm like all right you know I I love matcha but like I'm Gonna Save thousands of dollars by using this I'm GNA do it and then I just like sit there waiting for 30 minutes for my order to get filled it never does price Moves In the meantime and and I get screwed so it's a really bad uh user experience and and I think it decreases trust uh in products um but generally these apps are not really getting punished because it's not a very visible problem so again we are validating quotes to the best of our ability and generally not producing these and then the other issue you have quotes that technically can be executed but they just will realistically not be executed um so if your routing is really crazy you could actually find find Arbitrage opportunities within the routes that you calculate but we know that the space for MV and Arbitrage is extremely competitive and good chance that when you go to fill that order someone will have already captured that opportunity before you um so you might as well filter those out um also different liquidity sources are just more or less susceptible to MV um and you might want to account for that uh in in your routing for different types of pairs and finally some routes might just be too crazy to realistically be executed um yeah especially on a a faster blockchain where things are moving really quickly um these are metrics that we are looking at all the time in simulations when when we are building or improving the routing and uh we have definitely the best quote accuracy in the space across both of these types of uh quote inaccuracies uh V2 also has enhanced coverage and security what do I mean by coverage so the the amount of combinations of uh things you can do in the blockchain space it's just like growing at this crazy exponential fractal rate um there are constantly new chains being launched I mean you could probably launch an L2 in like 30 minutes these days if you really wanted to um there are all these different new types of tokens like uh fee on transfer Buy sell tax tokens um and on each of these chains there are all sorts of different types of uh liquidity sources or or decentralized exchanges so what we mean by coverage is you know as a DEX aggregator you in an Ideal World want to support all of these things but the complexity is just increasing at a crazy crazy rate and it is very difficult to manage um coverage and security go hand inand so this this is something we discovered um as well in in version one um so we we had this upgradeable architecture and it worked well for a while but you know back when we launched there were probably like five different liquidity sources and you know we we kind of saw where the the puck was heading but uh didn't necessarily expect there to be like hundreds of of different decentralized exchanges and you know we would roll out upgrades relatively frequently but then just the number of code Paths of you know things that could interact with each other uh you know also grew exponentially and it became very difficult to add new features without or while feeling confident that security would not be an issue that you would trigger some like random unknown code path so we designed a brand new architecture for our smart contracts where there is virtually no risk uh to the end user and we could essentially upgrade as frequently as we want um we are generally upgrading the smart contracts every two weeks now um so with these uh low iteration costs you can now increase coverage which ultimately ends up benefiting the customer um I've had this conversation so many times with with version one where uh a decentralized exchange would reach out to us and be like hey can we get integrated into matcha ZX and we'd run some simulations and we'd be like all right so this is going to improve pricing by one basis point uh you know in this one trade size bucket uh for like these three token Pairs and it's going to take a week of effort so in isolation those things are just never worth it and you know you kind of we kind of fell behind in adding new liquidity sources now it's extremely easy for us to do that in a very secure way um you know could do it in a matter of hours um so we just have all of the liquidity sources and we are you know keeping up um so overall V2 is built for security and for scale so what is this new security architecture how does it work um so yeah in V1 we had this monolithic smart contract that would get upgraded uh over time and everyone would set their allowance to this so yeah as I said adding new things became very scary because if anything went wrong poof billions of dollars lost um in V2 we use uh this new permit 2 architecture so we only ever ask the user to set very short-lived ephemeral approvals so their funds at risk uh or their funds at rest will never be at risk you can't be sleeping at night and like all of a sudden your your wallet gets drained because you actually just don't have allowances set to our smart contracts at all until the point that you are ready to to make a trade um there's also another smart contract uh called allowance holder which is uh more of a traditional familiar flow uh which I will get into in the next slide um so uh I already talked about permit to a little bit um bring low on time so I'm going to try to speed through um allowance holder uh is a kind of an intermediate contract uh which you set your allowance to and it yeah sort of just because it's like a middleman holding the allowance uh and you're not interacting with the the actual settler contract uh again this creates a barrier so that your funds at rest will never be at risk so with expanded coverage uh we've been able to ship more quickly uh V2 includes support for fee on transfer tokens um this is an idea that's been around for a while but recently really took off with the mcoin Mania so uh these tokens have a tax built into them if you buy or sell a token you might have to pay a couple percentage points um didn't think that would last but it did um and now that is supported uh in V2 uh in a very efficient way we have definitely the best coverage uh of meme coins out there um and we have also rapidly been add new blockchains uh so since launching about a month ago we already added four new chains and we have six or seven more chains on the road map uh by end of year uh ux and devx uh so these are just some kind of random pain points um that we discovered ourselves and through through customer conversations um this one is uh for all the the finance guys in in the audience um so when you know you have a swap feature in your app uh you're monetizing it you're collecting fees and all these different tokens and then you know you realize you have like a 100 different tokens 200 different tokens in your wallet and you need to sell them all into stable coins at the end of the month uh and this is a very onerous process uh most x aggregators uh don't really allow you to pick which token you charge fees in um it's hard to do in a gas efficient way actually um but yeah this is uh a a highly demanded feature that is Now supported in V2 so now you could just take your fees in like usdc or ether whatever you know maybe five different tokens and you don't really need to go through this onerous hedging process uh you're also no longer paying gas costs to hedge those tokens um and we also have uh enhanced quote validation so this builds trust in uh the quotes So that you can uh know that they are accurate we validate the gas costs very precisely uh fun fact we actually discovered that some of our users were literally using our API only for the gas estimates um turns out that is actually one of the harder problems when going multi-chain uh as well um and we also have a new field in the API called issues so it'll tell you exactly what went wrong and and what steps you need to do to rectify that uh for your users more of the story don't roll your own Dex aggregator get serious uh use use matcha use zerx uh thank you everyone for the time [Music]