A Web2 User Experience on Web3 - Evegny Vakhteev

you have to hold on to things okay uh i mean there's guru studio and founder and that's going to be like a technical architecture kind of talk about how we delivering web to user experience for our web 3 application so index guru we index in 70 daxes across multiple chains and like enormous amount of trading pairs tokens whatever we have and obviously it's kind of impossible to do it solely on web 3 because like as everyone know like the node itself under the hood it's a key value storage and you're very limited to what you can do with it how you can access the data how you can access how you can select and search the data of course it's it's decentralized but it's it's slow to write and kind of slow to read it's very costly to do boss things so to power up something like that you need some cushion layer in between web 2 and web 3 and that's the only way everyone proceeding right now so solutions wise we need indexing obviously so to get web 3 into web 2 then to get our web 2 web 3 application working on top of it uh let's say azure scan right i just stole the picture from some explainer from youtube but basically what they do is they indexing blockchain into into centralized db and then all of us can explore all those things and others can't just how it works or the graph the graphics cool it's uh kind of the same approach like indexing stuff but then the data layer itself is decentralized and you have like a pretty complicated uh way of contributing to it as well as uh like it's paid to consumers and curators so it the graph is really cool but for applications like ours it's not zay yet in case of like a db tooling analytics toolings around it so traditional db is offering much more tooling for to analyze the data so you can execute actual analytics jobs on your nodes and so on and so forth it doesn't exist in the graph yet here is how we doing it and uh that's pretty much what dex guru uses to get all that information so we we actually currently using three different dbs and uh bunch of indexation workers for every for every network we support as well as we parse in uh half a ton of different events from different dexes and put them in all those dbs simultaneously so we have radius which is like a cache for our real-time data so users whenever the hidden kid on our website can see those real-time strings really real time then we have elasticsearch where we have like search select and aggregation possibilities as well as we're executing some data analytics scripting on top of it and that code is actually executed on elasticsearch nodes so you kind of utilizing charted computer powers there uh that sets uh when you when the index in millions of transactions and trying to calculate prices of them like trending categorize traders and figure out if uh whales are selling a buying right now that's that's crucial thing to do also using click house uh and click house it's mostly for my for this thing called materialized views and click house that's that's like an uh pre-calculated view of your data based on your ingestions so click house is used on our website to calculate balances of every trader so you can see like when he was selling or buying and uh what's the top token holders and so on and so forth so we have like a zoo of different databases in here where and we indexing in all of them right now because all of them providing different tooling and uh when whenever we're indexing different chains we we are actually hitting different performance issues for example indexing ethereum is not the same as index and bsc uh much more transactions blocks blocks coming much faster so you you're figuring out next approach to it maybe you're switching db and so on and so forth so that's where you are at so what's actual pros and cons of course i already talked about it it's like traditional db's been there like for many years and uh they they're really good at things they already do that you can you can get as much speed as you want you have in-memory dbs you have charted dbs you can you can actually paralyze your queries across multiple nodes if you have replicas so like squeezing up the speed is actually what what what makes traditional db solutions much faster than basically grabbing it against web 3 even if we could request the data from web3 so there is all kinds of database as a service solutions from uh from clouds so you can actually instead of managing it yourself you're basically outsourcing it to aws or something and then basically just getting out getting getting everything out of it uh this that second the next thing is about like business scalability actually if uh you grow in your product you need to scale up your team really fast and whenever you're trying to find web stream developers or anything related to smart contract or anything like it like uh it's really hard to find people on that market as well as prices are really high but when you're trying to solve this problem you figure out okay i have indexation i have whole web 2 tooling so i can actually hire web 2 developers for everything what's not related to web3 so you can scale up your teams which solely focused on organizing back-end organizing data analytics and maybe doing some new apis and so on and so forth with web 2 developers uh yeah and all those two links which is available like executing actual queries and aggregations on database nodes themselves really complicated stuff sometimes like you have scripts running on the notes calculating some averages means so figuring out if data is biased and so on and so forth of course conservatives you centralize on everything right and uh [Music] what what what we're actually going to do with that because we all know that we want to go in direction of decentralization instead of centralization so how are we going to decentralize solutions which are needed for uh analytics and like in blockchain everyone understands that we have like tons of data like uh thousands of transactions happening every every block and every minute and even for uh traditional kind of uh indexation and database problematics it's a huge pile of data but if we're trying to do decentralization at the same time it's even more challenging and so on and so forth so what what we do to decentralize things and how we are planning to approach it so currently we are like a year in development and during that year we try different approaches and we see how to build some of the analytics on let's say on radius radius is key value storage as well and whenever we trans things on radius obviously if those things work in radius we can migrate them to something like layer 2 plug-in for the nodes we should work with level db same way it's working with radius and then we have something like plug-in for some particular feature some particular analytic feature or maybe set of them and then we can actually decentralize that plug-in which we're hooking up to every node that's pretty much how the graph approaches it but maybe we would need some more customization that the graph provides uh and that actually comes to a certain thing we have in mind that we need something like l2 but for data layer so nodes would be would be working themselves as a right now but we would have like plugins which which are working directly with nodes db like without any web stream api in the middle and organizing themselves into layer layer two layer two but for the data uh with uh additional features on how we can so let's say it would be layer two for the phy data or something like that so you would have actual swabs burns means from every dax possible already they are ready for querying in different kind of ways and so on and so forth so actual uh my talk was labeled as workshop i'm not sure how many developers i have in here any python developers oh really okay uh so to to get to the point to actual like developers problem here so this is uh like the cycle we do for every block so to actually get all the transactions from all the transfers as you see them another scan let's say for some transaction from the block those are all operations you need to do against uh like evm node so you need to get get lost block numbers and get all transaction indexes from the blogs and request all transaction transactions by their indexes so it's like a loop there then request recepts because you parse and recepts to get transfers as in basically intercepts you're looking for transverse event and that's how you parse in the same so if you're looking on something like [Music] this is a transaction this is gone and you see here ezra scan already did a nice job and actually parsed trans person here and you see tokens and everything so how okay it's too small sorry uh how the hell you would get it parsed out of your pure web stream out so from webster notes you're getting something like this and it's like on first recept logs and uh because we have so a little amount of developers here we can go through the code actually but probably some not useful but you guys can uh go and uh check actual github in here and uh you're just cloning it you can you can you basically have that portion sorted out so you can actually start indexing transfers into db of your choice and start analyzing them and those are years to 20 transfers and by by swapping apis there you can actually start indexing nft transfers if you want so that's the idea and that's why we need to have indexation because this is a way to get transactions for one block and we're converting it in in db we're converting it to the view where it's easily searchable or requestable by any key we we want to request it by so that's a complexity and index guru we have uh sets of those workers which are actually like working with dozens of nodes on different chains and that's how we get in like real time data uh in summary it's it's like uh 1500 docker containers right now which are running that kind of code all the time so yeah that's a question section any questions on so this work was something like with what celestia's doing kind of trying to so like a data lake i i hadn't checked i hadn't checked them but uh what we do so all data we index in we actually have as an api available like as a public api so you actually after after we already index that normalize the you can you can use our public api to go through transactions which are happening on old access across different chains and see like a pricing feed which we calculated for you already so we we're providing some access to our to our data already it's not like to also data and we we plan on extending our api so you will have like more access to raw data and maybe just uh kind of giving it to the world as a database interface makes sense in some in some way you know like yeah that's that's definitely what we may say yeah any other questions cool uh so here is my telegram twitter dex guru uh we are hiring as everyone here if anyone interested we sit in a pile of data and looking for someone to start analyzing it we're doing some categorization right now but looking for more things okay that's it then thank you