Vector Search for Decentralized Systems | Ben Greenberg

hi everyone my name is Jane welcome to the build Squad stage we're going to be here for the next few hours hearing very interesting speakers the first person up his name is Ben Greenberg he's a devell and he's also writing a book on this very interesting topic a vector search for decentralized systems come up here Ben o I'm going to throw the mic to you thank you uh good morning everyone if we can get the presentation on there it would be amazing yay hello uh good morning good morning it's really uh great to be here to talk about uh I think a really important topic which is Vector search and how to bring it into our applications and our utilities in our decentralized space uh first for we begin my name is Ben and you can find me uh basically at all places at huson rails.com I happen to like Kus quite a lot and I'm an oldtime Ruby on Rails developer so kuson rails.com you can find me basically on all socials love to talk about this topic with you later for the rest of the day or online at some point please reach out and say hi so let's uh set the our conversation with the right sort of frame of mind and to do so we need to really set our expectations which is at the end of the day data is basically garbage now that may sound a little weird to you data is basically garbage but what I mean by that is you know if you collect your trash over the weeks and months and it gets your house smelly and stinky and it's useless got no utility until you start processing it sorting it and Mak it into something that can be used as raw materials for new things in the same way data when it's just a massive heap of stuff that you keep on adding to all it's doing is adding to your storage capacity it's not actually doing anything for you until you analyze it process it and sort it and that's exactly where Vector search comes into play so we're going to cover a couple things in our short time together we're going to talk about what exactly is Vector search a brief introduction we're going to look at a decentralized paradigm for training AI models to help us with it some use cases in web 3 and then a big question which is what to do with all of the offchain data that we need for our applications and utilities as well so let's just get right into it Vector search for those who are not familiar is the idea of taking your data whether it's text audio video doesn't really matter what it is and turning it into representations of that data what's called a vector embedding to search for it through semantic retrieval which is a fancy way of saying searching for it with natural language using human words to find things now not just keyword search but actually conceptual search relational entity search uh conceptual search being able to retrieve back items that are not just keyword matches to what you're searching for and this creates a much more intuitive much more natural user experience in our applications in the things that we are building that's essentially what it is in a nutshell it it takes uh your data whatever your data is Dow proposals nft metadata transaction histories and converts that into what's called those vector embeddings and those Vector embeddings help create accurate search content recommendations fraud detection that's actually right fraud detection we're going to talk about that also in just a few moments and a whole lot more now what exactly is a vector embedding a vector embedding is really nothing more than an array it's a list it's a list of numbers specifically floats meaning it's got a decimal in the middle there and negative and positive integers that is all a vector embedding is now there's thousands of numbers for every piece of data some of our embedding models create thousands of numbers representing each piece of the data so they're very big but those in Vector embeddings each number represents a piece of information about that data but how are those Vector embeddings made how do you go from dowo proposals over here with text into a bunch of numbers over here how is that possible well that work it's not magic it's actually not magic at all it's just pure data science it goes to an embedding model an embedding model is a subset of AI models right so now we talk about AI you got to talk about AI all the time right it's a subset of AI models that takes text or images or audio data of some sort and analyzes it and produces the embeddings embedding models basically work like your brain works you are watching me talk right now about this and you may remember a conversation you had yesterday or something you read a few weeks ago you are mapping the data in your brain which is very similar to what an embedding model does Maps the data to each other it's sort of like a neural network which is really really cool and the way that that basically Works taking one example you may have heard of these three letters GPT you may have used chat GPT this morning at some point before you got out of bed I don't know the way this basically works is you feed the data into the embedding model let's say you're using an embedding model from the GPT class generative pre-trained Transformer and that the data you pass through to it gets chunked up into the smaller pieces called tokens those tokens are sent through layers of understanding layers to understand the contextual relationships the semantic relationships the the the the relational entities one to each other those layers are called Transformers GPT generative pre-trained Transformers and that's helpful to use for generating what comes next G P PT is a class of embedding models that have Auto regressive prediction which means that it can guess what may be the next thing in the data set which is fantastic for all of our code completion tools it's fantastic for when you want to help it you want help to produce the next paragraph in a paper that's where that comes from that's the embedding model process it's not magic it's actually a very simple kind of stepbystep process so and there not just a few of them there's a great resource it's a funny kind of name called hugging face and hugging face contains information on all the open source models including embedding models which are a subset of models as we mentioned and there's not just a hundred of them not just a thousand of them there's some nearly 8,000 embedding models produced by universities open-source developers companies governments all around the world and they're all a little different one from each other all a little Newan want one from each other so there's thousands we just covered GPT as one example but there are so many more of these embedding models so once you have taken your data and you've turned it into a vector embedding and the user comes to your dap and they want to search they go to the search bar that search gets turned also into a vector embedding and now what happens for a similarity search their query as a vector embedding gets searched against your data set which is a vector embedding to find the closest relationship right that's how that happens it's called nearest neighbor search or similari search so for example let's say you have a bit of heterogeneous data in your data set nft metadata recent transactions erc20 stuff Dow proposals all kinds of things and the user comes to your dap and they search on the search bar for something that search gets vectorized and then that search then Finds Its way to the closest part of your data that represents what they're searching for conceptually semantically ETC and then it goes even further into the subset of that data and returns only the most relevant results it's pretty cool and it basically basically creates a more intuitive more natural feeling experience for your users uh how does that all work though how does it find itself in what we call Vector space now I promise not to talk about too much math math is not my specialty either but you may remember from middle school things like ukian math does that sound familiar or cosine right those are used in real life to help determine the closeness the one of the more simple ones which also interestingly enough is one of the most commonly used ones is called dot product similarity which basically measures the sum of the two corresponding elements in the vectors higher sums are more similar lower sums are less similar that's often used for Content recommendations like you watch the British baking show you may like the uh Australian barbecue show that's happening with Vector search using dot product similarity so that's how closeness happens at this point my friends you are pretty familiar with uh the basics of what Vector search is what a vector is what a vector embedding is you're pretty good you got a good basis for it and what's really important what I really want to understand and if you don't understand this yet you you probably will by the end of this conference by the end of I don't know the end of today is that all of the AI generated content AI search AI content recommendations what you see how you see it what the data looks like that's all being shaped by AI systems and that's not going away that's only getting more pervasive more prevalent we're not going backwards we're only going forward Ward so those who control the models the embedding models those who control the data the compute on the networks those folks without overstating it are going to basically control the world which maybe this might sound a little acious but therefore we really must find a way to decentralize at least part of the process because this has huge ramifications huge implications for not any one area for all areas of our life of our society of our information retrieval of the ways we think about things so much is how the data is shaped and the data is presented to us so we really have to take a part in trying to find a way to decentralize that process which brings us to Vector search in a blockchain now Vector search and blockchain is already being used in many ways you may not be familiar with all of it but it's being used in a lot of ways to do some of the things you see in web 2 as well provide more semantic search for metadata give more friendly user experience on your adaps and also for advanced analytics and really fast and really efficient detection of fraud on our chains that's already happening in many ways but there's so much more that can be done as well so when we talk about Vector search in blockchain how do we train our models training the AI models is integral because that's where it all starts that's basically when you get to the vector search element when you're searching you're already at like step two or three four and five Training the model that is Step Zero that's Ground Zero so training the models instead of an essentialized Paradigm there's already been for years research and implementation of a more decentralized way of training it's called Federated learning you may have heard of it before maybe you haven't Federated or distributed learning but there is a traditional model of Federated learning which presents some problems in our context primarily it has a central aggregation node that coordinates the whole process you hear the word central aggregation and your brain is probably already spinning with all the issues that's present in that just to name a couple of them you have a single point of failure that's not good on a on a chain that's definitely not good in training AI models you have a central point of failure which allows for the introduction of Bad actors and remember AI models embedding models are not intelligent they do what they're trained to do so if you introduce malicious training or you do things that are nefarious at the training stage you are shaping the way users will interact with data and information going forward so there's a lot of importance to wanting to prevent Bad actors and of course another issue is just an issue that's prevalent all over web 2 which is the lack of incentives for people volunteering their compute why should I make my computer my machine part of the training process as a node what do I get out of it you may tell me I get the self-satisfaction of being part of something that's wonderful that feels great but I can't buy a cup of coffee with self-satisfaction I can't pay my rent with self-satisfaction I can't support my family with self-satisfaction it's wonderful Kudos but there's no incentive model there as opposed to a more web 3 context where you decentralize the centralized aggregator you spread it out the responsibility amongst many nodes you introduce smart contracts to regulate the behavior of the node operators and you introduce incentive mechanisms into the entire process the classic Model looks very similar to this and if you're curious and want to read about it recommend that research paper you can scan the QR code and and really dive into it basically of have a central node that sends out the global data each of the local nodes trains locally on their machine then they send back that up to the central node which Aggregates all the training and then produces sort of a um aggregated version of a training mod of a new AI model that's the centralized classic Federated machine learning model another alternative and this is only one example there are many examples comes from this white paper from zg which you can see as well if you want to scan that QR code the altitude here is just like wrecking havoc on my voice uh you can scan that QR code and you can uh see that model and what's really interesting there is you have uh a multi-node orchestration process so instead of a centralized node actor an aggregator actor I should say you have multiple nodes acting as the centralized aggregator spreading it out reducing the attack Vector possibility and making it harder to really tamper and interfere with the process okay my friends wherever you're watching this wherever you are you are now experts on this topic okay not really experts but you're familiar with the topic let's actually talk about how this might be used in a web 3 context for the next few minutes first of all fraud detection it is already clear and evident if you can use Vector search to see what's similar you can use Vector search to see what is dissimilar you can find anomalies in data patterns and as you could probably guess that's an excellent way to detect fraud and web 3 is not the only one using it in the web 2 context there's a company called revolute anyone familiar with revolute it's a British Banking Company they're very popular in England and the UK or they have a LoveHate relationship I guess like all banks and they have a product called Sherlock part of it is open source by the way not the whole thing but Sherlock detects within milliseconds fraud on a customer's account and they do this in almost real time millisecond time for tens of millions of transactions all day and basically what that looks like is if you're a revolute user somebody uh uses your credit card for something and you get a pop up within seconds asking you to verify the transaction if it's not your transaction you can dispute it in seconds that all happens with Vector search in the web 3 context we can do something like this using graph neural networks and Vector search because remember in a web 3 context a lot of attacks happen through Cil attacks and other forms of using multiple wallets multiple let's say burner wallets to uh conduct a coordinated attack which means it's not just enough to detect a single malicious actor you need to detect the connections and understand where you see a behavioral pattern across wallet addresses which means graph neural networks constructing a graph of connections of transactions is a fantastic way to do that so you would feed in all the blockchain transactions you would use a graph construction to understand where the elstic Clusters are happening feed that into an embedding model to detect the patterns the anomalous patterns and by the way all of this can happen in millisecond time not submillisecond we're not at that point yet but millisecond time that's pretty close to real time that's pretty good for huge numbers of transactions for millions of transactions a minute this can happen at millisecond time this is a great architectural model for in implementing that in a web3 context another great use case is discovering nfts this is a fun one for me right so let me just give you a quick quick example I'm on a website and I want to search for nfts that uh are like anime stuff so I search nfts that look like anime it's natural language right I get back no results that's so sad if I search search for anime what do I get back anime nfts what happened that's an exact keyword match search that's not helpful that's not not to shame that that that nft Marketplace I'm sorry to do that but it's not helpful I want to search with human language so using human language in my dapt in my web portals is just integral it's really transformative can't stress that enough okay so Vector search unlocks a lot of personalized uh recommendations content intuitive user experiences you can use it for defi apps or smarter yield and staking web 3 social to find better discussions participate in Cross chain asset Discovery and don't forget like great creating better developer experience for Builders they trying to figure out things in the docs much more intuitive user experiences in your docs but what about offchain data this is a big one because a lot of our data doesn't live on chain like for example Oracle data we are feeding in data from you know weather reports and New York Stock Exchange and Reuters you can use traditional Vector search platforms to do the exact same thing with offchain data because blockchain data is resource constrained not everything has to live on chain and I will tell you this may sound a bit audacious but use web 2 resources when it made sense because guess what friends we already are we're using notion we're using Discord we're using telegram we're using forel we're using AWS we might as well use web 2 Vector search platforms for offchain data as well it we're already doing it might as well do it now and there's a lot of different options there's pine cone couch base super base AWS all of them offer this as options just to show you one example with couch base they have a free tier free forever no credit card required you can actually bring Vector search to offline Edge which means if you're on an airplane you want to search on the nft marketplace you can do so offline mode using offline mobile Vector search which is pretty cool and you can discover more about that by going to that QR code so basically just to sum it all up for you a combination of offchain data vectorized with onchain data vectorized when it's needed together in coordination basically creates a more comprehensive Fuller user experience and I want you never to forget if you remember one thing from our time together data is basically like garbage it's only as useful as how you treat it data is like garbage now if you're curious about this uh I have a book coming out this uh in a few months uh called ve mastering Vector search for developers it's basically like a 50 page tutorial it'll take you through building a full stack application with Vector search I will never email you except to tell you when the book is coming out with a coupon code so you sign up for that mailing list I promise you the only email you will get from me is when the book is coming out thank you very much for your time have a great rest of the event uh thank you so much ben super exciting I know we're all looking forward to reading your book thank you uh next up we have an exciting talk uh from the co-founders of the monad foundation uh Unice and Keon uh they're going to talk about the layer one Innovations