On-Chain AI Inference: Securing & Scaling Blockchain | Jake Brukhman - CoinFund

all right hello everyone thank you for coming to this talk my name is Jake Brookman I'm the founder and CEO of coin fund and I spent most of 2023 uh investing in the intersection of AI and web 3 my talk today will be a little bit about um how we think about Ai and web 3 together and the work that we've done in this space first I want to address the the following issue you know um when we mix things in crypto like this a lot of times people think it's very hypy is AI and the web 3 intersection hype I would offer that it's a new category that is poised to impact blockchains and AI when I started investing in uh the blockchain technology around 2015 uh my first question was what can blockchain be used for Beyond just Sovereign cryptocurrency um really the story of blockchain is about coming to different spaces and categories and um disrupting them when we apply blockchains to finance we get defi when we apply blockchains to digital Collectibles we get the nft space so to me it's a very natural um phenomenon that blockchain would also be applied to AI but AI has been around for a long time and blockchain has been around for more than 10 years so what is motivating the intersection of AI and web 3 now why is now the time that this is happening and I would argue that the reason this is happening now is because we have had some strong advancements in the i space as you've seen with llms and generative uh AI over the last two years um the basic breakthrough there is realizing that large models when you feed more data and more compute into these AI models they become more intelligent and that is a fact that was not obvious at all 10 years ago even to the most plugged in AI scientist out there and the other thing that's happening right now is because chat GPT has become one of the fastest adopted um AI products out there people have had a chance to learn that there's now more in stake at stake than ever in terms of their data um products like chat GPT are poised to pull in more data than Google and Facebook um I would say combined in the past there's two ways that Ai and web 3 can converge we can bring AI to web 3 or we can bring web 3 to AI when we bring AI to web 3 we think of this as bringing AI models into a blockchain context so for example we could have llm assistants that are helping us discern the activities that are happening on chain and maybe even pulling analytics and insights about what's happening I put in a little picture of the the rabbit device because I imagine that in the future um we'll probably be asking assistance AI assistant like this a lot about what's happening in the crypto world when I think about applying AI to defi I think about um modeling risk or having models predict volatility or having an agent that is going around optimizing your particular investment U opportunities and satisfying your particular uh risk appetite and finally we've seen some really cool applications in security where um reinforcement learning models are learning to find vulnerabilities in smart contracts perhaps better than humans but the really interesting way that web 3 and AI I think converge to me is when we bring web 3 to AI because when we do that what we're doing is we're bringing web 3 Primitives to the pipeline that produces AI we're opening and democratizing it the so this means democratization of resources like big data compute and even capital that is required to fit some of these larger models it means open web three style marketplaces for training data models themselves and um inference it means working through some serious products and verifiable computation because this is required in order to create um kind of a trustworthy onchain as we'll discuss a little bit later um and also privacy like where are our llm assistants running are they running on open AI servers or locally and what kind of self Sovereign data Primitives enable that and finally we've also seen um AI products just put tokens into their uh sort of user stories as incentives which is another interesting angle of how to bring web 3 Primitives to AI when I think about this pipeline that generates AI models and this is an oversimplification of course but I think about people who have the talent to create these models and the mathematical and Technical knowhow and experience I think about access to data because this is a key primitive to create intelligence and models I think about training which requires really high-end hardware and a lot of compute to reach the levels of intelligence we see in something like gp4 by the way gp4 is a model with 1.76 trillion parameters it really requires quite a bit bit of Hardware uh to train and time as well think about inference which is the process of getting the outputs of the models um so that they could be used in products and I think about productization which is what are we actually doing with llms generative AI you know and other uh ML and one observation that is very obvious and natural to everyone here is that this pipeline is owned by a very small number of large big tech companies and in fact if you read some analysis of where um kind of Hardware is today the GPU Rich are people like open AI Google anthropic inflection X and maup and access to compute is very very difficult to come by um for other players but being actually open in AI is good I know there's a lot of nervousness out there about AI being safe but what we're also seeing is that most of the innovation in AI actually happens in open source this has been to such an extent that last year Google um leaked a memo internally that was really pointing to the fact that a lot of technical Innovation and optimization in llm models is happening in the open source world and Google didn't really have a special Advantage there other than of course their compute we also talk about AI safety a lot and when you look listen to Big Tech CEOs they say that you know safety is something that has to be guarded that has to be worked on internally and the world needs to be protected from um you know from potentially unsafe models but what that's saying to me is that these CEOs believe that this incredibly important problem can be solved by a few tens of people working in a private context 50 people at open open AI 100 people at open AI but to me the real way of making sure that AI is safe is to keep it transparent and to let thousands of Engineers work on this problem maybe even tens of thousands that's the real way that we get true Innovation that makes AI safe and finally the more time that you spend with Chad GPT the more it should become obvious that a single monolithic model cannot really work across all contexts because um people who are querying Chad GPT from India for example have a very different cultural context Than People querying it in New York and what's becoming more and more obvious is that AI assistant will need to be specialized to the tasks groups uh and purposes that they're serving and that is a world of millions of assistants not just one monolithic one how does AI actually go on chain what does this mean what this means is that the outputs of AI models become available to Smart contracts and when you attempt uh to do this that means that model outputs have to be um computed by a node on a decentralized network of course many of these networks are permissionless they are networks that can be joined by Third parties who we don't really know and the question becomes well we can kind of trust open ai's API to deliver correct outputs because there's a lot of you know traditional World checks on whether that output is correct you know open AI is legally liable you know they're um they're being held to account um in a social way but in a pseudonymous decentralized peer-to-peer Network we don't really have those guarantees and so in order to actually have an output of a model that we can trust we have to use a technological process of verification to verify it make sure it is correct and we are getting what we're paying for and we're not being attacked um one way that this has been done recently is through zero knowledge proofs right um we can use ZK to verify the outputs of models um in a way that will ensure their correctness and of course this also comes with a trade-off when you take a 1.76 uh billion parameter model like gp4 we just don't have the scale in zero knowledge computation today to serve that kind of um output and and to make it verifiable it's not possible today um however the state-of-the-art is probably like a few millions of parameters that we can verify using zkl today and the Insight that I want to give you about that is that there's actually a ton of little models that are incredibly useful um and really open up the design space of smart contracts like for example if you wanted to do facial recognition um you can very well uh fit some fa facial recognition model inference into a ZK verification framework and that means that we could do things like build wallets um that use your face as a security factor and that's really cool uh at coin fund our basic thesis is we think that web 3 can bring a lot of openness and democratization to the AI pipeline um and here's a little history of some of the things we've done in this space um notably we published our web 3 and AI intersection thesis in September of 2022 um and we've been investing in the intersection since 2021 so worldcoin was our first investment there this is a startup that's speaking to disambiguating AIS and humans um and I'll go through um some of these other Investments as well and more recently we we have a couple of portfolio companies pop who has gone into the area of agents and upshot XYZ has recently launched Aura Network which is a um kind of a decentralized network for machine intelligence we've done a lot of work um in this area at coin fund our most recent announcement has been around Bagel Network and we can think of bagel as a kind of open autonomous Marketplace you might think of it as a data layer for AGI or autonomous agents and you can also think of that as a collaborative technology where um think of it as a kind of a GitHub for machine learning data which could be data used for trading it could be used for rag um Bagel we LED Bagel seed round and they're hiring Jensen uh is working on an incredibly difficult problem of decentralized training of AI models not just distributed training but decentralized training um which requires verification of training sorry I'm not sure why this is doing that I think that Jensen is probably the only company in the world that I know of that has taken The Primitives of you know AI mathematics and ZK mathematics and has come up with a verifiable training scheme and of course they're also along the way working on democratizing the compute for training why do we want to do this this creates a really high level of automation long term and when Jensen thinks about the future they're thinking about putting compute in a configuration where models can be building models and we can make we can be making exponential progress in this area Giza giza. XYZ uh is a company that uses starkware uh and Cairo to bring AI model inference on chain so that you could use them in smart contracts and the way that they do do this they take this format called Onyx which is used to store neural network weights and basically um transpiles them to Cairo so they could be proven with ZK and Starks uh on Stark net what this does is it enables a very very practical approach to um using AI outputs in a fully verified way in the smart contracts and Giza has been out with their framework Orion now for close to about a year um people at hackathons use this and very very excited to see what people will build uh on this in the future but the very practical um sort of set of applications that we're actually seeing in the market right now is using machine learning models to empower defi applications again this is things like measuring risk optimizing your yield across across different opportunities and Giza is launching very uh practical products that people can use to um augment defi in this way cnry is a sass for highly performant zero knowledge proving and of course a lot of this proving is going to uh is today used for you know things like ZK rollups but I think over time it will go to zkl as well as we need more and more and more verifiable computation cury is not just about um not just about proving but actually helping developers and Engineers manage the entire pipeline for their ZK circuits and they support multiple ZK systems um and make the development around them really really easy and finally uh worldcoin the first project that we invested in into in this intersection is uh among other things creating best-in-class civil resistance I think this is an incredibly important primitive in decentralization and Sam alman's vision is the disambiguation online of who is an AI um and who is a human I also think that worldcoin has been very very misunderstood to me worldcoin is one of the most web three values compliant privacy Technologies and hands down the best implementation uh using Hardware of course for proof of humanity there's still a lot of open issues in the intersection of AI and web 3 um notably compute is hugely bottlenecked by companies like Nvidia Microsoft and so on and a big open question question is you know will web 3 open networks be able to Source enough compute to compete or compete in time and what's sort of encouraging here is that um there's a lot of work being done to move training and inference to commodity GPU Hardware um as well as some examples in the market of decentralized networks already running uh consumer applications that uh that require inference other issues might include the scalability of zero knowledge proofs as a verification mechanism um privacy is a huge issue because even a decentralized network when you send your job to be inferenced your um your counterparties still might see some of your private data so how do we how do we protect the privacy of people who are querying these Networks there's also a human element here right um historically AI folks have been very hesitant to adopt web 3 but that is changing as people like Andre karpathy and Elon are tweeting about decentralization and finally will the product Suite in AI web 3 be compelling um and will they make money that's my talk if you're building in this intersection of AI and web 3 we'd love to hear from you at coin fund we'll talk to anyone um about their idea in the space please follow me and our firm thank you very [Applause] much