Decentralized AI, from Open Data to AGI – San Francisco 2025

we haven't talked enough about decentralized AI open data distributed computation and transparent governance that the things that could actually be what democratizes a AI development and put the entire ecosystem onto a new course so I'm happy to be with a great group of individuals who aren't just talking about it but actually building it so um could we do a quick little round of intros we can start with you Vincent just who are you what are you doing hi um I'm Vincent and the founder of uh Prime basically a peer-to-peer platform for um Computing intelligence hey I'm Anna from vaana um we focus on the data piece so kind of self- sovereign data infrastructure for users to contribute their data to AI models in a privacy preserving way hello guys I'm Chi um from Ki uh we are a platform that focus on a ownership and monetization essentially it's a building a platform that we can ensure the transparent attribution and fair rewards uh to the endtoend air workload from data to model to agents hey guys this is ROM from open Ledger I'm one of the Coe contributors we are a blockchain network for AI we enable people to contribute data sets and build specialized AI models and we use attribution to reward people for the models that are built awesome well welcome everyone so we are closing out um ethereum San Francisco so they finally came to San Francisco and I have been ra ring about this for too long now that it's felt uh different than other conferences in the past in the fact that the worlds of AI and web 3 largely we've been standing on the same kind of fundamentals and foundations of valuing decentralization and many other topics and values but for a long time you know me coming from the AI world it's felt like there's been two bubbles right like there's been two bubbles that have not had a lot of overlap um these worlds have just kind of existed in parallel but e ethereum coming com to San Francisco I'm going to make a small joke about this if you're in the world of web 3 they kind of refer to themselves as CT like crypto Twitter and they refer to the A and crypto Twitter so if you're really deep in web 3 they also they refer to the AI world as SF just San Francisco they just cont they attribute the city of SF to AI but for for the first time it actually feels like with things like agents and things like that that we're actually starting to have more convergent conversations so I want to start off with like first principles what is decentralization versus centralization when it comes to AI um could I start with you Anna actually yeah um I guess on the topic of kind of like SF and this um meme of like how web 3 and AI interact the one that I think best captures the moment is actually the one where like the web 3 people are like web 3 needs Ai and then AI is like I don't think about you at all which is just the current state of things but obviously it's getting better um and I mean that ties into kind of like why decentralize things um I actually like to frame things in terms of like the properties you want so like self sovereignty and censorship resistance right so if you think far out um 10 years we have extremely powerful AI models that can do a lot of economic work um they might also primarily be used as a source of like information right and one tangible example there is do you guys remember when Google bard um kind of added some censorship where if you asked it for a historically accurate picture of a medieval King it would show you like an Indian woman and I think a lot of people are like oh it's this crazy wokeness of the 2020s but if we just got our information from AI that would just become truth right so whoever is deciding what data goes into Ai and what AI actually believes is basically deciding truth in society and also amassing a huge amount of economic power right open AI doing like $3.4 billion in Revenue as of last year all from just having these valuable AI models so the case for decentralized AI is really we're going to build this very powerful AI system and rather than having it controlled by a few companies who both get to determine truth and get to amass a huge amount of economic power that should be distributed among many um I think that's a high level kind of ideological thought there's kind of more we can get into probably on this panel from a tactical tactical perspective right I think a lot of people agree high level we don't want AI to be controlled by one company but you also have to be tactical about like okay what can we do in a decentralized context that you can't do from a centralized perspective um and from V's perspective we focus on really the data piece of that um but we can get into that more later sure I mean do we want to give any examples of like what can you do with something that's decentralized that you cannot do with centralized and or someone wants to do the rivers I I mean just as a sort of direct example where I think numbers can be really helpful right so if you look at the leading AI models today they're trained on around 15 trillion uh tokens 15 trillion words right huge amount of information and if you look at the whole public internet like you scrape everything actually love like the internet archive I've never been to this building before but they help with a lot of that so great um but if you look at um common crawl and then you look at fine web which is the highquality subset of it that people actually train on that's about 15 trillion words as well so the leading AI models today have actually fully exhausted all the training data that we have access to AI researchers call that the data wall and I think that is the very tactical example of like okay we're out of public data how do you get access to private data you can buy it from companies right companies are doing well doing that reddit's earning $200 million from selling data um yeah Apple's buying a lot of data but our approach and kind of the decentralized approach that I think works really well is actually having users directly contribute their data so similar to how when you park your car in a parking lot like uh the car parking lot doesn't own your car right it's still yours you can always take it back when you put your data in a platform it's still legally yours so you can go get your Instagram data your Twitter data any information you've contributed emails like all your writing and you can actually pull that um across all the different wall Gardens of big Tech and if you have 100 million users who do that which is obviously a big number we're like 1% of the way there then you can get to 450 trillion words of training data which is like 30 times larger than any single centralized company could do and you really can't do this in a centralized context right apple is not going to collaborate with Google and openai and every single other partner because they can't even legally pool their users data so I think that's like the the case for decentralized AI actually winning from a tactical perspective beyond that ideology absolutely and like let's actually go over to ROM how else can we use web 3 to incentivize people to contribute more quality data and perhaps build their own specialized models yeah I think there are two based like always decentralization has been used right one you find sort of like an alternate universe like there are centralized players who build a certain product and you create like a decentralized version of that and then use decentralization for bootstrapping or like providing resources um I take more on the second piece like not everything has to be decentralized like how can we use decentralization to bring in more resources how can we enable people to contribute resources and get rewarded for that right uh and especially AI if you take a look at it we are in a very initial stages um AI is more generalized now um as larger language models right as Anna was telling like most of the larger language models are trained on internet data this is this has very generic information but actually the once AI becomes much more bigger and once AI is getting used in various sectors you would want to access lot of propriety data sets right data sets that are owned by people data sets that are owned by organizations we want them to go ahead and contribute that so web3 could be a great way for them to contribute and get rewarded for that right and also make sure that their contribution still has them remaining ownership on the data right still has them remaining ownership on the data but getting paid as this data get gets accessed by the all the AI and that's what we want to enable we want people to like contribute data sets and build models out of that specialized AI models and use verifiability for them to see how data is used in a model right if I contribute a data set if it's used in a model I'll be able to see it on chain that my data was used in this particular model and I'm going to get rewarded so much for that usage right and then on the other side we also want to make sure these all these entire the contribution to the model usage to the rewarding is on chain is verifiable and it enables trustless that's again that's where web3 comes in right you want to in if you want to bring people across the globe and contribute something you want to have trustless I they can't believe in the concept that okay I believe in dram I'm going to contribute data for that they should believe in the code they should believe the trustless to be there so that's that's what you want to use be 3 as well to bring in that verifiability to Showcase to people how your data getting is getting used and another one is that again coming back to the first point how can we reward people like how can we use crypto as a way to reward people because if I'm going to let people get paid using Fiat then I'm going to need to pass through a lot of repats I can't like fund someone who's based of Nigeria another one based of Korea it cannot happen at all but crypto is a great way for people to get rewarded right so I guess that's how we need to see web3 as well uh it should complement the existing AI technology not really look at like fighting against them and like use pep3 prors to make I our own like how internet happened if you take a look at when internet happened it was centralized but then there was open source platforms like htps which is actually natively decentralized right that's what Ena worldwide web to happen and that's how we need a CI as well how can we enable web 3 to like build better AI our own AI so that that's what we talk about yeah and what are some of the most promising models of that it can kind of get messy in the worlds of data proin provenance and tracking and things like that what what are some things yeah I'll give you an example right so there's a team which is building like a solidity model on top of us cursor which is one of the largest larger language model for programming does not understand solidity solidity is a very Niche programming languages um like not many people have this data the data is not available on the internet so we went ahead and enabled a bunch of devs to build this on a platform where they collected solidity data sets codes code descriptions and a model is being built and we just open sourced that and now whenever this model gets used we will reward back to the data contributors very similarly there are more programming languages and natural languages that humans speak but you don't have the data on the internet it is among people so that could be a great sensive to have more models on various Niche languages another one I see is finance a lot of Finance data is not online is not on the internet like how people trade and all that is not on the internet so that could be a great sector that we concentrating on as well so Finance models Healthcare again a lot of people own Healthcare data sets it's usually proprietary people not willing to give it but they incentivise they're willing to give that so that that's another sector that you're looking at as well yeah and all these things sound very promising and they're all in effect maybe Chi I'm going to ask a weird one for you but can this actually work decentralized data ownership can this is can this be as effective as big Tech currently is and having kind of us in their grips like what in a world do you see this kind of being a probable likely outcome that we can actually achieve this as underdogs good question yes and no actually that's uh come to more to me as a more fundamental infrastructure problem um a little bit background about like how I was building or involved in the AI industry before um um I I did my PhD in Berkeley for U machine learning and then actually the first company I worked at was a uh closed AI company is Automated machine learning platform and then I moved to uh then this is where we recognize um you know in order for AI to be adopted in Industry data is is super important the data Silo problem data quality problem and everything is actually the huge blocker for uh ESP special Enterprises to adopt AI in their workflows and everything and then uh then after that company I I I join data BRS right now one of the biggest uh data and AI infrastructure company in the industry to lead uh data engineering pro products there which by the way is relevant to what you asked about whether it's feasible or not and now it's very fascinating that now uh we're in this panel to talk about decentralized Ai and decentralized hopefully like a data problem uh so when it come to Yes uh that means if we leverage uh blockchain correctly as a infrastructure and Dore the things that's important to store on CH for example like the you know like the ownership of let's say the data and some of the attribution information and even like the license agreement or license what license those uh data usage should be under uh those type of things uh it will be very good but if we are trying to bring let's say right now the contribution or like a real data fetch on chain then that will be extremely hard so that's why even we think about uh like decentralized the data storage like what ipfs and some other people trying to solve here uh it's a great solution to bring more you know like uh uh trustless and global distribution into the process but uh if we think about comp compare this to centralized the solution uh so what let's say awss 3 or data breaks type of company offer then it's far away to go uh uh when again when it come to you know we want to Fouch data globally and everything then the cost and efficiency will be a lot uh you know like a stale under performance compared to uh the one so uh again I think the the the most feasible way to uh push decentralize the AI to forward and at least have something uh feasible is to start with uh having the critical information being stored and uh you know like a tracked onchain and a lot of the computational heavy stuff still happened offchain but again even if it's happen off chain there's still way for you to add more trustless and uh uh verifiability in the process such as uh you know like what for example like a optimistic uh uh uh uh row up type of solu that you allow people to uh to to to start a a challenge type of thing uh in the process so again yeah I think from a infrastructure underline perspective uh it's still a long way to go but uh at least the blockchain itself as an immutable Ledger indeed can bring a lot of uh uh trustless collaboration and verifiability uh in the process which is a great technology uh to push the a collaboration and data collaboration and monetization moving forward yeah I would throw in like a kind of like maybe contrary counter that I think like pre-training is dead as IIA said data is kind of like dead in an organic sense and we're already way past Beyond like kind of like scaling with Organic data like kind of like user generated data and in a sense like more Reddit data won't lead to AGI or super intelligence um like having individual users contribute data very very like doesn't really scale like you you do want like whatever to not onboard like one user after each other you want to onboard like every like user using Code agents or something so my point is maybe it's like like ultimately scaling laws are true we're scaling through compute and we're now scaling reasoning models through synthetic data and we're won't reach like super human like level of like intelligence with human data like by definition but ra much rather through compute through basically self-play strategies so I think there's like specific domains where we actually haven't run out of data but like we haven't been able to use the data like basically for say robotics like we we don't have superhuman robotics probably anytime soon and um but there's like a bunch of like real world data we can use but actually much more even synthetic data and kind of like selfplay again like to for example get to robotics so I think actually my point is like to an extent that computers all that matters like like um it's like data is kind of like behind us and we are basically only bottlenecked by new like scale up in compute and scale up in Al algorithms I think it's kind of like what the history of like AI uh teached us and kind of like the bit of lesson um from kind of like the invent of reinforcement learning which is also kind of like now the the Paradigm uh again to move AI progress so I think there's actually maybe like a contrarian case where like the data wall wasn't really a problem we we fit it but like we've also passed it and I think um yeah like ultimately it might not be the bottleneck that some people make it out to be while ultimately still user the data like user interactions with chbt are incredibly valuable it's like where do they uh whatever like drop off in attention like what's kind of like the answers they're happy with or unhappy with like this kind of data I think is actually much more valuable is basically just like the data and the interactions with like a models is there any just to add to that I I also agree to that I don't think like user generated data is the only solution like I I would go with like the 8020 rule right 80% of the data is usually among uh businesses which have like propriety data sets we need to get access to because most of the user generated data is on the internet and that's already grabbed by these larger L large language models and they've trained these models with that uh what really kind of like brings a change is like how can we use business data and also a certain part of synthetic data generation right because we also explored that like for example like most of models that is used in Tesla is train on synthetic data how else could you actually train that model to drive through various scenarios you need to generate synthetic data for that pretty much goes with autonomous robots and all that so uh it's always large amounts of proprietary data and synthetic data that's that goes into a model and about 20% of that is used ined data which adds that additional value that is there like the the human touch to it that's that's what makes the models to be smarter so I would say it's a combination of more propriety data set majorly and then a bit of like user generated data that has to be added I think something a lot of people miss when they think about synthetic data is kind of like what is the human data that actually people started with to generate it so if you look at Deep seek for example one of the ways that they overcame the data wall is by first starting with really high quality data and then generating synthetic data from that and they actually hired a bunch of math and computer science phds to generate data and like their founder himself was literally like generating this super high quality reasoning data if you look at their parent company you can actually see job postings for these like data collection kind of data generation engineers and I think what that shows us is like this data wall is both an issue of like quality and quantity Where you want to figure out how do I get that super high quality data I think there's a ton of debate of like okay is it compute that's the big breakthrough is it research is it data our view in the long run is that it's very much data which obviously we can kind of debate all the time one thing I would call out about compute is I think that it is a much easier like capital expenditure to actually underwrite right if you think about some of the incentives for these hyperscalers right now and it is yeah it's sort of like something that I think a lot of people want to believe which is okay suppose we could spend a hundred billion dollar on a training run from a compute perspective I want that to work you know and I think that's really how the AI big AI players see it today so I think I would just call out that incentive as a cautionary tale to thinking it's just compute that matters I would add that basically to an extent like obviously you kind of like need all three like aspects like from data be it synthetic data like extremely valuable prary data compute algorithm breakthroughs I think one actually key pieces like to to your point is um that the most valuable data is sitting say like in hedge funds right like which are Harding this data buying it for many billions of dollars like analyzing all the like real time information in the world like satellite data they won't just like uh sell it to some like blockchain Network right it's like they're making hundreds of billions with that data and they'll obviously further monit like internally make use of the data I like maybe my my caution is more in the sense that I think the most valuable data is like very hard to unlock because people are making a lot of money having that data and I think uh they they'll use AI to unlock that value like even like the state has like incredible like amounts of like data that they're deploying AI on to whatever like do intelligence better like like literal like uh kind of like State intelligence but I think like this this data won't reach the public domain or or AI models anytime soon so I think that's maybe like one counters that like I I agree with that maybe you can yeah let you see yeah yeah so one thing I just want to add a concrete example to actually also give a point on why just relying on synthetic data is not going to uh uh work uh is when it's again come to the quality of the data uh actually in one of the podcasts that youas had a few months ago uh he break down about on uh how they're using how they train the Gru and the one thing he mentioned is uh they purposefully actually removed all the public internet data after 2022 um because there are so many fake data generated on internet that's in the end of the day including those data have already degraded the performance of uh the foundation model they're training so then they have to fall back to find those uh real human generated high quality content and data to train the model so in the end of the day it doesn't matter if it's a synthetic data or human generated data or whatever the data sources come from it's what matters is like how those data is high quality and representative of the pattern that you really want to find or you really don't want to try the model on so that's is actually the key uh here I think the incentive you called out of basically like companies that have valuable data don't want to share it right is exactly what goes on and I actually think there were like three AI video companies in and they're all kind of like racing each other to get the best training data set and their Engineers are like okay what do you think they're scraping what do you and then you're trying to get what other people don't have I think the real unlock for data ownership is that uh you actually don't need a business's permission to get the data right the user can just go and get it themsel and so you're able to kind of sidestep that incentive which to date I think has led to not that much adoption of a lot of kind of decentralized data type approaches no just to add to that I think we all talking the pretty much the same point it comes down to like the model that we actually building a certain model would need lot of like user generated data to make it much better in the in the field that is being built on a model like let's say that's built for autonomous robots might need more synthetic data less of the human generated data and certain model which are niches like the programming language model that you spoke about might need data sets that comes from propriety businesses businesses that actually own the data which needs to come forward and contribute so it all always goes down to the model that people looking to build and what kind of data sets that you want to collect for that particular model so it's it's a very Case by case driven solutions that we need to look for I think one thing like I would add even more like looking into the future is that like like also what we did recently and what and everyone is doing is is actually generate really synthetic data with verification like basically say in code and math let's like check if if for example the reasoning that that get you to like the solution is correct to basically scale towards like superum reasoning and I think like my ultimate point is like really that I think from the like for the next like 10 years I think what will get increasingly important is basically strategies to scale to like superum data and that can't really come from humans so for example it's like if you want to like analyze all of the sciences and like understand every piece of of uh like of like whatever like every molecule or every like piece of science like you you do need kind of like a machine approach and and it I think at some point basically it's like hard for the humans to keep up being able to generate valuable data and I think it even leads us into this future like like the models are already like at pretty good levels it's like we really need basically uh ways to smash through kind of like the those human limitations um to reach like much more super human data the human limitations are interest I was talking to someone from one of the big Labs um AI teams and they were saying they actually don't want to train on bio Publications because there's like errors in like basically Publications it's wrong and so if you train from that you end up with the wrong stuff and then the question becomes okay then what is the right source of truth right is it generated by humans is it kind of collaboration but yeah it is interesting that some of this human error we don't want to propagate that how do we actually do that and I think it's like it there was some cool experiments in this direction of AI already being able to verify s like human papers for example it's like a lot of them have erors even very famous papers and like AI spots those errors very quickly and then you can kind of like you can whatever like ass certain the like the quality level of like each paper actually through kind of like checks and verifications to know basically which papers are wrong like outright like fraudulent like scientific like Foundation papers like an Alzheimer's or whatever where then like a lot of the other SS is like basing their assumptions on like completely wrong science and I think that's actually like one of the biggest values for for AI is like to ultimately improve our like epistemics and and our like more like scientific Corpus yeah so we are coming to a close here and I like that we're trying to land this plane with super intelligence um Vincent I wanted to make a Shameless plug cuz I've recently uh partnered up with prime intellect and I wanted to kind of try to end by asking everyone in a sentence or less um how you actually Define AGI or ASI um just quickly because I don't it's it's another thing like a decentralization like we these these terms that we kind of throw around so maybe Rah I'll just end with you yeah AI is basically like achieving human level gentle intelligence right that's what people call AGI but I think AGI will not happen to be like very yeah you want say something one sentence one sentence okay one sentence AGI I think will be not generalized it will be very specialized to one particular sector I think for Tesla with Tesla we already achieved AGI with autonomous vehicles so that's my take so okay it's really trying to get to a standard definition of it all right G um AI to me means that um from from from day one a human actually will be um you know paired with a super intelligent almost like a coach life coach and uh uh uh yeah a person who can do more things than human and then you can really do more focused on creative stuff uh rather than the daily work that everybody including us are doing right now cool okay uh I think most intelligence and economic value is produced by AI would be kind of the most concise definition and Vincent I'm not going to ask you for your definition I'm going to ask you this is our Prime intellect um Vincent could you uh explain where we are and maybe explain this really quick like I think basically um like maybe just like an in advant it's like I think people are almost like the better way to put it is like underestimating of like how Wild the next like 100 or 200 years might turn out and like the next like centuries ah um Eternal y 2K cultural time trap forever is that what you're saying no like so basically I think maybe my very answer said I think like we we might literally hit kind of like super like within the next decades and I think that's actually quite a big deal in terms of like how it changes our world and I think like there's like a bunch of factors that I think people would think are like super intelligent that we are probably like on the verge of a re reaching like yeah lots to think about okay well we are out of time so thank you so much for joining us thank you thank you so much [Music]