Crypto x AI: The Future of Decentralized Intelligence - TOKEN2049 Singapore 2024

so before we go any further let's just do a very quick round of intros so just state your name the project one line on what your what your project does and let's start off with one question why should people care about decentralized AI all right uh hi I'm himansu tagi I'm one of the core contributors in the sentient uh protocol and the sentient project sentient is building AI with Community contributions so that we have models which can get you to build applications at Power with perplexity you can build uh you can use our models and build applications on it and when the applications use those models the contributors get rewarded two lines what's the question why should we care why should people care about decentralized AI why should people care about decentralized AI well first we should Define what what is decentralized AI but let's say broadly people should care about decentralized AI to make sure that uh we have ai that is not governed by any single any single regulatory Authority it has it has the regulatory uh freedom to be used by in different geographies with different alignments and with different options hey everyone I'm Shaun I'm co-founder and CEO of Sahara AI also an associate professor at USC um Sahara AI is is creating a blockchain platform that really prioritize AI sovereignty and Providence that basically create a sort of corporate system to empower new Revenue sharing mechanisms on top of these AI access we are creating and speaking about why we care about decentralized AI uh my take of that is trying to narrow this problem down to the problem of uh decentralizing the governance of the AI related resources right we're talking about uh who created data who own the data who control the compute power who is developing the model who is consuming this AI capability these are different parties and we don't want the majority of this play this like stakeholders to be really dominant on controlled by centralized providers speaking about companies like open AI Google meta right we want these governance to be maximumly in a transparent way to be distributed into different stakeholder including you and me so then we can create a healthier and more substan substainable economics on top of that hi everyone my name is Naraj and I'm one of the co-founders of ritual the goal of ritual is um or what ritual is is a decentralized blockchain that is AI enabled and allows developers to easily integrate different AI workflows into their decentralized applications why we think I should why I think decentralized AI is interesting is because it combines two of the great Technologies of the last 20 years it allows for AI to now be trustless in nature private decentralized and to apply all these great tools that we've had with crypto uh you know the crypto rails like being able to tokenize things attach um ownership and and um you know data portability to different typ of assets and a and a whole other you know range of things that I'm sure we'll um we'll discuss uh later today hey everyone I'm Trevor the co-founder of sapen it's good to be on stage with some really good technical Minds here and at sapan we're building our one liner is we're building the world's largest network of human data labelers we believe kind of in the Revenge of the humans where the Frontier data that's going to matter that is going to be demanded to get to that next stage step change in model performance is going to come from humans we think you know if you look at the iceberg of data right now the open web is just the tip of the iceberg underneath that you'll you'll see things like um Enterprise data that's not really accessible for these models yet uh I heard a stat JP Morgan has 120 pedabytes of data um privately and Chachi pt4 was trained on one petabyte and so there's a lot more under the hood that we still have to train on top of um but below that there's even a bigger ice and we think that's um kind of the end game which is unlocking all the knowledge from humans and bringing it um to to the machines and that's what we're trying to do at sapen cool so what we've heard about is you know we we want censorship resistance we want transparency we want it to be more of an equitable economy so let's dive in a little bit deeper thinking about the different parts of the tax stack what should be decentralized and what part should not what should be decentralized and what what should not uh I'm sure all of us will have quite different answers for it I I for me the most important part that must be decentralized is the governance the things about the alignment of AI the think about decisions on who contributed what to AI that that is the part which I would like to decentralize and over the last so many years we have learned about the tradeoff between decentralization and having trust free compute uh these are two different complimentary things which should be comp combined judiciously for example if you are if you are building highly performant Ai and you you want to compete with the projects like perplexity like open Ai and have ai of that quality and I don't think inflence has to be decentralized in a way that it's geographically spread out but the ownership of that influence can be decentralized at the same time if if you are building building very new AI applications where inference determines something about something about your protocol something about uh more financially important decisions then I think it may make sense to decentralize inference as well but for sentient our training our inference is centralized anyone who builds the model they centralize it but the ownership and the governance of how those models is used and the access to the model is decentralized yep I think um that's a great question because I've been like debating a lot in the past few days with on different side events about why some parts of the um the the Landscapes of AI shouldn't be decentralized for example one one uh one big things I've been pointing out is the data com um it's the compute shouldn't be decentralized too much um because all the progress has made in the past three years is coming from building better GPU chips and then putting distribute together turning into the data center and then use this to power The Scouting law of the a training and here's the progress we have today right with all these like gbt models and other large models and we shouldn't go back and then say hey let's decentralize this um whole compute architect right but then I think the governance and ownership of the compute could possibly be decentralized meaning that um we don't want the the huge centralized provider like Nvidia open AI Google work closely with GPU data center players and then they control all of the um usage and even the pricing of this compute power we want this data center still physically be there in a centralized way but we want every single GPU hours could be possibly owned by different stakeholders we think maybe a potential way to uh change it is to tokenize distributed compute power push the um Logics um um of how the transactions be made onto blockchain so everyone can transparently audit what's going on with the compute right and this basically goes to influence fine tuning retrieval augmentation Generation all sort of the uh further AI capability everyone been talking about today um where I see you know maybe kind of adding on to what you guys have said is um you know I'm a crypto Maxi so I you know theoretically want everything to be decentralized but you um taking it from a more practical standpoint you know I think everyone on the stage is probably making some tradeoffs today bringing in centralized pieces into certain parts of of their products so that it's easier to go to market and faster to build up uh you know base of customers and base of users um and I think you know when thinking through what are the most interesting like ways to think about decentralized Ai and its use cases are just what are the enabling things that you can't do in centralized AI and I think you know one thing that they had mentioned was the ability to create assets the ability to create a decentralized asset that's Global accessible by anyone in the world and now can plug into the world of Defi and gaming and you know all the other great um Sub sub sectors of crypto is just something you can't do with you know a meta model or uh or you know Chachi PT or something like that um and so I think through like what are th those super interesting applications and I I think a lot of them are on the stage you know ability to do governance with models ability to own models ability to do um you know turn data into an asset and distribute it to people the ability to you know create these types of applications and so that's really how I think about this decentralized AI question and you know what should be decentralized yeah I have a little more of a practical answer on you know we're bias towards human rhf data and fine-tuning data and I think it's underrated you know humans chat GPT wasn't the first um model at scale it it was the third Facebook and Google had internal llms prior uh but they didn't really work and then when open AI kind of experimented with dumping large amounts of human data labelers in these data sets it kind of like magically started working really well I'm simplifying it a little bit but um to me the reason we want to decentralize uh human data uh production is the same reason like uber wanted to decentralize taxis it's just better for everybody and especially the supply side which is the drivers or the human labelers right now there's a giant economy of millions and millions of human data labelers that get paid about $1 to2 dollar per hour mostly in the Philippines in Southeast Asia and they're doing low-level data entry tasks for big Tech labeling companies um as proxies to the big models um but I'm not going to focus on like the negatives of it it's to me the opportunity we see specialized data labelers as huge opportunities for regular people uh we see a world where AI workers are bigger than any other gig workking Marketplace combined bigger than all the Uber drivers and the door Dashers because these models are craving more and more human expert knowledge and whether you're a pancreatic cancer researcher drawing bounding boxes around tumor cells and cancer cells to train models how to detect cancer or you're a solidity developer labeling ether scan data to help train a a model how to like do price predictions or something I think it's super interesting to say hey we can actually monetize our knowledge our specialized knowledges to these machines and so we have kind of this new symbiotic relationship with them where they need us more than than we need them in some sense okay great so I guess so far on stage we've heard about people talking about different parts of the stack whether it's model data Trevor you mentioned a lot of that compute whether it's training or inference let's kind of move on into models a little bit and double click there so right now you know there's a lot of Open Source debate that llms are getting a lot more commoditized open source llms are getting a lot more cost effective it's very transparent people can customize them and there's a lot of people in the community that think that this open source approach enables higher quality of models do you think llms are getting commoditized and if so what would be the mold of projects after that llm as in open source LMS are getting monetized llm in general are getting Comm monetized llms in general of course are getting monetized and there's no doubt about that everybody wants to interact with an nlm right now I mean the monetization is secondary once you see a value of that size value is created of that size monetization is just an indicator of that value I mean this a found this is a foundational thing which is so breakthrough every kid uh starting from 12y old to every adult and to 40y Old all those guys are jumping into AI shifting want to learn about llm there's no doubt about llm monetization so would that llm monetization only go to closed Source model or is there a space left for open source model I think that's a more interesting question for me and we'll talk about that in a little bit as well okay so if I just talk about llm monetization for sure there's no doubt llm monetization is there maybe the Nuance to that point is that the number of use cases which have started working and which have captured people's imagin are not as many as one would think so far if one imagines an AI application the the best the best thing that comes to mind for everyone is chat GPT and there are surveys and you can go to places like India most people won't know what geni is llm is but they will know what chat GPT is but that's because that's the application that has captured everyone's imagination that llm can become gateway to knowledge over the Internet the previous search is now M the other use cases which are making sense are very hard use cases like coding coding assistance but whether there be more use cases that can be monetized I believe there are many I believe everyone is trying to discover but there are not too many winners right now for example I'm pretty sure customer service will take a big hit uh Salesforce may have a huge Victory there that part will happen What Happens because of this math problem solving in this whether Innovation will be outsourced to LM I don't believe that we are close to that right now that's so so the use cases part of the llm monetization is just getting discovered very early but everyone is looking at that they will enter every use case it's just like website Center every use case P you believe that llms will be getting commoditized and so what would projects then turn to as their mode sorry as their Moe as a mode for the llms for projects in general okay so this is a great question so how does an AI application work is it all llm is it all single llm or there's more to it so if you go to a foundational model and ask it a complex question does it just spit out the answer in all the details and the answer to that is no there's a lot of lot of reasoning steps involved before you get the answer there's a huge difference between a perplexity and a GPT model and if you just take perplexity of the shell uh GPT of the shelf and try to create uh perplexity out of it you may not succeed okay and worse still if you take llama off the shelf an open source model and try to create a performance application like perplexity prompt engineering Rag and all those gymnastics you can try the answer is no it will not work so there's a lot of difference between this foundational model which is a which is a very genius piece just gives a quick answer and an application which works and the art is in how models are combined how how intelligence steps are brok broken down into different parts of the model a planning Model A reasoning model a function calling model there are variety of features of a model that are used to build a high performance application and that's the Gap at least for now that's a very exciting space and that's the Gap in which now o1 has started entering and that's the Gap where everyone will enter for now so there's a foundational model which is like huge knowledge database captured in this uh 405b or whatever one TB one TB parameters um and there are other units of intelligence broken down to give you an application like perplexity and that's so so there's a lot of space left to monetize Beyond just the L got it very cool um Naraj what do you think about commoditization of llms commoditization and if it is what should be the mode that project should be working on that that L like LM model Creator should be thinking about project developers who are working in the space some of them rely on Tech what other things can they rely on if llms are getting commoditized okay and Hopey I don't repeat anything because I I could only hear half of what you were saying there's like yeah the echo is really bad um okay um so are LMS becoming commoditized I think yes certainly I think both it in part due to the big open source mov over the last you know kind of one to two years which is really not the case four or five years ago when all this stuff was being developed in corporate Labs uh and and less of like a public interest and fervor in in open source Technologies I think there are questions around how capable are these models and how capable can we push these models with their existing architectures um all of the existing evals that we've had every single one keeps getting broken um and these models are just becoming more capable than anyone you know whether the public or AI researchers would have thought you know in the last couple of years so I think over time I think each kind of architectural shift will become commoditized um I think we need to see what maybe many of these companies are thinking about which is are there new and interesting architectures you know thinking about is there something after the Transformer um other types of gen models that we can kind of Leverage inside of these models to do certain types of tasks um so in building a mode I think they're I think we're going through a period now we've had this like very big new technical Innovation and people are thinking about what are the next directions that we need to take in the market uh and really I think a lot of these are going to be driven by these companies having these large profit motives and you know a large need to go and generate cash in just that will be the Big Driver of I think a lot of the progress because the space has become so financialized that's going to push the development of new models uh and that will push the modes but for me to say will there be a strong Mo with one model or another in in 10 years hard to say but I think crypto incentivized models will could provide a formidable competitor to the de centralized close Source you know for-profit uh models out there so I want to add to this point uh real quick I think in in the whole process of L being monetized right now there's two par are pretty dis dis encouraged these days actually the model developers are pretty dis encouraged because because you think about you develop prop model these days very likely you have to bundle your model with some serving platforms like you exclusively deploy your model checkpoints on certain um GPU serving providers and then you start monetizing there and your business model essentially is adding a margin on top of the GPU cost and that margin basically represent how much your model uh your propor model is uh priced and we are seeing super competitive uh market right now between all these serving providers so that margin getting really thin that's really discouraging to model developer and then another party in this process is um the application Builder who using this proprietary model endpoints to really Serve customer and take revenue and they have a huge say about how much they want to pay to the uh model serving endpoints which are the model Developers uh and they also accumulate the final user feedbacks through the apps they can decide whether I'm going to give this user feedback to the model developer to further iterate a better model so they have the control of many things and I think this kind of relationship really need to be changed in a healthier way to encourage a better ecosystem for model developers to to shine right one uh analogy I know when Naraj was an investor I think I heard him talk about this you you hear about the hourglass of crypto there's like the the or even the internet like the thin middle is the protocol where it wasn't really value acrel mechanism and it was kind of like the infrastructure below and the applications on top where the value accured and I think that's another way to ask the question you asked is kind of where is the value going to acre in the AI stack or even to put it in a different way where is a sustainable competitive Advantage so is it in compute probably not you can just the next guy is just going to buy more chips and make bigger data centers and now you're not as as as competitive as him and what about the algorithms well there might be parody with kind of the rhf algorithms and the Transformer algorithms as well so I go back to data I think to me that's probably the last chance these businesses have for a competitive Advantage if you're JP Morgan and your entire Services business is going to live inside of a free llm um data is the only thing you got and you're even seeing this trickle down to startups now where the verticalization of these models kind of like verticalized B2B SAS was kind of the Venture story of the last I don't know 15 years it's kind of the same kind of thing in these big legal what's it called Harvey legal uh legal data and they have like some data mode of some kind and I kind of go back to what I said earlier about there's the open data that's that's easy now everyone can get that and crawl that um so it's going to be what's under that not just creating new Frontier data but getting access to reddits data and paying big licensing fees or figuring out how JP Morgan can like monetize their own data somehow in a new way I think that's kind of from an investment standpoint where you want to be thinking about placing bets yeah and maybe just piggybacking a little bit on that um obviously giving data contributors a way to uh participate in the E AI economy is great but we also know that data is not equal the obviously data that is a lot more high value can you talk about how decentralization will help us procure that high value data the EO again the how to Value decentralized data so not all data is equal there is data that is pretty low value there's also data that is you know Frontier data that is really helpful in getting a lot of the AI models going how do you think about decentralization helping us procure such high value data yeah I go back to kind of my opening statement around expert Knowledge from humans that is the next chapter it's like getting inside of specialized human brains and then somehow compensating them for it appropriately so like I said right now the equilibrium price for low-level data this is a stop sign this is a baby a a lot of that's being automated and B it's it's very low cost uh data creation or data structuring so the next level is like much more less scalable it's like getting inside um we actually did a project at Sapien for uh a large Asian education company who wanted us to bootstrap a network of Mandarin speaking mathematicians because they had five million dollar contracts out for math gpts an algebra GPT a calculus GPT um there's an interesting reason why they needed that we could talk about too but uh so now we have to go and find highly skilled more highly demand paying um mathematicians who speak Mandarin so what I'm saying is like this is I think the more valuable data that's going to um help these models like get out of this little lull we're in in terms of plateauing and get to the next to the next level it's just going to cost a lot more and it's going to require a lot more um kind of logistical planning from getting humans reasoning to the machines one last comment on that that I heard from one of the co-founders of scale AI talk about was why we don't have AGI yet and one of the main reasons is the data on the open web where most of this these models are trained and I'd love for the smart smart guys to uh uh comment on it as well um it's kind of the result of human reasoning so it's kind of the output like you think about what to write on the internet and you write it you don't write about how you came to those conclusions and how you reasons about it so when the machines train on that it's not a surprise that they're kind of like one-dimensional and they don't get the reasoning part down so I think that's another kind of meta question here is how do we give these machines more reasoning data to be in a learning standpoint how how humans think and learn and reason and and bring that to to these large data sets as well all right cool um let's jump a little bit to compute right now there's a ton of distributed GPU marketplaces out there a Kos render do you think that this will be a significant Trend in five years time will distributed GPU marketplaces be a significant part of the market in five years and if not why not uh for me I'm looking right now for cheap compute which works distributed or centralized whatever it is I want to save some money on the thousands of experiments that we any any AI company would do uh that's very important so if distributed or not to become useful today at the volumes at which bigger Cloud providers are useful one very important thing is that it should work you should have that software layer sitting on it there are many applications where performance are not such a bottleneck for example if I'm fine tuning a smaller model that's quite possible to do on a smaller compute but give me the software layer that gives you feature parity with a centralized provider which is the easy part I don't know why the distributed compute networks are not solving the easy part first build a good software layer over it uh Fork from Lang chain or whatever and build a good software layer then we can use it right away whether it can be sustained uh I have uh not I'm I'm not sure maybe not in this kind of applications the hope for sustaining distributed comput is the new kind of applications will come mostly driven by crypto I think I think crypto AI will be the big consumer of distributed compute as well I think just the Enterprise AI use cases the big model building use cases will be it will be hard to compete with players like AWS and Google right now uh in fact someone mentioned if a Transformer architecture would change none of that will also change because of the kind of money that has been ped to optimize compute for those few architectures the whole TPU architecture of Google is so narrowly optimized for Transformers so many of these things are difficult to move now the architecture of the models are difficult to shift I I think if you compete on those kind of use cases decentralized compute will have a tough climb so they they will be very useful for new use cases which we don't see so far and I think um this might be a question that might not be even valid in 5 years because at that time we might have a model that is small enough and capable enough that we don't really need uh a computer network to power like what we're doing today um so we don't even need to worry about hey I need to connect all of our gpus on different data centers on different server racks to power a big enough model um and I think the main reasons to have a computer network uh to be to be established is the fact that you you need to to keep transferring the compute informations across all of the nodes in order to do some synchronizations for example facilitate a type of updating of individual preferences to a central uh foundational model that is also open and transparent to every single one to use this is kind of a collective intelligence basically idea yeah um so if the question is will we have GPU marketplaces in 5 years I think the answer is yes we'll have both centralized versions like we have today and then we'll have the decentralized versions as you as you mentioned but yeah I I agree with both of these guys that um are we going to see like 10,000 gpus in like Asia and India working together at the same time to like work on a model like probably not I think we'll we'll probably see the decoupling of the compute and the and the intelligence to some degree and um you know people can contribute their pieces to a network of intelligence or um we'll see more like kind of local clusters start up and I think we're seeing this evolve in the decentralized GPU networks today already um the utilization rates are still pretty low but I imagine as we start to see you know more people that are developing these Crypton native models come online we'll start to see the patterns of how people actually use these together and then also you know will we start to see these these like distributed inference plays where it's you know multiple computers working on these results or is it you know the results inside of a te and then get the result back and it's uh you know the the machines are larger that are providing the results so a lot of open questions but I think um yeah that's kind of how I I would dice it out nice so recently we've also been getting a lot of insights into the financials uni economics of open AI open AI seems to be the main company in the space that's been making money but is still unprofitable in a decentralized world how do you guys think about profitability I can start first um I think this is the problem space that me and Sahar most excted about because it's so tangible and so burning um if you look at today like I said the whole life cycle of AI model development it starts with where you get the data probably from Individual users of AI or application and then to the model developer start putting together compute resource data resource and the recipe of the model to create a sort of first version of the model and then you deploy it start integrating into application and from there you start having user feedback and you start this whole iterations many times until you get to a good product Market fit for your application right and if you look at this whole process right now the dominant party is the application Builder because they have direct channel to the end user who pay for the apps and then they control the revenue they can say hey this is how much I'm rooting to pay for the data providers um this is how much I want to hire paid for the salary of the model developers and then for the compute provider these days we seeing the saturations of the compute powers and the market of compute is actually very crowded these days so um if you look at this whole life cycle I think it's not very sustainable because eventually you will see the super app which are basically companies like Google meta maybe open AI down the road who control the super power apps will become the dominant of all of these Upstream stakeholders they can price the data they can price the compute they can hire all of the good model developers and then if as this time goes you will see the super Earth star automating part of the humans jobs and cutting out our job opportunity and create a very different Society down the road and the question now is would that concern any everyone sitting here um if this happens in 5 years I think people will have very mixed answer if you are feeling really burning about fact that uh your life and jobs going to get impacted by this like dominant cental as provider I think you should be part of our community of decentralized and trusted AI in the sense of um start asking to protect your user data protect your compute protect your uh knowhow about model development and a way to a trieve to that stage I believe is to use a blockchain's technology and privacy protection methods to create a copyright mechanisms to recall who has done what to what extent in this whole life cycle and then you can use that prominence infrastructure to to create some new economic mechanisms of both ping resource to build AI model or to give out a shares of the model to different stakeholders to power some more exciting and more healthier economics yeah just to just to build on that I think it's not clear where the revenue is going to come from the on the one end I think um there's some there's an argument to be made that the consumer's going to win and we already are like for for example we when we did our logo we got like a quote from a branding Agency for some crazy amount in the UK and then we're just like you know let's just go to Dolly and check it out and we we just for $20 a month got this logo and we don't need designers and agencies anymore so who's winning who's losing there we're winning and maybe the the uh the agency's losing but you can also look at it a different way where you're seeing like startups now where one or two people can build like Venture back companies we saw this in crypto with companies like Unis swap um getting to venture scale with just a handful of employees and a relatively low amount of lines of code written and so what I'm saying is you can get potentially get a lot more leverage and the barf for quality of startups and design is going to go up and we're going to have more startups um but we could still have those hyperscale startups that do really well with less people and that's generally good for um Venture investors as well so to me it's unclear interesting himansu why don't you have the last word yeah I don't think in terms of unit economy and profitability I'm an academic unfortunately but I must say that there are things bigger than bigger than these things this is a completely crazy thing happening we are this is not a value stealing thing we are pushing the economy and unit economy and money will catch up with where AI is taking us it's not the other way around uh it's not stealing value from the past you are seeing completely crazy advances in robotics things which were done by Boston Dynamics over 20 years are now recreated by newer companies in 2 years one and a half years things which were Unthinkable in terms of foundational model for robotics is making such a huge progress and uh it's completely new value being created economy will follow it's like the internet era where it's where where suddenly the capital was not not not not the uh not related to real estate alone and that was not the most interesting investment and it turned out that there is this new crazy thing called software and now this is new crazy thing called AI which is changing everything there's no doubt about economic value creation I don't even want to bother about it everything is changing every the demand is so high who captures that economic value that's a difficult question to answer right now but whether valued would be created or not no doubt about it money will catch up with it it doesn't need to catch up with the money all right I think that's a beautiful place for us to end on let's give a round of applause for the panelists thanks a lot everyone