Debate: Oracle-Aggregated LLM Outputs Are a Myth | Chi Zhang, Nalin Mittal, Chirag Dhull

[Music] uh my name is Jill I'm from chain link labs and today we'll be talking about uh is llm aggregation a myth or a reality or in simpler terms how much of AI can be decentralized what are the use cases and its pros and cons um so first before we're jumping in uh let's do some quick intros so as saying I'm chag I've been with Ching class for four years uh before that I was at AWS and Microsoft working on multiple machine learning and iot products G hello everyone um Chi uh I'm the CEO and co-founder of Kai and uh kaii is building uh the F first the purpose build uh layer one blockchain for AI and uh my background come from more the AI and uh data uh space so I did my PhD from UC Berkeley focus on machine learning and CA and then uh start my career as a funding member at a Automated machine Learning Company and then uh join data burgs to lead the product management for their data engineering Solutions and now kite hey everyone I'm Nolan uh I lead web3 product at Google Cloud um I've been with Google for about eight years um and joined the web 3 team about three years ago so we're building uh developer infrastructure data and services for web 3 Builders so node infrastructure index data into big query we have a great faucet a lot of you probably have used that um and we're just trying to build all of the things that developers need um and also working closely with protocols um that that need developer infrastructure um I've been involved in crypto since 2013 um and I'm based in the Bay Area awesome great to have you both so the stakes for AI are really high as more and more of our lives are the key decisions are are being made by ai ai agents llms on a day-to-day basis uh and we understand that particularly many of these llms are centralized with the data the access the control um the key question would be what parts of AI could be decentralized what are the pros and cons of a decentralized AI what part of the stack can you decentralize we know about decentralized infrastructure but beyond that what else could be decentralized so CH I'd love to start with you first uh to share your thoughts about pros of decentralization for AI sure um actually uh it's a fun that right now I'm building decentralized AI while I come from the centralized AI word because I have been building uh the we to and data special infrastructure product uh for about uh uh uh six seven years before uh kite but uh what makes me feel super excited about decentralized AI is uh first of all what what it does that mean uh I feel like that's more when it come to decentralized AI it's talk about the trust the transparency and the ownership um because to me I think centralized AI is a lot of times great in terms of efficiency um but when there's no transparency in in the process and also there's no ownership what the staming is when centralized organizations or companies like open Ai and a bunch of other are using the data from internet or using the other you know like a stuff to into the foundation model building process you don't know what happened and nobody really gets anything uh from the process even if you are the creator of the data so that's to me is where I felt super excited about decentralized AI to bring the ownership and uh you know uh transparency back to people but again uh uh it's uh not when it come to the idealist a uh decentralized AI a lot of people talk about running everything on a fully decentralized uh environment such as running the model on blockchain or even like Trend model on CH such as the thing so to us we are trying to strict the balance um but the key Focus for example for a kite is really trying to focus on the proof of uh attributed intelligence and bring the ownership and a monetization back to the uh broader Community for AI yeah yeah I think you have to kind of break the discussion up into two parts right there's the decentralized like training and then decentralized inference so on the training side I think it does make a lot of sense because you can as you said like attribute properly and incentivize the right people and compensate them fairly for their data on the infr side I think it's it's a much different story I think centralization is more pragmatic today because you know if you talk to Enterprises they're hesitant to even you know use like the big commercial models because they don't know what's happening with their prompts their corporate data Etc you see a lot of Enterprises even running like open source locally um because they don't want any data to go out so like decentralization is like the Other Extreme where it's like who knows where your prompts are going right like on the inference side so I think it's uh it's going to be a while before like Enterprises are comfortable doing that um at least that's that's my perspective on it uh talking a little bit about AI agents now um truth terminal was launched as a way uh to provide fact checking in a very decentralized manner to all the uh false information that's uh that's spreading on social media these days however it kind of took a wrong turn where it started investing in uh meme coins um so where do you see agents play a role from a decentralized ation perspective are you all scared about AI agents uh being decentralized or what are the applications or use cases would you say are probably the right ones or not the right ones for AI agents in this space yeah so you know at Google we're thinking a lot about how we can give llms access to onchain data so you can use llms to query the blockchain um and understand what's happening on it um but I think like truth terminal is like a pretty interesting example right I think most people are not going to feel comfortable giving an agent a lot of money like maybe a billionaire can give an agent $50,000 without thinking about it letting it gamble for him but like I think the average consumer is not ready for that kind of world yet um I think people are just understanding what agents are I think they're going to go through this journey where like okay you have an agent that does some work for you you get comfortable with it you understand how it works you understand like how you're in the loop um and like you have control over what's happening and then you'll start like giving it okay here's 100 bucks to buy something or 50 bucks to make a reservation or something um but I think it's going to be a while before you like connect it to your bank account or give it like tens of thousands of dollars and from a mainstream adoption point of view yeah so uh for for for us uh our perspective is always about uh whenever he come to AI like AI agent uh is that more for AI agent for web 3 or you know like a web 3/ blockchain for AI so when it come to the uh AI for web 3 uh that's in some way it's no-brainer uh because like uh n mentioned you guys are trying to give uh G access to the oent data so that they can help the crypto people or like the web three industry to make something easier and as we all know that how at the moment early stage the crypto user experience is that will be very helpful for us so uh that in that way I think that's very good for example we saw a lot of the partners that we work with are using uh AI agent to write code right like the AI coder but more dedicated to crypto so they helped them to write smart contract and uh then do some simple you know like uh uh portfolio management and everything of course ways come to very high stake management money then they that that's a big trouble uh or early stage as well and then when it come to the uh uh blockchain for AI agent the best thing we consider that uh G to happen W uh with a agent and uh uh blockchain is actually blockchain offer offer a great payment real underlying for the agent to agent payment because at this moment as we know like uh it's very hard for agent to get like a KYB or like a you know like a know your know your agent type of uh uh thing to get access as a to a credit card or so so how do you pay agent and all the stuff where where the account goes to so in that case crypto for example the proo programmable wallet offer a great way for you to even like a use a onetime payments onetime wallet for every payment in the process with a even real time and a low lowcost manner yeah I mean I I do agree like once people do start giving agents access to their money or like allocating some amount of money to an agent the agent to agent payments are going to happen on crypto rails I think it just like makes a lot of sense for that to happen and you'll also have the transparency and the communication between these agents so um I just I think it's we're far aways from actually seeing that come alive that's true a lot of things other things to fix first I don't know about uh if people are ready to give money to AI agents or not I was speaking with somebody in the morning today and and he told me uh his uh his somebody built an agent on their platform and that agent was already 200% up in this volatile market so I don't know I would love to see your show fans how many of you would be comfortable giving $5,000 to your AI agent to trade on your behalf there you go there you go yeah three four four hands of like 20 people it's not bad it's like 25% people already so probably in a year that'll be like 50% me this is a a crypto audience right so like I think the risk profile is very different here than like a common uh mainstream audience but it's exciting to see that people would be willing to do that that's the beauty of uh by the way crypto industry people are usually more willing to be adopt of a future technology so so tell me a little about proof of AI does that solve for that like would it make a person like me more comfortable using an AI agent with proof of AI yes and no yes it means like uh we track the attribution so the uh every uh agent for example like the contribution where like their parents agent SL models inference will be cryptographically verified and uh recorded on chain so that means you can always go back to see what exactly happened in this process who transfer what money and know uh you know like if the agent is giving you a verifiable or verified AI output so in that way that's definitely bring more trust but uh in that but we're not solving the financial side of the concern like the what if you give your private key to the agent and that the agent somehow lqu were being fooled by other people by other developers with a hack um then that is something I think it require advancement of uh other Technologies as well yeah and I think Beyond just the uh trading Bots uh side of things the other valuable use cases that you mentioned were one was obviously on the coding side uh solidity so I'm sure that's going to come up more and more uh then the others could be security audits Etc and and making the whole Space more and more secure and AI governance is also I'm hearing using agents for Dow governance Etc yeah since you talking about governance and security would love to ask you a question uh what do you feel like the role of uh Oracle uh in the you know like the in the AI space especially whether the existing data and other stuff can be utilized in the decentralized uh uh word yeah so on the Oracle side the way we look at oracles is is uh an intersection between web 2 and web 3 where uh oracles traditionally have been great at providing a lot of data uh for consumption on the web3 side from different types of smart contracts DBS Etc and then uh more and more financial institutions are now working with chain link on uh building on chain Financial applications so recently we did a research involving Swift and euroclear where we looked at um solving the problem of unstructured data so as you realize that there are terabytes and terabytes of unstructured data in the world and uh how could now you use llms to help these financial institutions make that data unstructured and then also bring it on chain for any onchain execution so we started looking at that and then we realized that uh because some of these unstructured data is like corporate actions which are like hundreds of pages of PDFs and some of this data can be very Niche now because of the the data could be Niche there is a higher chance that the llm could hallucinate now the llm could hallucinate How could an article play a role in perhaps getting multiple llms to provide the same answer to you and then comparing the answer in a vectorized format to then see uh which llm is hallucinating versus which is not and then picking the right llm to get the right answer um on chain and then that's what we call the golden source of record um and essentially this really helped the financial institutions we were working with uh feel much more confident in using llms because now they almost had uh a greater sense of security that um for all that Niche data that they are now making more structured they have multiple LM output being aggregated and the correct answer being provided so we thought this was a great uh kind of a uh an example for uh leveraging some of that centralized aspects of llm but making them decentralized for a higher confidence and uh answer to be put on the blockchain I don't know if you have any uh that I think it's interesting I think uh you know doing the aggregation for writing on an oracle Dr risks to some degree but whenever you're dealing with non-deterministic outputs like it's pretty risky to call it like the truth and put it on the Chain right so like I think it helps drisk but I think you need another layer somewhere if you're really going to call like an llm output what is what's going on are we done is that is that the you've awakened the spirits it's quite distracting I'll just say that um so I think you need another layer right like some evaluation or something before you put it on the Chain um not just simply aggregating I think it does help it's like one step um and you're seeing like I think a lot a common cognitive architecture now is like to qu send a prompt to like 10 different models Rank and sort and evaluate the results um and then use them and move forward that way so I think this is like part of what you're talking about but I don't think it's the complete solution by the way that's is where we think some of the things we are building can be helpful here because at least the one we can help provide this end to endend again like tracking of uh attribution from the origin of the data to you know like how the data go into the model and then the model being used for inference or using the rack but of course when come to something like uh uh n mentioned about deterministic or n deterministic this is where like a again not ready yet but some interesting proposals from the crypto crypto Community about like a ZK verified model when it come to oh I'm pretty sure that this is the model you're using type of uh type of uh proof can be helpful to bring more trust in the process but uh the Oracle one is a very interesting experiments to bring more you know like a confidence yeah yeah and talking about just a little bit about ZK proofs and just trusted Computing in general what are your thoughts about the whole aspect of uh IP and um ethical aspects and transparency of data being used in llms any any final words I know we are about to come on time um yeah I don't know if we so yeah I mean I think there's obviously concerns right I think uh protocols like story are doing a really good job of like putting IP on the chain and then like attributing throughout the different usage and compensating people across the value chain so I think like it's a very important thing um and and we'll see this being addressed by like a lot of players going forward because you can't um do AI effectively without without proper attribution yeah um actually uh yeah to to us that's why that drives us to build what we are building because we consider ownership and monetization is a real thing that can be addressed by blockchain technology at this moment with some advancement uh some other disinterested AI Advantage may take longer time to build but ownership and monetization is really something we can achieve soon if we do that correctly awesome uh and yeah my final thought there was like using um leveraging blockchains as like a source of Truth for ensuring there are checks and balances in place where you avoid the problem of garbage and garbage out where agents are talking to each other and they're just feeding each other garbage data so maybe leveraging the blockchain for for that cool I know we up on time but really appreciate uh both you both's thoughts today thank you you thank you guys [Applause]