Decentralizing AI Our Next Great Challenge | Illia Polosukhin - NEAR Foundation & Erik Voorhees

Ilia and Eric and margarite I know my gosh I feel like uh this is my first time M seeing so I appreciate your patience and understanding but this is a big one I mean like this is this is a big one this is going to be a really fun chat so I can't wait to have them up here while Briana and team are setting up awesome well have a wonderful time and enjoy enjoy the panel hello everybody is this on so I'm margarite D Corel I'm the CEO of blockade Labs we use AI to create Virtual Worlds so think of it like a mid Journey but in 360 and and 3D for environments and I'm here today with Eric VES and Ilia and I actually am big fans of both of these guys since I started my crypto Journey um followed Eric's work early on 2014 he's always been this advocate for freedom and actually Eric I'd love for you to give a little intro on your history my history sure hi everyone uh so yeah I'm Eric Vorhees and I have been in this crazy crypto world since 2011 um fell in love with Bitcoin as a way for humans to completely opt out of um government managed currency and I felt like that was one of the most important things that could ever happen to humanity and then Along Comes This mysterious guy named satosi and and suddenly it exists so uh totally fell in love with it and have been um building various projects projects in this ecosystem for 100 years now uh early on Satoshi dice bid instant uh more recently shape shift um and now I've started to get the bug on uh on the AI world so um yeah we we'll talk a little more about that thank you IIA um you know we've known each other since 2018 I think when you launched your protocol and his team has always been world class um I don't think there's been a team that's hustled for me and tried to help me more than your protocol um so anyways I had a joy finding out your this deep background you have in AI in addition to your expertise and decentralization um could you share a little bit of your story yeah so for about 10 years before I got into crypto uh I was working in machine learning uh and deep learning AI depends how you call it and um I always thought that the new way of of interacting with computers new way of kind of computing in general will come from advancement in Ai and especially natural language and before kind of getting into the startup game I was at Google research and we were building models that actually would were powering answering questions directly on google.com as well as few other places and we were facing this challenge which was the models the state-of-the-art model were not performing enough they were too slow and so a whole group of us ended up building Transformers which is the core technology that now Powers chat GPT mid journey and a lot of other kind of advancements in Ai and I left to start near AI which was an AI company uh we were teaching machines to code which in 17 sounded like a science fiction and now seems like yeah pretty reasonable idea and uh we were actually doing a lot of crowdsourcing we were having students around the world in China in Eastern Europe and kind of other countries doing work for us writing some code writing descriptions and we had challenge paying them and this is where kind of my crypto Journey started in 18 where we're like he well we need a global payment Network that is kind of easy to use that's Works globally uh in any country for people who don't have bank accounts and you know Bitcoin and ethereum sound like the systems like that but even back then the fees were already higher than what we paid people kind of uh per task and so we started investigating the space and that's kind of when you know we went and interviewed a lot of people uh in the space overall and and kind of started near protocol as a way to challenge and address that thank you thank yeah just incredible background um Eric you've recently so with your crypto background you've recently pivoted or not pivoted started contributing to AI augmented augmented um and I remember we were talking not that long ago and you're pretty passionate about your reason to jump in and start contributing um I'm curious if you could share with us and maybe to help set some credibility for why AI has a place in the blockchain sector not just as a token cash grab which we've seen a lot of opportunists maybe pivoted in the bare market and try to have this new narrative and story but there actually is a case to be made that AI is a very important part of our blockchain evolution yeah well okay so let's paint like a a dystopian picture for a moment uh imagine a world where uh everyone is using um a large centralized AI service that comes from one big tech company and that tech company is uh heavily regulated and controlled by a government um everything that humans speak to the AI and get back from the AI is recorded by that company and SED off to that government um this means that everything that people know and think about and talk about and care about becomes centrally controlled censored uh in small ways and big ways and um ultimately that seems like a a potential for a totalitarian Nightmare and it's also like not very science fiction it is sort of the default position and we're not very far off from it uh right now so um the way you solve something like that is not making sure that like the uh the board of that big tech company is structured correctly it's not to make sure that like the state regulations are done um intelligently the way you do it is you provide optionality for people and that means that you have to have a decentralized foundational layer for uh for AI so for the same reason that I love decentralization in blockchains because they allow money to be permissionless uh the very same principle holds with AI with intelligence so decentralization allows intelligence to be permissionless um and I think you know things like mathematics and money and intelligence and language uh should not be controlled by any single Central party um that makes me kind of like a radical extremist which is uh which is really sad but still important and um it's really cool to see how much interest there is now in in the crypto world like helping to teach the AI world how to do permissionless systems and uh there's certainly going to be a lot of like grifty AI by crypto projects so watch out but um at the core AI can benefit from decentralization because it can bring the attribute of permissionless to intelligence thank you so if we were to have this sort of networker system and an ideal one how would you and Ilia I imagine you've thought about this because of near protocol and your AI background how would you design such a system what would the components be yeah so I think like obviously depending on the timeline there's different components that are needed but maybe starting from like a a future um where we want to end up is a fully Sovereign operating system that has like access to all of your data doesn't leak it out to any other company it has an ability to transact communicate establish connection with everybody else be that other individuals communities Brands and uh it is able to generate experiences for you so I would say like over time we expect that the current way of you building applications where somebody frontend engineer builds a specific thing will also start transitioning more into fully generated experience for you individually all of this you want to first of all happen as much as possible on your side you don't want your private data to go to some centralized party uh you don't want to you know your queries your applications to be defined by somebody else and so we do need local models that are able to run inference uh directly on your device and so there's kind of on Edge Edge intelligence uh Dimension to that you do also have need for Community models for models that are uh represent knowledge and understanding and able to make decisions and uh potentially even allocate money and incentives and this is where kind of decentralized inference comes in and provable inference comes in uh as well as a lot of other pieces and so there's kind of a whole spectrum of you know from local models to inference on infrastructure side we also need better foundational models that are open source one of the kind of interesting challenges right now in the AI space is that if you're building a model and you're open sourcing it you kind of don't have a good business model because anybody can grab it and run it and you're not getting any attribution and kind of revenue from that um and so creating an incentive to improve this models is as well something that uh crypto is actually doing already through few different projects now the there's a whole experience layer right where right now you know the content we consume is coming from you know different creators there is a machine learning model that Tik Tok runs for example and that machine learning model for for not China is optimizing on on their profit and for China for example optimizes because they have regulation on like for kids for example to learn more and so the fact that like somebody else decides what your feed looks like is as well challenging so we'll see transition there where the content will be generated for you potentially from creators kind of putting different components and new characters in new places and then all of this compos for you and rank for you based on your interest based on your uh demand and I mean you're actually working on in that space as well so there's kind of across the stack a lot of different components and I have a huge diagram how this should all work but uh there's a lot of also challenges across the space as well is this is this on your road map yeah so I mean I put out this vision of like fully Sovereign operating system uh a couple months ago and so we are kind of building a lot of the pieces in that direction as well as we working with a lot of startups in the broader and near ecosystem who are building different pieces of this experiences to then kind of bring this overall like novel way of interacting so yes amazing Eric uh were you following the news at all with the Google's Genie conundrum with so for people don't know uh there's a release where you were prompting and you were getting almost everything else except for what you were prompting or asking asking for in a lot of ways to try to be more correct or be inclusive and so some of the conversations that have come up from that um were you know what is the source of Truth who gets to control it who gets to decide it where does it come from how so I'm I'm curious if you've thought about how could decentralization could help with that problem yes well so who a lot of you are probably on Twitter and have seen this crazy Google Stu uh but yeah to give a simple example it's like you you prompt in for an image of the founding fathers of America and you get back a bunch of like um Asian and Native American women and it doesn't make any sense and um it's I think the the the real danger here is that you think that you're talking to a computer but you're actually hearing back from humans and it's kind of fraud frankly right you're you're trying to use like machine learning algorithms you're basically trying to use a language calculator that that is programmed to do statistical inference on language and what you actually get back is a whole bunch of bias wrapped from people um who believe that that's actually what you should know instead this is obviously very very dangerous and I think that the the answer to this again is not to regulate Google but is to just make sure that there are alternatives right so if you go and you type in something to Google and it's totally absurd and you go and you type in something to an open source uh decentralized AI alternative and you actually get back a more reasonable answer cool like that's how the competition should work um that's the best way to ensure that you know truth and intelligence are approached because no one should have a monopoly on truth truth is something which is approached by mankind with iteration and and experimentation so um it's really it's really about the having Alternatives and um avoiding some kind of you know centralized Monopoly that just controls where everyone's information is coming from maybe to add to that like fundamentally this models represent what data you put into them and so there is somebody somewhere deciding which training data is going in and which additional biases to add at inference time and that decision making right now is completely opaque right this decision making also can include that if you have this email when you're using chat GPT the output should be different from this email like we don't like unless you actually like you know test it in different ways like you wouldn't even know that this is happening and so making sure that there is a community kind of curated data sets which people understand kind of the like the source of this data and how it went into it will be important and like in a way the way Wikipedia is an open encyclopedia where people continuously updating and kind of interacting and up like and you know arguing what's true right we need simp similar thing to happen on training data for this models to be able to actually kind of discern that otherwise we will be kind of in control of whoever is deciding which training data goes and like for a specific example there's somebody somewhere writing in Python script that literally decides which documents should be dropped from training data and like that's because on internet like if you look at the overall scrape 50% of the time people say that Barack Obama is born in Kenya and 50% and in Hawaii like that's literally statistics and so there's somebody somewhere deciding on how to deal with those cases and like right now those decisions are completely opaque and are affected by ideology of a company that doing it by government regulations of that of that headquarter of that company and many so there's like no there's no way a centralized company can actually do this right like it's the same as like social media censorship for like social media companies trying to censor or content there's no way to do it right because it's so much different from different jurisdictions from different communities and so we do need similarly kind of how we're trying to bring social media to web 3 and make it you know that the layer of uh moderation happens on the front end we need the same thing to happen on data and kind of build models of this uh like Community curated data yeah we're we're not very far from situations in which the the bias isn't just organic and generalized so that everyone that puts in a prompt gets back something that's clearly wrong uh it's not very far to realize that soon we'll have like targeted bias we're like you know you signed up with open AI with your email your email is all over the Internet and easy statistical connection you know tells the machine that okay you are a trump voter um now the answer that you get back from openai might be very tailored to either like a pro or anti-trump kind of stance or try to subtly or overtly like change your mind on something um and I I would hope that regardless of what people think of of someone like Trump that they would realize that there's something fundamentally um fraudulent and scary when that kind of thing is being done uh and at at minimum at minimum those kind of changes and biases could if if the company was being honorable could just be stated clearly right like here's how we bias what we tell you here and we do this because we think these things are important and so here's how we bias it and here's how it's expressed that would at least be like honorable but that's not done at all uh you can ask the prompt inside the chat GPD there's a way to get it out so like you can see the system prompt but you don't cannot see the training data so that's yeah the the people that are a little more sophisticated can often like get the the chat system to tell them like where the biases are which is cool but I I don't think we're far from um you know those things also being constrained so that the amount of intelligence and information you can get is highly curated um and and pretty scary well the even scary example for me is if you're running right now one of those companies that doing that offering like chat GPT like service and you have a huge GPU bill right and you yes you're making money from subscription fees but not everyone can pay the subscription fee and so the natural thing for you to do is to Auer ASE a cheaper or free version but monetize it through advertisement and the reality is advertisement auctions are actually faster than the token production right now from this models and so you can actually run an ad auction on every single token as it been produced pretty much completely like changing you know small things like oh which car should I buy like instead of seeing what statistical model tells you being like completely affected by advertisement and same you know for political affiliation or anything else and so I think like that's even it's like to me that's like the most realistic scenarios is this going to happen because like so much economically like makes sense and at the same time it completely biases people into like whoever pays more to affect your opinion on specific thing thank you um I so in hearing you talk I had the follow-up question thinking about if we're putting with the open source AI all this data so transparently and potentially even using blockchain in some of this uh instances isn't that rather dangerous and risky too especially with personal information and what can be done to build a secure way to have this data without people being you know at risk yeah so certainly like the most the most safe from a personal data perspective is to be running inference locally on your own device and there's a lot of ways to do that that are not super technical or hard um and then you could do it without any internet connection and you can just be interacting with the the llm directly on your own machine machine um if you're trying to be really private that's the way to do it um unfortunately most people even though they might say that they care about privacy they often don't in their actual actions and they'll just use whatever is easiest or cheapest and so that's where we get into a lot of the problems so with that where where is the open source Community really being targeted by regulators and where where are we most at risk at seeing a stunt and Innovation on that front Okay so uh most people don't know what happened last fall um with the executive order basically the executive order U from the US white house uh stipulated a number of things that are to to start coming into effect uh this summer I think in June um these include things like requiring lure for various models these include things like um restricting how many parameters can be in a model like it's basically telling the population that like you're not permitted to do certain types of mathematics um obviously that should be that is unconstitutional obviously obviously and yet it still happens with almost no fuss from anyone uh these regulations are going to start being imposed and they're only going to get worse because that's how these things work and um another of the the key points is that uh the government will be considering whether to uh whether to include the The open- Source weights of models as a dual use munition um for those who don't know or weren't like around in the 9s uh strong cryptography used to be uh illegal to export and to use in many contexts cryptography which is just mathematics used to be considered a munition and it was illegal to like send cryptography or to use it across across borders there was a huge fight back in the '90s about this and now the exact same thing is happening but with um with AI intelligence system and in these models so at minimum all you guys should be aware that this stuff is happening and again the solution I don't think is to talk to politicians about it because you know they don't ever do anything useful the solution is to actually build permissionless systems that you don't need anyone to give you you know uh permission for that's what crypto is really really good at and so that's what gets me excited yeah maybe to add here like the the reason why they're trying to add this regulations is because they're afraid of con quences of introduction of this machine learning models and the reality is like those kind of regulations are not stopping anyone like we are living in a global world where the fact that like you regulate in us doesn't really stop everybody else in the world to like get access you know train these models and use them for evil and malicious use but even just to like tell you an example like misinformation doesn't need Advanced intelligent tools the example I really like is a journal whose article was published on major uh news Outlet was getting receiving like kind of questions from other people because they're like why did you publish this and and when she went and saw it somebody doctored the page and posted that on Twitter and everybody thought that she indeed wrote that right so the examples like this like you don't need the like Advanced intelligent tools like misinformation been existing for you know well thousands of years but like on internet been very prominent for for decade now and what we do need is upgrade our quote unquote Society operating system the operating system that we are all right now relying on information when we look at it kind of on face value we don't like similarly what happened with transition from HTTP unencrypted content on internet to encryption content where before everybody saw your passwords if you were in coffee shop everybody like anybody could actually replace your page when you're visiting it like we repl we kind of replace that with encryption on end to end we need to have similar thing happening now across the whole society how you interact with government how you interact with each other the fact that like right now when you're making a video call it's not encrypted we're not exchanging Keys is like highly limiting and leads to like 25 million stolen recently uh when somebody thought they're like on video call with their colleague so we need to actually upgrade our operating system and like all the tooling to be cryptography first and like blockchain plays a role as like a ledger as you know coordination identity layer all kinds of things but we need to do it now like this is not an AI problem it's like you know people will use these tools they'll figure out how to use them people right now scamming and uh you know you get a voice voice call that says you know your spouse got kidnapped and actually just recorded a short like 30 seconds of them talking and now can like mimic their voice right that's already happening and like none of the executive orders address any of those problems so we actually have an opportunity here as crypto industry to address this problems this is not like AI problem it's like literally the society problem and we have the tools to do that so I'm like encourage everyone to actually work toward that I feel very inspired from that um Eric and um Ilia we only have like a minute or so left just really quickly what are you working on today that is working towards that future that solution that you feel more optimistic about um what are your contributions uh yes so I as I was getting into the AI stuff I learned about a project called Morpheus which is a an open source decentralized AI network uh so since the fall I've been contributing to that um and yeah that that's what I'm putting my time into yeah I mean so as I said like our global vision and and long-term plan is this fully Sovereign operating system and so we have kind of both the blockchain part of it which is you know near building out chain abstraction and really kind of creating really easy interface to bring people into cryptography we have you know uh the most users now actually having the private key cryptography without actually knowing that they have it being able to be on chain and transact at the same time Edge models so we have Edge intelligence uh which is starting to build up like how do we make models smaller while still as intelligent run them on device and finally working with a lot of startups to kind of as advisor to like help them bring their vision of kind of AI and web 3 together uh to FR thank you guys so much thank you I hope you guys have a good eth [Applause] Denver sorry I'm