Fireside chat with Dominic Williams, DFINITY, and Kartik Talwar, ETHGlobal I Pragma Singapore 2024

[Music] want to bring on Dominic here who is the founder of definity in Internet computer and he currently he is the chief scientist of definity foundation we're going to have a discussion on kind of how we actually can think about exploring the possibilities with blockchains we're going to look at AI on chain how we make chains more accessible and a lot more in between so without further Ado please welcome Dominic on stage hi good so I want to start off by you kind of giving us a a refresher on the world back six seven years ago um when you came up with definity and interet computer what was kind of the original goal and what made you start this as a project and uh how does it compare to where are now um actually it was incredibly N9 years ago so if you go to the way back in you know the internet's Wayback machine and um type in definity doio you'll see the original definity website from 2015 that describes um the vision of creating crypto ledgers that can scale infinitely and could potentially host something like a search engine or Gmail um so the definity project was you know I was part of the earli E theum Community sort of 2014 uh uh on was and there was this phrase World computer and uh you know I had my own interpretation of the phrase World computer and decided that needed to be built and the vision essentially is that you can extend the internet so that as as well as the Internet connecting software it can host software but of course it's software superpowers it's smart contracts and uh I I saw in sort of 2014 2015 when this kind of stuff started um we first started working on the stuff that smart smart contracts really a new kind of software and this new kind of software has amazing properties so it's hosted on the work obviously um one of the properties is that it's tamper proof a smart contract is guaranteed to run the logic as written against the correct data and in a sense you can think of that as meaning that smart contract is immune to a Cyber attack you don't need to protect it with a firewall or anti- malware to stop it being in infected with ransomware or something so you know for me that was a big deal uh obviously a smart contract is always available it doesn't crash it's not like a software running on a traditional server um you can make it autonomous which means that you can get rid of human controllers either by making it non- updatable or putting under the control of a dow it's composable so I saw that all these properties existed and so my conclusion in 2015 was inevitably um smart contract software I don't think it'll be called called smart contracts forever it's a very silly name came from Sao um but you know Network resident blockchain resident software will actually become the default backend software uh eventually and it's just completely inevitable because of these huge advantages and so I said well okay um if it's inevitable let's bring it on and and let's try and uh create a blockchain that you know um has the speed efficiency and ability to scale with bounds um that's required for a world computer awesome so fast forward to now we have a functioning World computer or Internet computer and in the last two years especially we've seen a massive wave of AI applications and a big part of that is what people are worried about who gets to control these models and this is a perfect example for where blockchains can come in or just generally large computer actually helps so what I'm curious to understand is kind how do you think about AI actually getting more U on chain natively and what do you think are the possibilities here yeah so so as always you know we tend to see things differently to the rest of the crypto industry um and you know for us like when people say what's the intersection of crypto and blog chain um you know we see AI running on chain and we also see AI building on chain so um first of all uh um we think AI should run a smallart contracts and yeah it's true that you can put it under the control of a dow and so on but there actually reasons why Enterprises are going to want to do this so uh you know we were already familiar with the problem of creating a hot wallet you know creating some kind of web 2 website which has got a private key inside the backend code on Amazon web services um that maintains a hot wallet sooner or later someone will hack into that take the key and steal all the crypto well AI is like that but with data so if you if if you run want to run your own Ai and increasingly people are doing that because they don't want to upload all their data to you know open AI or anthropic or X and whatever um the challenge is that uh you know probably you're running your own AI because the data is very confidential like imagine a law firm you you know a law firm with a one or 200 Partners or something and uh this AI maybe has a context database and it's downloading all of the emails with clients all of the previous legal filings relevant case law legal tax and that you know the partners want to speak to this Ai and uh get answers to assist their clients or prepare for trial um it's a will be extremely useful for them maybe they built it using llama 3 or something but the problem is if it gets hacked the law firm's over and they're going to ruin the lives of their clients because in order to be useful that AI has to have access to all of that super confidential information and that super confidential information will be in one place so if you put it on Amazon web services or some kind of traditional Web Two Tech um you've created a hot wallet for sensitive data and of course uh that's what cyber attackers will Zero in on there's no way of create keeping web 2 Tech secure you can spend you can think you're a security expert you can think you've securely configured Linux and your database and you got a firewall and your anti-intrusion monitoring software and your anti- malware um there's absolutely no way you can't be sure that some software patch some zero day exploit some something won't let an attacker in it's just a matter of time really and you know cyber secur is about to get a whole lot worse with uh AI because you know AI can already hack um systems very effectively but you know GPT is locked down but if you can get a hold of an unlocked version of GPT you can use it to already hack it's been demonstrated you know systems very widely so um you got this problem like you know you this Law Firm you want to run a AI model securely you don't want it to be a hot wallet for your client's data whereby if it gets hacked cuz it's running on web to all of those confidential emails with your your clients all of those legal filings um and so on make it into the public domain blowing up your law firm and ruining your clients lives so you're in this kind of what do you do well our answer is that you run your AI as a smart contract on the internet computer it can it's it's to some extent privacy preserving and you can also run a private internet computer to control the nodes and even that means even if someone has harder access they can't get it mostly for curiosi is there a distinction between what running it as a smart contract means is this inference we're talking about is it training is it just calling it or is it all of the model storage or all the above well we think actually there's going to be a mix of models I think you know uh sometimes um people run centralized models and just keep a vector database in the smart contract but but we're also very keen that AI models run a smart contracts so so far we've demonstrated um uh image classification running as a smart contract cont and we've demonstrated facial recognition running as a small contract so if you're from the ethereum world you have to reorient yourself a bit so the internet computer currently is is running the equivalent of about 1.5 million ethereum transactions a second so that's just measuring it in terms of the you know virtual machine instructions executed okay so like instructions and flops instead and it's and you know it's capable of much more so um you know I forget how many was instructions are involved with you know like image classification inference but it's it's like in the billions I think it's like 20 billion or something so you you you know you need in order to run AI as a smart contract on the blockchain itself you need like a special kind of network that's much more efficient but currently we we're sort of stuck at um we're stuck at facial recognition and I'll tell you why the internet computer runs each smart contract it it it's like an independent software unit um and there's parallelism um but it runs each software unit each smart contract uh in a 32-bit environment and so it's got 4 gig basically so when you've got like so it turns out that you basically can just double the number of parameters to see how much memory you need so the fa the facial recognition needed two billion parameters like weights AI model weights so that means you need four four gab of memory to run the smart contract in so um you know if you look at llama 3 it's like 8 gigs but within weeks uh internet computer hopefully will be upgrading um and the smart contract execution environment will get a 64-bit and the memory bypassing wasm or or how no no it's going I mean so we're very lucky so uh you actually um the the the web assembly inventor uh a early definity guy um so we've always you know um played very well with the you know we've always known how to use web assembly and push changes into the standard and being aware of changes coming and so on so um wazzy time which is the yeah that's moving there'll be a 64- bit version soon yeah and CU this is like now just me personally trying to understand because you're now representing floating points specifically differently or is there actually a benefit or like is there actually some Innovation around getting the 64 bit yeah so that's no that's uh you know um it turns out the there's not a problem with floating points the determinism right um I promise everybody won't be technical but we'll just do this no web assemly processes floating points deterministically no we added like we spent a lot of time optimizing the internet computer smart contract execution environment for AI so for example we added simed instructions forget what it stands for like single instruction multiple D I forget what that stat right but essentially allows you to do multiple calculations in parallel a bit like a matrix calculation right so um you know there's been lots and lots of optimization and in fact uh since the first demo which is image classification I think we got almost like a 10x speed up for running you know for AI inference um but in order to get like large language models onto the blockchain running a small contracts we we need to increase the memory because they have more parameters so like you know the smallest distillation of of llama or one of the smallest is like um you still need like 80 gigs gig eight 8 billion parameters yeah by the way it's still a very powerful model you can do a lot with it and it's going to be incredible to have that kind of uh model running as a smart contract and um we've got other Technical Innovations coming which we think is going to make it very easy to boot up um and very inexpensive to boot up smart contracts with AI inside like memory mapping parameters and stuff like that so there's a lot coming and and also just in in the same scenario how how does it work when you have to delegate something to GPU because a lot of this context will be CPU based for a lot of these inference but can you actually pass this on to a cluster of gpus No so at the moment um the so the internet computers you know ethereum like runs on the cloud right like people create proof of State validators in the you know and then you interact with it by the cloud so internet computer I suppose is a more like puritanical project we and and you know it's all Sovereign hardware and basically in order to add nodes they have to be like their their um specialized Hardware built to a standard there's a bunch of reasons why that's necessary to to run them with the protocol but um there's no gpus on the notes we're trying to get to the point where we can spec credit a spec for for for no machines or gpus um but it's okay because uh inference is actually pretty efficient so you know when you're talking about like running llama 3 on on the Chain um for a for a legal company or for a smart wallet like you want to tell the wallet you can map the requ yeah like if rap Bitcoin hits 100,000 Swit you know sell half a usdc swap half of usdc also like you don't need to optimize for like I don't need 500 tokens per second you can you can be okay with 20 like you know like GP you know like open AI is serving you know millions of concurrent users um most of these smart contract applications aren't doing that so it's fine inference is actually fine without gpus all be it of course we want to add them um what it does mean is currently um you can't train the models on chain yet you can put them after and then you get to use them okay but typically you won't do that like typically if what you'll do is you'll take a pre-trained base model like llama 3 a distillation of llama 3 or F or a whole bunch of other ones now you take the distillation and then what you do is you combine it with a vector database and that contains the custom cont you know the context right and so when you want it to do some inference for you you first take the The Prompt turn that into a vector database query basically turn it into a vector you get all the content which has got the shortest Vector distance and then you prime this the the chat stream the session so you say hey llama 3 here's some background information now answer this question but of course the user doesn't see that they just see the question they've asked right so typically you know that's how you'll do it anyway like you but but at the moment if you wanted to create like a Laura like these kind of patches or incremental fine-tuning training you you you still had to do that off train you can't do that which is okay which is okay it's still very you know still very very useful because you know uh the smart contract tells you look this is the um hash of the AI parameters so you know exactly where you're running um and you when you run the inference on chain you know you're guaranteed that it was this spec it was the AI with this code hash with this parameter hash that ran this input prompt and It produced this response so and it's it's hack proof you don't have to worry you know about and also has a nice side effect you actually get to uh map and prevent hallucinations because you will be actually able to detect where the hallucinations are happening and correct for it well ter of hallucinations is it's it's a feature of Transformers unfortunately so but I mean that's another subject but um I think the um yeah so have a trace for like where what be really nice is like onchain training but for that we do need gpus so the challenge with gpus is they're non-deterministic it's not actually the floating points it turns out it the when you ship AI calculations onto gpus it's like all these different majes and the calculations are performed in parallel and the act and then you know the results are G to more calculations and so on and the sequencing of those calculations is non-deterministic and that's why you can't have blockchain nodes or gpus yet but it turns out that you know obviously we want to so we got a team of people working on making the gpus deterministic obviously we're going to sacrifice a bunch of performance but that's okay right you know if we we lose half the power of the gpus but we can optimize we can start doing trading on chain it's okay um but it turns out that there's quite a few other fields that want to to find ways to make GPU calculations deterministic one of them is fully homomorphic encryption so um there's a bunch of people trying to make fully homomorphic encryption uh practical by optimizing your GPU hardware and they have exactly the same need you got to make it deterministic so I think you know it's a problem that will get solved I don't know when it's going to get solved it's not it's like unsolved you just get to optimize it and make it faster right so that the goal becomes performance not binary is it possible or not yeah I think look yeah I think look I I think with respect to like why would you run an AI onchain um you know you're not trying to be GPT you're not trying to be clawed you're not trying to be Gro right um there's a specific sort of you know bunch of use cases one is like you want to run a custom Ai and it's incredibly important that it's immune to Cyber attack and people can't store the data or can't corupt it for example as per the case with lawyers another one is you know maybe that it's a smart wallet and you want to you know talk to your smart wallet and say look I'm on the beach if Bitcoin hits this price you know swap off for usdc that kind of thing right so again you know if in order for you to be able to trust the AI it needs to be a smart contract um and you know ultimate performance you know isn't why you put it on the blockchain the the performance has to be sufficient right but the reason you're putting on the blockchain is because you want to be immune to Cyber attack you want to protect the data in this case you're getting better guarantees around sustainability or or it's always available I you know imagine like in the case of the lawyers the law firm these Partners discover with the AI instead of taking three weeks to prepare for trial it takes three days so then the lawyer leaves it he doesn't start preparing until 3 days before trial but he goes to his Ai and it's on web to and something's gone wrong right the hard drive has crashed and he's lost all his data it's been encrypted with Ransom I don't know and now the AI is not available but he's completely screwed because without the AI it takes three weeks to prepare for trial but he's only got three days because he thought he'd have the AI so resilience is incredibly important so you know people can put AI on the blockchain um to benefit from these security and resilience properties and things also one other thing that and autonomy in the case of a wallet of course well that and also like people will start doing custom AIS that they would like to make money off of and they get to put that model and you get to say this is very customed totally model that specifically solves this the other last one is you know if you look at the internet computer you can create a social network with just one piece of serverless code serverless code to us is a smart contract right um with these properties um you know you have to have a special kind of smart contract that can process HTTP and so on um it turns out and also the way that you you know the way that you describe an application a web application um is very abstract because in the case of the internet computer like the logic runs in persistent memory Pages it's called orthogonal persistence so you know you just describe your abstractions and the data lives in the abstractions so it's ideal for AI and uh there's some other Innovations coming that you know when you upgrade a smart contract there's some new languages that produce that'll be available in about a month this functionality which have type safety so you when if you're transforming your data structures you get some safety in that the data the transformed data won't be lost so basically the AI can you can talk to the AI we' got this running of course talk to the AI it creates a system for you gives you a URL on the which is basically right something you can use to interact directly with the smart contracts right that's your web application you can use that web application put data in it and you can actually um up you know have the AI upgrade the running system and we think that that's really the future for All Tech like in the future if you're talking about like stuff that's like you know an information system like Salesforce like nobody's going to go and like rent this thing and then try and customize it using the hooks instead you'll just go to uh the AI and say hey give me a CRM and it's going to give you a CRM which is your own CRM custom to you yeah yeah you control all the data you're not uploading the data to a third party service provider it's your smart contracts your data you control it you got the keys and now you just keep talking to the AI and it just keeps an up dating it and customizing it and adding the features that you want this is going to be the new paradigm so one of the other really interesting things about blockchain is when you can make it an endtoend stack it's also the ultimate stack for AI to build things on and if you think by comparison about web 2 how people do it today how would the AI do it the AI the AI would you can't just upload a piece of abstract code the AI would have to go to Amazon web Serv provision a server pay for the credit Cardis yeah get to get an sign up for Amazon web services put a credit card in create some instances provision servers yeah set up databases patch them configure them get on a call with the sales prep that enterprise software company kubernetes set up cyber security um load balancing I mean it's a nightmar right there's no way the AI can do that and then imagine how on Earth would you um upgrade that running Sy system safely um as how could you trust that it's done it safely how did you trust that it's done the cyber security uh in a way that going to so look it doesn't work really I mean it can assist like Co GitHub copol can help you right but it's not going the whole way the future is you just talk to the AI and it modifies stuff unless it's something very specific like a you know like a social context into what it can and can't change so uh you know that's the other I think the other dimension you know that that we're going to see blockchain excelling as being the platform for AI because it simplifies R&D yeah and looks like you're already doing a lot of these experiments and something is enough things are live on the Chain right now for for you to kind actually show that this is actually possible which is which is awesome uh my last kind of question to you would be um I know we'll have to uh keep this very high level but uh in this world models exist on ICP people get to use them and we obviously want other chains to also be part of this and you kind of have a whole platform around this where for chain Fusion how does that work at a high level and uh how do you get to make this work with multiple chains uh so in the last year uh the internet computer has um buiness upgraded this called chain key uh capabilities so uh smart contracts on the internet computer can uh sign transactions on other blockchains like Bitcoin and ethereum without having a private key the blockchain itself act as the private key of course behind the scenes it's using a complex assembly of threshold cryptography the world's only non-interactive dkg key sharing and all this kind of stuff um but essentially it allows a blockchain um to be a private key so now you know you can have a small contract that generates a bunch of public keys and can sign for those public keys so for example the public key now with the chain Fusion tag uh you know you can ask a small contract give me an ecdsa public key for for an EA you know ethereum account for example and um now you can sign for it so the actual internet computer network is integrated at the network level with Bitcoin and ethereum Bitcoin was the first uh ethereum is a more recent integration and you know there called chain key eth and stuff like that anyway the point is that um you can do smart contract cor level integration so it's possible to have a have Smart contracts on the on aium like D5 smart contracts in ethereum and you can build the rest of the infrastructure like the user user interactive user experience anything which involves uh AI storing and processing large amounts of data you can build it on the internet computer and you know smart contracts on the internet computer can interact with smart contracts on on ethereum in a trustless way um using cheny cryptography you can even actually there's a special kind of Dow you can use on the internet computer called a service nervous system um and you can even put like smart contracts on ethereum under the direct control of a Down internet computer along yeah alongside the user experience and the AI and the data and whatever else there is on so uh you know that's the world computer vision that you just keep on building um this stuff out and amalgamating stuff um but it's a very effective way of like if you want to fully decentralize and get rid of web 2 and you know it's worth remembering like nearly all the hacks and things that go wrong uh in this space apart from re-entrancy bugs in the evm um apart from that it's nearly always some kind of Bri or web 2 it's a human controlled areaa yeah you know and so there's there's a reason for decentralization and if you really want to fully decentralize your projects and make them more robust um and put them maybe under you know the control of a dow but automated control not like decentralization theater when people they vote on a piece of text and then the engineer goes the administrator goes away and uses his back door key like full fully automated fully autonomous decentralized control um yeah internet computers is a great option for that amazing Dominic thank you so much for this amazing conversation and looking forward to seeing more AIS on chain yeah thanks thanks C thank you [Music]