Harry Grieve I Decentralizing Compute for Machine Learning I Gensyn I Pragma London 2024

[Music] for this next interview I want to bring on Harry who's the co-founder of Jensen and we're going to talk about all things that they're doing how do we actually decentralize compute and the opportunity here that lies with crypto and AI so please join me in welcoming Harry on stage and we'll go right into the next chat so this is uh this is a topic that uh is increasingly getting more and more requested for all of our all of our events um and it's natural because as somebody who's kind of doing computer science this is like the the most obvious And Timely overlap that they see so our goal here is to kind of think about how do we perfectly blend uh and think about the opportunity with crypto Andi um but the thing I want to do is start off with it is um maybe getting a bit more introduction on yourself and and Jensen so uh what is it that you're working on and how does it work and we'll kind of start with that awesome cheers so Jensen is a peer-to-peer machine learning compute protocol in essence it allows machine learning Engineers researchers and also other machines so artificially intelligent agents to train neuro networks and allows them to do this at extremely High scale so basically internet scale but also at a lower kind of unit economic cost than if you were to use say AWS or even some of the custom GPU clouds um it came out of my co-founder and I uh Dr Ben Fielding meeting in 2020 and kind of having this issue to you know in our respective career trajectories where I was leading a machine learning research team at an applied AI startup here in London we were predicting disaster events for insurance companies and the cost of compute that we were incurring was marching ever upwards and it was becoming quite clear it was around the time that uh gpt2 was released but we thought there was going to be a kind of explosion gpt2 gpt2 yeah so gpt2 uh I think it was early 2019 it became clear that you know these models were generalizing and I looked at it and I thought okay our computer costs are going up these models are getting bigger you know there needs to be a larger system to accommodate this we are getting squeezed on price this doesn't look good for the world um so that that was my kind of like runup to it my co-founder Ben he was in uh Academia so he was doing a PhD in neuroarchitecture design and he basically had to build new neuro architectures which would compete with you know Labs like Deep Mind open AI Etc but he had to do it with four gpus under his desk in our University research lab and it was basically impossible because Google you know had TPU clusters you had all the stuff he couldn't compete we met each other in early 2020 and we kind of bonded over two things we bonded over this kind of pretty Stark like economic problem which was emerging but we also bonded over uh funly enough uh Edward snowden's autobiography which had just come out and this idea around there be increasing capture around a lot of the internet and we thought these models are getting more generalized the prices are going up and there's lots of oligopolists emerging we need a solution to that and that was the origin Point awesome well so what does it mean to uh for it to be internet scale and um and you obviously talked about margins which I want to get into a little bit later but kind of what is the scale that we're talking about here of course we're seeing fights over GPU resources you still can buy offthe shelf Nvidia Mach uh gpus but um what does that alternative look like and and uh what is the scale of Jensen right now yeah so to kind of talk about the vision it's obvious to kind of everyone here that machine learning is basically the next Industrial Revolution it's going to dramatically change the unit economics of everyone's jobs it's going to free a lot of people from labor that they you know otherwise wouldn't want to do it's going to cause a lot of societal change it'll be interesting to see how it plays out in the US elections this year for example um but it's also got very large consequences for kind of the fabric of the internet itself so right now we exist with an internet that's basically quite static you know you have the client server model if you want content you call a server it's quite static you know there's just text there get back we've reckon in the next kind of two to three years it's going to evolve to a much more Dynamic model where instead of knowledge on the internet being stored in kind of static form it's embedded into the neural weights of machine learning models so if you call a server it will be a machine on the other side a kind of you know machine intelligence receiving your prompt and giving you back the best result and that has very kind of start consequences for lower down the stack because if you control the machines which can produce those models which serve you your information and suddenly you have control over all the information on the internet and you know the last kind of 10 years has been quite instructive about what happens when large companies start to control the internet so the long-term goal what's at stake is basically a new kind of Internet of um of machine intelligence when we think about where we fit in there we provide two things we provide the ability to train a machine learning model across any machine learning compatible device and when I say that I mean any device where it makes like economic sense to train a machine learning model you're not going to do it on like a Casio calculator you know you're not maybe not going to do it on you know some like you know a smart fridge or something but you might do it on an M3 MacBook for example if and only if there's a decentralized network you can attach it to where you can receive a yield for the kind of computational cycles of that device um that creates kind of five really hard research problems so the first problem is how do you verify the work's done without increasing the cost dramatically the most trivial kind of approach to this is just do like Bitcoin you know Brute Force like proof of work just replication makes it too expensive nobody uses it we have a solution which lowers the cost significantly so the overhead is kind of very very low double digits the second question is if you want to compare results between devices how do you do it in a way which is reproducible you know if we have like a Nidia a100 GPU and then there's another nivid architecture b00 and you want to compare the results between the two the way that the tensor floats and the calculations of a tensor cores are performed actually results in very slightly different outputs so if you want to do kind of cryptographic hashing which is obviously the name of the game and decentralized protocols can't do it so we have to solve that problem the third issue is more of a machine learning one how do you optimally distribute chunk and decentralize a very large neuron Network across all the devices in the world there's really interesting research out of Deep Mind like the deloo paper other papers like veruna other papers like Alpa which try and address this and it's probably one of the most interesting research areas in machine learning right now and they say the fourth Point's around how do you kind of build the blockchain component of this particularly around data availability you know there's lots of kind of storage considerations where do you store proofs how do you minimize the overhead no one's going to send around you know you know a sort of terabyte size you know uh model architecture with weights you know to every single node of a network how do you optimize that I me the fifth and final point is how do you you know basically connect those four pieces in a way which is game theoretically secure so that people don't get you know robbed on the protocol and people don't cheat uh so that's what we've been doing for four years uh and right now we have an internal version of a network and we're looking to launch the private version of a network to early adopters later this year that's awesome and and also such a really good way to think about the set of kind of open problems here um how much of this is well actually so I have a couple follow-ups so the first one is in terms of actually implementation versus kind of the the the the openness of the problem itself are you already done implementing these fixes and it's just more about testing this with with this private test net or is it that these are just kind of the ways you have to address this thing but it'll still be some work to actually make sure this can happen on the network itself yeah I think that's a great question so we have a version of it internally I was using it last night you know to to trade in small model um the the kind of three main areas that we're focusing on are one the overall systemwide game theoretic uh security one of the you know as you as everyone in here who's you know built or is building a a crypto protocol can a test security is really hard to prove kind of you know before you do like a a proper main net launch uh but then when you do a mainnet launch it's kind of all out there and you know uh so start of slowly kind of opening it up and managing to test it it's one one of the big pieces and that touches all those kind of uh components I spoke about the second thing that we think uh quite strongly about is that kind of data availability piece um so anyone in here who's building kind of like a data availability layer I think there's definitely a rising demand from machine learning protocols for for that sort of service so far we've had to build our own one internally because we haven't found one which you know externally which kind of makes makes sense for us but it's something that we're and maybe just to dig in which aspect of data availability is more important here just the fact that you want cheap storage or is it the fact that like in the case of machine learning like bandwidth matters and you have egress and Ingress considerations uh what's important in this case uh for us to think about yeah there's basically four things which are important so the first one's like price you know like it just has to be you know cheap um the second one is band so you have to be it has to be you know it has to be at least like a some minimum specification given how much someone's willing to pay for a machine learning job on the network uh the third piece though is the data doesn't need to be there permanently doesn't to be per like perpetually stored but it also can't be kind of in you know ipfs par Lance like unpinned by the person who originally uploaded it because then that breaks her verification mechanism so one of the issues we have is like if you at something like file coin the minimum kind of deal I think they call them deals and file coin that you have to have is like 180 days but if you're someone on our Network who's just wanting to like fine shown you know Mist 7B you know you don't want to like store it for like half a year right you know you want to store it for for less and there's ways that people could build those systems on top of it in a way which um you know on top of like fcoin which make it easier but we haven't seen one which is kind of plug-and playay for our purposes yet and then the final one's just fault tolerance you know around you know eraser encoding or eraser coding you know how do you basically ensure that you know if one of the nodes falls off that it doesn't break the whole system because some of these jobs can be very expensive and if a data component Falls over that that's that's not great um so yeah so we so that's the kind of second thing that we focused on quite a lot just the the kind of data availability piece I'd say the the third and final piece is actually just the decentralized model training itself so it's our you know we know from kind of speaking with a variety of folks in the kind of machine learning space there's an even at the more centralized companies like open AI um you know Google deep mind there's a push to you know design architectures which can train across you know lots of decentralized compute or at least distributed compute in different data centers but the local paper by um by Deep Mind which I mentioned earlier is a a kind of good example of that we're basically at a point where as the computational requirements to get you know emergent intelligence increase and increase further you basically need more and more transistors like in one place and it's getting to the point where you just physically can't fit them like in a single data center um I don't necessarily think there's like a kind of cap to this like the scaling laws haven't broken yet and it's quite clear that you know if someone could make intelligence in order of magnitude smarter and it would you know create more economic value for them they're probably going to want to do it so there needs to be a solution that basically activates this kind of you know world scale Compu Network so yeah um I think to summarize like the three kind of core areas we're working on just now the security piece the idea around kind of you know almost like Communications but also data availability and then thirdly the the machine learning research around how to optimally Shard you know and split models across the network awesome um I guess one thing that has been maybe explicitly mentioned which I assume is the case is that it is a network where anybody else can distribute their gpus or any resources to help kind of let you train and uh and the intention is that after you going to work out all the kings that you open it up and anybody can join and and provide the hardware is that is that correct yeah so one of our core principles is that the network should be decentralized and permissionless I think there's there's a lot of reasons for this um I'll give you kind of top free um the first one is in the very near future like this decade we and we want to to build a network to support this we think that the vast majority of Demand on Jensen will be from machines it won't be from like a machine learning researcher like my co-founder or like someone like me what do you mean by that so like um artificial intelligence or a machine intelligence which exists in a permissionless substrate somewhere so it could be on ethereum which is providing economic value maybe to one of us or maybe to another machine being paid for it somehow probably in crypto but wants to improve the quality of its service because its objective function is for it to you know provide more value or to you know to just maximize its you know it Wallet balance and it um it needs to call a network in order to basically update its parameter State such that it can provide more value um or you know even expand its its parameter state it needs a permissionless network to do that it needs a it needs a permissionless money it needs permissionless networking and it needs kind of permissionless like like data storage you know the compute piece is is the bit which is missing and that's why we're all in on on building it um but we think the vast majority of of computes going to come from machines kind of retraining themselves and you know when we think about what that also the knock on effects of that it needs to be permissionless there can't be a human in the loop um so that's reason one reason two is I think there's just a kind of fundamental Liberty around rounds like a liberty piece here like you know I think what Satoshi did was kind of basically give people their their money back in a way you know give you the keys to your money back which is good it's very similar to like Concepts around like negative versus positive Liberty it's negative Liberty you know it's like uh you own the keys it's yours you know no one else is giving it to you it's very kind of like locky in property rights from the 1600s it's you transform it you you know you find the Bitcoin it's yours no one else is giving it to you um we're moving to a world where our you know organic neurons in our in our brain they've always been ours but now we're augmenting them with these artificial neurons but unless someone does something those artificial neurons aren't yours they're owned by open AI or they're owned by Deep Mind or they're owned by whoever is creating the brain machine interface I think that's like pretty negative and one of the kind of solutions to that of what it's not a Panacea is to provide a permissionless you know peer-to-peer Network where if someone wants to train a model which does a certain thing for them and augments their own intelligence they're able to do that and they have the means to do it in a way which doesn't have to be sanctioned by some higher power I mean that's an awesome uh prediction if we're if we're betting on uh more machine to machine um inference or or even training um you've already obviously hinted about just the fact that it needs to be decentralized is is a a great answer for why crypto needs to be involved here because you have those economic sort of guarantees but um what other overlap you see with crypto and AI here and do you have any kind of bull predictions on where we see that space going yeah absolutely I think of it in kind of like three layers so at the Canal's core layer you've got the compute layer we do machine learning training but you could also uh do inference for example and there companies doing that um so that's the compute layer and then the next layer is kind of I would describe it as like the machine learning operations so to speak um so that's stuff like you know model attribution if you train model in your data how much you know of the parameter space is you know is a function of the data you contributed and how do you get paid for that also ideas around like you know Guard railing if you want like how do you kind of make sure that models are orchestrated in a way which you know um conform with or are aligned with with what you want so there's a bunch of stuff there there's also I'd say like the some of the data stuff fits fits in there as well um and then there's I think there's lots of people building at that layer right now so like when we go to hack pH we see lots of people like building in there then the next one which I reckon is like maybe three or four years out is once machine once these two bits exist in kind of anger and then there's a you know there basically is something which would approximate artificial general intelligence and its permissionless and it's on a cryptographic substrate and you're predicting this in the next four years is that is that what I'm hearing yeah i' I'd say so for sure yeah um the um the next layer is how do they interface with the real world and what's the kind of you know mechanism by which like very like trivially if a machine intelligence wants to mail something to someone how do they like physically make the post office Do It um it's like a you know pretty like dumb example but like use the fatex API somehow yeah yeah exactly but you know when when when when like a person in like the real world has to actually do something you know what's the kind of mechanism for that happening um I think you know there'll definitely be um they'll def definely be kind of web two companies which are doing that but I think it's something which is because I think a lot of the intelligence will or most of intelligence will exist in in web free or crypto I think it's quite kind of you know appropriate that there's those mechanisms in the cryptographic space as well awesome um you said the the private kind of testing is uh is in the works um when can we see a broader public access to Jensen and kind of what's next for the protocol yeah so we we like to kind of you know over commit and underd deliver it's the the kind of name of the game for crypto protocols but we um we're doing the the kind of private testing uh this year and then we hope to do the public testing probably next year although it's it's it's kind of TBD um I think when we think about like you know what what's next we're very interested in kind of two areas so anyone who's kind of thinking sort of deeply about the optimal ways to kind of train models across lots of um you know lots of De centralized devices that's something we we think very deeply about and would love to talk more about but also anyone who's kind of providing any of those like incil um tooling that that we spoke about like I I think particularly the data stuff we weirdly like we're like a a compute protocol like we don't think about data you should be using something else we should be using something else right and like it's just it's something my co-founder and I have always kind of like we've kind of like grudgingly like built this thing internally but you know we've only Built it because it's there hasn't been a kind of external Solution that's become apparent so anything in those two areas is something we definitely love to chat about awesome well har thank you so much for this amazing conversation and uh can't wait to have a a better than AWS Services out there that's decentralized yeah my pleasure cheers awesome