Putting Your GPU to Work: Opportunities and Challenges | Lindsay Walker - Lilypad Network

back to our next speaker I would like to present Lindsay Walker hi my name is Lindsay Walker uh I am the product and strategy lead at lilypad uh lad. Tech today what I'm going to be speaking about is uh putting your GPU to work um we're going to talk a little bit about the research that our project based our work off and I'm going to talk about the opportunities and challenges that I see in the landscape of what I call high performance compute Network so Lily Pat is a project that is working to make it easy for anybody to run their AI models or other type of compute jobs on a network of decentralized nodes so the question I'm asking here what do you do with your GPU in a post proof of work world don't be like Voldemort um Harry's asking what happened to him professor and he said and he said he didn't reach his Roi before the merge but this is not a bad thing it means that we have a lot of compute resources available out there so what are the opportunities and what are the challenges for the vast network of high- Performing compute infrastructure that's available across the globe not only in individual miners but in data centers what do we do with all those gpus of course the answer is De centralized compute networ n Works our vision is to give users access to all the models that end users need with confidentiality and integrity in the research that I've done uh there's different ways that you can categorize types of compute networks there are networks that do GPU rental so you rent it by the hour and you can configure and run what you want on there there's also API endpoint networks they provide the models for you uh you make an API call and you get On Demand GPU compute there are networks that bring on groups of trusted nodes and this is honestly the majority of it right now um and there are some networks that are trying to do trustless compute so instead of having an SLA or some kind of an agreement uh they use Game Theory and uh cryptographic mechanisms to ensure that we can rely on this network they're networks that do te or confidential Computing to verify jobs and there are networks that use other verification meth methods and there are different projects that are targeting generalized compute run anything you want on our Network versus specialized they may go for a vertical like gaming or AI inference some advantages of decentralized systems you can unlock a market of data centers that are sitting with 50% idle compute resources so for any given data center 50% of those gpus CPUs what whatever you may have are not being used decentralized systems are dynamic they can scale up in scale down they're resilient and felt tolerant if one fails there's plenty across the globe to take to pick up it means efficiency again we can use those re resources that we already have you don't have to be like Voldemort you're not wasting your compute and blockchain brings payment rails and transparency to these systems to make it possible so let's talk about high performance compute traditionally these clusters or networks um were put together in one geolocation it means they had really really high networking speeds that means that not every single compute task that we used to put on HPC clusters are are great for decentralized systems because we have lower networking speed it means we can't do instantaneous inference like a chatbot as well there's also a lot of latency because it means we have to move the data to run say an AI job to the compute node that is running it that doesn't mean there is a huge opportunity here there are things that are really well suited to these decentralized systems anything that's batch processing if you need to run the same inference jobs many times over a large set of data so you have a genome and you want to use an AI model to uh analyze it it's a large set of data we're making the same call over and over great for a decentralized network any jobs that's massively paralyzable it means we can split it up across the network and these decentralized networks also sort of have this opportunity of putting custom uncensored or fine-tuned inference jobs that you can't find elsewhere which is something that Lily Pad is working really hard towards uh we based some a lot of our work on a paper called modum uh mechanisms for outsourcing computation via decentralized Market I'm going to talk about what this paper uh presented and then some of the things that we're using and then some of the limitations so the summary of this paper it outlines a decentralized system for outsourcing computation it's a network of crypto incentivized actors it deters actors from misbehaving with mediators or validators and enforceable fines no consensus mechanisms needed for that they're not super efficient it doesn't require Global trust and there are decentralized roles in this system job Creator solver resource provider directory kind of it's mostly just storage mediator which we call validator and a module creator let's talk a little bit about what these roles are job creators are the end user you in uh you will kick off a job usually that might be an AI inference job uh we have validators validators check those jobs to make sure you actually ran AMA 3 you didn't go to Google and try to find the answer to it instead solvers match the job creators who have jobs with the compute resource providers resource providers are the ones who have that high performance compute the gpus CPUs what what have you and then module creators are a pretty unique thing to Lily Pad module creators are the ones that containerize and put the types of jobs on the network that means you can put a fine tune model on there and that module Creator can monetize it every time that job is run they earn a little bit of the uh the fees generated on the network it's a game theoretical model um we don't need to have as much security as say consensus mechanism we don't need to validate 100% all jobs done by All actors most of the time we just need to economically Inc disincentivize what we don't want we want to make it easiest and most profitable to just per perform the job as intended um this is difficult in a game theoretic model there's a lot of collusion that goes on so it's not as easy as it sounded we we've been doing a lot of work um experimenting with this seeing how how it runs in real life in our test net one thing we make a lot of use of is statistics based job matching so our resource providers um they have statistics we record them they go on chain for a lot of transparency lilypad job creators get bids by resource providers by price and by requirements if you're a job Creator and you want to use a resource provider you can look at the machine specs how much vram um how much RAM how much CPU is available from this resource provider but also statistics about pass performance how successful are you how long does it take you to run modules this makes it possible for the best actors to get the best jobs but it also makes it possible for those who maybe you don't need the fastest resource provider and you're you're happy to pay less money for that um it makes that possible as well we also have statistics on our modules so again those are those containerized and packaged compute jobs that are put on the network by module creators we collect information about the average tariff flops for the module run and the time to run a job resource provide providers use this onchain information to calculate how much they're going to bid for a job we have module run information so people who are running the jobs know okay what's a success rate how often does this run right what type of U machine specs does it need to run on resource providers also can allow list and set prices for per model or they or not so these are some of the game theoretic mechanisms that we enable to enable the best actors to rise to the top and um earn the mo most rewards from our system now I want to talk about why now why are we doing and why are we creating decentralized compute networks why didn't we do this 5 10 years ago well there's a lot of idle GPU according to the United Nations University in 2024 the global GPU Market size was 634 billion that's a lot that is a lot of market cap in data centers more than half of the gpus are sitting idle at any given time on average so now we have this really great mature blockchain technology we can create more efficient open marketplaces with better prices there's a huge demand for compute we have the tools to do it and we have the we have the resources they're sitting there doing nothing we have the crypto and web Community these are open communities that can keep up with a speed of AI Innovation I am floored every day when I hear about the new models and new agents being built especially in the web 3 Community the web 3 Community has a ton of technical Talent eager to build and accelerate the growth of the AI ecosystem our Universal and programmable payment rails that is blockchains with smart contracts are really good for things like AI agents um they don't have to you know have a wallet we can use smart contracts to enable them most importantly um in the last like couple years there's been huge advances in parallel processing and containerization uh the Cuda toolkit in rock M um being able to run what you want on those your ml jobs your AI jobs that's new that's just emerged in the past two years uh we have Docker enabled orchestration so it enables us to sandbox and coordinate jobs on these different nodes and and a project that we call that we use called bakalow enables these workloads on untrusted Hardware to run across a wide Network there are some really big challenges and that's why I'm excited for the number of projects that we have going on in this space these dpin networks for AI four enormous things that we have to solve to make these possible especially if you want to have truly trustless networks trusted networks actually are also a good opportunity as well but we need to build a reliable decentralized network of nonhomogeneous compute nodes that means we have all these data centers with idle gpus but they're not just Nvidia we have AMD we have gpus with arm architectures we need to make sure our Network can run on all of these different types validating is hard it is hard hard hard and it's a cool problem so if we are running a job usually it's easy to check if you have a deterministic job 2 plus 2 should be four yeah I can check that with jobs like AI you get one put you put one input in and you get a different output out every time that's what non-determinism means it is really hard to check if you don't know what the output is supposed to be um like I was talking about with our statistics a little bit it's hard to build a hardened and enforceable game theoretical model so your actors are doing what they're supposed to do and getting incentivized for it and then security is a big thing um you want to build a product your data needs to be secure and the modules the compute jobs you're running on these nodes need to be secure you don't want malware showing up on a bunch of compute nodes so I'm going to dig into these a little bit building Network nodes this is really hard in order to capture the entire GPU Market at 634 billion the software needs to work on different architectures and different devices different drivers it means we need to also the way our ours works you know that game theoretic model we need to collect information and validate from all that different Hardware that means a different different software for each one different drivers it's hard to configure so we install our Lily Pad software on each of all each of those nodes it needs means that we need to be able to configure it for each of those nodes and then we have those modules which are the packaged compute jobs that are sent out over the network all so must be written and configured for all nodes on the network so this is one of the things that I'm really excited to tackle at lilypad we've start we've started with certain um Nvidia architectures but we're starting to look at different architectures how can we make this work on everything and unlock all those idle gpus validation is hard there's different ways people go about it I kind of look at it in two ways you can do pessimistic validation you have to check everything you assume no one's doing what they're supposed suppos to you check every single job run and it can take a lot of work some of the ways people do that are generally cryptographic proving mechanisms use ZK proofs fully homomorphic encryption that's when you encrypt the data going in you act over the data encrypted and you send the data back encrypted or confidential Computing where all of the Computing happens in a secure Enclave on your uh node there's also optimistic which is which is what people tend to do uh just because it's less expensive computationally and energy and everything you have to run Less jobs we assume that jobs are run correctly but we have some sort of random challenging we can use some probabilistic methods with this as well you can do GPU profiling when I run this job on this type of uh GPU what does it look like how much power is it using that when it Cycles through what are the T flops doing what is you know how much heat is it giving off uh an Nvidia SMI gives some really great information for that but it's a little bit harder on different drivers semantic validation you check it does it make sense this non-deterministic output from an AI model does it make sense I ask you what is America's most popular cat if if you say Garfield you're like okay that makes sense uh if you give you give me a Snoopy semantically I don't think that person ran the the high parameter AI model I was asking for so I've done a little research on that and there's an interesting landscape but it's not very big the projects I've looked at and this is something Lily Pad is thinking about as well have a narrow set of types of compute they can validate well we want to run any job on any node a lot of these projects only can validate like a set of five llms I have a method for this but it only works for these five types of llms uh there's a really cool project that can ml flows but it's limited to specific Hardware uh some some projects are experimenting with ZK proofs but these are really computationally intensive and expensive so it might not be a good alternative it might be if it's a big enough problem though the other problem Game Theory I've talked about this before we want to prevent cheating we want to prevent collusion it should be impossible to game the network ensure if you're running a job on the network you're getting what you're paying for we don't want uh certain actors to unfairly win more jobs than others so that's a big problem to solve a fun problem to think about we want to prevent collusion between actors some examples of what this might look like I talked about our actors before we have our job creators our resource providers and then we have validators a validator could collude with a resource provider and un fairly just send it all the jobs we have to make sure that that can't happen or the solver the one that does um oh that was the the solver collusion uh validating could be you validate a job that didn't actually run security is huge right currently a big complaint is most llms are you use uh openly you get that those free credits for it they're collecting and using your data if you are a paying customer that wants to use AI in your Enterprise use case absolutely not even a question you have to protect intellectual property and personal identifiable information that means there's three types of data you have to protect data and Transit getting the data in and out of the models data in use that compute has to be has have integrity it has to be confidential if you're running it on a decentralized node or you can use a trusted node data at rest not as much of a problem we at Lily Pad we kind of just send the data and we say we're not storing this this is yours it's yours to deal with but we could also Outsource it to a storage provider and say hey you guys solve this problem of storing it we also need to make sure that models have integrity if we're allowing these uh AI inference jobs to run on every computer we also don't want it to be setting up a bot net uh on that computer or something else that's really bad so we want to make sure models have integrity and or make sure that we have a really strict barrier between the models that we're running and uh the systems that they're running on a couple things I'm really excited about I think uh breakthroughs and advances in both of these things means that uh we'll be able to rapidly Advance trustless decentralized networks fully homomorphic encryption like I said before the data goes in it's encrypted it's acted on encrypted it's sent out encrypted it never gets decrypted the and the node can never see what's going on it's really complex and expensive to compute I read somewhere but this was an old paper that it's like 5,000 times less efficient it might be in the hundreds now but regardless it's it's a lot less efficient um and there's a lot of other little details it actually means the cipher text the data going in and out is a lot bigger than the original te also kind of known as uh confidential computing uh it's not a secure Enclave the the decentralized node running it has no access to it there's protected Keys within this Enclave the problem with that is usually these secure enclaves only work over CPUs there's not wide availability eg a big decentralized network of tees that are acting over a GPU I believe there's two types like the Blackwell architecture I forget what the other one is there's two architectures that actually enable tees over gpus however I bet you in the next 5 years some of this stuff is going to be solved it's going to become a lot more efficient and it'll be it'll mean really big things for trustless decentralized networks so yeah I think the right now we have a lot of trusted networks which is great it means we're offering other options over big expensive Cloud providers where you can save 60 to 70% on high performance compute costs what I'm excited about is when we can see Trust secure Dai networks with advances in fully homomorphic encryption trusted execution environments and ZK proofs I want to give special thanks to our amazing engineering team um these are the people solving these really hard problems implementing this very complex uh theoretical paper and thinking about all the implementation details that weren't covered so big thanks to our engineering team at thead and I'm Lindsay Walker I'm the product and strategy lead if you want to learn more about lilypad go to lad. Tech follow us on Twitter I am on Twitter and I'm also on Blue Sky we also just released an API so I encourage everybody to go test that out thank you very much thank you so much Lindsay uh we actually have some time for questions if anybody would like to feel some um there are a couple of comments on YouTube if you want me to s tell you a little bit about it so Andrew said yes uh we also need it to prevent single point of failures of a if a nation state or a vendor decides to censor or give unfair advantages to competitors yeah that's a great Point um that's why decentralized networks are are really really great um you can might be able to cut off access but with a decentralized network there is no single point of failure right there's tons of nodes out there that can still do the job for you yeah that's beautiful and then uh Jay said we need to decentralize compute to ensure everyone can compete and build AI based applications yeah um it it and one thing that Lily Pad does really well is we allow you to run any model um we it's really easy to deploy any model so it means if you it's super easy to put it on the lily pad Network and monetize it okay thank you so much for your presentation we look forward to seeing more thank you go we