Decentralized AI: Beneficial Scenarios – San Francisco 2025

Well, here we are again. Those two you already know. Uh those two you might not even though if you paid attention to my talk you would. Uh the these are the co-authors of the gaming the future book that I mentioned at the beginning and Mark Miller has been a security pioneer uh for a really long time and is a foresight senior fellow and Christine Peterson um co-founded forite institute the organization that I now lead uh and is credited with coining the term open source software. So we have a really interesting panel.

I think we want to get into a few of the questions that like I broadly touched on uh during the intro talk uh and maybe also in like your unique perspectives on that issue because you're all coming from different fields and so it's really interesting I think to hear your specific perspectives on this. So if we can just have a fire around and like keep the your comments short and sweet on you know where do you think the future is headed? Is it going to more towards more in a centralized versus decentralized framing? And I know this is like not cutting it, but you can probably provide more color on it if I just give you these like very hand wrappets. So why don't we start with Max and then we literally just go and trickle down here.

Where's the future headed? Um, one one thing that I really would like to do is to operationalize decentralized versus centralized on like in in what way are you talking about? Because when you say decentralized versus centralized, it's like a a pattern matching between does this look like a distributed network or does this look like a single node network. But I think uh I like the way that some uh basically business strategists would think about it. So I would think about the different levels like the stack, the AI stack, the application layer, there's the layer of the models themselves, there is the model serving platforms, there's the uh there's the actual hardware too.

And so you could talk about uh how centralized or how many actors there are at every layer of the stack. And I think it's not clear. I mean, it's I think it's clear that the I would be very surprised if training in the future was very decentralized. I suspect there's a lot of returns to to centralization there, but on the others, I think it's it's varying. And I can elaborate more, but since you wanted it to be rapid fire, I I'll pause there.

Cool. Okay. I like the stack distinction. A I'll Yeah, I'll extend to what Max said. So, I feel like u both the worlds are going to exist in front of us.

uh the clear winner will definitely the one where we where users feel that they are more empowered, they're more secure, there is more privacy to their data to to to the way they interact with AI and uh I feel like um you know there would be different use cases where each of those technologies would be beneficial in different aspects. Um, but I feel like the the the winner or the one that will go really uh in the long run would be the one where there would be more security and privacy for the users interacting with the technology. I like a lot of the themes we're all discussing. Uh I think I I um I want to focus on this issue of the inevitable arms race in each separate domain between offense and defense and trying to project those arms races. trying to project those arms races uh into the future based on what we know about the underlying nature of the of the system and try to get a sense of does the offense side or the defense side naturally get a decisive advantage in the arms race?

And for those systems like I would argue at the computer security level of abstraction in the digital realm uh where uh defense does eventually arrive at a decisive advantage over offense. And we know this because all the principles were disco necessary were discovered most of them by the by the end of the 1970s some of them in the '9s. Uh and essentially we know everything we need to know technologically to build secure operating systems provably secure operating systems uh languages and cryptographic protocols. Um adoption-wise, we've only gotten the massive adoption for the cryptographic protocols. In addition, uh the big unsolved problem back then was the supply chain risk.

We got that solved with blockchain as a first solution. There's other solutions possible. Um the physical realm to take the other side of the spectrum is one where naturally the offense has a decisive advantage over the defense simply because it's easier to destroy than it is to defend against instru destruction. And historically, it's always been the case that the solutions we've had to in the physical realm to that ease of destruction was to build some kind of more or less centralized institution. And the more centralized it is, the more dangers of corruption.

So the hard challenge in the um across all of these realms, but especially the hardest one being in the physical realm is how to arrange as decentralized as possible means to bring about a long-term decisive advantage of defense over offense. Cool. Yeah, I'm not a computer scientist, but I'd like to just ask a question, which is um are we sure that large language models, the way we're training them today, is this the ultimate for intelligence in machines? Is might there not be other ways to do artificial intelligence programming? Are you looking at me, Mark, like you want to answer the question?

Okay. Well, great. What's the answer, Mark? Okay, I got Okay. Um, so the answer is probably not.

Uh, Yan Lun, one of the, um, winners I think of the touring award uh, for the transformer models. Um, he is very strongly of the opinion having been having been a major creator of these language models. He is very much of the opinion that to get to AGI you have to start with something other than LLMs. I was just hearing from a uh Ravi for um Ravi Pa foresight the um just hearing about how miserable these LLMs do including the ch the long chains of thought at solving blocks world planning problems when the blocks world the logic of the blocks world problem is encoded in other words so it can't simply do pattern matching against the text that exists about blocks world problems and they're awful at it. Uh is that it?

Okay. Yes. Well, that was I was sort of hinting at that might be the answer. So, uh if that is the answer, that's going to affect this question of decentralization versus centralization. These LLMs take a huge amount of compute, an awful lot of data.

Can we find other ways? And if we do find another way, how is I think that might lead to a lot more decentralization potentially. So you think really like the I guess like Max and Christine, you both kind of agree that like it's in the training basically and like you know the training will mostly affect whether or not we'll have a centralized system or not and and the types of models that we're training. Um I guess I would I would elaborate I guess and and say that the the question of centralization and decentralization is different at the level of the uh the supply chain for training the models versus the models themselves. So I think that how I like to think about it is the incentives that shape the distribution of models.

I mean one one question is like why do people open source their models? What does that lead to? What is the future of that? I think that the um I think it's pretty reasonable to expect that they're not there's not going to just be a single model provided by a single provider. But um you know if you think about like different innovator dilemmas and different economic incentives, it could well be that maybe AWS or Microsoft just wins the uh training infrastructure race and they all have their nuclear power plants that train all the AI data centers.

Or it could be as Christine was hinting at, maybe there's probabilistic machine learning and we have we only need a laptop to be able to do it. But um that that's hard to say in the next five years. Okay, let's get maybe into some of the possible downsides, upsides and how we can build for them. like you know when you think about like you know the future in general like and you all have very diversified perspectives on centralized decentralized and that like hand wavy um even just like trying to disintegrate them but like what are you specifically worried about looking ahead into the future and like what do you think we can build today to account for that like if you think about differential tech development or something you know like what's something that you want to point people to um in addition to like the specific risk that you're most concerned about uh whoever wants do you want to go first what scares me the most would be cyber attack just because it's much easier to uh to twiddle bits than it is to twiddle atoms. Um so and I think we're very badly set up for defense right now.

So what I would suggest is how about if we do distributed red teaming and compensate folks for doing that and we need to start right now. Uh I have one take on that as well. um there should be a clear definition on what we are defining as centralized, decentralized, open source, closed source and so on. Like for example, Allison, you had a slide on deepseek. Now, one of the things with with that everyone has observed is that you know it's it's an open source model, but if you ask it a question about tenement square, I think one of the uh historic events uh uh from from the region where the model comes from, it doesn't it refuses to give you any answer about that.

So uh is it fair to still call that model an open source? That's that's like one. But it allows others to build cysts like to basically use that infrastructure to then build other models that allow you to ask questions about the tier deep. Yeah. The the um the thing about deepseek being open source is that the means by which the weights were arrived at has not been open.

there is no open ability to reproduce a set of weights that are like the deepseek weights. uh uh so I would say that that's partially open um uh with that out there uh and given the fact that there are genuinely trained open large language models like Llama uh I think we should look forward to there being genuinely open uh um uh training systems where they're trained in the open from open corpuses And where all of the rand of the random inputs are either made pseudo random or logged random inputs so that exactly the weights arrived at that were allegedly arrived at by the open process can be reproduced deterministically from the same corpus and the same random inputs to create the same weights. Makes sense. Just getting back to the problem, you were really worried about cyber risk and cyber attack. Anyone else want to throw something?

I saw a hand I saw a head nodding for that one. Like I think we can all agree we're worried about that one. Anyone else want to throw one into the head and especially how can we build for it now? I mean my lightning talk was pretty much around that and it feels like uh um I said early on as well uh that I guess uh whichever model whichever approach uh uh or whichever technique we pick up however we build up the models whether LLM is the right way or not moving forward whether is agent and everything I guess cyber security or the security the way uh individuals interact with it the type of responses they give how much can we trust them really having that trustworthiness and and security and sense of peace when using the technology. I guess that is going to be the core in in in in making sure that it evolves in a proper way.

Yeah, I might uh add uh I mean there there's a lot of things that that I could enumerate that that are potentially concerning from the biological risk side to the cyber security side. Um and and also to I I'm really concerned about how people can make sense of uh like we're social creatures. We evolve through cultural evolution. I think it's pretty bad if most of the content we can't distinguish whether it's you know generated from someone who has different interests than ours. Um but uh in terms of the second part of your question which is you what things or what kinds of things can we build or do about it?

My uh I I would take the gist of of my talk to and and present it here. I would want to for us to broaden our sense of what we've been calling technology to things that are not, you know, uh the cutting edge or sexiest uh tech. And I would include things like uh um almost like uh uh psychotherapies as a form of psychotechnology as like a very crucial thing to raising the sanity waterline. And like I was talking about KYC controls, it's like is a protocol for figuring out if the person you're talking to is an AI or not. Is that a technology?

It I think it doesn't have to be programmed in software to be considered a technology. And so I would I would push to lots of those kind of traditional things as things that we can build to address, you know, the bio, the cyber uh and ourformational hazards. Yeah. Um yeah. Uh so when I talk about we've solved the engineering problems of secure computing at the level of security per se uh there's uh two things that I left out.

One is uh we have not solved the adoption problem except for the crypto protocols. uh the the adoption of the crypto protocols uh through open- source efforts in building them and and and um human eyeballs verifying them. Um that was the great success of the cipher punks, a small group of open-source activists uh winning the first crypto war against the US national security system. Um uh that's an extraordinary accomplishment and part of it is overcoming exactly the resistance you just mentioned which is people were using HTTP. HTTP became the entrenched protocol for a while specifically because of the US government export controls during the crucial emergence.

Um but even though most people didn't care, the people who cared got enough people to care by publicizing the dangers that eventually over many years there was this market replacement with secure protocols. um uh the issues of the economic uh and sociological things that keep the massively insecurable infrastructure that we've got with terrible operating systems for essentially everything all across the world. We need to address understand how to arrange the economics and sociology uh so as to overturn that. I guess could I add color to that? do is came from uh so so a week and a half ago I was at this cyber biocurity conference and I think this is now public so I can I can talk about it a bit but um you know I'm all for red teaming red teaming is definitely great when we you know make it specific and operationalize it but one of the things that they did at this conference was they pulled up a combine so this is one of these huge uh pieces of agricultural equipment that that uh plows the fields and lets you sew seeds and they had someone on the stage uh in 30 minutes basically uh remotely access the combine and uh rev the engine.

So there was there was basically a remote vulnerability. But the core thing about that interaction was uh like first is this unusual about this particular combine. He's like nope it uses uh like the CAN like a CAN bus which is a standard protocol for for cars. and he's like, you know, we've redteamed this. We knew that you can do this, but every single car and every single piece of agricultural equipment uh relies on this fundamentally uh vulnerable protocol.

And so you you can do all the red teaming you want, but unless there's actually a law or something that really causes the upgrade of, you know, from the CAN protocol, like you're not going to fix it. So um so there's a practice in um in computer security of um uh uh combined the bug bounty and responsible disclosure. There's some word independent but they're often combined. Um does everyone here know what that combination means? Okay.

Or maybe maybe just say a few words about it. I'll just say a few words. I'll I'll I'll leave aside the bug bound. probably obvious uh responsible disclosure is when a red team uh discovers a vulnerability and writes an exploit that enables the exploitation of the vulnerability. They disclose it to the proprietors of the software that is vulnerable with a deadline uh the um typically like six months.

uh in six months you you have to either fix it or demonstrate good faith effort enough good faith effort that I'm satisfied that you're really trying to fix it or at the end of the deadline we open source the attack enabling the entire world to now attack you and bring you down and that has been we've seen that that is an effective way to get companies to fix their software so max in your situation Are you saying like they knew about that vulnerability? They had red teamed it before. They found out about it, but it was just still prevalent in the systems and they were just relying on not many people knowing some some version of that. It's it's that even if you tell John Deere or whatever that their their software is built is vulnerable in this way, the problem is bigger than than they can solve. It's like, you know, it's like asking uh a particular website to rewrite the whole a whole networking protocol because the the canon is just like a univers.

It's like a standard across the whole industry. So you you can't really expect you know a company of 50 to 100 people to rewrite you know like HTTP. Okay. So now we've got uh the existing LLMs are quite useful at writing code not reliably. Uh they're getting better at um creating proofs of theorems.

Uh but still there's a long way to go. But when we think about this arms race between the red teams and the companies that are about to get smashed, we should factor in that AI amplifies both sides of that. And in this case, that's uniformly beneficial. You help it helps the red team find the attack and and disclose it uh hopefully responsibly. uh it demonstrates by virtue of the fact that AI helped them find the vulnerability that other AIs that are not beneficial inclined could easily find the vulnerability as well.

Also, it even works without the deadline and it helps the defenders uh get onto a technological treadmill uh where they are better able to rewrite and fix and adversariously examine their code so that it meets uh the you know the the needs of the code but now has fixed the vulnerability that the red team identified. All right. Well, maybe we'll leave that one here, but that's terrifying. Um, and I want to also move towards, you know, like if we think about decentralization and if we think about like open source, you know, like we might not just care about it for some of the better vet teaming that it enables or some of the better like, you know, open system design that it that that it like tends to um that it tends to come with, but also for distinct benefits of open source, right? like and I really want to hear from you.

You having been involved so much in the early like development of the open source community and of the open source software community um and it having been like such a striking also feature to uh influence much of our work at foresight like you know how do you think about distinct benefits that could be unlocked with more decentralized open AI development if we don't just think about red teaming AI but like if we now lift our gaze a bit and think about okay what what could that what could some of the benefits be? Yeah. Um if we think about all right what will the centralized efforts focus on? Well we know they want money let's make money another that maybe maybe equal to that or just under that is uh military uh let's get very good at defense and maybe also even offense. So these are the interests of the centralized efforts and they're going to be running as fast as they can to do those things because that's what they do.

Um so if it's more decentralized, what other more uh creative or interesting topics could we could we address? Well, I would love to see these powerful AIs we unleashed on the topic of human aging and longevity and disease. Right? The human body is the most complex system that we know of. So let's take the most uh the systems that are best able to grapple with complexity and throw that power at at medicine and see what we can do there.

That's what I would like to see. I think in general just like a like somewhat more of an incentivization and direction of like you know AGI towards like AI for like actually solving really interesting scientific problems that like affect all of humanity would be would be awesome. uh and maybe go with Above next uh just because you've said uh not as much on the panel yet. What do you think? Um I mean there is one thing that I wanted to ask Christine as well.

Uh open source institute they came up with a report on what what really open source models really mean. Um so uh I mean uh the one thing that I keep stressing on is that we should have a very clear understanding of what centralized uh AI systems would mean. what decentralized systems would mean. We don't want to land into a situation where there is a hybrid where you know a corporate comes in and says that hey we are decentralized because our training and everything is pretty much done on nodes spread across the internet but the model is only accessible to certain certain uh set of people or certain set of industries and so on. So really having that definition well definfined well set up I feel like would give people a lot more confidence.

uh for example open source software is very wellnown that anything whose code is available there is a license attached to it about how anyone can modify it how anyone can use it and so on it's it's pretty wellnown but at this point uh things related with large language models or AI technology in general what would really open source mean what would really decentralize mean I feel like it's still not that clear okay I'm I think I'm we exactly disagree um uh open-source software when software needs needs to to be something that a large community can have high confidence in its reliability and security. Open source is the best way to go there. Uh two examples of that the one practical high-erformance proven secure operating system um uh is open source and has been and open-source with the open ability to rerun the proof that the operating system is secure. So as a mechanized proof that is also something everyone can rerun. Uh I think the the mistake you might be making is this thing about everyone can modify it.

Everyone can fork but there's still uh for each each of those forks there's a competition for reputation of reliability and there's openness of the sociological process by which that fork is maintained. there's there's limited commitment authority and it hinges on the reputation of both who the committers are and what the open process is that people so also crypto protocols are uh open source and then you know the modern meaning of crypto i.e. uh blockchain, cryptocurrency, and especially smart contracts across that entire industry for which there would normally be, you would think, many benefits to making things proprietary because there's this huge externality of of efforts that people are pouring millions of dollars into. Nevertheless, the the ethic that's completely prevalent across that industry is develop everything in the open.

All right, we have to leave it at this. Um, Max, do you have a final word on this? I think the there are a number of exciting things. Maybe one thing that I'm quite excited about is uh taking seriously the literature on pedagogy and making it so that each person can have, you know, access to the Bloom's two sigma effect where you can get really personalized good learning. Uh, I think there's a lot of lowhanging fruit there and for me that's something that I'm excited about.

Awesome. Well, if you want to discuss more of this, and we only uh managed to scratch the surface here, we have a workshop that's starting in 10 minutes, I think, downstairs where we're going to dive into this a little bit more collectively. Um, and yeah, thank you so much. That was really fun, and I'm always happy to chat more, especially about the 4 piece that's been coming up over and over again in the past few few days. Thank you guys.

That was very fun. Thanks. Thank you, Alison. [Music] [Applause]