Collective Intelligence vs. Superintelligence? – San Francisco 2025

Hey y'all. Welcome. Welcome. So Chris and I were both active in intentional community. This is a big influence on how both of us see the role of technology uh in uh in all of this in building meaning together uh getting things done together uh and uh institution design.

Yeah. Um and I think there's always an interesting uh contrast in approach between those who would look to the hard rules of instit of institutional design uh the kind of mechanistic design and those who would look to uh something softer, something more values or virtues based. For example, we often see the design challenge taken up by technologists uh who are interested in supporting democracy, supporting healthy governance. They often see it as how do I design a good system that people will fit into well uh robustly robust to the worst person and so on. Uh so rarely do we see the design challenges.

uh how do how do we design an institution that is effectively just watching what everyone's doing and support that? How do we design an institution that follows the group uh uh rather than uh defining the group or structuring it? Right? Which also goes to what what is the goal or what what what is what is a community or an organization directed towards? uh the more the the lower dimensional the uh the measure of of a desired outcome, the more likely you could have some kind of, you know, calculus of what is good, the more you could aim for some kind of optimization.

uh the more you have a very highdimensional space of of what is potentially good or what outcomes are um the less likely you are to be able to have that kind of uh mechanistic guidance towards uh what's desired and the more it becomes the the messy uh uh the messy moral problem of making trade-offs between competing goods as much as uh especially advances in AI have made it possible to treat institution design as fundamentally open-ended. Uh there's still a bias to treat voting as the for example as the fundamental unit of democracy and voting as the problem to solve. Uh you see it up and down Dows. Uh one way that Chris and I align is to see the intentional gathering as the fundamental unit of democracy. And when you change the unit you're designing for, uh, you change, well, you change everything.

Um, okay, voting's a nice problem for computer scientists to approach. Uh, it has all these great security concerns that you can design for and around. And it's very immunable to the Mchavelian conception of institution design, which is to be robust once again to the worst person, be robust to the to the most awful person. So, um I have a a beautiful cake and look uh this cake is is full of spikes that only hurt bad people. Isn't that a great feature on a cake?

This it's how you think when your uh democracy design, your community design problem is framed fundamentally as an adversarial one in which security uh for example is is one of the core things uh you're focused on. When we get into intentional gathering as the fundamental unit of democracy, you adversarial nature isn't one of the things you're leading with. You don't have to worry about spikes on a cake and and you are opening up to well, what does everybody here need to build meaning more effectively together? Um well if you look to the literature on uh like deliberative democracy even even that part of it uh on kind of agonistic democracy where actually conflict is seen as at the heart of democracy and yet what are the properties you're looking for to support that that kind of democratic uh policym or decision-m It's uh voting won't get discussed in that literature at all. It's all about um you know certain certain virtues of public discourse around uh the decisions that need to be made, the policies that need to be made.

Um a discourse that needs to recognize well somehow has to overcome loser like you've got to get losers consent. Uh, and you need a mutual assumption of good faith. Even when you have adversar an adversarial um setting or tone, which deliberation sets up, great. You still have an agreed upon ground rules that you're uh no longer assuming with a lot of adversarial security first design, right? And you know, there's this balance between a decision needs to be seen as on the one hand it is binding.

you can't ignore it because you don't like it or or fail to respect that that is indeed the decision. On the other hand, uh it needs to be seen as provisional. Um it it won't be once and for all. Um and uh there there needs to be an orientation where there is uh conflict, where there are um different interests are in conflict. There needs to be an orientation towards uh like well an open-mindedness because there needs there needs to be a hearing of what what the other position is.

It can't be dismissed as uh automatically uh not a moral position um not not worthy of of of being in the discourse. Um but there needs to be an orientation towards uh uh recognizing the overlap and alignment because in any contrasting conflicting positions there'll be there'll be quite a lot that is shared even where even where there is there is also a lot that is different. Um and that's a that's a stance you can take. If I choose to identify all the ways in which we disagree fine that's what we'll see. Well, but if if we if we choose also to emphasize where there is alignment, well, look, it turns out there'll be there'll be a bunch there.

And that's this is kind of the raw material to work with um in in any kind of deliberative democratic decision-m. I like the focus on deliberation a lot because you're already seeing the AI space get uh influenced, informed, and excited about supporting deliberation, supporting facilitation. The idea being uh we can have uh AI facilitators, AI assistants who help people get into better arguments or learn things better or or or suggest more humanistic prompts for for discussion. Uh but what if the point of uh deliberation, what if the point of facilitation uh is to do it? There's a whole set of skills.

I don't hear anybody disputing that, hey, AI will be great for giving everybody some kind of personal noteaker. There's not, no one's really uproar on the labor side or ethics side about that. On the other hand, I haven't heard anybody say um you know, public schools uh in America are funded a little bit by performance and um attendance. Um funding public schools is good. Um humans are inefficient learners, especially um young children.

So what if we replace them all with agents uh to increase GPA? You know when you and increase funding for public schools uh we don't hear that because we understand that some things are about the point some things are about the output some things are about the process and I haven't heard the conversation in society about what jobs what work is about the process. Um, you're starting to hear it now in arts, right, that a lot of the point of art isn't the output. And and it's because of uh um uh because of advances in image generation that that made that conversation possible. But what about Yeah.

What about facilitation? What about deliberation? What about running a meeting? Uh what about um yeah, running a rewarding meeting, one that people enjoy? um uh running an intentional gathering that's energizing.

Um what if that what if the point of doing it is the process of doing it? What if uh everyday practice with democracy getting our hands dirty in it is the fundamental um hygiene for being a responsible steward of large scale democracy? Then what's the role of AI there? Right? And well, let me add something that is adjacent but relevant into the mix here which could be um uh you know you talk about democracy, you can also talk about justice and here as a speaking as a as a lawyer um there's always this this kind of tension between on the one hand substantive justice and on the on the other hand procedural justice.

Um, and there's something in how humans, societies have evolved systems of justice that there's this recognition that you have to trade off some substantive justice in order to achieve something that has a uh that can be more robust in a in a multi-interest environment. Um, and that's that's a kind of fairness or procedural justice. Um, but it's a real trade-off. Um uh and so I I feel well and I could bring in something else to keep playing on this this trend this this kind of tension between the the process and aiming for the substantive outcome. I mean if we look at you know another great fruit of human civilization like our systems of knowledge production um whether that's you know institutions of science or or other again there's the what's key there uh is is the is the process it's it's those virtues or values that are seen as um uh as more likely to orient towards truth.

um that that need to shape those that the institutional design or and the values that that are held held in those. Um so uh so this this becomes a I think a a good theme when looking at uh AI and and how we will begin to incorporate and interface with it. um to what extent can it support process our process as as humans and individuals and societies um rather than simply kind of jumping us to the substantive outcome or answer that we're looking for. Yeah. I mean, yeah, just like we haven't heard, hey, let's replace all grade schoolers with agents.

I I I haven't heard um you hear this all the time in social work work uh that my job is to make myself obsolete. My job is to put myself out of a job, right? My job is to build capacity in my in my community or in the people I serve. We're not hearing that from AI. Um and it's got to be in large part because that's uh that's a bad uh business plan.

It's more of a civic um technology perspective. Uh but in so far as so much of uh what community, institutions and collectives need is to be composed of experts um in shared leadership. Uh I'm more and more open to the idea that the role of AI in those contexts, the role of AI in anywhere where um the process is what you're cultivating rather than the output. Uh that the role of AI is to make itself obsolete by being such a good steward of its members. All right.

[Laughter] A guard rail for for AI or for AI alignment is uh uh render your services obsolete uh in in the most elegant way possible. And look at those buildings so uh cleanly off of um Wendy and I as subtraction. Right. Right. Yeah.

Uh how how to how to do less? Uh well, it it's not about uh it's not about maximizing um uh maximizing one's own impact. It's about it's about understanding what what is trying to be served. Um and again if if that's seen as some some kind of substantive outcome uh there's some endpoint that's being driven towards okay that'll take you one way and then you're you should be very concerned about whether that outcome is really something you desire. Um if instead it's more of a a processbased um and it is going to be easier to incorporate more of a values based more of a virtuebased um kind of moral framework where one is processoriented rather than outcome oriented.

Um I guess to make it so uh you run an eco uh you're you're a co-founder of an eco village in Portugal. I learned so much about chickens from Chris. Um, uh, the community within the community. Um, and all of the Yeah, being it's nice being able to have be an observer of a a smaller chicken community's drama as you're navigating your own. Uh, I imagine it gives one lots of design.

So if you could just snap your fingers and have a lot of technologies um available to to your community as it navigates building meaning among everybody and making better decisions as a group than it than it could as a set of individuals. Um how would you put a lot of these um our shared ideas in into practice? Right. The first thing I'd want would be uh uh support for process because this is this is something that it it isn't intuitive. Like let's say there's some kind of conflict or dispute um like it it unless unless you have a bunch of lawyers or otherwise involved in in dispute resolution somehow, it's not going to be instinctive what those kind of principles of natural justice are like.

what what uh what the right kind of steps or process would be to um not just to get to a a place of potential resolution that is from a from a kind of legal or moral perspective um uh fair but also one that is psychologically acceptable and satisfying to those involved. Um so these are these are these are expert domains. They're not something which is likely to emerge uh from us gathering as a community with with starting from scratch and trying to figure it out for ourselves. And so that kind of process support being being guided through uh a fair process for resolving a conflict for instance that would be immensely valuable. It's it's hard to find and often in community those with those kind of mediatory skills or whatever are are super precious precisely because they're it it's not obvious and yet it makes a big difference.

Uh I guess this is just the beginning of how design changes when we're focused when we shift our attention from a good output to the work involved in cultivating better humans. Um thanks so much. Thank you, Chris. Thank you, [Music]