Funding the Commons | S Process Funding

[Music] [Applause] [Music] andrew is a researcher at uc berkeley's center for human compatible ai founder and president of the berkeley existential risk initiative and adviser to the survival and flourishing fund today andrew is going to introduce the s process an algorithm that enables multiple funders to simultaneously delegate the optimization of funding decisions to overlapping groups of trusted advisors this also allows the advisors to debate and coordinate in deciding the total amount of funding any particular project should receive welcome andrew hello are you hearing me fine corolla you're great great great thank you okay um so thanks for having me here thanks to everybody who's attending and i just want to call out um a preference i have for how to use the chat which is that i would like us to i'd like to field questions during the middle of my talk i don't want to have to wait till the end if you don't mind um so um uh carol could you could you hype up if if a question comes in the chat could you say andrew there's a question and then pose the question to me right there in the middle sure i'll interrupt you at will please thank you very much and to anyone who's participating also um i would like people here to ask questions of the form wait what is that like if i say something and you don't know what it is and it seems like it's important to the talk um just pipe up and ask what it is because you're a representative sample of the audience and i don't necessarily know what everybody here knows or doesn't know and i want us to have a conversation not just a delivery of content so um so please do ask questions for clarification as needed um and with that um yeah thanks jamie um with that i will share screen um i'm gonna want to use which screen this one all right um do we have my presentation about to enter presentation mode and now in presentation mode you're in presentation mode great thanks okay so uh thanks again everybody my name is andrew my friends call me critch and my critches call me andrew i'm a research scientist berkeley this is this is a thing that kind of came up on the side but became super intellectually interesting and and just generally motivating because of the good that it can do to uh bring funding to public goods um the work here is is i would say joint at this point with jan tollen who's the skype co-founder and and funder of a lot of the grants most of the grants you make also oliver habrika from litecoin infrastructure who i hope is here in the chat um and other people from barry and lightco including eric rockstad alex flint ray arnold and ben pace thanks a ton to you guys i don't think we could have gotten this off the ground as fast as we did or maybe at all without you so uh the history here is that this this s process thing that i will define the whole talk is going to be defining that um it's a it's a process i designed initially for the berkeley egg central risk initiative to make grants uh from donations made by yon and we eventually we looked at it and we thought this doesn't really need to be part of an institution to work and it's maybe even better if it's not so so we created this virtual fund um that coordinate that helps coordinate donors and people that are advising the donors um to to make grants we've made 16 million dollars in grants so far over five rounds over the past couple years uh the donors for the grants have been yantal and jed mchale and david marble i think we're gonna we're gonna almost certainly distribute another 10 million or so in the next couple of months and then next year probably more um and they've been great the organizations i mean certainly they've of course been sort of uh selected from a cause area that we the organizers cared about but the process um can't apply to any cause area um so we're trying to this with this process we're trying to push the pareto frontier of collective intelligence fairness and trust um there's going to be a few ideas that i want you to come away with from this talk that maybe you already have them i want to emphasize them more or maybe you've never had them before the first is the following when we aggregate intelligent decision making we don't want to aggregate actions we want to aggregate information and marginal value functions are a more intelligent type of information to aggregate than money so i'm going to be convincing you that we shouldn't be all just throwing donations into a collective pool we should be throwing marginal value functions into a collective pool and then using those value functions to decide on our donations the second thing i want to say is that funder final say options are great um if you are helping someone make grants and you let them have the final say over the grant that is less stressful for them so there's lots of people in the crypto space um right now that all of a sudden have a bunch of money and want their friends to help them decide where to do it and i just want to encourage people to use funder final say options in their processes the next thing i'm going to want to emphasize is discourse the importance of discourse we're all pretty numerate here uh i think and i and i think there's a tendency to try and use numbers to summarize away all of the trust issues or um coordination problems we have and i think numbers are huge help a mathematician that's my phd i'm now in each department at berkeley uh so i like i like computers i like numbers but this course um is not to be dispensed with we need conversations i was really pleased to see in carl's talk the the implementation of like publicly recorded invisible conversations and last is that repeated simulations during conversations can be a huge um hugely useful tool in helping people project what's going to be the outcome of this conversation and the s process is something that uses this kind of simulation discourse combo along with also this funder final say principle and the marginal value function aggregation so that's an overview of where i'm going with this talk um now i want to focus on a specific scenario for you guys to think about imagine you're looking at 30 organizations and each one of those organizations has different growth prospects in your evaluation or the evaluation of people you trust they have some capacities where if they get 100k they're going to run an event if they get a million dollars they're going to hire some people and maybe you even know who they're thinking of hiring um with 10 million dollars they're maybe going to expand to different teams or commit to five years of operations so so these different organizations have different kinds of plans depending on how much money they're going to get and not just plans but ability to execute those plans so what is the optimal amount of money to give to these organizations the the answer depends on marginal value functions so for each organization you can draw a curve that indicates how good is it to give an extra dollar to this organization and for most practical purposes there are diminishing marginal returns to giving to a particular org i don't mean to claim that there's dimensional mark diminishing marginal returns to giving to a particular cause area or service for example there i believe there have been increasing marginal returns to the influx of money and talent into the crypto space um however uh for any particular organization there is often a sort of bottleneck of once once the staff and network of that organization is overwhelmed it's not as useful to like pump them full of more money relative to some other organization that doesn't have any money yet um and so um for the purpose of this talk and for the s process we usually assume that there's a dimensioning marginal returns to giving right now uh heaps and heaps of money to any particular organization and if you if you draw those curves for yourself if you say okay how valuable is the first dollar we give to this org is it worth a dollar our how you know maybe this org right here we think a dollar to that org is like only worth 60 cents compared to a dollar given to our favorite org so you can put a little y-intercept here on your graph um and that helps decide the shape of your curve there's another point you can choose which is the x-intercept which is what's the point of which i think it's just not really worth giving more money to this organization right now it's just kind of pointless that's the uh that's the x-intercept and you can mess with the concavity of the curve you can make it convex in whatever direction you want to indicate something like wanting to give lots of money up into a threshold and i'm just going to share with you the app that we use for this so you can see what the experience is like from a user perspective so here is here are those curves i just showed you from the slide and here is user xo maybe this is um xavier uh some oscarsson um and if you see here as xavier turns down the shape of this nitro this curve for the organization called nitro or the nitro project you can see that they get less funding allocated in a world where exo is allocating a simulated budget of say just over 10 million dollars okay so if you think the marginal returns curve per nitro is shaped like that that makes you want to trade off like these other organizations election election works are now looking a little better and they should get some more money but they shouldn't get all the money just because their curve is higher and the optimal amount of money for them to get is calculated by the app here so if you have intuitions about you know um do i want to really definitely push nitro out until they get their full whatever this number is here almost two million dollars um or do i want to like kind of drop them off really quickly so that they get the first little bit of money but not necessarily a huge amount you can express that with these curves and then the algorithm tells you the optimal amount of funding that that each organization should have taking into account all the organizations you're considering giving to there's trade-offs here and one of the trade-offs you have is the future you might want to hold on to funding for future grant making rounds and you might want to express that in a curve so we have a special curve here called hold where we just indicate okay you know if we're giving grants and it feels like each dollar we're giving out is only worth 20 cents given to the best um the best giving opportunities we can find um then you know we should probably hold on to that money and wait for the future so you can express all those things with marginal value functions now back to the slides um now i want to talk about what happens when there's multiple funders because we're into decentralization here and each of those funders is capable of expressing a suite of marginal value functions indicating where they think money should go um and those functions are can be sensitive to each other so for example if everyone else has already over funded an org that you really like maybe you don't want to give more money to that org maybe you think they've got enough on the other hand if people everyone thinks that an org is really popular and thinks someone else is going to fund it you know they might all sort of stand by and hope someone else is going to fund like the anti-malaria foundation or something everyone recognizes is a really good way to save lives and then no one funds it because everyone thought someone else would do it so we don't want that and we also we also don't want weird games where everyone's sort of holding out on the bystander effect where it's like i don't want to spend my money on the anti-malaria foundation because i know everyone else is going to find it so i'm going to wait and then see if it gets enough funding and then if it's good i'm going to fund my pet projects that i think no one else recognizes um but you know if no one funds the anti-malaria foundation then i'll like jump in and save it at the last minute this is a stressful relationship for funders to have with each other it's kind of like a game of chicken where you know everyone's trying to hold out on funding the obvious things so that someone else will do it and in the s process we're trying to eliminate that ordering effect that hold out feeling the bystander effect the pylon effect all at once using marginal utility function aggregation so as i said you could you could choose an order for the funders but this introduces trust issues um and it matters who's the final player in this game uh and it also sucks for the uh organization because you you end up with the first funder who came in and thought about you maybe fully funded you and you don't get any funding from the other people who are satisfied now that you have enough money but would have funded you if uh if no one else had and so you you have this false sense of insecurity where you think you just got this one supporter but actually there's a bunch of people who would have stepped in to support you so we want to eliminate that problem as well um so at this point i actually can't see the chat because of the screen chair and i'm wondering if there's any questions that have come up that would be good and if not i'm only you you're free to go on they're asked answered in the chat great um so a better thing would be to uh aggregate everyone's marginal utility functions and run a simulation where uh each funder in each simulated funder represented by the funders marginal utility functions chooses to hand out say one percent of their budget or point one percent uh so in the first step of the simulation simulated funder one gives a point one percent of the budget to the best place that she thinks money should go right now based on her utility functions now the second simulated funder sees where that money went and wonders okay now where should i give is it was that enough to fully fund my favorite place if not i'm going to give a bit more and then the simulation moves to the next step in this way essentially nobody goes first uh everyone takes turns in simulation giving up 0.1 percent of their budget and there's not this anxiety about whether i should wait or go soon and if and you know no org gets fully neglected because if by the end of the simulation there's an organization that that everyone likes but no one's funded yet well their utility functions are going to pick up on that and fund them um okay now i think we can do even better than that and that's what we do with the s process we not only run that simulation once we run it many many times while people talk to each other and adjust their curves um so you'll have situations like founder one says whoa looks like you guys are giving a lot to organization one i found them to be deceptive in the past when i dug into their annual reports and found them to be exaggerated uh and then the other funders can make an update of their beliefs on that and sort of uh adjust their opinions i think this is really really important i don't think numbers can replace discourse yet uh and we should keep having lots and lots of conversations with each other while we do funding so that's that's part of how we run the s process each funder inputs their idiosyncratic values for how much money should go to each org and then we have a matrix that highlights disagreements like okay it looks like um you know funder xo here uh really really likes nitro but everyone else thinks nitro is no good um and so maybe there's a conversation we should have when we have our group meetings um people can say nitro tell us why you think uh or sorry exo tell us why nitro is so good and then they have a conversation maybe maybe xo changes her mind or maybe everyone else kind of comes to like nitro a bit better and there's that exchange of value and information um so now let's go to what happens when you have uh a funder who wants to delegate through other people um the simple solution here uh let's just imagine you have a billion dollars you're trying to find find people to help you give it away you'd love to just hire all your smartest friends or the people that you respect the most to decide where your money goes but maybe the reason you respect them is that they're busy doing something else and it's so so you can't just hire all the people you want to help that you want to get help from so one thing you could do just hand them all a million dollars and say here each of you have a million dollars give it to one of these 30 organizations please um i think then you risk the toe stepping problem where they can each over they can pile on too much to one org if they don't coordinate or they can bind bystander effect away from an org if they don't coordinate so you want to instill some kind of coordination and the simplest most efficient way to do that is for them each to draw marginal utility curves uh like we do in the s process uh now i think there's an even better thing to do which is what we do in the yes process can i interrupt we have a question a question from the chat of can you explain how these funders are setting their and submitting their marginal value functions is it quantitative or qualitative or are there defined structures um yeah so um in any given usually in our grant rounds we're focused on a particular cause area um so for us it's often been existential risk which is reducing the because there is basically reducing the risk of human extinction and so that's a quantity that people can argue about and say you know i think this organization is you know reducing risk by x percent per dollar or x percent per million dollars and even though it's very hard to ground out those claims because you don't get to wait for extinction events to happen and then collect data from them and then report back um it is at least something concrete that people can argue about and so that i think the the s process works best when the meetings are organized around a particular kind of goal for a society that people can debate concretely about the the mechanics of uh hopefully that answers the question um there's not an imposition within the app to measure your values in existential risk it's just if that happens at the level of discourse rather than in the app itself the app just measures the value of each dollar given relative to the value of what you think the most valuable dollar given is you call the most valuable dollar worth one dollar and then each other dollar if it feels worth less you measure it as as being worth less in your subjective judgment okay um so uh coming back the best solution when you have a bunch of recommenders people that are recommending grants to a funder is to first ask them uh to enter their marginal value functions in a nas process instance like our app then watch them debate and update their positions for a while so you get a sense you get the vibe from each person like this person has really thought things through versus like this person hasn't thought things through they're just like winging it or this person is really smart but they haven't put any time into it this time so i'm not sure how much i trust them and then once they lock in once the meetings are over they lock in their marginal value functions you as a funder can then enter your own marginal value functions for the recommenders so you can use the same process for funding orgs for deciding how much funding should go through each recommender so here's an example of that there's this is just like a fictitious numbers i made up where say yan is referring to these recommenders a marginal utility curve indicating how good is it to pump additional money through the judgment of xo versus va versus if um if you think someone's initial ideas are really good you give them a high first dollar value and if you think their ideas are going to keep being good when you keep pumping more money through that person you give them a large last dollar threshold and as you change this the threshold here where you want to give your last dollar we call that the last dollar threshold that changes how much money passes through each recommender which you can see here at the top left but it also then flows through to determine how much each organization gets and so as this person g z here i think this is g as this person g z here is getting less and less funding that's changing how much money flows through to the organization but as you can see it's not a really drastic change and that's because the other recommenders are looking at the same reality they're all looking at the same facts of the matter about what's good and so if you don't pump all your money through gz that doesn't have a huge impact on how much money these organizations get which to me is a sign that we're getting at truth we're getting at some kind of mutually agreeable understandable facts about where the money should go uh which i find exciting um of course we can have bubbles and filters and stuff like that that make everyone collectively wrong we've got to watch out for oh can i interrupt again with another question yes we have a question here from juan what are your thoughts on making claims made during discourse more verifiable or explicitly conditioning the impact of the claim on some factor he mentions that i've personally seen a lot of false or exaggerated claims in discourse only reputation systems both negative and positive which lead to really unfortunate consequences and funding systems so basically have an explicit factor built into any adjustments that happened during discourse after initial assignments were done yeah that's really interesting so uh a thing we do is definitely you know at at each meeting we raise open questions that should be looked into so that people then go and investigate and find out if those uh claims are uh check out um but it's interesting you could have an actual formula built into the system that penalizes automatically participants for having exaggerated um we don't do that automatically it is an interesting prospect i do think that there's um uh a natural amount of penalization that happens when the funder looks at everyone's uh work and recorded conversations and then decides how much they trust everybody um i would currently be reluctant to add an extra penalty on top of what i think this funder will socially naturally implement but in a decentralized setting that might be valuable and i'll come back to other ideas in that area further to the end of the talk surely that is not a fully satisfying answer to how to penalize exaggeration but it's what we have okay so um so that's what the funder interface looks like um i guess i already showed you that a little bit next is now what happens when you have multiple funders now you've got the same you know you've got a few people interested in a similar cause area um this kind of similar group of experts are really you know trusted to evaluate these questions so everyone wants to hire them up to their foundation but they're all too busy but it's fine you can just share an s process instance and watch their debates and decide how much money you want to flow through them and the s process just handles the issue of bystander effects and pylon effects and ordering effects for who gives money when so um i think the most interesting thing about this is that um again the funders don't have to play chicken with each other there's no i'll i'll hold off you go first um okay uh so here's what that looks like in the app i will switch now to what it looks like when we have multiple funders and this is how we've usually been doing it for the past few rounds you can see here um we have three funders uh jan jeb and david and these are hypothetical organizations we didn't want to violate anyone's privacy by putting up the real orgs but the numbers here are modulo some noise that i added for uh to make it difficult to recover exactly who anyone is these numbers are real numbers that were entered approximately speaking by real participants in the system and you can see here from this from this graphic that um because of the you know you can click on the the yawn note here that this is john's evaluation of how much he trusts the different recommenders this is this recommenders or expression of how much money and when it should flow to each organization and then all that gets aggregated together in a in a sigma node here which takes a second to load which you can click and you can see here okay we have let's look at um this uh or well let's find one that's getting here we have electroworks it looks like electric is getting money through three different recommenders xo and uh and a tiny little bit from bo here and then also a little bit from va and um and then xo is getting funding delegated from yan but also a little bit from jed and uh it looks like none from david so we have this flow through and as you can see there's a big difference between how much each funder is giving but the funders can still have a conversation with each other at the end and share their impressions of like who did good work evaluating where these grants should go um so that is what it looks like uh in app form and again we always have this matrix here that can be used to draw out disagreements uh and importantly i don't have we don't have we don't make our conversations our recorded conversations public um we do that for basically to make it less scary for the recommenders to sign up and participate so if they if they have positive or negative views of any organization they're not scared about people getting angry at them on the internet but i hope we move to a world where everyone can express their opinions freely and they don't have to worry about that sort of outcome when people uh really express their evaluations of organizations and i hope that uh the rollout of of uh you know public goods funding activities like bitcoin and the s process will make it so that more people can get involved in expressing their opinions about where money should go so that people are less triggered by hearing those expressions it's just normal to have positive and negative evaluations of orgs and nobody is freaked out if you think someone shouldn't get money okay this is coming up close to the end um of my slides but i have a few more things to say um i want to make a comparison to quadratic what i call quadratic matching because you're kind of matching donations with other uh with other donations uh in the quadratic funding space uh quadratic funding elicits honesty from participants that's what's exciting about it the formula for marginal value of making more donations tells you that you just want to give exactly the amount that you believe the organization should get but it leads to sub-optimal allocations because of these bystander effects these pylon effects the ordering effects um all that the people are not coordinated in terms of how much money should go to any particular organization in total the s process solves that problem by aggregating everyone's marginal utility functions into a process where everyone kind of acts simultaneously and it in practice seems to elicit a high level of honesty we don't have a theorem proving that it elicits honesty and in fact we know that there's small incentives to deviate from perfectly honest expressions however those incentives are very small and in practice they don't seem relevant um so we are pretty happy about that um and it would be an interesting research problem to actually like precisely define and measure the honesty incentives in the s process and if you want to do research on that you can email me it's also a feature and not a bug that the top level funders are not forced to communicate with each other in any particular way so that it's easy for them to decide to participate without being freaked out that someone's going to get mad at them for what they what they're expressing so um so that's a key that's a key part of the process there and uh it is something i would like us to do in the future or at least consider doing is adding quadratic quadratic matching agents to the s process itself so you can have a little agent in the simulation that watches the the recommenders and quadratically matches their donations so now you get the benefits of quadratic funding and the benefits of the optimal allocation process the optimal allocation outcome that the s process gives you um so these these these the principles of quadratic funding and the ass process can be combined and i'm pretty excited about that prospect um there's a bunch of research that i think people should know about uh in the area of transparency in game theory uh game theory a lot of economics is built on game theory and then presumes opacity and the part of agents participating in the system i think this is a fault modeling assumption that causes people to trust each other less than they should which leads to load trust systems which leads to low trust behavior and i think we should stop and we should start modeling economic actors as partially translucent i'd encourage people to read about translucent game theory or open source game theory as an important uh shift in how we think of agents and economics i want to say belief aggregation is important we should not merely aggregate the behavior of giving money we should aggregate the beliefs behind the decision to give money um and here's a couple of papers about aggregating beliefs or what happens when you don't which is that people with true beliefs end up winning a lot of bets and people with less accurate beliefs lose bets and kind of get screwed over which sucks for them and i think it's an unfair world um lastly uh i think mechanism design underestimates uh the cost of lying most formal mechanisms for listening honesty just assume that the agents in it are fully opaque it assumes that there's no computational cost of constructing a lie it assumes there's no reputational risk to being caught in a lie later and it assumes that the agents have no internal morally experienced distaste for lying which is a whole other thing that's that's a lot of people experience um so i think when we're designing these mechanisms we should be accounting for the fact that honesty uh is cheaper uh than lying in a lot of circumstances and i think that there's uh there's room in the economy to gain greater trust between people and institutions by recognizing that not everything is adversarial because it's not that easy to lie so i mean it is easy to lie in some ways and people lie and then they get caught and you hear about it but you hear about it and that's part of what makes it costly to lie and i and i want our mechanism design to be sensitive to that fact so you don't accidentally assume too little trust between the participants and thereby lose out on optimization potential so in summary number one we should not aggregate money we should aggregate marginal value functions number two funders should have a final say or the recommendations given to them so that they have an incentive to enter the process number three discourse is not a substitute for numbers please keep having the conversation part and lastly repeated simulations during discourse can turn one-shot games into iterated games which makes them more cooperative in a game theoretically and it also makes them more transparent so that's it uh please think about it and uh keep funding uh the public goods everybody thank you oh lastly i'm sorry i'm got the footnote we have a job opening we want to hire someone to manage and develop the s process app full time it's been a side project for all of us so far but we do want to hire someone or collaborate with someone who wants to do it at their home institution that's fine too and you can contact me or sff or oliver at these email addresses thanks thanks andrew we've got one more question for you before before we let you go if you have time and that is how much are the grants and the decisions described here purely perspective so based on how funds will be used against some plan versus iterated prospective pseudo-retrospective so keep funding them so they keep having similar or greater impact to what they've had in practice the recommenders in discussions always consider retrospective um we we use these grant the s process is designed for funding things that are that sort of uh have it's sort of like there's a bucket already there that maybe already had some funding and can get some more um it also works for prospective funding but it's really good for retrospective so um in practice we're always thinking about the and the recommenders are always thinking about wow these people have done a good job maybe they'll do a good job in the future or these people have done a good job they should like get some slack at their org to like take a breath as a reward okay thanks again to krich for taking us on a bobsled ride through the curves of the s function and i think my new rap battle name is going to be most valuable dollar i'll think on that over our third coffee break so i'll see you again in about 15 minutes or at believe it's 2140 coordinated universal time see you again when we reconvene [Music] [Applause]