Funding the Commons | Impact Evaluators

[Music] [Applause] welcome back to the funding the commons virtual summit for those of you just joining us this event is centered on new models of public goods funding you can add your speakers questions for the speakers in the youthglobal.tv chat and we can continue to discuss these issues and projects in the telegram channel also linked in the chat we'll be kicking off this session with the talk by evan miazono evan leads the research team at protocol labs and his additional interests include lock picking and citrus fruits today he'll discuss retroactive public goods funding for the broader protocol labs network welcome evan evan i believe you're still muted all right i think i'm all set now uh thank you for the introduction carlo and uh yes i'm going to be talking about uh what we've been calling impact evaluated internally and what what purpose we hope they serve what we'll be doing with them and uh hopefully tying together some very interesting threads that have come up over the course of today's conversations so overall there's a very real problem that protocol lab is trying to solve and the hope expectation etc is that the solution comes in really two parts and i'm planning on talking through those two parts and then explaining what you should expect to see coming out of protocol labs in the near future and how to engage with us trying to collaboratively build solutions to those problems uh in terms of the the problem i would say that it's one of proportional upside you'll notice some similarities between this talk and the talk that puja gave earlier we're both kind of approaching the same general problem uh from in two different ways two different subparts of the same problem and i would definitely recommend watching that video if you're catching this uh if you missed that one and overall i would say that this particular problem the one of proportional oxide is one of two very related ones and it will start with me coming back with some context of what protocol labs is looking at specifically we are working to empower and accelerate teams outside of protocol labs the organization we think that this is a fantastic opportunity because the dependency graph of crypto projects is very large but also very dense not all projects have projects have revenue streams but there are some clear dependencies between those that do and some that don't the ones that don't being some of these public goods that we've been talking about all day and we think that there's a really robust opportunity to share wealth where incentives are aligned um id referencing targeting back to bonds talk we think that paretotopia can robustly be achieved in this kind of a network and so we've been thinking through a lot of these types of mechanisms how to connect these types of mechanisms to help people start public good type efforts uh we have a few uh funding mechanisms that will let people de-risk initially starting on these efforts and also um how do we empower them to continue doing these this is specifically what pujo was getting into earlier today but there's a chance that with grants or bounties uh you might have this feeling that you're barely laying down enough runway to uh cover your current activities i've seen for public goods like academic research this happened where fantastic labs uh tend to have bad luck because they took on a project that was too risky and they were unable to fund their lab and had to shrink down based on either the whims of grant makers or the effectiveness capital effectiveness of tech transfer offices which can end up being fairly capricious and so i don't think that this problem is solved well at all in academia it's definitely not solved particularly well for open source and so uh i think that there's definitely something more to be done here in addition to grants and bounties which do serve an important role i would say that for funding a network you want these two properties to be satisfied you want teams to be able to have runway to take on things that have high risk high reward when i think about executing on project in space it's very easy to see that many people are not able to take the kind of risks to pursue things that they think are valuable because they don't have the privilege or the resources and so being able to provide that some support network like thing to let people span that gap is very important so they can take on things that have higher expected outcome however it's also very important that teams be incentivized to do those very impactful things because even if you're unable to it may not be the rational choice to go and do those not that it necessarily needs to be the rational choice but it never really hurts to have the prospect of high payoff in these situations and so i would claim that puja's talk should have convinced you the answer to this first question is yes we can use things like milestone bounties um and grants to give teams runway the second question the answer i think is a little less clear if you're here you probably agree that this is possible um italic highlighted some great ways in which the crypto community is really well suited to public goods funding i totally agree you have things like computational control of value which is harder for groups outside the cryptocurrency space to do deal with effectively you have strong belief in the transformative power of technology and you have what has historically been a fairly exponential growth of value capture and wealth and hearkening back to juan's talk i would highlight that pradotopia looks like a place in which there are so many low-hanging fruits there are so many valuable things um that there aren't enough teams to take on all of them and you see huge amounts of collaboration in the space as opposed to competition and i think if you look around in uh web3 that's really what you see and so the question becomes how do we go about answering this second question with a strong affirmative and i unsurprisingly the answer that i would give is um public goods funding specifically retroactive funding my claim is that if you uh if you want to empower people to take risks you need to fund proactively if you want to reward impact you need to do it retroactively if you incentivize impact proactively than what you're actually doing i think in practice is incentivizing the appearance of effort not the appearance of impact which is an important distinction and you can imagine that a combination of these funding mechanisms could empower teams to not just take on risky efforts but also be rewarded proportionally for them um this plot on the right shows what we would predict what a hypothetical team without a revenue model might be able to gain from some initial grant some bounties and reward from what we're calling the impact value and so the first component from the solution of the solution for these impact evaluators that i would say is a variety of retroactive funding mechanisms these are things that will sound very familiar because most of what have been covered so far today could be characterized as retroactive funding mechanisms i'm not going to go into depth i will simply say that uh if you haven't seen them you should definitely go and watch owen's talk from earlier today on bitcoin grants and quadratic voting you could use quadratic voting as an impact evaluator you could use badge holder voting and in the example of the retroactive public goods funding from the optimism team the result oracle is an impact evaluator these impacted values are just things that are after the fact evaluating and rewarding impact to teams through various mechanisms uh there are definitely uh well there are definitely more representative slides of retroactive public goods funding but i definitely had to go for one of carl's slides at the hamsters naturally um in terms of s process i unfortunately had to miss much of andrew's talk but you could additionally use things like eliciting marginal utility functions for uh determining how to allocate rewards retroactively uh it does seem to work far more effectively prospectively but in terms of developing a wide spectrum of these types of mechanisms because it's definitely worth considering um additionally we had an interesting experiment protocol labs did starting in the fall of 2019 there was a project called source credit that we helped bootstrap where data was pulled from github discourse discord into some global graph the different contributions were collapsed using uh some community-driven subjective agreement on how we should or how the community should quantify different types of contributions be able to collapse all of this down into a single contribution graph run a modified page rank algorithm on this and then assign some reputation score called cred to individual contributors of the project and then grain was this fungible token that was generated and given to contributors in proportion to their credit score with the idea being that you could computationally allocate value in accordance to the value that people provided to the project and the experiment that i mentioned about source credit uh is that for about two years protocol labs funded source cred through the use of source grid those of you who are familiar with the retroactive public goods funding will recognize this as very similar to a project token and i'm very interested in what it would be like to build better and more robust project token examples um source code very aggressively dog boots tool but they got to the point where they are effectively independent and for quite a period of time protocol labs supported only through the purchasing of the project token in terms of other impact evaluators that you might expect to see moving forward protocol labs is experimenting and prototyping a committee quantification impact evaluator where outcomes are listed uh rather teams participating in the network opt in to participate in this process and summarize their outcomes and then at the on a quarterly cadence some evaluation team that consists of people within the network will go through and evaluate the amount of impact uh not too dissimilar from the badge holders and we'll determine how much how many resources should be allocated uh we also expect that there will be other experimental impact value evaluators that we generate um another one could be something that looks more like a block reward analog where instead of minting tokens we create some uh general pot of uh assets define some deadline some maximum reward some kpi and uh it's important that we have some sort of upward concavity so that as teams compete to maximize the kpi or as teams try to maximize the kpi they do so collaboratively rather than competitively so because as uh teams achieve outcomes the reward is strongly non-zero sum and so we can generate this like build in this kind of non-zero sum effect into the reward itself and then allocate using something like uh either direct contribution of kpi if that's something you could do you could also apply something like shapley values in order to generate a more subjective evaluation and so that's a general set of a bunch of different types of impact evaluators you could have the second part of the solution to allow projects to be incentivized to be incentivized to pursue very risky things is to acknowledge that not any one of these impact evaluators is likely to be perfect for all possible types of work all possible types of efforts all teams and so the goal will be to take these retroactive funding mechanisms and develop them into a network that leverages the fact that protocol labs is building around its projects a network and so uh the first thing worth clarifying is that when one of these impact evaluators quantifies impact what they're doing is they are choosing some metric for themselves that is likely a subjective metric um in the case of protocol labs this is probably likely to be the pl mission generally or it could be something related to a sub aspect of it around increasing the ownership that users have over their own data for instance and each of these impact evaluators is going to have to find some way through some people to subjectively quantify the impact it was made against uh some goal to reduce this to numbers uh in order to motivate that this is reasonable and plausible i would highlight that this kind of subjective evaluation happens all the time in a very centralized and permissioned way in normal companies through things like performance incentives or payments and equity where the evaluation is actually done later well partially done directly immediately through the allocation of equity and then later through the evaluation of the market on that equity the other component that's very interesting about mixing these together is that we should expect there will be multiple impact evaluators in any given network and then we should have multiple funders each of their own mission or goals participating into and funding some impact evaluator uh each impact evaluator would have its own mechanism and uh operation and each one would then reward multiple teams and so you should have much better coverage of all the different types of work that you would like to incentivize through these and we're actively working through and trying to develop a mechanism to mix these together well it's worth noting that the goal of this system is to incentivize teams to pursue the most impactful work they can and as a result retroactive funding it needs to be something that people can expect and count on and so while trusting that for instance protocol apps would reward a project for uh retract or would retroactively reward a project for building something that was important i think it's far more valuable if we can together build a culture of retroactive public goods funding so that we can give teams the confidence that if they do something that is very impactful they don't have to they priority know that the team that they're having an impact on has an impact evaluator or one of the teams they're having an impact on and so i think that having events like this bringing people together and developing new solutions and making sure that these solutions are widespread and effective will do wonders for increasing the amount of public goods that are created not even in the very long term and so in terms of how we are going about scaling our solution and uh ways in which you can observe and participate in our efforts i'd highlight first that we are uh setting some very ambitious goals around what we want to accomplish within the next year in terms of impact evaluators we definitely want to have scale we want to have allegation of significant amounts of funding and we want to make sure that people involved in uh participating on these processes whether they are funders or recipients we want them to uh effectively we want them to find these effective uh quickly going through the last few notes i will say that we have some open roles we are trying to actively build and support more of these research roles will look more like identifying and defining mechanisms startup operator we'll look more about operationalizing and running these mechanisms we're definitely interested in cross-pollinating so if you're interested in running experiments um either co-funding impact evaluators are asking us to go fund impact developers please let us know in terms of like trying to improve these there are clearly gaps i think there are some very good questions about how these can improve and uh what we should be doing to make any or all of these impact evaluators together more effective and i would suggest that you reach out uh if you have any comments or questions on any of these here's my contact information and i don't know if i have time for questions we ran out of time for questions unfortunately but we can also ask them retroactively after the event in the telegram channel and i'd like to thank evan for walking us through the impact evaluator which sounds like it should hurt but is actually a super interesting journey through innovative funding mechanisms so thanks again to evan [Music] [Applause] [Music]