AI4PG: Bridging AI Research and Public Goods Funding - Buenos Aires 2025

[music] Hello, thank you for the introduction. The next billion is Shafi, the co-founder of uh Gainforce, but I'm also happy to take that recognition. Um, today is not about gainforce. Today is about a really serious topic actually, but I'll try to make it fun and colorful. It's about bridging AI research um to public goods funding or I thought about an alternative title how to stop running in PG funding circles and I think this is something that our space deeply needs and I hope this talk will something that comes um resonates with you.

So the master plan for today, don't tell anyone, is we're going to fix PGF research today. And the day we do it is first there are three lessons that we can learn from AI research that I want us as a community to understand. Number lesson number one is we have to eval. And lesson number three is we have to build bridges. And I'm not a person who talks likes to talk a lot.

So today we'll release three things that will help support these free lessons. But first let's talk about Gitcoin. Gitcoin has been a pioneer in this space and in many of us when we think about PGF funding we think about Gitcoin and the mechanism design that it pioneered and Gitcoin just recently released um Kevin maybe is here in the audience this really beautiful graph of a new era of PGF which I believe we are in as well. We in an era where okay web free public goods funding is not just quadratic funding anymore but this really beautiful exploration of many many mechanisms including quatic funding retrof funding AIdriven funding new design space new design space right but when I look at this for me this looks like a random walk um so you can simulate a random function and it will look uh it will go randomly somewhere and this is exactly the same graph that we are likely facing right Now and this is really really bad because we need to know where we are going when we're designing these this space right we need to understand are these mechanisms even making sense we can't continue and saying try out this cool new PGF mechanism bro trust me bro it's really really cool and I think today my talk is trying to aim to provide a road map on how to fix this I think one of some of you passed by today's um entrance and you saw the history the beautiful history of mechanisms designed for public goods funding. That's something that actually is a really great example of the commons and the power of the comments.

Um I designed this figure, this timeline like last month, just two days on my couch because I thought it's time for us as a community to get together to understand also where we coming from, not just where we going to. and looking at a beautiful history of our field. And believe it or not, our field has 11 Nobel Prize winners introducing concepts like public goods itself, game theory, um, assurance contracts like contract theory, right? And I hope that what we're working on is actually providing somewhere here in the audience the next web 3 based Nobel Prize winner for economics. So that's where we have to go as a community [snorts] and for that to go we need to understand what has happened before and where we are standing as a community together right now in the broader context of mechanism design and public goods funding.

And so that um that graphic doesn't look quite like this. This is a little bit more detailed but if you go to the entrance and just explore and look at the the people who came before us and the people who are coming now please understand that we are actually standing on the shoulder of giants. We're not inventing something new. This is not a new mechanism design space. This is something where we need to learn from the past.

This is not how the slide should look like. This this was a bridge emoji. Let's see how it comes. So I want to thank Vitalik um for opening up a new era in PGF. Vitalik's article earlier this year on human as a steering wheel and AI as the engine is I think one of the first not if not even potentially the first academic and non-academic attempt of trying to create a largecale public goods mechanism uh funding that is AIdriven and these ideas and the concept that he proposed is beautiful.

So you should check this out. This is I think a fundamental article in our community that is opening up this new idea of suddenly taking AI and AI research into the design space of public goods funding. And I think this new dawn of public goods funding has a really great opportunity for us to learn and not repeat mistakes. Um which is this is a bridge emoji again the question marks connecting it to AI research. There is tons of AI research that has been out there from Stanford, Google DeepMind, Europe that we can learn from as a community when we are now designing these new AIdriven PGF mechanisms that we should not ignore because they help us explain what is good and what is bad, where we going from and where we have to head to.

And sometimes when I'm an academic and try to also support initially deep funding and so I feel a little bit like this like you're coming with a lot of like this AI research but then it's it's somehow not getting through and today I'm trying a different approach. I'm trying to tell you why this could be really useful for you to consider when designing these mechanisms and why this space deeply needed needs it. First of all, currently when we are running an experiment like Gitcoin round or optimism or retropf, this is how we're running the experiment. We run experiment with fancy mechanism and this is not how you run an experiment. In fact, this is how you have to run an experiment.

We run an experiment um with a mechanism to test a certain kind of hypothesis. And we are lacking step number one, the fact that we need to define assumptions and the hypothesis. And the most important step on the back that actually creates this engine of improvement, the evil function. We don't have an evil function in our space. And I think that's something we as a community need to fix.

Number one. So lesson number from from the AI community. We didn't start so I'm an AI researcher. I did my PhD in AI systems. As Sarah already said, we didn't start with transformer models.

We didn't start with GBT. We in fact very boringly 2009 started with a million images of cats and dogs and other stuff and the [snorts] reason we collected this it's called the imageet uh competition is to evaluate how good image classifiers are doing right [snorts] so um Stanford collected uh 2009 this really large data set and then we ran a competition with classifiers predicting different kind of ways how good like top five accur top error accuracy they're predicting cats and dogs giving the image right And you can see this graph here. It looks a little bit boring, but it's actually the start of the deep learning revolution because this blue one, Alexet, um, provided a new mechanism, a new algorithm, um, neuronet networks back then that beat outbeat all the others and you can actually see and compare the difference in accuracy. And that's why people were excited and they they were so excited. It created a whole research field.

Look at the uh exponential decre increase in papers published in this field in machine learning. One of them in 2017 turned out then to be the transformer paper. Right? We need something like this in PGF. We need an EVO benchmark where our whole field can coordinate and aggregate and compare each other and measure the distances between mechanisms to mechanisms.

And it's really hard to find something like this because Julian correctly said um funding is governance is ideology. But we can also look in AI research and look at how they are comparing actually LM models nowadays. And LM models are actually compared through pair wise comparisons and ELO scores. So there's a platform called large language model arena here. And basically what they do is they show you two outputs of a language model left and right.

They don't tell you if it's like gemini or clo but you have to tell which output was better or not. And this is something we are currently testing out with protocol labs and David Gask um and I are designing PGF arena where you see two different kind of mechanisms randomly you don't know what they are and you can see the distribution of funding left and right in the different kind of topics and you I'm talking about you right in the audience you tell me which one you like more and [snorts] in that way we aggregate the preferences of our community and we figure out the data of what might be the right algorithm for you It might not be the objective ultimate truth for all of Ethereum, but it's the right solution for you. And if we aggregate this cleverly, we can evaluate it. The second lesson I want to provide is called the bitter lesson in machine learning. And it's the fact that data and compute are king.

It comes from Richard Sutton who's the inventor of reinforcement learning who spent his whole life improving machine learning algorithms but then was outbeat by just collecting more data and putting in more compute right and that same lesson we are experiencing now it's the deep funding bitter lesson so in fact after 10 months of deep funding we have only collected 627 data points of low quality how can we talk about distilling high quality human judgment if 64% of all the data collected manually is just one single person labeling something and there's not even a comparison or check if this person is correct or not. Right? And the fact that some judges have 10 times more weight than the others is is just that doesn't scale. So this is the bitter lesson that deep funding is right now experiencing that the competition they are running because of the lack of data is just statistically invalid. Is there a better solution and we think there is.

So the second thing we are proposing to the Ethereum community today is we need to aggregate way more data. 627 point data points will not revolutionize public goods funding. In fact we know where the data is. It's in the forums. It's in the discord chats.

It's in the telegram groups. We need to download that. And we can also learn from AI research how to download this. In fact, Audrey Tang, the first minister of digital in Taiwan, has introduced broadlisting algorithms that can extract this data um from just unstructured data into context switch databases. And today we're releasing broadlisting.

org, which is a large scale database of 20,000 claims just from Ethereum magician for you to download in CSV to analyze and that is just uh 30 times more than deep funding in 10 months.

[applause]

With this data set, you can explore it. You can track trends of what people are talking about in different topics automatically discovered by the LLM. In fact, we can we created in a pilot project with Octant a list of just we don't even know what projects are in EUM. We just let the AI analyze the whole whole form. We extracted 1,600 projects and the LM was able to automatically cluster topics and subtopics of each project.

So we invite you to uh explore this data set to give us feedback. But this is the first release of this public good for the community. Number lesson number two and lesson number three and that's I will end with this is that we can't do this alone. We need strong ties to the academic community. We need to be there where the academics are and we also need to invite them to our field.

And you're thinking maybe why would a professor come to our field like nerdy web free people. No, we are sitting in every mechanism academic stream. We are running their dream experimental engine. In the past, when you go through the entrance and you look at all these game theory Nobel Prize winners, they would have dreamt to be part of our community to run their ideas, their theories on Ethereum on our machine to test out if it works or not. We have the capital, we have the creativity, and we have the platform and technology to run these ideas that they have.

They are urging for an invite to come to our space, but we're not making it easy for them. So, in order to make it a bit easier, Maria and Sage and I together with funding the comments and many other public goods funders opted, ours um running a grants round. It starts very small, $10,000 um per grant for AI for public goods. But we also do a neat trick. We pay $800 per reviewer.

So lots of academics came. In fact, we had a huge um actually not a huge uh review application. Once I treated it on X, you see we we after two weeks we had only 30 applications from the reviewers. But that's where lesson number two comes in. You need to go where the academics are.

They are not on X, they're on LinkedIn. Once you post it on LinkedIn, there's an explosion of academics who can apply. So this is a lesson for the field that please don't just treat and think this is how you reach like professors like go to blue sky at least or go to LinkedIn. Yeah. So to cut it short um and to give you like the free lessons again, we need to have better eval functions.

We need to have larger data sets. We need to bridge our community with the academic community to really make um this space the space that we all want to be and then hopefully in the near future one of us will win the next Nobel [music] Prize in economics. Thank you so much. [applause]