Reputation Based Impact Attestations for Funding Allocation - Bangkok 2024

[Music] hi well hi everyone I'm actually really happy to follow the panel because I'm going to be talking about one of the challenges that they've been that they've mentioned which is how we're actually allocating this funding my name is lamu I'm the founder of metrics Garden labs and we've been exploring how to measure and evaluate impact in the web 3 space for a little over almost 2 years a little bit over a year and a half and definitely resonate with what Eugene and with Ana mentioned in terms of there's a lot of confusion around what actually impact means and the difference between impact and output and one of Let's go and one of the ways in which we've been approaching how to generate more data so that we're actually looking at what the impact of these grants is is by looking at different data sources than what we've been leveraging right now and so we've been running an experiment in leveraging impact attestations and at attestations so that we can improve data generation we have more information to feed into how we're allocating the funding and then improve that to set a little bit the context one of the things that I often times hear people say is there's not enough money I want to get a grant but it's so difficult to get a grant there's no money for the thing that I'm doing and I want to give you some numbers so that we can actually put things into perspective so between 2023 and 2024 uh as was presented in the state of web 3 Grand 2024 report there was over $93 million that were allocated only by seven ecosystems this is putting aside the 61 million op that optimism delivered in 2023 and 2024 and the 50 plus million ARB that were granted in stip and ltip so this is a lot of money that we're put out there and it's been very difficult to identify which data we're leveraging to actually understand if this funding is actually being put into good use and this is one of the main challenges that we see in the ecosystem we have the grant operators and all they want all they think of is how do I grow my ecosystem and they are meant with this inefficient allocation of funding it depends a lot on what are the goals that they are actually trying to achieve how they incorporate those goals when they are designing their grants programs and then ultimately as well what is the data that they have to evaluate if they are giving this Grand funding to the right projects that are going to generate the right outputs but also the right uh impact to their ecosystems and then what are the strategies that they are following and what we have here and what we've identified on Metric Garden is that this is a data problem we don't have the data that we need to actually design the grants programs that are going to be the most efficient in achieving the goals that we want and this data problem is broken down in several pieces one of them is there is some data and it's very limited we tend to skew and focus only on quantitative data which gives us a very partial vision of what we're actually trying to achieve what is the outcome that we want to see on our ecosystems and then it tends to be skewed to one source which is usually GitHub there are some ecosystems that have started to leverage onchain information as well but given that we are in this highly technical space most of it comes down to GitHub data and this is one of the things that I find really interesting because when I was doing user research last year precisely looking at what are the metrics that I could use to suggest an evaluation framework for an ecosystem I asked some of my developer friends like what does a GitHub star mean for you each of them had a different answer so when we're trying to create this metrics for GitHub Stars equals this thing within my ecosystem we really need to think again of how are we designing these metrics the second one is we have a limited understanding of the product Market fit of the different things that are happening in our ecosystem this also leads for us to understand in a very limited way what are the actual needs that our Builders have in the ecosystem and therefore what are the things that we should be focused on in funding and then the third one is because of all of this lack of data and limit to only quantitative data we have a lack of visibility on the value that has actually been generated to the ecosystem stakeholders whether this is our Builders or this is our um or users in our ecosystem oops wait sorry and with limited data we're able to make limited smart decisions and so what is it that we are looking into from impa garden and metrix garden we want to increase the type of data and the amount of data that we have that we can leverage to make decisions and so what we're looking at is doing that by adding standardized qualitative data into the analysis that we are generating for funding allocation the reason why we believe that qualitative data is so important is because now we're able to put these numbers into a context we're able to generate these metrics that are giving us more insights not only into how many additional users did we get into our ecosystem but how did we get them what type of value they are generating to the ecosystem in which context did this happen and then also second one is we are able to tailor the type of information that we're generating we are no longer just looking at data that was pre-generated that we have no idea or no control over how this was created but we can in tailored way generate information that is useful for our own grants programs and our goals and then the last one is we're looking at interoperable data which means that we're building this uh database in such a neutral way that it can be used among different decision makers and it can be shared as well between different ecosystems and this is why we're using impact attestations and why we're leveraging attestations so the first one is we want to make sure sure that this qualitative data that we are generating is captured on chain so that it can be leveraged to whether it is automatedly be used by a smart contract like call um and then Define if an allocation should be made we want to make sure that this data can be leveraged in an onchain state which is uh how several of these Grand programs are operating and so the way in which the impact attestations are built is first uh which is what impa Garden does which is the project that I've been working on is first we leverage EAS on the back end EAS is a multi-purpose primitive which enables us a lot of different which allows us to do and store a lot of different data so first we're able to store it onchain or offchain we've been storing all this information onchain it gives us the ability to have multiple configurations depending on the type of privacy that we want around that data we can include images as well one of the things that we were exploring actually during The Residency is collaborating with gain Forest which is a regenerative project in storing information that was reported by their people on site of trees one of the like they focus on um mapping different areas so this is one of the things that can be added into the impact attestations Beyond just lines of data then second which is most important is we have data standardization which is essential to be able to carry out analysis of the information one of the main problems that we've had in the last years with qualitative data is that it looks very different everyone reports the information in a very different way so for us it's essential to have this data standardized so that we can actually analyze it and anyone else can verify this analysis and the third one is that this is permissionless It's transparent and it is traceable then the second important point about how we're leveraging impact attestations is we're using the reputation of the people leaving these attestations and there's also three reasons why we're doing this the first one is this increases the value of the data being able to know who is generating this attestation and how relevant they are in your ecosystem gives a lot of strength to why this information should be considered in your decision-making process something that we're trialing out as well is we're enabling peer-to-peer attestation based on skills so that we can also create more reach social graphs to make better decisions depending on the specific topic that you're looking to fund second one this enables risk reduction one of the big problems I'm not sure who attended the c um Workshop is we have a lot of cbls going on around in the space and it's a lot easier though for us to just capture all of the data and then based on the reputation of the person leaving the attestation to give a different weight to those voices that are actually relevant in our ecosystem and this also enables social penalties for people that are lying we're still at a stage in our ecosystem in which it's small enough that we can enforce these type of penalties if someone is trying to benefit from our grants programs and the third one is you have a reputation that is portable depending on what we are leveraging as the identifier for the members in your ecosystem whether it is that they are logging in with GitHub and leaving their uh scorings with GitHub whether it is that they logging in with a wallet or a different social media account this is information that we can pull and leverage into the social attestations and as I mentioned this is what we've been working on at impact Garden so what we've enabled is we create standardized and onchain qualitative data to inform the funding decisions and strategies for different grants programs and our platform looks like this in case we understand that projects are in different stages not everyone has a social graph not everyone knows who is relevant in their ecosystems or what they want to listen to when they talking talking about so we have one module in which people are able to attest in a peer-to-peer way to the different skills that they recognize in others then we have another module in which we ask them to rate the different projects or the different contributions that have been generated to this ecosystem this is a very and I'm going to go into the exact questions a little bit more but this is meant to be really easy we want people to continue and F feel go through the process so that we can generate this which is the data that we're pulling and the metrics that we can generate from all of these different information and data points that people are generating and giving to us and so one of the first uh experiments that we run was with optimism in which we created this very easy process for delegates and badge holders to go through and attest to the benefit that they have derived from specific governance toolings so we asked two very simple questions How likely are you to recommend this tool to someone else in your position and how would you feel if this tool cease to exist we also included this open-ended space so that people could give us more insights into why they chose these answers and this is the translation of this two little questions that you saw so first we're trying to understand is your product something that people are actually using or or is it a necessary evil like I think in the web 3 space we found some players in the ecosystem that they are monopolies they are the only ones that are the provider for this one thing to this ecosystem and you just have to use it even if it's a terrible experience second one we want to understand if your product has product Market fit is your solution actually needed have you found your Niche and this are two questions that I that we found are really relevant for ecosystems because you want to focus on funding the intersection of those you want to fund those projects that are actually enjoyable to your users and to your community and that are actually needed in your ecosystem instead of wasting more resources this experiment only ran for two weeks but we're excited at the results so we were able to generate over 700 data points from I think it was like 50 to 60 different High context users within the optimism ecosystem and we also saw three different platforms that were leveraging our information to display it in their own platforms this again oh well here also optimism already had a a basic reputation layer and we were using foraster so that we could also more easily identify who was whom in the ecosystem and the reason why this is really exciting is because this was just two weeks of data Gathering so we want to look at and explore what does this look like if we just running this all the time this means a lot more data to make better decisions so what's next for impact attestations and for what we're building at impact Garden we want to run more experiments and we want to generate more data the first one is we want to once we have more data due to the neutrality of the data that we've been generating we're going to be able to run more we want to be able to run simulations right now all of the people that are leave that leave an attestation have no particular weight added to their own social score but this can be something that is trialed out so once we have this Bas layer of data we can start changing the weights that each one of these members has when they are leaving an attestation and we can explore what would it look like if we funded only the projects that have this type of participants receiving attestations what if we funded only that other type based on the skills based on the reputation that they got from someone else just having the data gives us a lot of new opportunities the second one is we want to make it a lot easier for people to live their attestations we don't want people to go to a different platform so one of the thing two things that we've been exploring is enabling this through a telegram butt and a a telegram mini app but also just embedding this experience into the websites of the projects so that we can continuously continuously capture this data and then the last part that we're exploring is how do we read and gather insights from all of the written um information that people have been sharing right now this is what the space looks like when we're working with mechanism design in this space we've mostly been focusing here in how we make the decisions how people vote who votes but we want this to change to something like this where we're actually first starting with the data what is the data that we have available to make decisions based on this data then who should be making decisions what are the type of processes that we should be L ranging to make these decisions and then even what are the type of analysis that we're going to be able to do so that we can actually improve the whole funding Circle and that's it so this is our uh QR code to our platform and this is a QR code to the telegram group of impa garden thank you [Music]