Measuring impact of open source software (Web2) - Berkeley 2024

[Music] I'm Andrew a package management nerd and I've been uh working in the space of trying to understand and extract data from open source projects to help with solving loads of different problems around trying to understand them and how to best enable and improve open source software for for quite a few years uh work out what the impact of Open Source is and like what is the reason to do this kind of analysis so there's there's lots of different challenges involved uh in studying open source software but that in uh kind of attacking those challenges you then start to unlock some interesting uh abilities ultimately what I'm trying to get to uh is that if we have good data we can then use that data to make good decisions which then can improve open source software and because everyone uses open source software in all their software that ultimately improves all software which can ultimately improve the world right that would be a really good thing to have lots of challenges so as soon as you start going to look at open source data you realize that every different software ecosystem has different formats different files different names for every kind of uh piece of metadata even though it's all pretty similar every dependency network is ultimately the same with within each programming Community they just use different formats and different ways of describing them if you can have a system that can work for everything then you can build tools that work for every Community without having to understand each one of those communities necessarily I've built a system called ecosystems and it basically gives you a normalized layer of metadata for E for software packages and repositories and then everything that goes along with that and it collects data from something like I think it's 35 now uh software package managers normalizing that data out so that it is in one standardized format and then uh hundreds of millions of Open Source repositories from GitHub gitlab bitbucket code BG and hundreds of other smaller forges as well just to give you uh an idea of some of the numbers the uh the database is about 12 G uh 12 terabytes with about one terabyte of just indexes to be able to query that on a you know a fairly performant basis and all of this is available to download absolutely crazy numbers that are happening in open source and it's literally uh the the growth of every ecosystem is accelerating right now what kind of things can you do with it where you can find critical open source projects within a community you can go and discover who the key maintainers are for any particular software uh ecosystem or for all software ecosystems and then you can also connect this data to other kinds of networks so I've got a few little case studies where things have uh this data has been used to enable new and interesting analysis one way is um looking at software paper mentions and you get some absolutely crazy visualizations it's probably really hard to read on this screen but uh there's a lot of links between papers that you don't you don't normally see and this worked for three different kind of package managers python uh bioconductor and R uh with all with the same code so we were able to put this together in two-day hackathon which was uh enabled by the kind of normalization data that I had already put together another example um I've linked to the slide here of uh Josh bres who does a security kind of analysis of all of Open Source uh and he is kind of Blown Away by how much um how many versions 100 million versions of packages have been published which is if you were to try and review every version of every piece of software that's being published right now you're never going to be able to keep up because there's more and more every day the gra on the right looks like it's broken but it's not it's just absolutely shrunk so this is uh the number of maintainers that uh exist on packages basically most packages most open source software only have one maintainer which is a really scary number when you look at how many people depend on all of these things and that graph there is like oh there's nothing there it's because it's literally just one tiny axis on the right another thing I did is I looked for funding data I wanted to find out which projects describe how to fund them uh I mined the funding yaml files across all uh packages and um as well as GitHub sponsors data and if the pack some package managers have metadata around funding links as well and I found some interesting numbers there you've got like only 22% of critical which is kind of like you know the top 10 ,000 open source libraries have some kind of way of describing here's how you fund my work which is a tiny amount really considering how all you need to do is to drop uh some file into your repository that says here's how to fund this thing and the top 1% which is you know like what's that 100,000 projects uh is only 14% I'm also working with open Collective at the moment to try and connect the dots uh between funding data and uh activity data on open source projects it's an ongoing project uh but to give you a kind of an example we can look at financial data of the projects how much money they receive and how much money they spend and then compare that to the activity data on the project so do they get more contributors when they receive or spend more money is this kind of data then because open Collective supports all kinds of different projects I was able to just normalize that out and go okay well here generalized activity data for all these projects and we have generalized funding data for all these projects what are the trends there uh eventually we'd like to get to the point where anyone can just dump the an s bomb in there here's all my dependencies and have open Collective be able to say well here is this Financial State of all the projects that you're using in your application be open source or close SCE uh some things I got on the road map are doing really in-depth license uh analysis looking at more um security data the open ssf scorecards a prime example of like trying to understand their best practices around CI and different kinds of security uh methods that people are taking in maintaining their software so yeah thanks very much for listening uh car is now