Building Truly Open AI - Berkeley 2024

[Music] the version to talk about document processing so today if you want to create a document you have multiple options of how open will be the product that you will use to do that uh and Google Docs in the modern world is open AI API right if you build your company around open API you will be killed eventually when open decides to change the pricing um or or they will decide to build the same feature and cut off the access to any body who's competing with them and then M and Lama what we call an open source AI is open source in the same way in which Microsoft Word is open source right uh in the sense that you do have the model and you can be completely offline you can Tinker with it you can fine tune it Etc but first of all you don't know how it was created you don't see which data was put into creating M or L what sort of back doors were potentially placed in there uh and you cannot recreate it right if you want to introduce a Fe to it fine truning is as far as you can go there's no way to um to redo the process of creating a llama with some changes that you want and the only model I know today that is in any way comparable to Liber office ismo which was announced I think less than a month ago by Ellen Institute of Technology which has open data open training Loop so you can download all of it spin up a cluster and fully reproduce it get exactly the same model Tinker with the the the way you want I will refer to Lama and mistl as open weights model and open weights model surely Drive Innovation if you remember what's been happening when stable diffusion came out it was complete craziness like every day top one news on Hacker News would be how someone built something new completely outstanding using stable diffusion uh and there's even bigger case for truly open Ai and for that I will do another bad analogy in 2024 the internet of tomor will be powered by artificial intelligence and on the current path if nothing changes much we will be in position where between Google open Ai and couple other behs they will control the tier one models that will be the true power of the next internet then there will be open source models the same way that we have mastedon today for social networking they will exist people will be able to Tinker with them they will always be one step behind uh and and there will be no way to penetrate this world Garden because once it is established once AI becomes the the way we interact with the internet the feedback loop of creating data um of capturing user attention and uh and information will make them unreachable nobody who builds open source or open solution will be able to compete with them uh and I want to one now that we know that that there is this problem and I hope you guys agree with me that there is a problem here what would be the building blocks what would what would it take for someone to go and what if we say hey starting tomorrow we want to start building truly open AI simultaneously and in parallel with the world Gardens something we failed to do in 2010 as a Tech Community with social networks um so there are multiple building blocks one is compute and computer effectiv means you need to have a lot of money if you don't have a lot of money today you will be irrelevant on the in the AI space so Academia small startups they will not be what will create this power unfortunately and then there is data uh data uh there are there are many sorts of data you need for machine learning I will not talk about some of them but I want to talk about what I call Smart data data that um that in factly differentiates your model from the competition um you can build a lot of interesting Creator you can build a lot of interesting systems where you incentivize humans to either create data or Supply data or annotate data for you uh and uh on near uh we do have we have had an experiment since 2001 with data annotations which has moved through many many steps and as of today we have an extremely sophisticated data annotation that works in a fully decentralized manner with people annotating data uh and having incentives for that uh and and you can extend this idea first of all you can scale it significantly you can extend it to not just annotating data but also Supply data right so for example Twitter or LinkedIn they have fully World Gardens but you can easily create an incentives for people as they browse the internet to supply this data into the common pool of information and naturally Twitter and linked data belongs to people it is Data created by humans and it belongs to the society as a whole um then you do need long-term incentives so this is an interesting problem that exists today that even though mistl exists and Lama exists Ando exists all of those are artif of the current market where the the attention and hype you get from publishing an open source model creates more value for you than the cost of keeping the model Clos but that will not last forever and you know stability is a good example where incentiv suddenly stopped working and stability is now struggling and the same will be happening you know Mist will not be publishing their models forever because they are for a for-profit company they need to be making money uh so you need you need to find long-term incentives we're publishing open source models truly open models will be forever mean the most meaningful act action for the entity that is driving uh The Innovation finally you do need in that entity to have tier one AI Talent AI people do not work with people they don't respect this is an interesting thing about them is that um if you look where people who are very good at where they go they always go to the labs that have the best people they go to Google they they go to open AI they go to Facebook that's where you know hypotheticals Yan Leons ilas and the um Ser zedis of the world work right so you need if you don't have people like that it's going to be much harder tangentially and completely unrelated it happens that the paper that introduced Transformers to this world was written by one of the co-founders of near um um yeah that's uh this this is where it would have been nice to say more but I cannot so thank you uh and thank you can we give a round of applause for Alex