Arpeggi Labs 🛠 Use and contribute to the largest on-chain open-source music and sample library: ARP

[Applause] [Music] well i'm excited to uh get this talk off with kyle um so yeah so we're gonna be chatting about a lot of cool stuff here um our in particular how to use and contribute to the largest on-chain sound library um i know there's like a huge topic right now as nft music so you know a lot of people are excited to build with that i excited to do a ton of stuff with music um so i'll get out of the way i'll let kyle take over and um yeah i'm excited to learn more about art excellent awesome all right thanks andrew and uh jacob um all right so my name is kyle i am the cto co-founder at arpeggiolabs um really excited to talk to you guys about this um i love hackathons i've been a part of many hackathons in the past and i mean i think music and crypto are probably like the coolest two things that you can work on right now uh either as a hackathon or as your full-time gig so you know as a as a software engineer myself i'm this is kind of a really cool opportunity to talk to you guys about what i think is that some of the technology that's going to really revolutionize the way music is is made in the future um so i'll jump in uh quickly i'll just talk about arpeggiate arpeggiolabs aka arpeggy we've put out a platform and a protocol but the summary is we're transparent open source and collaborative music creation platform um and that's basically a combination of a platform uh which is our website our peggy studio our home page our explore page which i'll show you in a moment plus an underlying on-chain protocol and this protocol is what allows us to enable uh permissionless remixing of sounds um it tracks attribution of sounds it can enable royalty payments and splits payments out to contributors of sounds so everyone who makes a sample that gets used in a song can receive credit and payment for that song um and i'll get into that in a moment um that's this is what i'll be focusing on primarily is the art protocol because it's something that we've intentionally designed to be open source collaborative so that developers can build tools music creation tools or registration systems on top of it so that the ecosystem this sort of collaborative music ecosystem can grow and grow and musicians will have access to the best tools and the most sounds um possible so it'll take it'll be helpful i think to just quickly go to the arpeggio website and show you guys a little bit of kind of in practice what does this look like what does this mean so you can find it at our peggy.io i actually encourage you guys to follow along check out this site yourself um so let's just take a look at what that looks like so our homepage has some marketing information at the top to kind of explain the concept you can see here this is a screenshot of our peggy studio which is kind of our primary platform product this is an in-browser digital audio workstation that's free to use that people can the musicians can use to basically take sounds that are registered to this protocol and create songs that then get re-registered back into this protocol so that every component of your song uh is is reusable for other artists to use slide ahead a little bit i'll show you how you can browse the actual sounds that are currently inside the protocol and these are sounds that you all have access to these are all cc0 sounds which means that you can use them for literally anything you want to right now and i'll explain that in a little bit um every item currently that is in our platform on the platform side of the of the site is actually tokenized and turned into an nft but to be honest this isn't actually required this is just sort of a choice that we made for just as a way to visualize and to easily represent each sound but when you guys are building if you guys are building on top of arp you don't actually have to use this you don't have to tokenize things if you don't want to and then this is kind of the most interesting uh part of the platform which is the fact that we track the our protocol tracks the usage of a sound as it gets you know say say you have a sample like a short sound of a guitar being played you can take that sound use it in a in a loop uh that loop can be used in the song that loop can be used in multiple songs right that sample can be used in multiple loops someone could take a clip a snippet of that loop and use it in a new sample and then that sample can be used in new songs and this is kind of the thesis behind the audio relationship protocol all right so with that said let's take a look at what kind of content we actually have in arp you can find this by going to arpeggio.io explorer uh or you can go to our arpeggio.org and click the explore tab on the left this will open up sort of an explorer of all of our sounds that are in arp you'll see it's grouped right now into songs loops and samples these are the primary um uh types of odd of media that are stored in art um so you can listen to a few of these sounds so just click play here and it'll start previewing it on the right this is a full song um so this song is available for this song is cc0 it's available for you to use and remix however you'd like and i'll talk about that a little bit later here you can see loops these are more discrete components of songs and these are also available to be used in remixed and then finally samples um these are short one-shot samples which are just little sounds they're usually less than a few seconds long and if you are a producer if you make digital music you know you're probably familiar with downloading a pack of these samples from uh say a site where you pay for them or maybe a site where you like splice where you pay a subscription and you download you know as many of these as you'd like or a site like reddit where you can download these for free illegally you can pirate them essentially which is actually really really common in music production because of the uh the restrictions that have been placed around uh usage of sounds um so these i'll just actually quickly show you our patty studio because i think it's quite cool again this is a free to use app you can go to our pc.io studio to kind of get a hands-on experience with what what this kind of ecosystem looks like let me just clear this track here but basically you can see on the left you have access to all these samples and you can drag them into your uh your studio here and create create loops i'll just play you can hear kind of how that sounds when you're done you click mint song the resulting song and its individual tracks will get registered back into uh into the protocol so the reason i'm talking about our platform is because it's a good example of the kind of thing that you can build on top of arp but by no means do you have to use anything that we've built on the site what i really want to talk about today is the actual on-chain code that i think you will find a lot more interesting is a lot more easy to use this is where we get into the arp protocol what does that actually look like and now this is actually inspired by zora so i'm sure you know the people who are participating in zora hackathon you all are probably probably familiar with sora um azure is a protocol for selling and buying nfts we were inspired by that model basically they have smart contracts that are deployed to different blockchains that can empower nfts to be bought and sold easily we took that idea and we said what if we made a protocol that was deployed to different blockchains that allowed media to be reused permissionlessly and that's kind of where our the origin of arp came from so essentially it's an open source on-chain media library that's on polygon it's comprised of one smart contract which i'll show you guys the actual code for if you're curious it tracks attribution usage of sounds and the audio the actual audio data so the actual wav files for all of the sounds that you all heard is stored on rweav which is if you don't know about our weave it's a really fantastic product they've created a it's called a weave which is like a it's kind of like a blockchain but it's designed for decentralized file storage storage and the advantage over ipfs for example is that it's guaranteed to be permanent you can also use ipfs actually you can also use uh you can actually also use you know s3 or some other data store uh whatever place you want to store your wave files on um we just prefer our weave because it has a it has those guarantees of permanence and persistence and we really want that we want the media that's registered to the art protocol to be around forever so you can always track it and go see it and find it and listen to it all right so let's actually get into what does arp look like and how does it work so here i've linked our docs docstarterpg.

io if you want to follow along this is where i recommend recommend going um if you go to docstarterpgio you'll see we have our faq page here which explains what arpeggio is on the left click build on arpeggi and this will give you information about all right i discuss arp is a composable on-chain registry for music primitives it's decentralized it's transparent it's trustless again very similar to uh inspired by zora um so let's scroll down so again if you build on arp it makes your media trackable attributable usable and remixable and it can potentially even add utility to nfts which i'll talk about a little bit later um again the license for all this media is cc0 right now that means that you can use this media for whatever you want but okay now the fun part the actual code so we've open sourced the code you can find it in the github repo um if you click this you should have access to our public contracts repository where you'll where you will find the smart contract code for v1 of arp so just click v1 go to contracts audio relationship protocol and you'll be able to actually look at the the code this is deployed to polygon right now and it's also deployed to mumbai which is the polygon testnet um so if you guys have written a code to interact with smart contracts before then this should be pretty straightforward essentially you'll use a tool like say you want to do this in a browser you want to build a browser app like a web app to interact with this contract you will use a library like ethers.js or web3js one of these javascript libraries you'll create a new a new instance of this contract object and you can call one of the two methods that are on this contract that really matter which are either get media or register media it's really simple it's just a read or write essentially let's say you want to build something that uses media from arp you'll primarily use get media and i don't want to go over all the details of the api if you guys are interested you can find all that information here that's the beautiful thing about blockchain programming is that all the code is pretty much open source by default um but i've specced out a little bit more of the information about the the parameters and kind of why they're used and what you should set them to be if you'd like to register media that means taking a sound or a sample that you have and make it available for use in arpeggi studio and on all the other platforms that are built on arp all you have to do is call register media and pass in the artist address which is the person who created the media pass on the data uri metadata uri which is explained below here and these other these other parameters and it'll become available instantly all right if you all have questions or want to use arp please follow up with me i'd love to answer questions about this and improve the documentation here but i just wanted to give you a link to this so that you um have a place to start if you really want to get nitty-gritty and get into the blockchain side all right um now i'm just going to quickly go over some some of the cool things that you could build with arp for this hackathon for example this is stuff that i really i find super exciting because it's a mix of music and crypto here's an example you could create an automated dj mixer that makes an infinite mix like a permanently just a permanent mix of uh like a essentially an infinite dj set using on-chain stems like i mentioned earlier every single stem in arpeggio is registered or every we call them loops every loop or stem is registered onto the blockchain as along with some metadata about that stem you'll be able to read from these stems and then potentially mix them together in interesting ways and make a a song player we have the bpm we have the key all of that provided so this would be really a really cool project we have an inbred another option would be to make an in-browser step sequencer that takes samples from arp so again all these one-shot samples that we have like we have a lot we have a really high quality now we have something like over a thousand sounds registered to arp right now and a lot of them are very high quality submitted by the community you could take these and create a step sequencer that would let you program say drum patterns you could program you could make a synth sequencer and then at the end if you would like you could register those loops back into our for people to to reuse in arpeggi studio or in other tools um i'll say one shot samples here because i think just to be clear what that means another option is an in-browser sample editor so if you really wanted to get your hands dirty with signal processing and use filters you could create a simple web app that reads any sample from our processes it in the browser allows the user to you know to modify it to pitch it to you know reverse it do whatever and then re-register the resulting sample back into arp with a reference to the original right so you can have this kind of like uh um chain of attribution that carries through where samples are just getting modified and chopped and screwed as much as you'd like i'll have some suggestions at the end of this talk for how to actually edit audio because there's it's really quite powerful in the browser nowadays next is you could build a desktop application if you really wanted to that allows dragging samples from arp into your favorite digital audio workstation like ableton logic for uh fruity loops um this would be if you really if you really wanted to like start using samples that are on chain in your favorite daw that would be really powerful um this one is if you wanted to get more into the marketplace side and integrate with zora you could build a splits distribution smart contract on top of arp that automatically rewards sample and stem contributors of a song for example uh by using the zora asks 1.1 uh contract which i i can link to this uh as well um this is this is actually available on sora's website if you'd like to find the ask 1.1 contract plus our petty's art protocol so this would kind of look like this diagram that we have on our homepage let me find it so imagine you have a graph of audio that looks like this so a sample was used in the loop and then it was used in a song you could write a smart contract that would let you potentially say give a tip to somebody who made a song or sorry to all of the creators of a song and it would basically funnel back through all the contributors at each step of the way so it could you know you might give 50 of the tip to the artist who made that song and then split 50 of the tip to the contributors of loops to that song and then you know for those loops you could also distribute some percentage of that to the people who made the samples that went into those loops so this would be you could use this uh ox splits uh hyper structure to do this ox splits is a really great um open source uh system for splitting on chain income i'll put a link for that here you can kind of start to see how this is one of those things where like this wouldn't really be possible in web 2 right layering these different protocols on top of each other in this like totally uh native way this is the kind of thing that we've always wanted to design around and it's one of the reasons that again zoro is so influential um in kind of our protocol design last and this would be the most intense but really quite powerful is you could make a vst type plug-in which is like a synth or say a filter or you know an equalizer that could be tokenized the actual code of it could be tokenized and then you could actually integrate that plug-in into our pudgy studio this would require you to work closely with the arpeggio team but it's something that we're really interested in doing and if you want to make a plug-in that you know if you want to make the first tokenized plugin talk to me that's pretty much it for um everything that i wanted to cover right here um actually i'm going to link a couple of uh sites for folks to use to work on um if they want to work on you want to work on browser-based web audio processing apps i have some experience now doing that in different ways with both web audio api and with a new uh framework called elementary which i highly recommend um but both of them are totally valid um i can talk about the pros and cons if people are curious that's it i want to open the floor for q a awesome uh it looks like there's a question in the chat already which is um has anyone harped on audio visual creation with arpeggio or maybe hopped uh like just visualizer and outside stuff like spore style yes it's a really good idea um and because everything is uh again everything's open source like that's you could totally do that right now you could take any of the wav files and make a visualizer wrapper on top of it if you're interested in making visualizers um we would actually love to talk to you and we could uh we could potentially work together to integrate them into our native token um we've talked about literally just expanding our you can see kind of our song token has this sort of a technical breakdown view that shows the composition but we've uh we're definitely we have plans to essentially upgrade this to support like a variety of visualizers that you can just flip through so if you're interested in doing that um i'd love to talk and be a great hackathon project and i can show you how to uh i can give you a few example uh stems um if you'd like to to work on something like that this is all using like react and you know you could you could literally use whatever you wanted to for that you could use like pixie or i think like uh something called like 3js different tools for 3d or 2d visualizer rendering cool yeah it seems like they might want to so if you want to get connected with kyle i can just stop yeah if you guys have an email thread after this just dm me your email or you can dm file your email or something yeah i'll paste my email um okay perfect and then uh another question yeah same person um you know what if you interpret music from the blockchain something like sheet music is that something that's possible i guess are they midi files or what's possible yeah it's a great question um we have right now there's a sort of like a arpeggio format that we use to represent the composition so that's sort of the sheet music composition it's very similar to midi but it was developed it's kind of like a way that was designed to be ultra lightweight for storage on the ethereum blockchain originally and we've upgraded it now to support a lot more features in other words we could open source this uh this format and we're planning to open source the format right now it's not open source unfortunately like it's it's quite a complex i would say like you know any you can imagine any uh audio format has a lot of little edge cases in it um but one option is open source that format another option is we could actually build a midi converter that takes that format and that converts it to the best possible midi approximation um so unfortunately not available right now but a good question and in the future it'll come obviously that would be really useful for your visualizers by the way i just wanted to add like i think the fact that most visualizers only depend on having access to the uh waveform of audio imagine if you had access to the actual sheet music how powerful that could be so yeah really good question sweet and i think you mentioned this a little bit would be cool to drill down and how you guys are handling licenses and um and like just ownership in general and all that stuff yeah cool um yeah thanks for the compliment um it is everything is cc zero right now in our in our in art and we're kind of keeping it that way we'd like to move towards cc by which is the same as cc0 but you need to give attribution to artists we feel like that really fits our ethos at our pudgy which is essentially make your things available for use but trust that the people who are going to use them are going to give you credit for the stuff that you made that will bring value to you the more your stuff is remixed and it'll bring value to them because they'll have access to more things to work with um and right now if you think about the traditional music industry like you can download all these royalty free samples from different platforms you have to pay for them or you have to do it illegally but once you've downloaded downloaded them and use them there's no connection between the sounds that you that you're using and the artists who have created them originally there's literally no reference there at all that's something that we kind of want to we want to challenge that because we think blockchain tech is the perfect place to have a permanent you know attributable ledger of um even even fine-grained things like that like who used what sample and what song like that stuff is a perfect use case for uh for blockchain so let's say cc0 is where we're at right now for everything because it simplifies simplifies things uh massively for us um but we're considering potentially eventually how do we how do we take some of these concepts that we've figured out with this very simple cc0 licensing system and move them towards more traditional music licensing systems you know where each song has a number of rights holders might be the producer it might be the record label it might be the artist it might be the singer it might be like the guitarist you know and kind of expand from that from there but currently cco cool all right well if there's no other questions uh we're actually basically at time perfect cool yeah you'll find also all this information out of io um um yeah that's pretty much it great well thank you so much for being here kyle really appreciate you taking the time um and again this talk's gonna be uploaded so if anybody here wants to review it or anybody is watching this later from home you're already watching it on the live stream but um yeah of course get a resource there and you can just go to the global.com youtube page um and then yeah thanks again kyle for being a part of this and excited to have you be part of capitalism with us awesome thank you so much for having me thanks for watching everybody