Livepeer 🛠 Unleash Web3 Video Experiences with Livepeer Studio - Yondon Fu

foreign [Applause] [Music] thanks everyone for joining me tonight my name is yonden Foo and I am one of the co-founders of the live pure project so life here is a decentralized video streaming and processing Network and I'm going to talk to you today about unleashing web 3 video experiences with live pure Studio but before I dive into live pure studio and all the different video experiences that you can build with it I want to talk a little bit about open video infrastructure and what we've been building at live here and how that enables live here studio in the first place so just to kick things off today 80 of all internet bandwidth is consumed by video streaming and this shouldn't be too surprising to many of the folks in this room that are used to consuming content online on various social platforms and this number is really only going to increase in the future so 80 is really just the start of this trend however if we take a look at the status quo and the landscape of how internet video works today it looks a little something like this so we have this closed box that's powered by web2 applications and web 2 infrastructure and many of these names on this slide should look familiar to folks so in internet video we have dominant players like YouTube for on-demand video we have twitch for live streams and increasingly as of late we have Instagram around short form video with their reels feature so Tick Tock is also in that category as well at the same time when we think about the infrastructure that powers all of these experiences we have some familiar names we have AWS we have Google cloud we have Azure so a lot of these hyperscalers and cloud service providers and you might ask the question well why highlight this is this really a problem and I would make this claim which is closed infrastructure so that box that I just showed on that last slide leads to closed walled Gardens and this is particularly important when it comes to video I would state that because video only exists and can be experienced within the boundaries dictated by whoever controls the infrastructure so imagine the scenario where you do have portability around all the video data on Instagram or all the video data on all these different platforms if you can't actually play back that video and you can't actually distribute that video effectively to a global audience then you're still hamstrung right and it's really only these small number of Giants and the small number of cloud service providers that are effectively able to provide that infrastructure at scale so your ability for your content to exist and be experienced by the the rest of the world is dictated by the terms that they set and some of the implications that I think are interesting here are one developers are unable to freely experiment with building new video experiences without either one of the following exorbitant costs so this infrastructure is really expensive to run so either you pay a gatekeeper for access to it or you build it yourself but in both cases you're paying a lot of money or in the case where you're relying on a gatekeeper for access to set infrastructure you're subject to the threat of a single party that may or may not have any accountability mechanisms to flip a kill switch in order to make your content available and accessible to the rest of the world or not and second what I think is interesting is video creators are actually defined by the platforms that they use instead of being self-sovereign so uh I've a colleague that likes saying that he wouldn't want to be a twitch streamer or wouldn't want to be a YouTube streamer but he would just want to be a streamer and he's defined by the act that he performs and the content that he creates but that's really not possible today right because you can only be a streamer if a platform like twitch allows you to be a streamer you can only be a streamer on YouTube If YouTube allows you to be a streamer so the situation that we're currently in is because infrastructure is not particularly accessible you have to play on the terms in the playing field dictated by whoever controls said infrastructure so we think a lot about this at live peer and what we think the path forward is is open video data and open video infrastructure and an example that I like going through of the possibility of what the future could look like in terms of unlocking new and Rich video experiences I often cite a pair of applications from the lens ecosystem actually so for those that are not familiar with lens lens is a decentralized social graph protocol that's built on the polygon blockchain and what's interesting about their approach is that they've created portable social graph data and I'll get to why this is relevant for video momentarily but before getting to that what I have on this first slide is a screenshot of an application called lens tube so it is you can think of it as a decentralized YouTube where the social graph data and all the social post data is portable and shared as common infrastructure amongst the same set of applications and we can see here that we have this video experience that looks very much like YouTube except it's built in lens tube but if we look into this next slide we have another application from the same ecosystem called Leinster and this is actually the same exact social post data and it's surfaced in just a slightly different way so it's the same video content it's the same text content but it's presented in a different user interface and and the reason this is interesting is because we have the advantage of data portability which allows developers now to build specifically for custom preferences depending on a user base so it could be the case that in scenario number one it makes more sense to build this video consumption-centric application experience so something like lens tube might fit that user type a little bit better but something like Leinster if you're less focused on consuming say long-form video content you're probably not going to want that video to be taking up the entirety of your screen if you want to scroll through a feed really quickly that's really not optimal for that so something more along the lines of what Leinster has built might be a better fit for that so what's really cool here is that this is the same social post data that's shared and interoperable and now these developers can be really creative around how they build custom experiences and new experiences for different audiences instead of a single monolithic application having a single developer team trying to cater towards many different audiences simultaneously but never truly fulfilling the needs of any of the audiences particularly well so where does video come into play the reason that I like highlighting this example is that in both of these cases video is still featured prominently here and imagine a scenario where you're unable to watch the video in Leinster as well as you can watch the video on lens 2. and that would detract a lot from The Experience right so even though you've created this new interesting product experience for Leinster your kind of hamstrung because someone tries watching the video content for your social post and it just doesn't work particularly well maybe it looks a little bit squished maybe it keeps buffering uh you're not really gonna paint a very compelling picture of your product if that's the case so this is where a guarantee of experience and portability of experience comes into play where we think that the future of rich media applications is going to be the combination of these two things one is data portability so this is something provided by projects like lens where you can share social graph data but second is this guarantee of experience and this portability of experience where you can go to these different applications and you can be guaranteed the same high quality video consumption experience as some other application in that ecosystem and we think that really the only way that you can do that is with share shared public open video infrastructures to provide these guarantees to more developers so they can spend less time worrying about well is my video going to buffer and more time on how unique can I make my product experience so many of the people in this room might already be familiar with decentralized storage and you might be asking the question well if I already have decentralized storage and I'm able to upload say an mp4 file to ipfs our weave so on and so forth isn't that enough we've already seen examples of applications that upload MP4 files to these systems and then play back in the browser don't get me wrong you can get pretty far with that approach however the reason that I'm choosing to highlight this right now is that if you take a look at the existing web 2 experiences that consumer consumers know and love there's a lot going under the hood and a lot of complex Plumbing that makes that YouTube experience that Netflix experience that twitch experience so seamless that low latency that interaction that you can have in a chat box because the latency is so low there's so much that's happening in order to make that work and while decentralized storage is certainly going to be a part of the picture when it comes to building these applications we think we need more and why do we need more because we want to avoid this we want to avoid this constant loading indicator because this is what kills product experiences and to just illustrate the flow of how this buffering and this loading indicator might manifest itself is let's just say we have three people and person one wants to play a particular video and let's say that they're just using ipfs or they're just using some decentralized storage system they're going to have some CID they're gonna have a hash to reference the video and that provides data integrity that's great but let's say person a person one wants to watch on their iPhone person two wants to watch it on their Android different shape different form factor person three wants to watch that same video on a low bandwidth connection and everyone in their household is also simultaneously watching Netflix so they don't exactly have a lot of bandwidth despair so the basic approach here would be your device would send a request to said storage system fetch the file back and try to start playing it this is probably what's going to happen someone is going to say that the video keeps buffering someone's going to say that the video takes so long to start playing you press play and seconds might pass by and nothing happens and then it starts and someone might say that the video looks really squished and it just doesn't look right because the video that you're playing back wasn't created for the particular device type and the format that you're playing back on so all of these are just examples of how that experience isn't just quite right in that scenario and this is particularly problematic when we think about what the experiences of the future are going to look like where if we want to be truly creative with how we can leverage media on the internet we want to encourage people to experience experiment with new ways of consuming that content so regardless of what you think about say tick tock or the recent Trends around like recommended algorithmic video feeds what was particularly novel was this vertical scroll auto play mechanism and that was one of the first times that when people started using that in Tick Tock where they experienced this product where videos just continuously played one after another short form and it was pretty seamless and because of the recommendation algorithms it was also really engaging but the only reason that format works is because you can watch those videos those videos seamlessly one after the other just imagine a scenario where you watch video One video two and then all of a sudden video three just halts you're instantly removed from that product experience you're instantly losing engagement there and I highlight this because it's going to be challenging to create these new product experiences if you can't guarantee the experience of any individual video at the same time so just to round things out here I I highlight all these things because why not just upload and Playback MP4 files well some of the things that we want here we want support for a short time to playback that can be hard to achieve with big MP4 files we want to support things like adaptive bitrate streaming so this is actually a term that refers to what services like YouTube Netflix and all these uh web 2 platforms that we're accustomed to use it basically means that I'm going to take a video that you upload and I'm going to create all these different versions of it so that no matter what device you're on and no matter what internet connection speed you're on you're going to get the best possible version of this video you can only do that if you have some level of processing involved uh the next one I think is pretty forward-looking uh and interesting and actually not too far off but we want support for things like video enhancements so I don't know if anyone follows uh kind of like the latest trends in artificial intelligence but uh open AI uh a major r d lab in this space released their whisper model recently which is really effective at automatically transcribing the audio contents of a give-in video or generally a media asset and then what you can do with that transcription is now you can also insert that as metadata into your video and now you have automatic closed captioning you didn't have to do anything yourself and that just works once again that's not going to work unless you do some level of processing and lastly if we want to support things like live streaming unlike static file content live streaming it's Dynamic files being constantly generated so that's why the industry has specific protocols to support this such as rtmp and hls and lower latency variants such as llhls and all of these things need to be supported in order to truly create a good live streaming experience so in summary you can just upload and Playback MP4 files but you probably want a little bit more so I mentioned all this because what we've been working on at live here is what we refer to as the open video protocol and what this open video protocol enables is tying it back to some of the things that I was just talking about it allows anyone to generate videos with the best codec selection for a desired video experience so going back to Adaptive bitrate streaming it allows anyone to generate the best possible quality versions of videos so it doesn't matter where you are in the world you can get a good playback experience allows anyone to generate videos with the best container selection for a desired video experience without getting into the Nitty Gritty details videos are often packaged up into different containers depending on how they're going to be consumed so this is enabled on the network by media servers that provide this service and lastly it allows anyone to deliver these generated live streams and or generated on-demand videos with low latency and short time to playback for viewers and this is all supported by media servers that participate in this protocol and in this network so putting this all together we have this video infrastructure Network and this is a screenshot from our Explorer application that just gives us a snapshot of some of the relevant stats that you might care about around this network such as how many fees are being paid in in order to access all these Services how much actual minutes of video are flowing in in order to access these services and that brings me to live pure Studio so live pure Studio which I think is going to be most relevant for anyone hacking this weekend is what we refer to as your gateway into the live peer protocol and the network and it is also the video toolkit for web 3 applications that want to build these rich video experiences that I mentioned some of the features that studio gives you includes end-to-end live streaming so starting from ingestion so you have a streamer that wants to stream that video you can ingest it you can transcode it on the fly on the network in order to generate all those different Renditions that I mentioned and then you can also play back all those Renditions seamlessly in your browser in your application we support video on demand also referred to as VOD similar to live streaming this is end to end so you can upload a file have it be transcoded into all these generated versions and then play it back seamlessly in your application and then a couple other cool things that we support here that hackers might be interested include viewership metrics have you ever wondered how many people are actually watching the content that I put out there how engaged are they can I use that information to update How I build my product experience and lastly video nft minting so today where we've seen this explosion of activity around nft marketplaces and people using nfts and as an interesting monetization tool for creators and a lot of these nfts have been linked with interesting text assets image assets and we think that the next wave is going to be around video similar to how in web 2 we went from image only sharing to this explosion of video sharing so what we've built are a set of sdks and tools that allow you to easily go through the video nft minting process without a lot of pre-existing knowledge so just to walk through some of the things that you'll be able to do with live streaming with a single curl command that looks something like this on the left you can create a stream so that you're ready to actually start live streaming and then from there you can use any standard broadcasting tool maybe OBS which is a pretty popular tool all you need to do is enter the rtmp URL supported by Studio you enter a stream key that's generated for you when you start using the API and you're Off to the Races you can start streaming we've built this dashboard into Studio which gives you a lot of high level information about your Stream So this includes the health of your stream what the metrics look like it includes a preview of the actual playback of the stream so you can actually determine for yourself is this working properly is this actually being presented properly to the viewers for VOD for VOD we wanted to make it really easy to have you take your existing assets regardless where they are and have them be transcoded via live peer so that you can get the best possible quality of experience in as few lines of code as possible so if you have an asset that's already say on decentralized storage whether it's in our weave Gateway or an ipfs Gateway doesn't really matter if your asset is already there you can submit a single simple JavaScript query to one of the API endpoints for studio and have Studio pull that data in transcode it and make it available for seamless playback if you don't have the asset living in another storage system already you can do direct upload as well you can simply upload and have one of your users paste in a file and then that will be uploaded to studio for transcoding and Playback viewership metrics simple JavaScript query to an API endpoint for studio and given the identifier for an asset that you've already uploaded you'll be able to get viewership information you'll get a sense of what engagement looks like and lastly video nft minting so for video nft minting we've created an SDK and tooling that will simplify this process so that given an asset that you want to create a video nft for it can be exported to ipfs it'll be uploaded and transcoded and made available for playback with live peer all of that metadata information can be packaged up into ERC 721 compatible metadata objects and then at the very end once all of that is ready we also provide a really simple template for you to try out minting that video nft on polygon so this is a template right so this is really just meant for you to get started and play around with the nft minting feature of course you're likely going to want to customize his experience for yourself and you can do that using the SDK and tools that I mentioned but this template can help get you started just so that you can go through the flow get accustomed to what the flow looks like so going into this weekend we have a bunch of features in studio that we think will be really helpful to hackers that want to embed video in their applications and there's a lot of hacks that we're excited about and some of the things that I'll mention here really are just guidance and for inspiration of course we look forward to seeing what everyone will build on their own but just some of the things that we think would be cool for people to explore and think a little bit about include live stream experiences around nft drops so nft drops today uh oftentimes are kind of individual experiences where an entity drops and you might participate in the drop you might try to get a particular nft in a collection what if you can actually have a live stream experience where people can participate in that drop together in a live fashion and create a more social and communal experience around that live streams around nft Pack openings so I don't know if anyone here is familiar with the concept of an nft pack if you're not you might be familiar with this other genre of videos or content that you might have found on YouTube or elsewhere which is I think it's called like unboxings where these content creators will have mystery boxes that they'll open Live or in the content that they share and then they don't know what's going to be in that box but it ends up being this really entertaining and interesting experience for Their audience because their audience gets to watch them go through the process of opening that box it might seem a little bit silly if you've never kind of like dove into that rabbit hole content yourself but it is really interesting how engaged people are with those types of experience and the reason that I mentioned that is because well physical unboxings make sense you have a physical box you'll open it up you don't know what's inside what's the digitally native version of that what's the web 3 native version of that and we've already seen a lot of experimentation with nfts and some interesting Concepts around being able to create an nft and then call a smart contract function in order to generate additional nfts that you don't know what they will be ahead of time and once again this is very easy to be an individual experience but what if you could make it a social and communal experience what if you can live stream that what if you can have people participate live an extension of that would be what if you had people say live bid and participate in the pack opening process so that you can see a content creator that you really enjoy you can support them by bidding on the packs that they're opening and that's part of the experience for everyone a couple others to round out the list video messaging apps messaging apps that use video as the native content form factor in order to communicate with either another individual or in a group video recording tools so think something like Loom except web3 native allowing you to record and own the content that you create and then monetize that directly with your audience something else that might be interesting is web3 video newsletter subscription apps so we've seen this huge explosion of newsletter writers on platforms like sub stack in the past year two years and text is a great form for that right but we also know that vloggers on platforms like YouTube or streamers on Twitch video is their native content form so they might not be the well most well suited to write a written newsletter but they can create really interesting video content on a recurring basis and what if we created nade web3 native video newsletter subscription apps for them so that they can regularly schedule content to be released to Their audience and have a direct monetization relationship with their users With Their audience and lastly just coming back to the theme around nfts we've predominantly focused on static content for nfts but live streams are a great example of dynamic content so what would it look like when we start embedding live streams into nfts and having collection of live stream nfts and once you start exploring that having an explorer that can Surface all these live stream nfts so that what's interesting is that I can go on Twitch today and see what streams are live there I can go to YouTube live today and see what streams are live there but there's not a very easy and good way to have a composition of the two and to have those two Services be interoperable but what's interesting about nfts is that they can be a shared data layer across all these different applications so what if you built an explorer that allows you to surface all these live streams across different applications and then you could build experiences around that so those are just some of the things that might be interesting for people to think about and hack on this weekend but of course uh everyone here is probably way more creative than I am so I'm sure that there are much better ideas out there than what I listed so encourage people to just poke around so to round things out uh for eth Bogota this weekend we have a set of prizes for people that integrate the live pure Studio API uh first place five thousand dollars second place three thousand dollars third place two thousand dollars we have this hacker quick start guide so feel free to scan the QR code and that'll take you to this notion page that should contain all the relevant resources for you to get started poking around with studio and also exploring some of these other hack ideas and last but not least join our community so once again scan the QR code and that will take you to a portal that will give you access to the various Community uh channels that we use and this is a community where we're not only Technical and we're building out this protocol and network and if that's what you're interested in of course we welcome that but it's also a community that's exploring this question of what are the rich media and video experiences of the future going to look like not only for web 3 but for the internet as a whole and if that's interesting to you definitely jump into the conversation there so that's all that I have for today thanks thanks for your time and if we have time I can take some questions but I'm not sure how much time we have left I don't I don't know I yeah we'll we'll start with him we'll come back here yeah yeah great question so I think the first example that I would cite is the one that I already signed into slides this application called lens tube which is built in the lens ecosystem so that application is live you can use it today I think it's at like lens tube.xyz if you go to the live peer Studio page and scroll down a little bit there should be a few other references to applications that are live using live peer today um a lot of them you'll see the trend is user generated content so we have applications uh building user generated content platforms focused on filmmakers or um lens tube I already mentioned so a lot of user generated content so I would start there you had a question yes so the Studio Lightyear studio also supports recording so you can automatically record the live streams so that they're available afterwards and I believe you can choose the storage location of your choice so uh if you want to use centralized storage go for it if you want to use something more decentralized you can do that as well but that is supported in the service sorry could you repeat the question uh okay custom link so I believe that short answer yes I believe that you can do that today already but it might require a little bit additional development work on your end uh something that uh actually one of the team members that is here this weekend is playing around with is making that easier so you can easily create say like sub domain URLs so like let's say that you have a domain that you want the playback URL to be hosted at right now you can do that but you have to do more work yourself uh and I think there's a couple things that we can do to make that easier if uh you have a particular use case that that's important for definitely come find us talk to me or someone on the team because we'd love to hear what you need for that all right well if that's it thanks so much for your time and uh Happy hacking