Global Video Content Delivery with Storj and Livepeer

all right thank you everybody for joining us uh we're here to talk about uh globally distributed video transcoding and streaming with a combination of storage and live peer so my name is John Gleason I'm Chief Operating Officer for storage labs and I'm Eric Tang I'm the co-founder CTO at live peer so for those of you who don't know about Storage storage is a decentralized cloud object storage provider we have a globally distributed network of 22 000 nodes in a hundred different countries and those nodes are operated by five or six thousand different people it's one of the most distributed and decentralized networks out there today of course we're completely open source and because of the characteristics of our Network we're really optimized for use cases like video streaming when you have a distributed Network like this there are a lot of things you can do to make it go really fast we're optimized to take advantage of all of the bandwidth of all of those distributed nodes that are all over the world we are able to do things like long tail cancellation to make sure that even though we're Distributing files over a lot of different places we're always getting the fastest nodes to deliver our content we're able to minimize coordination we don't have anything in the critical path of file delivery to make that go slow and of course the parallelism being able to transfer pieces from lots of nodes at the same time gives you a huge performance boost it might be helpful to talk a little bit about how we store data on the network and then we'll talk a little about how we stream it out every object that's stored on storage is first encrypted it's encrypted with keys that only you have so your keys are your data every piece of data stored in the network is encrypted so that no node no provider no one at storage no one running any node in the pure in the environment can have access to your data every encrypted object is split into lots of pieces as few as 80 and as many as thousands of pieces depending on the size of your file and those pieces are distributed over our entire network and then when you want to download your file it's just the same thing in Reverse you download the pieces you re-encode them and you decrypt them now that's a really fast process and our software makes that really easy and if you want to see a demo of just how that works we have a little demo that shows where your pieces are uploaded on an ongoing basis these uh presentation materials will be available afterwards so you can grab the QR code and check it out of course storage works with a lot of different providers in video management and collaboration space because video streaming is such a great use case for us and one of our Premier Partners in the video streaming space is live peer huh so our slide didn't load okay so Eric do you want to talk a little bit about live peer and what you do in the steering coding space yeah yeah for sure hi everyone um life pure is a globally decentralized video streaming Network that we started out um in 2016 2017 uh as building very closely with the ethereum core developers at that time we were building video streaming into the Swarm Network which was you know this kind of the search provider for for ethereum and over time uh live pure really started as a focus on a globally distributed video transcoding infrastructure and for those of you who are you know like what's transcoding well transcoding is actually the the kind of the linchpin mechanism that makes video streaming works at scale on the internet so you know when you're watching a Netflix video and it starts a little grainy and then it goes to kind of a high high resolution view that's actually transcoding working in real time to make sure that you don't have any buffering in the video and like in your viewing experience the problem with transcoding is that it's very very expensive requires a lot of computation power and it requires a lot of data bandwidth to to kind of move that data around that's why working with storage has been really exciting because there's all this data and video data is being stored inside storage and live peer can help transcode that and make the video streaming experience even even smoother for the end user so one of the great things about working with someone live live peer is that we're solving both sides of the equation so if you want to have a video streaming service first you need to start with your video you need to transcode it you need to host the transcoded pieces somewhere and then you need an Ott player to stream those pieces so you can actually watch the video seamlessly and that Ott Player is taking account of how much bandwidth you have it's downsizing the video to optimize your experience for the bandwidth you've got and so this is a great example of an end-to-end solution where a customer can actually go use R2 products in conjunction together seamlessly just as they would Amazon or Google or any other provider and have a great solution but the main benefit for that solution is it's about 90 percent cheaper when you combine our two products than using a comparable service on Amazon we have a joint customer who's very excited to use both of our products together because they dropped their bill by a zero they lopped a zero off their bill going from Amazon to storage and they're going to take it down to another 20 percent by adding trans coding and they'll give their consumers their customers at the end watching those videos a better experience in the end of the day so we're going to walk through a transcoding lab today to actually show you how this works and because we have conference Wi-Fi we're not going to attempt to do this live however you can join us at the uh the sponsor showcase tomorrow through Sunday we'll have people in the booth Eric will join us there as well and you can actually test this out try it for yourself and see how this works so um we'll walk through uh just what the steps are in the presentation today and then we can also opens up for questions and so the main things that we'll show you in this uh in this transcoding lab are how you create your tran your live credentials and so the credentials in live peer are how you get access to the system I don't know if you want to talk a little bit more about under the hood what goes on and yeah so so when you're uploading video um kind of into the into the live peer Network we're transcoding uh that video is actually being chopped up into small bits and distributed around the world to these transcoding nodes that are running gpus on them gpus are very effective and efficient in transcoding video um and it's about um 50 to 100 times more efficient than CPUs um and and using life here to transcode video represents the same amount of kind of savings as as you compare to something like Amazon so all that video is being transcoded and there's a protocol that runs on the arbitrum network that makes sure the transcoding work is done in a secure fashion so that when so when the transcoded job kind of returns the result back to a storage bucket in this case in storage you can be sure as a developer or as an end user that's using the application that the video that you're getting is the one that you have that you have uploaded but now is you can stream it in an optimized way so when you're interested in sort of following up on what we've been talking about today everything we have is in our documentation so you can get the materials you can use this QR code now uh and there's a step-by-step set of instructions on how to do everything that we're going to talk about today and uh one of the things I love in web 3 is is using projects that really work and work really well and I've used Amazon for transcoding in the past and I've used live peer and this morning um I I was just uploading and downloading a video and walking through it and just making sure that everything I had the flow down it works phenomenal and it's fast and when you think about that all of that transcoding happens you upload a video and it's it's done in two or three minutes and you've and 90 less than Amazon and the results are amazing so everything you'll find that you need is in this lab so your first step is going to uh to life peer and that's live pure.studio and you'll create an account so when you create an account on Live Crew Studio then you have an immediate access to create an API key and you want to talk a little bit about that experience yeah so you just go in um you sign up for live pair it's free to sign up um the live pair the life pair Studio application is a is a gateway survey so the life here Network think of it as like a inferior for for the ethereum blockchain and once you go there you go to the developer tab and you can create an API key and then you're ready to go so once you've created your live peer API key next you need to create your storage API key and you can go to storage.io and that'll link you to our satellite interface and just as live peer has an inferior-like interface we have an inferior like interface where you log in and create an account live here comes with a thousand minutes a month of free transcoding as a free tier that's right and then storage gives you 150 gigs of storage and bandwidth also so you can do your whole Lab you can do this testing without committing to anything and you can see how well all this works for free so when you go into storage uh you'll just go down to the access grants which is where you create credentials and you can create a set of S3 compatible credentials so you would use this application just like you would any S3 endpoint so live peer is great because it works with Amazon Google Microsoft Wasabi backblaze almost every single storage provider out there Plus Storage as a decentralized storage backend and so once you've created your credentials what you'll want to do with storage is just create a bucket and upload your first video you can use any S3 compatible tool you can use our CLI you can use our clone or you can use the web interface and just drag and drop your video into the browser one thing you'll find that's really important is when you're creating these credentials when you're creating an interface it asks you to set an encryption key that's done client side even though it's in the browser it's still client-side but don't lose that key because when you set that encryption passphrase that key is your data and so if you lose that and you try and go back and get it later all you'll be able to do is download the encrypted asset which is super unhelpful so remember save your key and once you've got your access credentials your key is actually encrypted and embedded within those credentials so the next step is you run a a script and within that script what your setting is uh you're setting the live pure API key you're adding in your storage uh S3 credentials and you're setting both the the source of where you uploaded your file and then you're setting your destination where do you want the transcoded assets to go and then there's a little section there to set which transcoding uh levels do you want and so do you want to talk a little bit about the importance of establishing those credentials and the the formats yeah yeah so it's important to set the storage key correctly so that you can the video can be transcoded and put back into into your storage bucket and when you're setting the transcoding profiles we call them bitrate ladders that's where you can set kind of based on how how you how you want your users to consume the actual video you can you can specifically set these settings so for example if you want if you think a lot of your users are going to be on mobile and you know they're going to be walking around and have shaky bandwidth connections then you want to make sure that you have a maybe a fourth 480 or 360 P version of that video so it can work very well in low bandwidth conditions or if you if you if you know that your video is going to be a 4K full full HD and you want people to be experiencing those 4K full HD profiles you want to make sure you set that as well so you just set up all the profiles there's also a default one if you if you think you know um it's just kind of like a regular video on demand use case you don't need to set you don't need to set anything and the live PR network will pick a default solution for you so for example if you were at a conference you may want to go all the way down to like 240 or or 120 yeah that's right right on all right and so once you set this up it's relatively straightforward you run the script and it will go through and it will transcode your assets and so what you'll see when you sort of are looking at the storage interface you can either use our web browser or cyberdoc or any other asset you'll see your original assets that you uploaded so the video that you're going to transcode and then when you're done what you'll see is you have a long list of these smaller objects and what's happened is the transcoding process should I tell them what the transcoding process is or do you want to actually tell them I'll say a little bit so the transcoding process you know we talked about kind of it breaks up the video into small chunks and then he transcodes them into different versions and what you're seeing here is the video be the one video file being represented in those small video chunks we call them video segments right and in online video streaming I use this thing called hls which um which is a standard that's established by Apple they own the phone so they kind of own the standard and and what you're doing what the um end device the viewers are doing is they're downloading each of the little segments in sequence right so and and then here's where the magic comes in because if your bandwidth connection all of a sudden drops the video player is smart enough to start downloading the lower bitrate version whilst continuing to play the same video so you may see the video become a little more grainy but you won't see the buffering and that's that's a really important step to make sure users continue to stay on your website and viewing the video from start to finish one of the great things about live Peralta is that you can you could have this Dynamic bitrate with hls um but also if you just have a website for example and you're only doing the uh the video you can also encode into just standard MP4 files and so instead of having lots of little files you can have the whole file and the cool thing is that when using the player in either model you can bite range seek so you can skip ahead and it doesn't download the entire video it only downloads the byte range that represents the segment of video where you're watching so it's really fast so you don't have to have that caching in uh and long loading experience so one of the the next steps is once you have that video transcoded you just need to create a shareable URL and we have a service called link share and linkshare just creates an HD HTTP string that represents access to a file or directory so you can share a whole directory in this case this is the directory of all of the video assets together so that you can send it to a player and then the player is smart enough to know what to do with that now you can use Link share to send an individual file you can use it for streaming video you can use a lot of things but it works great with the live pure Ott player because again when you're putting our two products together our the storage service simply stores the the data and allows it to stream out efficiently and really fast live pure handles all of the transcoding but also offers the Ott player that knows what to do with all of those tiny pieces and so when you take that shared URL and again you can use a variety of tools to do this and you can script it however you want in your application ultimately what you get back is a streaming video a player that you can embed into your application uh one of the the interesting things about this is um as you're sort of using storage plus live here that that whole savings experience is really good but sometimes people think well is it is it good though yes I'm saving a lot of money but ultimately what's the user experience like and we have a joint customer and that customer does something like two three four hundred thousand uh streams is it 200 000 a day something like that it's a lot of video streaming they have customers all over the world and uh this thing works seamlessly um we have this customer is uh excellent we will be doing a webinar with this customer in the next month or two but it's it's a fantastic end user experience this works and it's one of the things that gets me really excited about web3 when I see something that that gets a customer happy who's coming out of web 2 who's dissatisfied with Amazon and sixty thousand dollar um Amazon bills and they're looking at you know a ten thousand or twelve thousand dollar bill at the end of the day after using a combination of our services and that's that's fantastic so when these videos are streaming one of the other cool things that's going on in the background is when you sort of bite range seek to a video of when you're playing that video is actually distributed out across thousands of nodes all around the globe and when you bite range seek you get maybe 20 milliseconds 30 milliseconds of latency As you move the uh the cursor forward and one of the great things about that is it it's not just going to a CDN it's not just going to a centralized provider like Amazon it's actually reaching out and gutting byte ranges from thousands of nodes and every time you seek it's getting 80 nodes it's finding 29 pieces and it's downloading loading those in parallel and then streaming that video right from your uh your laptop or phone it's incredibly fast it's incredibly Snappy and for all of that going on we're tapping into that latent capacity we're tapping into that extra GPU all around the world and it's delivering an end-to-end solution for customers so if you want to know what's next we have a Twitter live stream later this afternoon together we'll be talking about this a little bit more talking about the customer more in depth you can learn more by going to the documentation page for our lab you can join us tomorrow through the end of the week at the uh the sponsor showcase so with that if there are any questions please go ahead and uh we're happy to answer them space Oh like that um like price decrease and price Advantage like that's a miracle in the decentralized space do you think that's a little bit of Amazon just being monopolistic so I think it's probably two things and and you could talk a little bit about how the transcoding pricing works but uh the traditional approach to competing in this space is you build a building you stuff that building full of servers you stuff those servers full of hard drives you connect the internet and power and you call it Cloud that is an expensive thing to do and so to enter that space to compete against Amazon is a crazy thing to do when they have so much money and they have so much economy of scale so yes they can keep their prices High because they're one of the very few people that can do it at that scale and everybody else is competing in that space is spending hundreds of millions to billions of dollars to do that competition what web3 brings is the ability to not do that it's the first credible alternative architecture to compete in that space because what we're able to do is collectively tap into latent existing infrastructure all around the world 22 000 nodes and 103 plus countries sharing hard drive capacity with our Network aggregating that making it available to how many notes are you running right now yeah there's there's a hundred operators but you know tens of thousands of gpus around the world running and so neither of us bought a server built a data center or stacked and racked a bunch of power supplies and redundant internet it's like the Airbnb of of internet resources and bringing this together is basically a way to aggregate that resource allow those individuals who have excess capacity to participate uh as my buddy Kevin says they can be the cloud right they can participate in this economy and earn money while doing this and at the same time allow us to office a service that's differentiated and that's a much lower cost point and much better value for consumers and this is a great way for web 3 to bridge into web 2 use cases the long answer to short question yeah thank you that's it's incredible I mean I think another thing to really think about is a lot of people don't know that you know even Amazon as big as it is it only represents a pretty small footprint for the total amount of computation capacity out there in the world right it's less than 10 so there's a lot of computation capacity that we just don't think about on a day-to-day basis that are sitting in you know warehouses sitting in people's you know server racks and buildings right a lot of those things are not being used so um so so what we see on the live here network is not only the individual hobbyists that are you know running GPU miners but you know there are you know scaled and professional operators that actually have data centers and have latent capacity in those data centers they need to monetize that somehow so they plug into these decentralized networks um and that's a great way to to kind of hardness the the laden capacity around the world right uh yes I have a question um what are some of the successful uh business models that your customers are using um where end users are providing them Revenue so if I understand your question correctly that's who who is paying to use this service is that what you're asking for that yes that's correct oh great uh so we have a uh a range of customers right now we have one one of I think both of our most significant customers is um a company called innovo um and they're a a small to mid-sized video streaming company and they're one of those companies who they have that that that sort of narrow uh band of usage where they have a few hundred terabytes of storage and they're doing you know low to mid petabytes per month of egress bandwidth and there isn't really a a tier of pricing for them in the the main Market size for for them to be successful right if you're paying full Freight on Amazon and you're paying 23 bucks a month to store a terabyte and you're paying 90 bucks a terabyte for egress your Amazon bill for that use case is sixty seventy thousand dollars a month and that is just no bueno if you're if you are trying to create a streaming business however if you look in and say okay what does that look like if I use decentralized object storage and uh transcoding and you're looking at you know lopping a zero off of that getting anywhere in there from six to twelve thousand dollars a month or that same tier of service right it allows them to be very successful outside of video streaming for us we have a variety of use cases in web 2. um in addition we have you know backup storage and uh a number of things in m e workflow Beyond streaming media and entertainment workflows um and then inside of web 3 we're seeing a lot of stuff you know growing areas with with video transcoding is is probably the biggest for us because that's one area in web3 where there's a lot of data gravity and the other one is blockchain fast sync so we have for example a storefront called chainsnap.io it's uh with anchor the node running group so we have uh snapshots of chain states there that people purchase to spin up new nodes quickly and efficiently because nodes crash and you want to get one back online fast and the fastest way to do that is to have a snapshot of chainstate and so that's another significant Revenue driver for us because again if you're putting those chain State snapshots in Amazon and you're spinning up 100 a thousand nodes a month all around in the world you're paying that Amazon egress full Freight again it's 90 bucks a terabyte versus seven right so lap into zero off that bill if you have an event you need to restart a bunch of nodes fast you could have a hundred thousand dollar Amazon bill or you could have a four thousand dollar storage bill um you wanna talk a little about where you're seeing success yeah I I think you know the the kind of the cost advantage in this combined solution is actually quite significant for many businesses or many online communities that you know sometimes don't even have the ability to do video streaming at today's price point right so if you look at online video streaming it's pretty monopolized right there are a couple of big players out there um you know these big these big players also happen to be owned by the big cloud providers like Google and and Amazon and whatnot right so so so that that creates a big problem which is if you want to put your video on YouTube you gotta play by by YouTube's rules right YouTube we're getting to a point where YouTube kind of represents online video streaming which is a huge problem because they set the rule they figured they they said how content is being monetized and what you see is now that there's this um commoditization of online views right where people are trying to rack up millions of views per video on YouTube and each view accounts for a fraction of a penny and it commoditizes the the value of the video and and it and it and it starts to get everybody's attention right so I think There's an opportunity here for us to to start to think about like by um by knocking down the infrastructure cost it gives all the application developers an opportunity to start building niche video video streaming applications that actually serve Niche communities for the needs that they actually have right so in Novo is a great example they have a really Niche use case they have a niche audience and they don't want to put their video on YouTube and you know but if they were just using Amazon well that cost that cost of running business business is gonna rent them out of business right so um so um so yeah like I think in the next couple years what we're going to see is video streaming continue to grow the amount of video on the internet continues to grow and the amount of Niche applications Niche use cases uh will continue to grow and again that price point and and that value enables those smaller parties to actually have a real business model where they just don't have an option today this provides that solution which is great and you see that popping up in areas where there are oppressive regimes and people don't have access to certain um uh properties and and assets they can use these services to be successful over here got it hi uh so both of you describe gateways that are kind of web 2 like that make it easy to use both of your decentralized networks are there other gateways that exist besides the two that you mentioned and what's the plan to ensure that if one of those gateways gets uh attacked like legally or otherwise that the network Still Remains accessible for that kind of price and that ease of simplicity I'll talk a little bit about that um it it's true that you know gateways are are kind of potential single points of failures right um so so right now in the Life peer Network Live peer studio is the main gateway service that that allow people to access the life of your network but there are other services in um you know application developers in the LifeCare ecosystem that operate their own gateways right and they you know they know how to kind of um if they have a little bit of devops chops um they can they can spin up those Services we have um we have this thing called the life peer Catalyst which is um the the general Gen 2 of the life pure node and that Catalyst software is you know a video transcoding node but it also is a full media server that you know does everything from video ingest to service to you know to everything right so if you want it you could run you know a globally distributed Catalyst node cluster and run your own run your own Gateway service and you know and and of course in uh for for live peer Studio live here Studio has to follow certain rules right like if we got a dmca take down uh notice from the government we have to take certain content down right but there's always this alternative for uh for for alternative gate Gateway operators in the ecosystem yeah we're in exactly the same space right so we run gateways that are highly available globally distributed multi-region and all that good stuff um and again because we are running them as storage Labs we have to play by those rules as well that the dcma takedown notices and things like that and and um you know just as a responsible operator we have to do those things if someone else wants to run a storage satellite which is the entry point to the uh creating access credentials and all those things it's open source code and you could someone with reasonably good devops chops can do that today we have a a video on doing that and we have a bunch of documentation in the works to make it easier for third parties to go run their own and same thing with the edge Services which is the S3 compatible Gateway which talks S3 on the front end and storage protocol on the back end and does all of the encryption Erasure coding all that good stuff anyone can run one of those as well so you can also split the baby and go use any Gateway with any combination of satellites so if you want to run your own satellite but you don't want to host your Edge Services you can use your satellite with our Edge services or any anyone's hosted Edge services or vice versa stand up your own Edge and again the advantages there are just simply control cost if you want end-to-end encryption and you want to have absolute maximum control of your data all the pieces are there for you to go do that all right so we have 17 seconds left and we were told that was our last question so thank you very much for joining us again you can find us in the storage sponsor Booth uh the rest of the week um our social info is up on there so you can tap into the the deck when they post it and all this information is in the storage documentation and it's also in the live pure documentation so thank you