Streamr 🛠 Getting started with Streamr - Matthew Fontana

[Applause] [Music] hey everyone uh welcome to this workshop about uh streamr um in this in this workshop i'm going to explain uh what the network is how to use it jump into some code give you some ideas for what to hack so let's start off with some background theory so the network is a pub sub messaging system so off chain communication it's not a blockchain but it's built uh with a lot of the primitives of ethereum that uh can kind of natively be sewn into your ethereum based decentralized application it's fast and scalable so typically live data flows uh through the internet via you know google cloud some some kind of big data center where it's you hand over your credit card details and uh they provide this decoupling service between publishers which is usually your app or yourself that is publishing data instead of making direct connections to all your subscribers you just push it to one centralized point which ends up being some company's data center and they handle the the data propagation with the streaming network it's uh that middle that middleman is replaced by a topology of nodes that can be run by anybody um and uh yeah there's quite a diversity of the types of nodes that uh can run on uh that run the the streaming node software on the network and yeah here's a few examples here so what is the unlock here um so as i mentioned it's a fast and scalable messaging network for web3 uh all the good things that come from decentralization are here censorship censorship resistant and uh permissionless of course uh secure so every message on the network is cryptographically signed by the ethereum private key there is end-to-end encryption if you want it but it can also be you can has also have open data streams as well and the one part so this is classical networking technologies the data flows through web rtc if you're familiar with that p2p technology but the one touch point that we do have on the blockchain is the on-chain stream registry and that holds the uh basically the accounting of the streams and their uh access control policy so which ethereum identities can publish and subscribe to the streams so it's on polygon at the moment and it's the source of truth for the network uh it's it's a serverless technology so the nodes run anywhere that javascript runs including the browser and it's free to use as well so there is one very small costs of ascent or two to create the stream on the polygon blockchain and any sort of updates to the access control will require essential two of matic you can reach out to us for this medic if you wish and call us to sponsor this transaction um so then it would be purely free to use but at the moment there is that one tiny little cost up front of ascent basically and uh you can monetize these data streams if you wish so um you know owning your own data is a cliche but on the network it's it's technically provable that you do own your own data and you can sell access to that data if you wish and we've built what's called well uh there is a framework available called the data unions framework which is a crowdsourcing uh crowdsourcing and crowd selling way of of monetizing that data it is external to streamer but if you do build a data union then that's totally uh eligible for any sort of hackathon okay so then there is the access control which i mentioned so ethereum identities can have anyone or any combination of these permissions which grants them some sort of uh role in in the uh stream so what can you use this network for so there's kind of two key use case scenarios that i'll that i'll talk to decentralized communication and data sharing and monetization communication is any sort of generalized messaging between humans machines scripts iot objects nodes in a decentralized network anything you can imagine this uh bi-directional messaging can all flow through the the network um one one cool use case of course is quite hot at the moment is uh chat over over these web3 networks you can do this natively on streamer very easily so this is a chat app that we've been building um ethereum based every single message is cryptographically signed by the metamask account and the the group the group chats uh are essentially listening or uh are controlled by the access control of the streams so every chat room is a stream and the the participants in that stream are publishers and subscribers uh as per the access control of the stream um on to data sharing and monetization so you can share open data about absolutely anything it's a great thing to do because you can build the developer ecosystem on top of this data and essentially data wants to be free and as if you can make data accessible to as many people as you can that's there's a lot of public good benefits to this but as well you can also monetize the data if you wish a subset that you consider to be kind of premium you can make pay to access data streams that earn revenue when someone buys access a time-based subscription or a permanent subscription to some sort of data stream taking that a step further you can create very expressive data dowels that allow anybody in the world to contribute their data into a shared fire hose of of data that has this great data density that could be used for all sorts of kind of ai and machine learning and these data union smart contracts they can they're programmed in a way that when access is is sold those that those funds go back equally to everybody that helped contribute to that data set so it's a way to incentivize crowd crowd data so some hackathon ideas um so group chat on lens protocol would be really fun so lens protocol is like this uh nft uh social graph and we have a instant messaging group chat so uh combining the two making the access control policy uh talk to the the lens protocol is is actually quite low hanging fruit and would be pretty amazing to accomplish in this hackathon so that is that that would be awesome uh any sort of decentralized uh wallet wallet communications so if you know wallet connect uh you can kind of do the same thing with streamer very easily you can set up that decentralized secure pipeline between a wallet and a dap that are not necessarily in the same environment you can do those signed transaction and request communications over a stream then there's off chain multisig chat and conviction voting so i think gnosis safe here you have a bunch of uh multisig participants that are looking to sign proposals but they need to talk through those proposals so um having an off chain chat where you're absolutely certain that the the communication is happening over a secure channel and they can kind of get this pre-conviction before the final on-chain vote is very uh very useful so that would be awesome decentralized network communications so this is a big one but blockchains can actually leverage streamer to propagate the blocks of of their chain on on streamer um this is this is a very legitimate use case and uh not only this but in other uh decentralized or peer-to-peer protocols there is often a need for some sort of signaling server signaling server adds ad centralization but if you do signaling through streamer then you can combine two decentralized networks and have a end-to-end decentralized experience another interesting one is to validate validate decentralized wireless networks with your own proof of coverage algorithm so we've seen folks from the helium community in the mappers community specifically that are very interested in bringing in that coverage that real world coverage data network coverage data into the streamer network and it's a fantastic technology choice as the data transport to bring into the proof of coverage oracles of say the helium network or some other decentralized wireless network so this this is another um fun open open data use case essentially and then there are others like your metamask excuse me your metaverse character is moving through the metaverse it's creating this data exhaust so maybe push that onto streamer and potentially monetize some parts of it uh one that's not included here that probably should be is some notifications protocol that would be fantastic uh it's quite similar to chat but uh it's essentially a mecha you can have the streaming network as the data transport layer for any sort of scalable notification so if you wanted to send a notification out to a million subscribers that all have a certain token in in their wallet this can be very easily accomplished using streamer uh you could build multi multiplayer gaming on streamer as well uh it's uh sharing the state sharing the position of characters we've had uh chess apps built in the past on streamer and you can think of all different ways to include a streaming network there and you can also build uh these data unions which i uh mentioned which is these uh monetizable pay-to-access crowdsourced uh fire hose data streams okay so that was uh that was the start and let's uh get into some some code and some real world examples now so i'm on the streamer network website here streamer.network in the top right there is use core which is our connect so i'm going to um sign in on metamask this is the polygon network so it might ask you to change if if uh you're on the main net and uh here i see my dashboard um a list of my streams that i've already created and for this session let's create a stream so i'm to call this if online 234 if you have a ens identity you can also see it here and um [Music] then your stream id we will be for example matt fontana dot eat forward slash is online forward slash 234 but i don't have an ens identity on this metamask account so i'm just going to create it as is as mentioned this is an on-chain transaction it'll cost one cent and it'll take a few moments to mine what i'm gonna what i'm creating here is a stream that we can it's like a topic in pub sub if you're familiar and we are going to push some data into it and subscribe to that data in a different window so now we saw this the transaction was mined and the stream was created and uh i can just run through quickly some of the other things on this page there is code snippets uh which we'll we'll jump into shortly uh status uh don't worry about status this is just um this is just a ui thing most streams remain gray so you don't need to worry about making them green we have a live data preview so data live data points will flow through here emphasis on live so um yeah make sure they're live data storage so stream it does offer storage for four streams uh but it's a little bit centralized at the moment so uh we only have one one node here um and you can choose how many days uh the the data will be stored for so this is another transaction it's totally optional but it does give the resend ability so if you need the last message of a stream then you can get that there and we will be decentralizing parts of this in the future and then there's uh stream partitions which is like a sharding for very high volume streams you don't need to worry about this okay so next let's jump into a project that i have started so this is um very very simple so it's just a node npm project so you run npm init and npm install streamer client and i have the dot m environment package as well just to help me out storing a private key but that's it that's it just make sure the streamer client version is at least uh six and we are good to go so i have two scripts here a publisher and a subscriber and i'll make this a little bigger so what i'm going to do is going to go into our code snippets here i'm going to copy the light node js and let's paste it into the main function so let's have a look here so we don't need to we've already included this section so we don't need that it's just requiring the library and then for the private key i'm pulling this from the environment uh file so process.m dot private underscore key is how i've uh named that uh uh variable inside that file and you can look through the dot mb npm package for docs on how to use that if you are unfamiliar otherwise you can just paste in the private key from the metamask account that you were using and that's totally fine as well has a string there so this is our publisher script so we don't need the subscriber i'm going to cut that out and paste it into our subscriber function for now back to the publisher let's just have a quick look at this so instead of just sending one message let's send a message every every second with the set interval function great so the publisher script is ready so this um brings in the stream of client package so it's actually running a light node as part of your application uh we're authenticating with a private key that created the stream and uh then we're calling streamer dot publish stream id stream id is uh this guy here and we are publishing a data point hello world so at the moment uh data points on stream need to be of of json type so this is you know field hello world as the value and we repeat this publish every second in the subscriber script um we we do something pretty similar it's actually something i've missed out here let's need to copy over setting up the object and so um we do a streamer dot subscribe to this and we need to do something with the message so let's do something very unimaginative let's console log it great okay so now we're ready to run our publisher script and yeah and then we should see something happen in our preview as well so i simply run node publisher.js and a few things should happen from here so first we get some logging of connection into the network so this is very normal you might even get some some warnings from the rpc um this is all normal and fine but metamask also popped up as well on our preview screen and what this is is a key exchange mechanism on on the network so the publisher is basically asking this subscriber hey are you um are you actually authorized to view this this uh stream and so metamask says uh yes uh here is here is uh my signed uh key which says that i'm a valid subscriber on this stream so i'm going to click sign and uh here we are so we see the messages flow through as a as a subscriber so that's great and i can also show that if we run node subscriber.js we can create as many of these subscribers as we want and they will ingest that that rule stream of data so every second a new hello world is is being propagated from this publisher node script to our two subscribers so next thing i want to show you is that we have a network explorer it's sometimes useful for debugging and sanity checking it's at streamer dot network forward slash network dash explorer i have i can simply paste in my stream id and i'll click on the first returned result and it shows three notes and so this is the publisher node subscriber node and it's also this uh this browser tab so this browser tab is a light node in the decentralized network and it's helping it's helping to propagate data around so it shows these three nodes in zurg in switzerland which is where i am at the moment so that is publishing and subscribing um you can do a lot more with the streamer client so i encourage you to have a look at the docs play around with the access control while while i'm here one other thing that i can show you is that we can make the stream publicly readable so in the top right section there is this share button it's currently set to private if i set it to public and save it'll be another transaction but then anybody will be able to view or subscribe to the stream and in in that case the end-to-end encryption is is turned off um but you still need to be a valid publisher so if you want to add new publishers you can go to this share section and add in publishers or you can do this through the streamer client as well some quite straightforward to to change the permissions and uh yeah do all sorts of things here uh so the next um application is the streamer broker now you don't need to use this of course but uh if you are uh bringing in data from a different programming language or device that cannot run javascript or some sort of node environment then you will need to run a broker node and open up an interface websocket http and mqtt interfaces are all available and essentially from your application or device you start pushing into these interfaces of a broker node that you are running and that becomes the entry point into the streamer network so we have docs here streamer dot network forward slash docs streamer network installing a broker node you can run the broker node through docker image or npm you can even stake some tokens on here and earn some rewards uh but the developer use case is to then uh interface into that broker node and um uh use it as an access point into a network into the network so that's um that's the stream of broker and we also have the cli tools so this is a handy debug tool so you can do in the command line streamer stream subscribe and then your stream id for example and that can that can help you debug any sort of issues you might have okay so that is the streamer client the streamer broker um next i wanted to show you the uh our github it's at uh streamer dash dev so the action happens in our network dash mono repo repository and you can see the the packages uh here if you want to dig deeper into the code um and we also have an examples repo so if you have some trouble getting started we have some ready-made uh flows and tutorials here to to help you out uh lastly i can give like a quick little quick little uh demo of the chat up so um every single message here as i mentioned is is being signed by the metamask wallet we have a we have a clever way of doing this where we have a hot wallet stored into the local storage of the browser but uh yeah it's fully decentralized and and yeah you can build quite um quite amazing things on this token gating is something that we're working on but if you want to build your own version of this this is also really awesome and this would be uh very um very useful or the the way to connect to the lens protocol with their nfts so these chat rooms could be gated by a lens protocol nft for example and lastly uh to mention there is the data union dao which builds on [Music] on top of streamer so if you build a data union it's very much a valid submission and big businesses have been built on on this framework swash is a browser plugin uh demo is a is a physical uh raspberry pi style device and you connect into your car and and uh ingests the car data and and fi finds useful ways of uh sharing that data and so yeah this is the uh smart contract flow for these data unions it's a bit too much to to go into this uh session but i guess the takeaway is that anything is possible with these composable smart contracts that interface with the uh network and access control policies yeah so i think that's um that's about all um yeah hopefully uh you've learned a few things about streamer and uh yeah we can um uh yeah see you see you in the in the discord okay thank you