Ethereum Swarm Workshop - Swarm Fundamentals - Part 2 - How to Install Your Bee Node

are you here yes yes sam [Music] up to you i will stop sharing my screen yeah hello uh so this is we we come to the part uh uh the practical part of this talk uh and uh where where we are going to uh to run our uh swarm nodes and as we all know uh we are all contributing to the open source so um we will start with the source of the b node and i will share my screen so do you see my screen uh my browser currently yes i see and the idea is that here everybody joins you right yeah so uh uh so um first uh we are going to uh not to trust anything and uh we are going to build uh our node uh by ourselves uh from the from the code here uh there is another reason for it for that uh uh because we have the uh the latest release 28 days ago and since then we added a bunch of new features that will be actually released today so version 0.3 will be released today and we will be using the master to build our node and to have all the cutting edge now cutting edge features you can also see that we have a link to our to the documentation this is the documentation for the previous release and i have uh locally here running uh the up-to-date uh documentation uh but you can also um you can also uh go to uh to this repository and build and run your own documentation it is pretty straightforward but but i i already have that covered uh here running and and i will share important information uh video here so uh to build our to install uh from sorts uh we will need the go compiler git tool and make which is actually optional for for anyone that that has a problem with this so not to install or cannot install or would not like to um to have that that executed we will provide during this this demo binaries so i will actually build upload the latest binaries and you will be able to run your own node um so let's go you can go and where is the chat here i posted the links to the eva sphere b and even spear b dogs already in the chat yeah so uh cool so if you uh don't have uh golonk uh you can download download here and and install uh also uh git uh and and uh i will uh while you are preparing uh i will just clone the code uh jano is one question the b dogs gets a 404 the repo is not public the repo should be public ah okay so maybe you can link this one yeah so yeah this is the renamed uh repo this is my mistake then uh so uh this is the the correct link uh and uh and uh we will we will clone uh this and by the way if you're having a question while janus is working your honors can't see the chats but i will see so please feel free to put up your hand and i'll try and interrupt janos if possible yeah yeah that would be ideal so uh interrupt as soon as possible uh and we can execute a simple command make binary and we will build the binary for the current operating system and architecture it will be built in here so uh it's there and you can look at the the documentation make binary and we will right away copy it to the other to our part so that we have that installed okay and just to make sure that we are uh running uh that binary that we just copied uh so when we uh install some prop uh some some tool uh we can just run it and by default uh b uh presents the usage and and this is uh this is uh uh these are commands that that we can run uh we can start our node uh check the version uh print configuration so not much but start is the uh is the the most important let's uh uh check the version so and let's get out from this directory so we have our version we are prepared for 0.3 and this is the current uh git commit just to validate that that we are running uh with this state of the code i will paste that in chat and uh yeah so actually uh so there is a question about go version uh we uh we support only uh uh 1.14 and 1.15 which are actually uh upstream uh which have upstream support from the from [Music] compiler development uh and uh you would need uh at least 1.

14 to uh to be able to compile the code um and yeah so we checked the version uh we can then check what are our configuration options because if we start our node we will connect to the uh to the boot node which is the main cluster with network id1 so because our main cluster main network works with the release 0.2 at least for a few hours more we set up a dedicated cluster for this workshop with the latest master installed there so that we can all connect and and and and run uh the latest features i will go yeah we've got one question uh from presonium project uh asking the binary file is for windows 10 question mark uh the binary file uh is uh the binary file is for the architecture that you uh that your operating system is yeah exactly so if you are um running windows 10 and you do make binary uh then it spits out the binary for the os that you're having yeah but it should be byte compiled compatible with the other uh windows older windows source uh so uh we we will produce the the binary which should work on on on uh eight or seven uh and um yeah um yeah if you have any problems uh just just write and we can jump on on it to see how to run and yeah so we have a bunch of options there i will just point to the most important ones so we have addresses to listen so this will actually uh uh spec this specifies the address for the main api uh be main api which is by default on 8080. we have a debug api address uh this is the api for inspecting good uh b internals and it has to be explicitly enabled because it is it should not be exposed outside of your node because uh it can uh it allows uh for uh some malicious uh work by third parties uh and and this we have to be careful with that um we have network id which is very important now because network 81 is for the for the public cluster the main domain network uh we have p2p address and p2p address is the address on on which p2p uh uh listener uh will be started in order to communicate uh with other nodes uh and uh actually uh this is the uh the the main application uh interface uh debug api is the operator's interface for the application and p2p address or p2p listener is just for not to not communication we will actually not use this uh this uh this api at all uh and um uh yeah uh by default now swap is enabled but we will disable it for this cluster uh uh just not to go into the uh into the incentives right now so what is for those who don't know very brief yeah so uh swap is uh uh the the first step to incentivization uh so uh bandwidth incentives uh part and and uh we will uh we will see if we can cover it by the end of this this talk uh and there is another uh uh useful well uh nice to have uh variable defined is welcome message this is the welcome message from onenote to another and you can you could you you will see in the uh in log messages uh uh these welcome messages uh from other nodes uh janos i have one uh person who is putting on the hands i don't know if some of the presenters can give him the mic i for some reason can't yep you can talk please new script you can unuse no you can unmute yourself one two three okay please continue janos yeah thanks so uh well uh i think that this is uh just enough for start uh you can look at the uh configuration options uh in the in the documentation but i wanted to go through uh most important one so uh another thing you can provide configuration uh with command line arguments uh with environment variables and and the configuration uh file and um uh their names are all in relation and uh you can you can you can look that up uh here not to spend too much time on that i prepared uh one uh one uh one configuration file for this workshop uh that can be uh used by uh by you and that can be uh that that should be adjusted a bit uh and i will do that right now so i will share this uh this link with you and you can save it to your disk so i'm in the workshop directory i will use this command oh so i will download this to the workshop and and have this uh here so uh what i set is the uh boot node address so uh what what is the boot node boot node is uh the node in the network uh that will share information about about connectivity to other nodes so they are addresses so that we can join the network uh this boot node can be any uh um uh address of the uh it can be addressed to any net any node in the network so we are we will just use uh the one that we set up uh and uh and and uh and it it it is uh in some way uh uh a way of uh of uh centralization uh but actually we do not need to have uh this defined uh for for the on the on the cluster that is running internally on our infrastructure you can use uh you can ask for any of your peers uh to get uh to get uh the the address uh p2p address in order to set it as the boot node and uh and um and and join the network so uh the only trust that you would need to give is that the person you ask for is running the node that that you can rely on we will use this network id 2809 which is a current date for this network we will define a data dear you can set it to whatever directory that you that you want uh i'm just using my tmp and all of the state so all the chunks all the information about other peers and all meta information that we need to to to keep for sworn off to run will be started there and also uh very important by default uh for now uh keys private keys uh of your nodes uh will be stored there uh we will enable uh debug api uh in order to inspect i just set this password here for convenience but you should be careful with the with storing your password given that this is the temporary network it is okay but if you don't set the password here you you will be prompted to enter it every time when you start your uh binod uh i am setting verbosity to trace because i want to see all of logs which may be quite a lot but for for a demo it can be useful for you to see what is going on as i said we will disable swap and we will set some nice message so i will just amend a few things there in order to i will from so that you can see that you are connected from uh to my node and that's that's it then i will run b start and provide workshop like this and we are starting our node uh we have uh the possibility to use uh nut for better connectivity uh this is why we waited here to discover uh possible uh not by our local router uh before that we created a private key and derived the network address this is called the overlay address so this is the address of our node in the network we have the public public key of our node and we we are sharing disks in this underlay addresses so underlay addresses is the are p2p addresses that other nodes can connect and you can see that these three nodes are actually not possible to connect to because they are in the private network but nut manager managed to open uh open uh a hole in our router so that that we that other others can connect to my node so if i now share to you uh with my node address this one and you set it as the boot node you will be able to connect and you can see that after that we are starting our api listener debug listener and we we started to connect to our uh to the nodes in the network uh and we are actually uh seeing the uh welcome messages from the nodes in the network this is the default network uh the welcome message in the in the in the public cluster and we have uh and we are running so uh as you see uh i connected to atlas uh node uh already uh uh he is uh uh he he so so uh let's see if there are other yeah uh from goron so this is this is very nice um and uh let's leave the note running we will now just look uh what our connectivity is so we have our debug api and we go we can go to the api reference uh debug api reference that we can look at what we can see connectivity uh a bunch of uh a bunch of endpoints there inspecting balances checkbook validating some things around chunks status settlements but we are now interested in connectivity uh i just want to share to show to you that that we have uh open api documentation for these endpoints and that you can browse but for now i will just well uh type some uh http request here um so 6060 and and i will just check my addresses oops let's pipe to jq and you can see that this is the overlay underlay addresses and public key you don't have to parse logs to get those you can get this and yes i have a personal question about the public key yeah um is this my ethereum uh address can i use this one also to interact in the ethereum blockchain or not no uh this is uh this is a just a serialized uh uh public key uh ethereum address is derived by hashing uh this this data so it is actually a different uh different uh uh array of bytes um we actually so just a a spoiler uh public key uh can be used in communication to target uh uh target uh uh pss messages uh to encrypt them for specific recipients but let's not go into into into their uh right now uh but it has its purpose uh so these are these are our these are mine my peers but the list is does not say so much so we have the top topology endpoint which provides a bit more information so this is the topology in the debug api the same address we have the population of 12 nodes uh at least i am aware of that so uh my my boot node communicated the information about 12 nodes uh and all other nodes uh uh gossip uh this information so i i am connected to ten of them uh uh for various reasons one may be that it is just enough to construct a healthy kadamlia and we have here uh kadamlia beans so we are categorizing our nodes into into into into beans by their proximity uh so uh you can see that uh that most uh further nodes are there and then uh as we go to the uh being five this is the most the closest node that i know about so this is my neighborhood uh and uh not to go too much in in in theory there this is just something that i wanted to show to you that uh how we are connected uh to the network uh and we are dealing uh the overlay address so these are swarm addresses here we do not expose here underlay addresses which are p2p layered addresses okay so now we would like to share some files uh and uh you can uh for example uh for any of you uh that that are using mac os and and did not want to build b i can upload that binary so i can use uh the api which is files api and i can use uh this is the multipart form uh upload with curl uh and this oh no b and i am actually submitting a file uh or with multipart and encoding to the files endpoint as the post request and we can look at the logs where we actually yeah i have one question um this question is from victoria thought saying which port should i open on my firewall for a swarm connection with the autonomous you actually don't have to do that so you do not have to be exposed uh in order to be connected to networks you to the network you can connect to other nodes which are actually exposed already uh but if you would like to have better connectivity you can open uh port 7070 from your your well your computer on your router uh to be accessible uh by other nodes so in in that case um your node will be advertised with that that underlay address for other nodes to connect but it is not required uh you can connect to other nodes but uh none of other nodes will be uh will be able to connect to you if you don't do that uh so i see that there is a q a question so yeah uh yeah so uh we uh uh yeah this means uh that one of the protocol uh has a problem uh delivering uh a chunk to its destination uh i also see this and and and it may be uh due to some network problems there are a lot of things that that can happen there so uh a node this pier which is actually probably might be is it i think that this looks like me um got a problem uh to store maybe because i'm now uh uploading uh uh this uh this file uh and uh so we can investigate on that uh but uh uh yeah we can uh keep the network uh alive uh for the few days uh and that's uh yeah uh that that's not a problem so uh to get back to our uh to to to to to our upload uh we did upload a larger file and and it needs to so my node i i uploaded a file to my node and my node needs to push these chunks so first my file is split is splitted into chunks and every chunk has a dress like this one and every chunk can be measured uh by proximity to other nodes to the nodes in the network addresses of the chunk and addresses of the nodes are in the same form so they are the same vectors in the metric space and we can calculate uh which chunks are uh closer uh to which which pairs so we will deliver these chunks to their closest peers and this is what my node is currently doing so i am pushing chunks so this is still doing my internet connection upload speed is not so great so this is uh this is still running but every chunk is pushed uh to the pier that i know that is closest to this chunk and if this spear knows another pier is connected to another pier uh which is actually closer uh that that chunk will be forwarded to the closest peer and that way we are distributing uh distributing our trunks in the network so that our retrieval process and retrieval process is part of of the process when we download the file our trunk retrieval will be able to find the chunk by looking at at which which piers are closest to the chunks so this is a reverse process uh and i'm still uploading this uh yeah so uh if you uh would like to to download this uh this this is the file that i [Music] this is the file the address of the file that i i uploaded and when i push this chunk you will be able to to download it from the network okay uh i would like to go into something that is uh and this is the the api uh reference uh so uh this is actually uh storing uh presenting the the information about the api that that we have uh and we use the file api uh to upload the file this is uh slash files and and and we can upload specific bytes get specific bytes uh well by the error array of bytes so this is arbitrary data uh chunks uh specific chunk but this is this is more on a lower level if you are a dub developer uh you can use these two uh as uh for your distributed application to uh to implement some some specific logic because it is a lower level api and file api is actually storing metadata about file name and content type so this uh this is much much more useful for uh for sharing uh files and even more useful is to share the collection of files so uh we can uh upload a tar file which contains uh directories of files uh in order to access them over the the api and this so you can see that pusher did uh did its work so this this file is now pushed to to the network and you should be able to download it uh by your own notes so uh so if you uh if you go and uh and uh execute something like curl um oops i forgot the files so this is files api uh aha file exists okay so um yeah i already have the in the chats there is one jpeg image maybe you can use this one to illustrate downloading a file let's do that so pasting the address of the yeah okay and yeah so nice to see you attila wonderful uh and uh that is uh uh that is a nice demonstration from from attendees um i uh i would like so uh so we have uh five more minutes and let's uh uh let's upload a website uh and let's do that um without as as quickly as possible uh and i have here some code shared by uh the uh uh by our colleague dan uh it is the yeah uh it is uh uh an application uh that can be hosted in uh in in in swarm uh but it is uh uh basically a distributed application uh to use the api to share uh share snippets uh and uh i will just uh i already cloned this is the republic janos yeah yeah okay so dc yeah thanks this is the the and what we will do uh uh we will tie this and yeah that's it uh we created this site we will uh this we packed this site so we see uh files there so it has some assets in the ssd directory uh index.html referencing these assets and we can upload i already have this not to type so we will post this tar file this is the tar file uh to our uh dearest uh endpoint uh we will set the header swarm index document to index.html we have this index.

html and we want uh to serve this index.html on the on the root of our uh our uh address that we will receive so not to be able not to be required to uh to to type index.html and share those types of links so we already have our nginx and and the web web servers set up for this but we are here uh explicit and uh yeah let's upload it uh this is not so large uh we got this address and uh bzz so we will use bcz to access this and if you can see in the so i will just refresh and you can see that we are downloading our assets uh with um with this url so we have easy uh endpoint uh that can serve files under under this disk hash so this is a very nice application that you can that you can use so i i actually don't know how to use it but it is a nice presentation that that it loads uh uh the uh required assets um maybe i don't know if dan is here uh if don't do a one minute pitch of this app if he is here if not yeah dan will uh host uh the second workshop uh which will start right now uh but uh but yeah so this is so uh another thing that i wanted to share is that actually uh we have a few demo videos quite short uh on in the ethereum swarm youtube channel and you can watch that uh we actually used uh this uh swarm encryption uh in testing a lot uh for uh for testing the current uh release uh so i don't know how much we can step into into uh the next hour so a ring should we stop here or or continue no i think janus this this was perfect i think we can stop here we prepared a bit a bit more um but i think this is all good for the for the next workshops um so maybe just a couple of uh final notes to to all participants um thanks again for joining uh we hope that this was was useful and we hope that that's that this workshop triggered you to uh to start looking into our documentation and read the book of swarm and uh basically uh become part of uh of one of the first adopters of this revolutionary new way of building applications on top of the internet as mentioned as the swarm team we are we are very much looking forward to to collaborate with you collaborating both so far level uh if you have a cool idea for an application uh share it with us uh but also if you have if you just want to discuss this and maybe you want to have some more philosophical talks about what we're actually building um we believe that this is also highly relevant so definitely not only tech savvy people are welcome to to join us um maybe one more request is the link to our telegram channel the orange launch i don't have it at moments but maybe somebody can join this share this in the chat and then of course we have another workshop which i think is starting right now hosted by them and um this evening we are having the release of the swarm beta so if you want to hear more about the latest features and the development of our ecosystem announcements then you should definitely join there as well that was it for this workshop the next workshop is starting in half an hour so take a well-deserved break and [Music] yeah i don't see the link of the [Music] telegram group so i don't know invite new members can i do this no i can't okay um you can also reach us on uh get up you always link to github and i'm sure you all find your ways to reach out to us if you really want so thanks again and i hope to see you all very soon alright