Liberate Data Week Hackathon - Day 4 - Free Press & Publishing

[Music] unreal [Music] changing my ways counting the days i'm so messed up running away starting to fade cause i know the truth how could i know [Music] just existing [Music] [Music] coming home where i belong [Music] angry [Music] something's missing breaking the rules [Music] hello hello everybody welcome to the fourth day of the liberate data week uh today uh we speak about uh free press and uh publishing um it's a bit uh shorter program uh planned for today uh compared to the previous days um more to just give like uh quick updates what are ongoing activities around this wits one as the first talk or the discussion of today's session a discussion with a special guest was planned unfortunately these guests won't be uh available it's not possible for him to join us today so we will make up for this at the next occasions but we do have also uh some new things more uh uh then we announced um at the end at the end we will see already some results coming from the hackathon itself so uh the plan uh for today's session is uh we start first with dan who is going to give us uh explain us how uh how to host a website on swarm uh followed by uh zahur zakur will talk about uh how we are working on getting open street maps onto swarm and uh last speaker for today will be a quick demo from today who's been working on um pharaoh's ui web ui i had the chance an opportunity to see it yesterday and was completely blown away and yeah we should be sharing it uh with uh with you guys so um maybe uh now we can just continue uh then are you here please join me um i am here but i can't start my video hi guys it's all good it's all good uh no no worries okay even better even better so maybe then before uh you start with the talk it's uh i think what what is also important to say you are probably the person who uploaded most to swarm in most ways uh for the most longest longest time so you're like the the man then the man for this kind of things and uh how do you see it now in the past months even years uh this changed from the old sword network to the newest one network and where do you see it going well i've just been um spending some time with attila who's the bjs lead and i think like really that is the um it's the most exciting bit really so these javascript tools are something that we never really had before and so we were kind of like making do um but now we've got a bit more juice behind us then i think like it's gonna start to get really exciting really soon um it's it's interesting moving from old swarm to b because that b is a lot more efficient in a lot of ways but i think a lot of the edge cases have not been um found out where a swarm is a bit more of a mature software so i think we'll see in the coming months um everything coming together and coalescing into hopefully like a highly efficient and mature um infrastructure for building gaps and then to be able to do stuff like this what we're talking about today in terms of providing censorship proof and sovereign ways for people to distribute their uh important journalism and so on yes yes so you know it's like we both know like um services like netlify or uh ones and my understanding is a lot of this let's say front-end movement guys who are into static websites for example they do it because of speed do you think that song besides providing this censorship resistant uh platform uh will it be able to compete in the future uh also on the speed side of the things i mean i think that's always going to come secondary um you know it says in the book that b-line delivery is planned but it um with that then you lose some of the you know sort of taught onion router kind of elements of swarm so we we're never going to try and have this initial retrieval to be as quickly as possible of course like once you've got it on your local machine then it's insanely quick and in that way that will become much quicker so it depends really on the exact use case and i think as we sort of like start to re-architect where data is stored and how it's moved around the internet then we'll find that the these sort of like benchmarking models will completely change because it's it's a different paradigm and so it works in different ways awesome awesome thank you thank you for the answer so uh i won't keep you up more because i know you also are on a tight schedule and maybe we just switch now uh to to your talk how do we get a website running up on song indeed all right well um i was taking it to the wire on this one that's how you say i'm still kind of like halfway through preparations and and even the stuff is being built as we're using it so what i'm going to do is i'm just going to give you a general rambling overview of where we're at awesome awesome but i mean hopefully that that she's more in line with the hackathon element exactly that's the best we can ask you know that the info comes straight from the source of my desk this is what my guest looks like sorry for this [Laughter] i'm all right switching my camera off the floor okay cool so um let me try and make this rather simple eco here and you guys so this is for anyone who's not seen it this is what the ens domains app looks like and i've got this connected at the moment via metamask to the girli test network because ens updates cost around 15 a time at the moment so it's kind of expensive enterprise to do testing on the um on the main net but fortunately ens also works with girly go into my account you'll see i've got uh two domain names here and we can you can actually like buy these on girly as well search for them just like the normal domain site registry i won't do this now because it involves a one minute wait maybe kick it off now in s domains is is pretty good for the most part it works well i'm kind of one of the crown jewels of ethereum really before the d5 invasion so anyway if you go back and have a look i've been experimenting here with this domain that i bought earlier swarm swarm swarm.eath and then then if if that completes okay i'll show you but at the first time you come through this you'll have to deploy what's called the resolver contracts here and then this is the bit the links with swarm the content hash so if we go into the editing bit here you'll see using the bzz protocol and this calls and hash and there's other you can do tor and ipfs also now this hash relates to the manifest of the website once it's been uploaded onto swarm and there's bunches of different ways to do this uh the old way in the way that's currently in the docks is to go into the directory others so in here and then hopefully do something like this so tar the website without compression yep then upload that as you can see i've done that quite a few times before and this is just using the line pal amendments so this here you can specify in the headers the index document so normally that's index html right and also an optional error document so that's like if something uh if a file is requested it's not in the manifest then render this and then here we've got it index html which is how you deal with uh rooting for um single page apps like react to the song where the errors are dealt with with the internal router so you see we've got a reference here and the idea is you can navigate to that reference that gives the website here so what we would have done is taken this hash put that in here and update it messmaster is playing nice yeah and then oh i've got my beat set up at the moment the local version here you can see i've got the resolver options to be using the girly guardian point so hopefully you can see that yeah and what that means is it's using the girly ens contracts which interact with the girlie ens domains app and that gives us that means that when i go hopefully on here to thing it should take me to the right website so then the process for deployment here is you know you just update this somehow tar it again curl it again this time we're going to get a different reference so you can see this this hash is different to this hat because it's uh uploaded a whole new directory structure manifests and all of this here we would see two exclamation marks and you come in here and change the hatch now this is this is the step that take costs at the moment 15 worth of gas on the ethereum the good thing about this is there's no wait no need to wait for propagation with as is with normal dns once the transaction sinks the network it sinks let's see we've got two exclamation marks here so it's done the update and as i've said multiple times now this is extremely expensive so we've got a solution for this and so we're still working on it it's very kind of like bleeding edge at the moment so we're still like ironing out some difficulties and i'm using a dev environment but i'm going to show you what is the solution to this gas problem i'm just interested to see what's going on with my other registration so this is what i've been doing with that and i can show you here for the um extra technical this is us trying to work out how to do with uh b j s and this is probably a bit more instructive rather than just going straight away to the punch line which i'll show you in a minute so in swarm there is this concept of beets now feeds around are feeds are a collection of single owner chunks that uh have a certain index which we can determine um based on kind of like pre agreed rules so there's a certain set of rules that be the client and the gateway application abides by in order to say this is the latest version of my website and what we've done is when you navigate now to an ens domain or um a feed manifest hash from your experience b will go into the network and work out what is the last version of the website you uploaded and then display that so that means that you just need to update ins once with the feed manifest hash and then b will know that it's a feed and then go search the network find the latest version display that's the user and then when you need to update and just do a feed update and that will enable these to know there's a new version of the website which will then display so as you can see in this code here this is this is a good example also um how easy it is going to be to interact with swarm using the bjs library just using like normal es6 here you like require you just require the library create a new one with the relevant endpoint new instance of it and then this is node just going to say where my test site is i just showed you uploading the files from that directory recursively setting the index section so this is instead of doing tar and the call and uploads it to a specific reference which is returned from that command so already this is way easier and you can do this programmatically using javascript's trivial levels better then we're getting into the feed bit now which is a bit more complicated so it has a private key of course that's not a good way to manage keys but it's just an example the topic which is essentially name spacing for the feeds and then this is like experimental api bjs but maybe the feed writer create the manifest i've been talking about now what this is is this is a special chunk that is um created in b sorry created by ob in the swarm so when the finds this chunk it knows i need to go and resolve this speed now so i check and the way it's working at the moment is as simple as just incremental um indexes in the feed such as is there a is there a chunk of feed zero uh index zero if there is it checks one if there isn't chase two and then maybe it finds nothing and three and then returns two so that's what the manifest does and then this feed writer actually goes and checks like what is the last available index and then uh updates to the next one it's quite quite complex stuff where we can see this manifest reference here we take this and put it into ens then that is the last time that we'll have to update our website on ems domains run run command change that this little bit so you can see some changes reference the index little node scripts we're using there so it's uploading it to this reference so we can go directly to this one this is the same as what we were doing before with kyle you see so bzz endpoint which is kind of like the gateway and by the way i've got this just on a isolated local network now but if it was connected to the network the main net then all of you would also be able to go to this hash address now and find this website because it would be things like life and also um gateway.swarm.org and then the other gateways as they pop up and here is uh listing out the manifest hash so this is the manifest itself and this is what we'll update ens with as i said still seeking problems but forgive us that's not quite updated but it should that should be the latest version it's a single in the trunk itself it's not important because purposes of this so the idea is i mean you get this go into ens go back here just to reiterate edit the record bzz protocol put in the manifest address itself and send the transaction watch as that gets mined into the chain and then next time we go to this storm swarm swarm it will change with the feeder updates as the bugs get resolved then finally there's one higher level abstraction which is something that's hot off the press um so much so that i'm just running it here in a development environment and see if it's going to work we've got the swarm cli so i think i think this is a premiere i'm not sure um there's identity management so that you don't have to include that private key as a string or otherwise deal with it key management in your system it stores in an encrypted v3 so you can create your identity and this is the second time i've ever used this so there's a good chance this might not okay great so it's created a new identity then let's see what else we've been doing so we can upload the files here ah okay so it looks like we just used directly the feed upload rather than just the static one i'm kind of interested in winging it so let's have a look see if it works i guess the identity was enough so it's good and this actually even has got a status bar to show like this is how it's syncing into the network so first you'll upload to your local b and then that'll slowly push the chunks out into the networks where the eventual um place they'll be stored from us and then as we've spoken about this feeder mode and i'll do all of the steps above in one go very much looking forward to getting this since the continuous integration process in the swarm scene how that goes and this is important to note the when we're changing the website the root hash of the manifest itself is changing but the hash of the of the manifest for the website but the feed manifest hash is not changing and that's why you only need the one ems update so because if i run it again now and they'll both be the same so i've not made any changes and then if we navigate to this see the change is implemented like this probably won't work right now but in the future change this then go back to manifest and the changes will be shown as i said very much uh work in progress at the moment we're getting very very close to being able to um release this in a fully functional capacity so yeah watch this space and i think that will probably conclude my random presentation if greg or if you're there yes yes awesome thank you very much it's uh we didn't think before this talk and yes i was hoping i was hoping we will cover the fifth part because like i said you know every update is quite costly and uh actually if you also want to have like websites that update faster than 14 seconds uh faster than the block time on uh imperium uh then we need feeds so awesome and it's uh it's been great for me to follow it through because yeah i haven't seen it with the cli tool already done and i actually also didn't know how exactly to do it with the fits and topics so i up to now when i was publishing something essentially i was the one paying enos for every update awesome then is maybe uh the script that you have uh for uh creating the feed and uploading the website is it maybe available anywhere as a gist or something i'm using like development versions of the javascripts client at the moment so but yeah anybody like getting searched via mathsmost or b support or orange range and i'll pick that up i'm happy to go through it with anyone and show you where we're at really good thank you very much then uh for your presentation my understanding is that unfortunately you cannot stay with us uh for later the ama uh but i'm sure we will see you around uh in the next occasion yes i'm always ama gregor anytime aim it i am cool catch you later so um i will uh i would now uh ask to the stage is going to present his work on bringing open street maps to the uh on this one zakur are you here i'm here but uh i'm not able to start the video it says okay now i get it i guess yes okay hello hello uh can you hear me now yes we can hear you loud and clear because uh from the technical room i uh i was told that my sound sucks but it's actually probably just my voice you know not technology can't help with that so yeah uh you've been working on a pretty amazing things you know and we've seen pharaoh's already this week uh every time we speak about ferroes there's a bunch of new kick-ass features uh really uh really good really empowering um but today today we meet with a special purpose today we speak about open street maps on swarm by using pharaoh's public pots right yes yes so before i start i have to say that whatever whatever uh was shown by dan in the in the in this previous demo is super awesome and it's going to save a lot of money to me and i would say thank you dan i was about to do it uh otherwise so yeah yes as you said so uh the intention of uh intention of this particular session is to show that how um how open data can be pushed into into swarm and can be consumed by other decentralized application that actually make use of swarm so if you build any applications and store the application in swamp and it's totally decentralized except it uses some centralized service to consume some data right there are i mean we we have seen a lot of applications like that um one of the very notorious one is actually maps um even though if even i mean we make lot of you know software which is decentralized and this and that we don't have i mean a data provider as i say right i mean a decentralized data provider uh to you know power the apps so uh this this actually data need not be you know some special data that is being done by you know large organizations but it can be it can be anything right so let's say that i have like a huge collection of movies which i own and i want to share that with the group of people over the internet uh i should be able to actually share the data i should be able to update the data i should be able to give information to people so that they can actually consume that but there are a lot of other data which are which are very important to applications and i'll i'll speak when i when i give the presentation um yeah so this is what we are going to speak today so today is more about how to store the data and make it accessible to other applications and through this what we want to say what we want to communicate to the decentralized community is that uh now you have the tools right so if uh now you i mean you can actually put put the data there and you know consume the data in a very decentralized fashion without depending upon any third-party centralized services and things like that so this is like a big step and and we want swamp to be like you know like a good place where you put lot of you know data like this and make it accessible for public yes i mean you know for me this is one of the missing things it's like uh we speak about the world computer but uh then the data is being stored on amazon's and google's and stuff you know it's like we speak in the ethereum space we speak about nfts and how they're linked to digital arts and then digit that digital art is again saved on uh centralized platforms what does it help to me if i have a record of an nft but i cannot access the content of that nft so um it's um yeah i guess you know once we go into the decentralization sooner or later one realizes that uh we need to go all the way and putting data putting data uh on decentralized storage is uh this missing piece so um without further ado i will just ask you show us what you got yeah i'm going to just share the screen now and maybe don't you don't see my face for good okay so i have like couple of slides which speaks about this and how this was implemented and you know things like that so um you know as i said in introduction right it's very important that uh we have uh ways to store data access data and make it available to applications only then it becomes a real world computer as as as we say right i mean without the data you can only run certain kinds of apps and not go the whole nine yards right so um i think so this this is go i mean this if it inspires few people to put the data and maintain the data and thereby they can get i mean incentivized through through actually swarm incentivization and things like that then i think we have uh we have done some good for the society so okay so using pharaohs as i said on on on actually the opening day so you know the public pods are a way to enable uh anonymous anonymous data sharing or swamp so you you you actually create a user in fairways and then you create a part each part has its own key value pair and so you can just create as many parts as you want and inside one of the parts you could put all the data that you want to uh individually control and then you can make it accessible by sharing the part right so what are some of the data that you know that can be very useful to applications especially decentralized application as i said open street map you know the map data if we if we actually find a way to put uh you know osm data open sheet map data and also also have some way of um generating the tiles um which is my hackathon project actually by the way uh then we we had actually solved a way you know to access map data over the internet in a decentralized fashion there are a lot of other data sets i showed ethereum transaction in the last demo um where i just inserted 500 000 the first hundred thousand blocks and it can be accessible and you know i mean i have a list like you know we want to put wikipedia like thing uh you know in some countries wikipedia is really not accessible um you know like people uh people from that country can access wikipedia and there is like another project gutenberg which is one of my actual favorites where you put all the books uh which are which are free and in creative commons uh in a place and make it searchable so that you could you know you could actually go and find it uh another data set that i really really really like is imagenet dataset this is the go-to dataset to train any convolution neural network uh to train a neural network to recognize objects in a photo it can be any object and this is a beautiful data set um you know like put together by a lot of people over the internet and and it's like close to one one terabyte actually it's very very hard to download that much data and do anything on that you need to have a serious serious serious hardware to do that so if we have these kinds of data in swamp then i don't need to actually download i i just uh you know my application can access only data that is required and and it can just show the data if these data are indexed and accessible or so on right um okay how can we store these data as we said you know there are three different ways to store data one is you can just store raw data as it is in the file system and make the file system you know like public uh of course this is not much useful because um raw data means you know people again have to search through that and you know do something over that but this uh is useful for a few of the data like wikipedia and things like that where you just need to just take a page and just just render it in the browser and you don't need some very hanky funky very fashionable data structure to actually do this there are other data structures like key value and document store key value if you are using a simple index data structure document store that if you want to do or search the data using multiple fields right um okay so let's actually look about open street maps so open street maps data is made available to everybody so you can go and download the data and every week there is an update you can actually download the osm files these are in specific format um and inside that you will find uh usually they come in xml file and um they come in three okay so inside that you will find three different kinds of data one is actually called the node data node is a point of interest in a map it can be a building it can be a dam it can be a cafe anything right it's actually a point of interest uh so and then you will have information about the point of interest like what is the lat long what are the tags whether it's cafe if it is like a building what is the name of the building and things like that and then and then the second kind of document that you will find is is called a way way is um is a slightly bigger construct where it is useful to represent more than one node um something uh if you want to uh if you want to say like i mean a road or or a actually waterway or anything like that so which is like some um way uh and then you can put different nodes along the way and then say that oh okay so this is a way and there are different nodes along the way relation is a little bigger construct than way that where uh you actually say that um hey uh so i i um you know like like uh like uh uh um like actually a bus bus route which can have multiple ways and multiple uh nodes along that so right so uh so i give an example of these things these are it looks like that if you actually convert that into json so a node and then you will have an id every node id is very different uh and then you will have the lat long of that and the different version because people update this and then you will have you know actually the tags the tags will give information about this particular uh you know object it can be a node it can be a way it can be relation in relation you will have actually the members inside the relation i'm sorry so you will have actually the members inside the relation like uh like what you see here the node node node and then there is like way way way also like i mean if you take a bus route bus goes through this way and this way and this way like i mean it can go through three different highway and it can actually pass multiple nodes or a point of interest along the way right so these are this is how openstreetmap data structure is is uh defined so what what i do is that you know uh i have actually taken this data and um and and sort of index this using actually the tags and and actually the type right so what you can do is that you can actually build applications which can generate vector tiles by taking this data and this vector tiles can be done from the when you can have a server side application or you can have a client-side application which can actually do inclined right there a lot of javascript and other libraries out there in the open street map website which which which will actually convert this into vector tiles or or actually restaurant tiles right and then you can actually display in that particular environment so okay so uh i'm speaking about indexes and i want to give uh you know a small introduction about how i have built indexes you know just for you to understand and appreciate uh and to know a little bit more how to really use this um so what what i do is that so i so when you index a document uh i mean i i just created like um um like a big file let's say uh switzerland.json which has all the nodes and all the ways and all the relations of uh of the country switzerland right so each one of the lines here you see here is one json record which indicates either a node a way or a relation uh i mean it is actually a pretty big file usually uh like switzerland is like uh um when you uncompress it it's like 1.

6 gigs um and it has all the information inside that um so when you when you upload this into fair os what fair os does is that it splits this into smaller blocks blocks are something very accurate to uh how swarm deals in uh you know chunks uh but chunks are way smaller blocks are slightly bigger you know just to make sure that you know you could access these things without any big delay if you ask for something right so you upload this and ferro is automatically splits this um according to the uh you know this thing the block size you give when you upload let's say the block size is 10 mb or something like that if you upload a one 1000 mb json file so every 10 mb will be taken and it will be compressed and will be stored as a file so each block will be a separate file in swamp and all the metadata related which actually binds these things will be stored will be created by ferro s and that metadata also will be stored in swarm and when you index a file and and actually say that okay i want to index on two fields let's say i want to index on the field you know type and i want to index on the field tag let's say um what uh what i do internally what we do internally inside fair os is that you create a prefix tree or or you can say complex compressed prefix tree um or some people will call that radix tree uh for each of the field and which is like let's say um if you have like name n a m e and at the end uh you know the leaf node will have uh the place in which that particular document starts right so it will say that hey if you if you search for this uh this document is there at this point in the entire file it will say what is the block and what is the space in that and all these things uh even though it's compressed it will still show you that which is the place of that and similarly every field that you index will have a separate radix tree and all these are also stored in swamp so it's so nothing is stored outside everything is stored in swamp so one is the entire raw file uh is stored in swarm as as small blocks blocks are also compressed and each of the fields will have a separate radix tree uh and each of this the entire data structure will also be stored in swamp so if you if you actually search let's say if you say hey give me a document which has field one equal to blah blah blah it will go here and every every point it will go and load this particular part of the tree from swamp and then it will find out this last bit and it will it'll it will jump to this place and it will get this only this chunk from swamp so it's it's it's actually efficient very efficient in in i mean in a sense if you see like that so if you if you access something the index only that part of the index will be accessed and loaded into inside the memory and only that part of the uh actual file will also be loaded into memory and it will be used uh of course there is an lru cache inside that and it will be used so let me just go uh and show you a demo um so yes so this is uh in a place called api.fairoise.io so you can you know i'm doing from my local machine you can also do it from your local machine or you can actually install your own swamp and uh install your own farewells and import or or or you can access this public data set from there also i will give information about my data set uh so that you could access that in the end uh so uh so this is actually connecting to api.feroz.io so uh the username is osm um you don't need to know this but you can still access it by only using the reference so there is a part called osm so i'm going to open this pod and inside that you will see i mean a directory called switzerland uh we are slowly planning to have more countries and um and so on until we reach planet osm uh right now we cannot do that because uh it takes a little bit more memory and i'm trying to optimize the indexing to have very very less memory end and be decently fast enough to access in a user interface so in in this one you could you could actually see the actual file which is sw text.

json so there is um uh you uh if you actually download the file from uh openstreetmap uh you will you will actually get that in osm format which is xml i have written a small utility to actually convert this into json i will give the part of the utility also just some small 10 line utility which you can also write in python i have written in golang you can write in any language i think there are lots of utilities to convert osm files to json so it actually looks like this um [Music] it's a slightly bigger file so it takes time to load in vi so it looks like this so every row of line is a json document uh you can see the type here id lat long visibility time stamp and then actually the tags the tags will be depending upon what node what way what relation you have different tabs you have different tags so i have indexed this in um a separate database called switzerland i have indexed two fields one is type and another is actually tags stacks is actually a map index when you say a map index you don't have to specify what field you have to index you just have to say that hey index this entire tags so this tag is actually a map which means it has key value key value key value like that it has key value pairs so it will take the key value pairs and index those pairs automatically so um you can open this by using dock open with land it takes some time so i have already opened this uh here um at this place um so yeah um so let's let's actually go and see how big the file is the file is like sorry so the file is like 1.6 gigs and if you actually see the file here in this one stat sw tags dot json which is the same file so i have uploaded as 10 mb files and each 10 mb block is compressed into 1.6 mb so that that is a nice saving that you get it's like almost five times close to five times compression you can say so now you can actually um run some uh actual queries on this you could say uh count switzerland uh so it will count the number of documents it's like closely 6.1 million let's see here uh wc minus l switzerland is 610 6163 uh it's slightly different because uh some of them will be ignored i think if they're not good they're ignored and things like that uh but it is approximately six million so you could actually count uh for example [Music] um using an expression you can say you count only the nodes where type equal to uh no um [Music] right so there is like 1.6 million nodes and ways are like 4.

3 million obviously there are a lot of roads and waterways and things like that let's do something useful so um there is a command called find find will um uh will uh will give out it's i mean if you have used mongodb you will understand this it's it's like get all the documents which match this particular expression so it will find all the document where type mnet is equal to cafe and you can give a limit on this one so let's say i give a limit five right so it will just give me only five documents uh there are a lot of documents so uh what you can see is did i put it rightly this is the land let's see if there is an amenity cafe is spelling right okay a-m-e-n-i-t-y fuel amenity uh if you don't give the limit it will it will just give you uh the first 10 documents so um there is some issue i don't know what the problem is as you uh i think there is some oh okay so i thought okay not type it's tags tags equal to amenity cafe and you will see okay you will see documents which have amenity equal to cafe uh unfortunately there is only one document there i saw fuel just now if you el if i'm right uh this uh yes so this this actually says what is the this is this says that is a fuel pump um brand is bp and octane 95 yes and self-service and you know like things like that it'll give information about that so you could even count the number of uh fuel stations which are there in open street map okay uh so there are like 3495 fuel stations in this wing uh of uh using actual open street maps so let me go to geneva so uh i have already gone here but uh just to make sure that you guys are everybody knows geneva yeah so i go to geneva and uh open street maps shows me there is like there is a country there is a state called geneva there is a city there is a town village and all these things we want to go to the city geneva which is there in switzerland so this is having this node id and it shows all this information and with lat long and all other ways so this is actually a way uh if i'm right uh okay this is actually a relation so what you see here is a group of relations here a group of ways and nodes so there are like 18 members here so this is a node way and all these things so this forms the entire geneva right so using this data this is actually drawn as a raster tile so um let's see if we can get this um in our database so what you can do is dock get switch land um you can just give the number if it's id so you are getting all the relation and the map and all these things related to geneva there are different ways there are different tags uh tags are in different you know the name genny wise in different languages and this is the members as as i said this is the tags and this is the members members have all the maps and nodes that we see here um reference okay so this is not displayed uh like very nicely i think the display is like putting three dota within since it's a big number i think there is some parser here but you will get the actual data in the json uh it's just the displays like this uh but you get the entire um node and then the tags and then the all the all actually the members uh which is there in their geneva relation right i i think you get the idea right so this is this is how you you take open street maps that and then you try to you know like show it in your uh screen um yeah so i said i will i will show you uh the place where you can just attach it so if you go to this um uh you know my um my github there is this thing called osm there is a repository called osm i have put uh the file which which actually converts the given osm.pbf so this is the file that you download from street maps this is like in a protocol buffer format it's very simple to read this and you know print all these things in you know json so this is how i have done it and so yeah so uh this is the api gateway and this is the sharing reference of the part if you want to use the osm part um you just have to do part receive and this this in your uh in your inside your user and you will suddenly see a part called osm and inside that you will have the database called switzerland and you can access it exactly like how i did it right now uh i will put this in the telegram uh you know the support chat and if you have any uh you know any doubt or anything regarding this um you know i'll be there in the chat you can ping me i sleep slightly earlier than you guys um but anyway i will try to be awake and answer all your questions back to grigor awesome thank you thank you very much uh for this talk i think i think there's actually we have quite quite a lot of things to unpack here so um a couple of questions before we proceed with others now my understanding is that i can now as a if i'm developing an app i basically can just now connect like uh do ordinary api calls to this spot and get that data right yes it's as simple as that yeah so and now this api endpoint i can set it up myself running on node or uh you set up also api dot ferrus dot org uh which is i guess that's like infura for pharaoh s right now yes yes it is um i mean initially we need to have some kind of this thing for people to uh you know just uh hack around without installing without doing anything so this api is like a free service that we are giving for now but i totally get it i totally get it you know it's like i'm one of these people it's like where uh uh i'm a weekend hobby coder you know and then if i just spend my weekend on setting up the whole stack before i can start experimenting you know i get tired so uh connecting quickly to just api endpoints to see to get this instant feedback uh yes i i do need that and um i know quite many developers who will see the demo from my local machine i connected from a local machine no data is there a machine but i was actually executing in my local machine so maybe a bit of provocative question why only switzerland um i actually had afghanistan first and it's a crazy place has a lot of peaks and a lot of military control borders but i just thought switzerland is a nice uh place uh not i mean i have not seen it physically but i think it's very nice place but i used it because uh the size of the index is not small not very large so it's almost one so yeah no but you know it's like why i also mean this it's like i think we should use now this opportunity it's actually a call to action to others uh and it's a quite uh i would say also an important moment as until now within swarm we didn't speak about a new stakeholder and this stakeholder could be called like data stewards so it's not the developers it's not the node operators it's the people who know it's important to have certain kind of types and kinds of data uh in a way that cannot be taken away from humanity so um i think you know here maybe it's an open call uh to everybody help out you know claim your country help upload that that country index it and uh we then connect uh we then create a product pods i guess right yeah which is the the whole world so um if you if you would like to uh be let's say one uh chief swan cartographer for a country uh please uh please do uh reach out to us and this could i would ask you if you can uh stay on because um right now i would uh also invite today so it's a it's a good uh good way to also show uh today has been working on some really exciting stuff uh uh in connection to pharaoh's it's an ui and uh such an ui could be very helpful either for developers to uh work with uh the data sets the pods or for these data stewards that i mentioned yes so today thank you thank you for joining us hey hi hi as i mentioned yesterday uh yesterday i saw what you are working on and i think we need to share this already with the world you know to to see it and it's great continuation to what zakur just presented if you can show us what you've been working on during this hackathon sure so so i've been i've been following the who's work for quite some time and as i saw this beginning of this liberate week hackathon his progress i said okay this is super super cool but maybe people would need like a web front end you know instead of uh cli and uh so where where all these uh methods uh for the api are implemented in a referenced way and also to show other developers how actually to use this fair fair os defense so maybe i'll just share my stream you see it yes yes okay so we start with the user the user can new user can be created based on the username password and mnemonic and you just can sign up or you can import based on what you type inside your mnemonic username and password and then you can check all other stuff i have this already typed in so i'll just log in and now i have first of all i can select which gateway do i use or use a local local host or or the one from the and this list will be and can be populated by by by users anyway right so apps can have this predefined or or just allow a user to use whatever gateway they want so so one of the things maybe i'll turn the book on uh so one of the things is first we have like a list of parts i can create new one new bot one and i can open it close it and i can see its contents and i can sync and i can delete once i can also make a directories inside yes oh there's some problem [Music] yeah as i said this is we didn't uh the democrats are not uh let's try again or maybe i close no there's something else okay anyway i have a pod one ready and i have few folders inside with with files mind you that i didn't bother with uploading files because it's it's more how to how to do and deal with data right so now i have a pod one open and i have several tables here maybe i try to do another one yeah so step 11. so if i choose one i have 10 records here i can seek at the start and then i can just go through all the records i can put some key with value put it i can get it and fall as well i should get it also here maybe two x [Music] so yeah it depends how you search right from which index still till from start till end so maybe this could be like this and just get this one so these are these are key value stores right so app app can put the data into the parts and then we have like a documents again you can create them you can create them with the indexes and select this one and i put inside some sample data so this is already inside and now i can get it out right or i can get i don't know one two i think this is the data that comes out f4 does not exist five so let's try something because uh the hoorah added the pot and i'll receive this part yeah you got the part osm is the last part oh yes cool so yeah you see switzerland yeah i have seen the rent and now i can actually search for it right so what was the reference number uh just uh go to open uh openstreetmap.org and search for geneva geneva is the only place i know in switzerland yeah click the city there yeah you can get the index um i'm getting some errors i have to open it up ah you know where to open it it takes some time to open because it's actually a big database yeah it's open now yeah okay yes that's it see how easy it is to get the data yeah yeah so so this is basically like meant like to be a tool where you could manage your own data right maybe the interface of course the interface would have to be changed but uh um the idea is that you could manage like um you know like you manage the databases on [Music] my admin or stuff like that you can manage most of these data in such a way that's in this project right so basically basically we can see in future that people uh won't need any technical skills to actually create and upload data sets for others right yes so the the idea is but i didn't come to this yet right uh so this is where we stopped now uh so the the indexing and searching still needs to be done but the thing is here you can the the thing is you could just like specify some you know end point where you get the json and you insert it just like in get it from the some url you get it into the browser and then from the browser you insert it into your own pot which could be a really powerful kind of a shortcut right for um for normal users very very exciting to see it's uh we need to stress again you started to work on this this week right yes tuesday so amazing amazing how much uh you you were able to do in such a short time so yeah it's uh it's really it's really good um and i i can see how um it can empower people uh to uh create public data sets my my kinda you know my wish my a big dream is that uh we would also have interactions then uh when everybody can contribute to some data set and at the same time the hash we get the same hash in a very similar way as we get a transaction hash on blockchain and we can check that exactly the data that we submitted is part of the data set and if we think a bit about this uh it's like so many so many applications it's uh we see um how much people today are mistrustful uh how it's uh how their data is going to be handled and so forth um really nice thank you thank you very much today for this demo um i would i would actually ask you also to stay on um we have uh now we have come to the end of the demos i would invite uh for also closing discussion and uh sharing thoughts on uh free press and permissionless publishing i would invite gabriel uh if uh gabriel uh you can also join us awesome great uh great to see you again uh thank you thank you for thank you for joining us so um my pleasure it's um you know uh i'm very hap i'm very uh happy that you joined us because uh you've been uh actually through the years now a big advocate of permissionless publishing and uh um free press so uh uh you are a part of the iov team and you've been also uh working on these uh uh matters uh maybe maybe uh you can share your views uh with us and why you think this is important and uh how you actually tackle this at iov well thank you very much for for the invitation uh congratulations to all the demos and and all the participants in the hackathon and as you said greg i believe this issue is very important i thought it was very important before and now i believe it's even more compelling um i mean it's incredible what just happened the last couple of months you know we've been in in the bitcoin space uh working on smart contracts on ethereum on rsk uh understanding that there's no going back in in the construction of a decentralized uh finance uh and we always kept this idea that you know this should be like the cornerstone in order to build the fully decentralized internet as it should have been in the first place uh now i i'm just completely surprised and and i cannot believe the the the events that happened the last couple of months which in my opinion just make even more compelling and urgent the need of a fully decentralized internet with uh decentralized publishing and and permissionless and censorship resistant internet um i'm sure you you agree but the internet has become a geopolitical battleground in the last couple of months you know the the platforming of trump which on a personal basis i hate by the way but but it's just i i i mean in latin america it just shows every single politician and every single uh person that whenever these massive tech corporations want they can disconnect any president or any government or any uh public speaker or thinker with the rest of its uh fellow citizens if they have been able to the platform the most powerful man in the world because he was a sitting u.s president imagine what they could do with any other president in the world um and and i think we are starting to understand the the impacts of this uh i think even even jack dorsey was you know explaining that that he was against twitter having this power and and everything that you guys are are building or enabling to be built on to perform uh i believe is the only hope that we have to to recover the internet because uh there's so many uh issues that we could spend hours talking about um you know blocking uh applications from from being in the app stores you know this now has become a very powerful way to concentrate and to censor speech uh the listing um or blocking applications such as tick tock in the us as uh trump wanted to do or asking apple to include backdoors to spy you know maybe uh apple users in other words uh uh prohibiting uh companies public companies to list their shares uh in different equity stock exchanges um so we are in the same way that the u.

s and other governments started using swift and and the financial system as a geopolitical weapon now they are moving full on into the rest of the internet so imagine in the near future when we have a fleet of uh robot taxis right and where people uh don't know how to drive anymore because it doesn't make sense to go through all the process of just you know to drive for fun during the weekends because you're dry your car is uh self-driven uh if they can shut you down your gps or your your your google maps so that's why i'm so excited to see openstreetmap uh been um published and so on because every single piece of the internet that we see as a very naive and you know what can they do with this can become a geopolitical weapon in the future um and and this is becoming critical because i do not see uh you know a scenario where suddenly uh china and the us start getting along and usc school with china taking the leading role in the global economy and and what other uh countries or regions such as europe or latin america or india we do in this you know massive geo political uh power this is the reason why we at iov labs um two years ago acquired taringa which is the largest social network in spanish in the world because we wanted to make sure that we have a proof of concept to see what are the building blocks the infrastructure open source building blocks required in order to decentralize social networks and then sharing economies so this is why we built the rsk infrastructure framework or reef which are these building blocks which integrate with the centralized storage with swarm and ipfs decentralized communications so we can post our uh photos or videos on decentralized storage we can communicate to each other uh with this communications relate through the centralized network of nodes we can share the value uh with the content providers on decentralized nano payments we can manage a reputation imagine if you want to exclude a group like the ubers in in china you can just tell them you know you lose all your reputation in wechat or you know you don't like any particular group or a particular country and just erase their long history or reputation or just the platform the people from decentralized marketplaces so the identity part and the marketplace the decentralized marketplaces are also key components of the reef framework which will be used as a proof of concept by taringa and open to any other you know social network that is willing to to to be built fully open source and decentralized uh and i just i cannot believe how much we need these technologies now uh when we were talking last year you know it felt that we had like five years time frame you know it was it's very difficult to be built uh and the problem was kind of there's no one really uh desperately needing this uh everything has changed uh what happened with with gamestop it was again another massive red flag of what the current establishment uh can do uh if they see their incentives or threaten other power no they they do not care in openly just screwing the little guy uh because you know this this amazing uh reddit users understood that there was an opportunity for a short squeeze and i mean we could discuss this for hours but what's the value of having these massive hedge funds shorting uh uh stocks and and even i'm super upset when they were shorting tesla which is you know a company that might have its issues and and i'm not saying that everything is perfect but they're trying to solve global warming so what's the value of human society of having these hateful hedge funds shorting the only projects that is trying to change you know the the mobility industry and how we burn fossil fuels so i i believe [Music] that everything that you guys are are doing and and i'm super excited to to see and uh uh the projects are gonna come up from this hackathon uh i believe that they're crucial for the future of of humankind i believe i feel that we are in a turning point uh in our existence um that bitcoin uh and defy and the centralized technologies will be will be key uh it's like bitcoin showed us that we can have a decentralized permissionless financial system and now we need to take that uh those ideas and just expand them through every corner of the internet because in any single point where we have centralization we're gonna be uh looking at governments and corporations just using that power to retain their their power and just not enable a more open global and fair society so um as if there's a message that i would like to give you guys or or leave you is um this is not a nice thing to have anymore this is an urgent need and and we should take uh our work and our technology extremely seriously because you know when you live in a country where everything seems to be working like switzerland uh it's very difficult to convince people that you know swarm is very much needed or a decentralized social network uh or even bitcoin now when you live in argentina or when you live in venezuela when you are censored by your government we you cannot trust your institutions as i sadly believe most of us will realize very soon that you know is it used to be a problem in the emerging markets and corrupt governments but but now we're seeing that the corruption is everywhere and it was just that there would no uh enough uh power at risk uh that when when when the sheet uh touches the fund you know uh we're gonna be very lucky and thankful that pioneers uh such as the swarm team have been working on this for so many years because you know if we want to start building this in in a year from now it's going to be too late so all this hard work that you and the team and the whole ecosystem have been taking in the last couple of years i think are going to be extremely important for for our society if we want to build something that is better of what we had because if we do not do anything if we just retire if you go on holidays then i mean it's very clear that we're going into the the black mirror scenario no it's uh the guys in power will just try to get even more information abuse even more from users have more surveillance there is there is no roof for the amount of surveillance and control that they would like to have on global society uh so it's up to us it's it's a community effort that we need to work together uh uh as a swarm uh to make sure that that a better society is possible so just congratulations i'm very excited to see all the projects working on this hackathon thank you thank you it's like you said you know it's uh i deeply believe that not not a single project can make a change we can only make change through collaboration it's only only with united uh forces and where we see how we can unlock these synergies uh that we can shift the needle uh into the the other and it's like like you said you know it's um in this in this regard maybe also call it the situation with kovit was uh uh quite a quite a learning experience because uh you know when i look uh okay most of my life i lived here in slovenia and it's nice nice country all good but we basically covet made us realize how fragile everything is we are right now i don't know it's like for three months we have curfew curfew was a concept i read in the books from second world war it was too abstract for me to think of and slovenia it's such a small country you know if you take the longest route it takes you four hours to cross the whole country and we have restricted movement i cannot visit basically my parents right now without breaking the law and they are one hour drive away from me and it's when we see like these kind of things or like three years ago uh what was happening in uh catalonia in the sun in the middle of europe uh where depression was happening it's like it makes us aware it's a it's very fragile and as you said uh i also remembered like the beginning of the year i was uh looking at the things happening and the discussions that were started by the mainstream media you know uh i was just traveling turning on in the hotel cnn and they speak about censorship resistance and i was like okay which universe did i wake up already it happened so fast so yeah um yes and also i i think the the the fake news problem uh it's it's it's a massive thing that we need to find a way to to tackle because uh what i feel now is that regardless how smart you are how much research you've done it's very difficult to tell what is truth and what is lying and and we are so easily manipulated that um i also believe that there's a lot of work and contributions that could be done through this decentralized movement on just curating those news and and and and giving us back some some level of truth uh like in latin america there was a massive campaign against the russian vaccines now and people rather die than getting the russian vaccine and now the russian vaccine is the only thing we can get so every and there was some extra information so now everybody wants the russian vaccine but it's it's so easy uh to manipulate in pursuit of a profit you know a pharmaceutical lab a geopolitical uh just comment from our leaders and and and we've realized that our leaders lie openly to the public with just no no consequences so unless i'm not saying that we can fix this but unless we fix it no one else will because there's so much power and and so many um interests in not saying the truth and using uh the social media to manipulate people that we cannot trust them anymore we we know that they want to manipulate us uh so i it has become part of the playbook and it's it doesn't matter to which option one subscribes or anything everybody is doing it it's this global fight for attention and uh you know buying into this kind of version of reality that fits someone's agenda so yeah it's uh unless we have really uncorruptable and verifiable data technologies that put us in control um i don't think the matrix scenario will be good yeah so the stakes are very high and and everything that we've been building over the last couple of years it's becoming more and more uh of an urgent need so you know it's been so um a bit of my sorry a bit of my background comes from user modeling and this uh these things and it's uh um this was last year i was uh preparing for one talk and i was looking up uh on google scholar how many references bitcoin has uh satoshi's paper has and uh how many references uh the work of michael kosinski has he's one psychologist that also cambridge analytica draws upon uh his research and it's you know like his reference count is dwarfing uh what satoshi has and this shows is also uh harari is telling today's sport not just by governments it's uh also by corporations or in general is hacking the human mind and we might even think we might even think that it's only the apples and the googles and these guys uh who are bad but it's even it's even the small user using facebook ads who's part of the game who actually also is because of the technology and the environment we have that also the good guys are also forced to be the bad guys in order to stay considerate yeah yes and and the problem i think uh the underlying problem it's really to to the incentives the economic incentives that capitalism uh brings on to us right when you when you have shareholders that are only asking for profit and and to maximize those profits then you're basically telling those management teams that uh you know the ecology is not important that uh the humankind is not important that the way we live that health is not i mean if you work in a pharmaceutical lab your shareholders don't care how many people you saved they care how much people you how much money you make and and i this is one of the things that i also like the most about decentralized psychosystems that uh we've been able to create a model that maybe with in its perfection no it needs to be improved but it could present an alternative to just raw capitalism that we've seen in the past where things such as global warming are not factored in um so this is a problem as well now when when the world and the main organizations are just seeking to maximize profit as its only concern then it's when everything else falls apart yeah then it becomes a race to the bottom yes um i'm looking at the time we've been actually quite over time but i think we could speak for a lot longer however unfortunately i think we need to wrap up maybe if there's any questions from the audience i would ask this is the time to speak up and we tried to answer so any questions i guess no i guess no so uh with this um i would conclude today's uh session uh gabriel thank you very much for joining uh zafur then today thank you for the demos uh everybody thank you for taking the time and joining this call and please hit us up on our usual channels telegram matter most ask us anything we are here to help you and let's build together a new and nice future thank you bye so you