How to Integrate Ultralight Beam by Eric Tu , Dean Eigenmann, Greg Markou (Devcon5)

[Applause] my name is Greg from chain Save Eric we've got Eric over here and Dean need some status with chain safe we're going to present to you guys something that we worked on as a hack and realizing that there's a lot of actual like application to it and it's a little tree lightning so quick outline we're just gonna kind of like over everything and give you guys a basically the code walkthrough we have a full we have a sorry we do have like a running surface TK that works but it's it only supports bluetooth currently in minutes a working progress on the Wi-Fi direct if we want me to show you guys how to integrate it directly into like a sub tab right now we can be like outdoors we're gonna turn the TV music so you know they have this slide is kind of cheeky like this Wi-Fi a good luck downloading Xcode so over doing the Code walkthrough not an actual integration but if you have Xcode we can do that's like four games kind of like results are like me where did it happen so we were in New York and basically we have this crazy idea that we were gonna transmit offline theorem transactions over bluetooth and Wi-Fi direct and we did it but you know very naively put we started working on from that and then a dimension proceeded into you know dude go they were like whoa these by pnina there's no support for Bluetooth and these languages you know on like public and we're like less problems and then we realized there's no support for certain function either bluetooth on you know only on iPhones or Android devices so everything we realized really quickly that like the ecosystem itself within Bluetooth even white button wasn't submitted so we ended up moving more towards a swift and because you know basically what is ultimately we're a transport agnostic Manning so mobile ad hoc network for sending arbitrary data over the wire basically what that means is we don't really care whether you use bluetooth whether you use Wi-Fi direct whether you're using like you know hard code it really doesn't matter what we're trying to is we're trying to find a way to actually communicate over your networks that are we don't care about really the topology we like we only care about how you're transmitting data from one period to another peer and when you don't have a stable connection that's like basically the idea - it's like how can we actually make that work and how can we do it in the future and to do so we can actually like do kind of like relay with your transactions and or just any transaction really in a very unstable environment we'll get into the get inside show so quickly about minutes so man refers to continues self-configuring infrastructure list of mobile devices on that date let's see that's a lot of buzzwords I have very little interactive thing I'm going to show you guys are not to make the difference so it kind of sounds like a mesh network and I want a different different you know make the explain the difference quite clearly so a mesh network you can think of like a bunch of IOT devices that don't move is your camera movement all right probably not your cameras like staying still and there it's gonna be our camera you know same apply your your toaster your toaster probably doesn't move for that stove that's like all connected that's a machine or even your brother you know those Google like little rubber modulus like they're staying in one place and typically not moving and they're relaying this whatever the data is like in a very static way do you know what here's me a thumb and I'm gonna be there yeah it's nice you know this is technology that's sending that it's very low-power doesn't need a lot of work and can actually like will work on this up so in our example we are static we're not moving we're a match but as steam starts to move around crystal transmitting data to oh and somehow this camera is actually moving or not moving so as a picture right this can move to this this is a mesh network and we're chance winning today like Jane hi Jean hi hi ping and then wait team is gonna go through this wall where there's no connectivity where it's still a minute no I'm saying hi team hi Dean he's the messages are in queue Erik did you see my hi team you saw I mean oh hi team hi hi so right there you know these messages when he comes back because yes we have some like kind of we've got some issues I get up two ways that we can filter no traffic to hop path of least amount of hops and then we do that using like more centralized will be layers so some needed connect talk about that we'll get to the any mandatory special questions just wanna make sure we have the yeah why is this super useful let's think about this for a second so this is kind of this photo drink it's kind of when you're in yeah we're like totally cut off we're talking like zero internet how do you communicate with my number you can do it these are like non sensible tools that our disposal you know going over data and connecting to the carrier that sense right because they can completion in a large crowd all you need are a few people with devices going and you have a full-blown network and where you can communicate with somebody potential on the other side of the world as long one person is internet then we can be funneling all our requests through that one person to a bounce back outwards and you you know you can be like two kilometres away as long as there's people moving and carrying these messages around kind of like carrier pigeons you know like hey this like flying flying flying eventually get to their location where they get deep it's kind of a similar idea oh yeah in buildings another really good you say Smith's oh no no no like you ever been in a building it's there's so much concrete you just can't actually communicate with anybody like your phone's down like same concept you could just keep relaying it until there's one person with the internet and you can get out and one of the things I wanted like clear the differentiate hears like yes this is to like do an offline set up but it's also a way to actually get offline devices to actually connect to the internet and send them receive messages so that's where the like power behind this slick really becomes like a powering for people is because you can now like we can give Internet access well messaging access to the Internet to devices that could be a 50-story deep with no actual active engine as long as there's somebody who similarly you know like in places that don't have access to like get infrastructure whatsoever you know we can this could set up like like in like small communities of like to talk to people so that's kind of like my intro to kind of mesh and the useful applications to them we have any questions on that because then we're gonna talk about like the like cash flow like stack here the merciless is like the best time to ask about Li yes so you could yeah you probably could run it again this just managed some like we're using a mana simply to help yeah yeah he's probably key to run Cisco but yeah Eric's gonna die if you guys deep into kind of like what all this means and what we've done thank you Greg yeah so this is kind of our architectural diagram for ultralight beam we have I'll go into each part separately but kinda the flow is like if you're sending and receiving information like data comes into the transport it goes to the node and if the node supports like certain services of Emma button this and if a node supports like certain services that you know it you know and it will process that information if it doesn't it like propagates back out to somebody else who can potentially then you know in Reverse so transports its we I'm not sure if we're gonna stick with this name but it transports in ultralight beam are it's the communication protocol I guess so that includes things like Bluetooth Wi-Fi direct and radio but it could be like literally anything as long as you conform to like an interface that we need to provide and the interface is very simple this is what our interface looks like you've implemented the Englishman sorry that's our interface you basically have to implement two functions it's descent and listen function so I mean theoretically like you can do that for like any form of communication so you can even do over ultra high frequency low frequency radios if you want to implement that so transports they're interesting because in ultra light beam we can support multiple transports at the same time so if you have a device that's Wi-Fi only you have another device that's Bluetooth only you have another device that's Wi-Fi and Bluetooth the the devices that are like only supporting one transport can also talk to each other by going through the second one and that makes it super powerful so you can create like these massive networks that are you know in the macroscale or very short-range but then it connects like larger other methods perform like a huge magnetic over like radio or something like that currently for ultra light beam we support bluetooth low esteem for short distance communication which we will definitely learn it but um soon we're gonna focus on Wi-Fi direct and get that working for Android and so like we can do like a bunch of cool stuff yeah again now we talk about services so currently like ultra with gold with ultra beam is to be able to like relay these generic messages like these UV message payloads but so services are things like we want to make it so that ultra light being conserved like specific application type things as well so you want to be able to like as grave said before the relay transactions offline like on a plane or something like that maybe one guy's Wi-Fi because a rich in the bodies are change state channel you're gonna clean you can start the offline state channels within like a plane and send people money or you can just doing chat messaging so these are what like services are they defined like how when these like how to handle these messages if you're a node and nodes don't have to support all services equal support some services and they can't support it they just like sent somebody else who's on their peer list and maybe they can support it just keeps going and going yeah so relayer we have the concept of relayer that's a node that generally lives like on the edge of the mesh network like for example like in the like in a yeah like a festival or something like that there is technically like you know there is like cell signal but it's like super bad but it might be good for the guy who's like maybe a little bit further out from like some background and that guy's internet so you know you might try something a facebook message to your mom showing like your party but you can't kiss your like in the middle of the festival so you send it out through ultralight beam it goes like through a bunch of people and then it finally hits a guy who has internet and he relays it to the internet gets Nessa tracking this you know like tracing back to you so yeah generally these things like living on the edge of the network I just said that yeah so we have Bluetooth Low Energy right now you got it working on Mac OS it should be fairly simple to get it on iOS because we're using only libraries that are core apple and they're cross-platform we're going to do Wi-Fi direct soon it's going to be supported on Android and all that so that's gonna be dope don't go don't go to bed yeah yeah yeah so we were gonna prepare a full workshop and then we got here on the first day and we realized that the Internet of [ __ ] and so getting you guys to download packages and stuff like that isn't going to work for us so what we've decided to do is that we'll do we'll try to do a quick demo live of how one of the real ER works just to show that this isn't completely vaporware that we can actually send and receive messages did you guys download I figured you would if you guys actually have Xcode you guys can pull a repo and join our network too but yeah if anyone has a Mac with Xcode an internet they could pretty much pull and join you have any questions yes yes I was learning about the Facebook example so they start asking for messages but can they actually initiate communication in theory yeah but that's like a lot of work dean of inland coding this for like the last four months we've been spending like 10% of our time 20% because we have like eight jobs yeah you could we made like design is that like our rationale behind like our design is like we made it generic enough such that you can do whatever you want on it this is like we just provide you you know like a way to like just send messages but you can definitely encrypt messages you can probably build tcp on this you probably build like into HTTP on that's too probably there yes yeah so it's like super generic should wear girls it over yeah so going back to your music festival example thing so the guy on the periphery who actually has serviced this we need to be like running this I yeah so like everyone involved in that chain of yeah yes so we still have this yes of like the the idea behind what we've done is we're trying to make it as minimal as possible because that way he any any app could integrate this the protocol itself as like a secondary pop as like a like a layer on top of like an existing messaging service right so this isn't like we're making another chata this is like hey please use this as a forum to do your if to this is like this way to like toggle like hey I want to like use the offline mode so your existing like Java that's kind of like the design idea so in theory if you went into that mode you could essentially receive messages from other apps that support you friend so you can have your own the same interface like what in theory if you know Facebook would ever adopt it you know theory like whatsapp can communicate with like messenger on an offline yes you know so you can do whatever you want to the payload right so the message itself we don't define how do we don't we don't define no you do to the message but you can encrypt it however you would choose to to ensure that like we want to be as less intrusive as least intrusive as possible so you can do whatever one that you couldn't repetition easiest is yes just how do you so now you believe we have this thing called the gossip right now which is doing that we're working Dean's been doing a lot of research with Oscar Oscar on better routing paths so that we can do more efficient hops and I actually like have less hops throughout the network itself the problem though is that typically what you would in a festival is a little bit different festivals there's not people are typically standing still so we can actually benefit from the fact that you know I can assume that the device would probably last more than a minute and roughly you know like a 5 meter radius but in a busier crowd things get a little complicated because what ends up happening is you have people moving from like you can assume like we only have connectivity for like maybe 10 seconds before now the peer list might be changing and drastically changing so we have to find as a way to do an efficient like one like a DHT wouldn't work here because like it would be so aggressively updating all the time that doesn't work so you have to maintain like a really close peerless and finally better running hats that's what I'm working on but if you have to stop see yeah that's right like there's only one guy access to yep otherwise how do I know the roots gets him like I look for customized in magnets you don't know roots yeah that's a penalty so measurements have the efficiency that you can pre calculate roofs and magnets because we assume very short liveness of things or very short lightness of the peers we can predetermine roofs so there are certain ways to for every health pick the next best by acquiring their peers right before the hall but like if there is like 50 people between you and the internet you can do that right so I cast like potentially to like a list that's that's how it works in the worst case in the best case what we have right now is that you try to first linked to the peers who have a specific service that you need and then hop further on like the worst cases is having thing yeah yeah validation every message is like is there a pattern no they go sale let's say I realized somebody else's message especially on a stove yeah so we've talked about the idea of kind of like an act where you know once we've seen it we acknowledge we like to catch of it so like we're not gonna like push this if we receive it one more time we're gonna like ignore it filter problem being is some what that does is if the context that if you're local if the one if you're man that disconnects from like a larger group you end up pushing that really quickly and then the message gets dropped so we've played with the idea of kind of like expiry times and stuff like that but it's still kind of a big open question until we can actually deploy this in a bit of a blur and a larger scale to actually like kind of see like how long does the message survive you know until it can actually get to its end user one of the problems we've had right now as well as testing anything Bluetooth is Pichet they will be optimal if we could like use something like white blockers spin up all our notes and run it but that's close to impossible because of how Bluetooth works Bluetooth the way it works is like it's driver dependent it's operating system dependent and then those operating systems implement different api's and so like our Mac implementation would be really nice if we could just spin up fake Macs and test it but we can't do that because us so you can't spit out magnet VMs because it doesn't exist unless you spend on packing Tosh's which then usually don't have the proper Bluetooth drivers so the entire testing of this requires us to actually get together physically and test it which has been kind of hard considering daily living yeah we're gonna work on something better yeah if you have any mac books you will donate we'll take them so here what happened is we're both running a real heir with just a basic service that implements just normal messaging between Eric and I it's a essentially it's a terminal chat up that just spends a main channel its cents to a service directly so that anyone who would be connected to this channel would now just be receiving these messages and our Pro Cemetery which is pretty cool next thing I'm going to do is I'm going to show you the code does some of this stuff this pump is also too small you guys know how to use your any questions while dc2 message replay they give the prettiest partitions and then reconstructed you replay messages yes message to retransmission by yes if you like exactly yeah so right now what happens is a message will walk around in a network for quite a time until you know it's no longer choose to retransmit it it's a passive it's a very naive sending algorithm which I will show you here here we go that's some so this is essentially the same function and what happens in the first check is we ensure that the message I there has a recipient which is a device or a service which is like a characteristic like if you're in transactions or whatever the recipient is a identifier is identified by a public key so that one recipient can have multiple transports attached to it kind of how multi addresses work in the p2p and then what we do is we try to serialize a message because it's a protocol buffer that's not really that important and then we get to we iterate over our transports and our transports have specific peers and what we first do is we check if that if our recipient isn't empty we try to find a peer to send to that that has that specific PR ID and if we do that we return we've done a successful set we don't need to send it to anyone anymore what we do otherwise is we look for all the peers which implement that specific service and we then flood that message to every single member who implements that service and when we do a flood send we ensure that we're not sending a message to the person we received it from where the person who originally sent it so if for example Eric sends Greg a message and then great passes that home to me I won't send it to any one of those anymore but for example if America sends Greg a message and there's someone between right and I Greg may receive that message again because we don't have the entire sentence to really have the first because that's the original sender and the latest because that's the person we thought of from their optimizations we computed this and then we count how many peers we sent it to because with if we sent it to none now this is part of our naive approach we end up just letting it to every one of our peers which is kind of ugly but it works for our current implementation news case so that's pretty much architectural E versus what like Iraq with the geographical range talk about you have different listen there's other mechanisms yeah there will be different like ways up because for Bluetooth as well you do the limits itself on to eight peers so we need to optimize that we're hearing with best possible peers of a specific time with Wi-Fi direct you don't really have that you can start direct connections with I think almost like 16 or 17 devices at the same time bluetooth you're limited to eight so the way we appear right now very naive but there's open research issues on how to best appear with other members so because of the limitations of all these things like we have naive implementations but we know of course that we need to have better implementations and discovery and routing will be different based on device of course like if I have finding etc but yeah that's pretty much the entire scent function and this is pretty much the hardest part of ultralight being like when you implement a transport you don't need to care about this when you met a service you don't need to care about this we take care of that then I'll quickly show how [Music] transports work should in a very rough way because there's a lot of ugly code on top of it what happens is a transport asset implements this sending this listing function and they then have callback functions which is like the transport received certain data which is which the node implements so that when data is received the node tries to unpack it checks if it is supposed to handle that data and then passes it one layer up which would be for example your application so the node does all that checking for you so if you implement electrolyte beam you can be certain that you're only receiving packets that you know how to handle or that were sent to you directly and are not just packet - forward because right now what we have ultralight beam notes are all have a forward capability so that we can have this strong network so even like if you were building a chat app and someone else is building a game for example and those both implement butcher light beam what will happen is your chat app might actually really someone else's game at packets and that way you can build like a really strong network like just imagine in New York City or blog set and both telegram had all fine messaging you never waste data again minutes you just walk through it at this point so yeah that's pretty much what we have I'm gonna show the real layer code as well a lot of this code which we have external and the layer is going to become no native code later on that's an open pool request right now but real layers are really simple as well what happens is it checks if we have that specific service and then that message is processed by specific service so in the case of our chat app it was really just decoding the data that we had in our packet I don't know if I have that person joined it I don't have internet oh yeah see if we had a multi being you see it though it certainly the real Ayres will just be having a much three that they find in frustrating they were like opt-out so this is kind of there's this so what we're thinking of right now is like an accounting model and then we do some form of tit-for-tat that you wouldn't continue relaying messages through for someone given that they have used up all their bandwidth unless they for example start paying but the second we have some naive accounting model we can figure out the incentive models on top of that so that's why we started it as New York because we thought about this would be cool for you that also have payment channels on top of it so that's someone like you pay for relays which we can do and it might be it would probably end up being she preferred than having to get data into certain places yeah exactly [Music] are there any questions not yet like we're very early stage in spam attacks amendments are relatively hard to solve it's called laser so we were looking into more traditional amendment routing protocols and there's three of those essentially that are used for amendments because the problem the problem with a lot of these routing protocols is that they don't work the second you have any aspect of connections made very short-lived side of a quasar thank you yeah midnight routing is hard it's there's like as Dean said there's like three different kinds there's obviously trade-offs in like processing speed early yeah you can either have like very accurate routing or you can have where you would have to do a lot a lot of computations so yeah one of our next steps is going to be investigating like building out all of these different ones and do the actual artist testing them out and obviously doing original research on that I can see if we could talk about something better as well and also with when it comes to the real layers to kind of address like your question I like that ideally in a sales falling like an ER here is an example of an official it's a little bit easier you can probably get away with like with just like a mobile phone or a tablet you know kind of doing that and in against city or in a much more dense area let's say let's use the protest fine then right like a protest that goes on we have like a lot of people are running around ideally what you could do in those situations is kind of set up like actual devices that are meant to be real layers it could be as simple as like you know battery packs in a backpack and like very powerful computer running in their real areas that are intended for dense areas should not be a phone they should be something that you know can actively take a beating and are expected to take a meeting so you can get the throughput we need and again it's like specific it's for the specifics to say I want to contribute to this like you do have the kind of capacity attached to it which allows for the data any sort of yeah yeah yeah exactly so the beauty the beauty of all this is that you can choose which services transports want to use you can choose Bluetooth Wi-Fi you can also choose what type of a node you're running obviously in centralization laws would be cool to like you know Constance is like more of like a as a goodwill it's like a public good like you know when you need this it should be used i don'ts for we don't it be beautiful if this is like running all the time like a subway you know but in ideal ideally and realistically speaking this is something that should be using like very for the use cases and like concerts and whatnot to which like people are going in understanding the conditions and like knowing what like this is just like a very viable way to communicate yeah so ideally I didn't you've never actually had a real ER yeah like given there would be enough device coverage in a place like New York City you've never actually had a real layer because you would just halt until the destination was found that's also something that like guess I glossed over and plank you don't actually need ever eat the internet in these use cases we always say the internet so that it's like way like your phone connected can talk to people but in reality as Dean says if like at a concert you just trying to coordinate like where you sit where you're standing right and like that's what's really powerful is that you can say like where you are kind of like the same thing with like you know live protests or whatever you can like it's a lot easier to coordinate where you are when there's like a million people you know on the Raptors won the NBA title I was trapped in the streets of Toronto and literally did not know where my friends were it was awful and like that's kind of like where the use of days like this becomes very powerful and that's why I say it's very like you stay specific I don't think should be like widely deployed 24/7 rather like deployed agree on a location missus you guys have any sort of like countless mechanism besides like I guess what really Bluetooth Low Energy hack the device right it's like basically the biggest I think not troll box right because like if I'm at a concert a million message going from filter flying than my friends just know because so Bluetooth Mac's often aren't exposed by high level in the eyes because of security reasons so what ends up being exposed as a UUID that has changed for every device so there's devices have a public key probably kind of played live p2p so you have an identity that can be found on X transpose okay the UID can be set so it's you know the public/private key cops yeah is there any information like your the tendinous Indian your example about the Raptors downtown trying to find somebody else is there any information about when you receive a message which direction that that came from you could potentially use this for about our quarter naive friends aren't saying here's what we oughta size you're trying to get to where that message initiated physically because of all these like yes uh the inferior yeah if you can do that because a group of people I think there's two separate groups of people one at university one of the one of the University of Toronto where they were actually they're able to steer ethically possible where they were able to actually do in or offline GPS by bouncing off of Wi-Fi and Bluetooth signals to understand where they are in proximity to the building so it's definitely possible and I do know that you do have access to the range and vo - yeah I don't know yeah yeah you know if you give some sort of bounces something so you can it would have a size range yes si equal zero strength yeah but you have to do some like pretty nifty accounting to kind of like basically you'd have to like essentially like retranslate the entire shape right because you'd get some like really ugly shape and get some zigzag and you have to basically find click there's a straight line somewhere along with those joggers so bluetooth you're looking at with 35 the tops yeah yeah you only have like 35 feet tops you like realistically speaking you can probably expect to get like upwards of like 8 hops like if you're in like close together like this enough that's out immunes you can probably at least get it off see you like there's a it's a lot of data to be also transmitting across the wire because we don't we don't keep that the you know say we only took the last recipient in the first one we could pay you mean at a counter to some degree but you know it's also difficult yeah yeah for sure you can like attach like a bunch of metadata to like your your packet based on link replay the previous sender and then based on that you can decide like kind of which appeared to send it to next best right like using like the our our SSI example I guess generally signal strength doesn't necessarily mean like somebody's brother or not but I mean sometimes it does but like you can if you were trying to disseminate information really quickly right you can attach like our SSI information to like you're getting at it and then you try you prioritize sending it to like the person who's furthest away and look down on that and within that way you can like so much information like super quickly right because you know that guy's far away so that he tries sends his peers pops Brandon and you guys think about using radio in one slide talk about rabies how we're gonna do it as a troll transport you know someone's working on it right now actually control yeah because it's like you know those obvious the radio of dudes and like sent messages across Europe embassies yeah someone is working on a transport for that put a little cream here you can read your goes a lot further right yeah and you've already established that it's like you use the acknowledged thing you're talked about earlier and they found each other potentially have something within the radio signals trigger sort of on some sort of frequency that's all so whatever it's very light payload and then boom you've got a natural direct path to know you should do going on github there's there's research repository and if you have any ideas like that relationship with like any research interesting research ideas like we're trying to get as many people as well as possible to do comas what's that company that does I think they've done Bitcoin transactions over radio yeah yeah but the difference about their thing is you need external devices for their mission what's do you know the name of the company someone keeps sending it to us yeah we've kept team like we don't wanna do this this is you need separate devices okay so and it also says Bitcoin transit okay so it's specific to their device can you generalize no it's it's a generalized thing that then someone built Bitcoin stuff long but it requires specific hardware it requires your phone to be plugged in one at and the list goes on yeah ten exactly oh yeah it's patented it's closed source that's yeah well it's the bugger spike the main reason we're doing this yeah there's like obviously I think the umbrella well no now in Hong Kong there's like because firechat used to exist yeah when what happened there is the developer was told by the Chinese government to stop building on it because he's a Chinese citizen right and now there's like another one and they have an SDK but it's a closed source sdk and you need an API key weird that you can have them in our chat and there's like the other thing because they're both proprietary they're closed so that's written by us oh yeah average of Pi yet they're closed source they cost money to use it's like we don't want to pay for this we want to be able to like just download it and use it you just did yeah I think it's what I think it's MIT yeah it's all MIT like yes and that's the point right steal steal it if you run it use it like this is a free public good for people right this is and as you said you hate radios that just means we can as soon as we get any better bands and - play further yeah and once we get more like low frequency bands just like think of the possibilities here we can literally do like because why would I want to use the internet when it's attached to me I speak French there's a lot of possibilities here design space is huge it's awesome yeah wouldn't you meet Akash the unique idea the messages that you're providing around if you can run into an infinite loop with yeah yeah so you guys are caching with you we are caching right now in the naive implementation but there's a discussion by one on how we must do that oh right there there's also a question on if we need a build that data sync protocol on top of it something like TCP so that we can assert that messages are somehow received but the question then there is how do we do that with all the holes that are in between two points there's a lot of there's a lot of ideas that we are floating around but a lot of research that needs to be done to figure out a solution unlike this clever chassis to do that so if for instance you had the ability you also have a little frequency band we could and you could in theory set something up where the acknowledgments go over the low frequency bands that everybody can receive it and we don't actually use it to actually just drop the network and you go and use Wi-Fi and Bluetooth we can send way more you know or bigger payloads across and that that could be a way to like cheaply do it you know when the device receives message it goes hey I received it like fine thank you right but again these are all like open questions and obviously becomes harbor challenges and we're trying to make it so that ideally doesn't matter like you shouldn't have to have hardware I'm gonna have the challenge of like battery on a mobile phone Bluetooth and Wi-Fi direct you don't want that to be volume all the time because at one point you're gonna run out of battery yeah there's a bunch of those technical jobs should allow people to harvest the battery level three layers you see that that's actually a nice yeah that's right all right [Applause]