Urbit 🛠 Apps on Urbit

[Applause] [Music] all right everyone thank you for joining the workshop today um we have neil davis here it's going to be a half hour workshop um workshop is titled urban apps on orbit so the urban team is going to be obviously engaging in discord throughout the upcoming hackathon you can reach them there and also their own channels uh you'll be able to ask questions throughout you can i'll point your your attention to the zoom chat is probably the best way as we go and with that i will hand it off to neil all right thank you i hope my audio is coming through clear there so i would like to talk about what it looks like to develop on orbit architecturally i'm not going to spend a lot of time talking about the underlying programming language and some of the other details if that's something that you need to get into to participate in this component of the hackathon then there are a lot of resources i can point you at but what i'd like to mainly focus on is the architecture of the urban system and how this impacts the kinds of apps that you'd be able to build and how it would relate to something like ipfs or filecoin and other kinds of decentralized applications as as these things come out so as we look at orbit as a system um i'm going to [Music] enumerate a little bit of what urban does for you and try and give you a feel for what it is it sometimes gets described as a personal server which means something and i i think it's a correct way to say it but i think a lot of people miss exactly what what's trying to be communicated by that so in the first place orbit is essentially it's an operating system uh we call it or we generally treat it as an overlay operating system which means that it's running as a platform on uh within some other operating system and this is similar to the way that a lot of cloud services run for instance and it's built as a functional as in language platform so when you operate with it you can treat the main the main kernel as a persistent event state machine here persistence refers to the fact that events are written to memory uh before the output of those events is emitted which means that it is it is completely stateful and an event log is retained of the that yields your current state at any particular time it also means that in principle whatever state you get your operating system into is understandable and you can find exit paths and adaptation paths and upgrade paths that can restore it if you've gotten into into some kind of unworkable state there are a lot of advantages of having a functionalism language os in this sense that's the orbit os component and there's another part of urban which is the orbit id which serves as a cryptographic identity and so this is a unique point it's like it's like an address space point uh similar to you know like like a regular um ipv4 ipv6 internet address url uh you know like once but but they're not dynamic they're static it's like you know once you have that it is identity that you can treat as something that you can attest cryptographically cryptographically that you own and you can interact with this gives you true peer-to-peer decentralization and it gives you end-to-end encrypted networking and the ability to to build apps on top of these os level primitives so these aren't services and third-party components that are built on top of your operating system these are pieces that are baked into your operating system at the ground level that it has these capabilities and it also means that when you build applications what i may call apps or sometimes agents it means that they have a standard structure on the back end similar to you know if you're writing a a chrome extension it's expected to have a certain shape and interact a certain way because chrome is the platform that it will always be working on top of if you're writing a net application there's going to be certain requirements that that has and by defining and requiring a certain standard application structure for everything that runs on orbit it means that you're able to define very portable ways of interacting with data that are held by by other agents you can start to treat the entire operating system as an ecosystem rather than a set of ivory towers that sit next to each other and that you'd have to interact with that way now i'm going to to turn the system on its side and talk for a moment about what urban looks like for the developer this is a little bit complicated uh you can imagine that if you were drawing a a diagram like this for developers to take a look at for you know unix or a lot of other systems it would sort of be equally as complicated of a system but i want to start us in the middle there with that red block which is the orbit os that was the first part i was talking about this is where this is the state machine this is the the event handler the the processor that takes care of everything that you do on orbit and this essentially serves as a back end and an application running layer that handles everything that that goes on above it so these these letters that i have a b c d e these are components of the system that i'll talk about more in a moment they're called veins but they're essentially system daemons they're system services that persist that provide a certain it's like systemd or something where they're just they're they're persistent services that are available to you through one of those the the j one which is short for jail it interacts with urban id which is uh bootstrapped on ethereum and so you're able to pull the state and you understand the state of your own you know the cryptographic state of your own identity and the cryptographic state of any other urban instances that you are interacting with on the network and then on top of the the g the the user space vein this is where everything that a user does is interacted with as they do things on the system most of the time when you're building an application in orbit you only really care about the top parts of this you don't have to worry about what's going on on the host os or the hardware or everything down at the bottom however it's worth discussing this briefly because it does inform a lot of the architectural design decisions that were made that that make orbit what it is so urban is designed so that everything that runs on it compiles to a language specification called knock this is similar to a byte code or you know kind of what what an llbm or a jvm or you know other other systems do it serves as kind of a a micro lisp vm specification this is what the system runtime the the executable file is actually operating it's interpreting these instructions translating them down to the whatever the host os and the hardware layer needs to look like to actually evaluate those but we can treat it the same way we would treat jvm or evm or you know anything else that's a virtual machine that we don't care about the underlying hardware we just care what it does for us and we assume that that runtime is doing the translations for us correctly now when we write things that would interact with say you know ipfs most of those things are going to take place through the system veins which are the colored letters here right above arvo or they're going to take place entirely in user space which is where end user applications live urbit serves as a as a backend in many respects i described it at the beginning as a personal server this means that you have a cli or a repel interface that you can interact with you can work with it but for the most part you're using some other application to provide a front end for you most frequently this is the browser there are a lot of react based apps front-end apps that talk to orbit via a regular http api and so you're able to do a lot of things and just pure json put get requests and you don't have to worry so much about what you know programming on urban as long as you know the architecture that it expects right like like any other api you can treat it that way but we do need to talk a little bit about what the operating system does and how it handles these messages and everything and so i'm going to take the top part of this from urban os arbo up and i'm going to map it in a slightly different way which gives us a different a different view on what's going on and i'm going to briefly check chat okay nothing in chat all right so turning this on its side um what i'm envisioning here is that the central circle in the middle represents the the runtime system that's actually operating everything there's a ring around that that's in the brighter colors that's hoon arvo zeus and lol this is arvo which is the orbit os layer hoon is the language so this is the the components of the language and then zeus and lower the standard library and then on top of those run the system services and then based on those system services would be applications and end-user applications that we'll talk a little bit more at the end here so if we if we expand the sound we just look at what the system system services are these are essentially your your operating system primitives these are the things that urban as an os knows how to do for you that you don't have to worry about provisioning if you're writing an application that works on these some of these make a lot of sense putting networking it's udp-based packet networking like i said end-to-end encrypted so this is handled by one of the one of the veins which is ames uh there's a timer vane which does you know wake up said at a guaranteed you know aft after a guaranteed time there's a file system internal to an urban ship you know you could think of this the same way you would like a it's not an s3 bucket but you could think of it as similar to that right like it's it's it's a way of representing all of the data on the system as a tree uh there's a terminal interface uh the relative sizes of these are are drawn from the size of the code base in terms of the lines of code so you can sort of see basically how prominent different parts of this system are or conversely how how relatively simple it is to implement different parts of this there are server services client services user space refers to the apps and having a standard interface the way you would for a net or a chrome extension there's some cryptography there's some various tools for doing that most of them are architected towards urban id but there are some other cryptographic primitives that you can use as well so largely that's what this is built around as a back-end system this makes a reasonable amount of sense you might think of a few other system primitives that it would be nice to have but if you're looking at a system that's going to act as a as a server instance or behave like a server to the things that are interacting with it you have a lot of things that make sense here you don't have things that at the current time don't make sense to put into an application like this so you don't see for instance machine learning algorithms right these are these would be relatively inefficient to do on a vm style system of course we do this all the time on the cloud as things currently stand but but you take my meaning there you want to run things like that as close to the metal as possible there aren't visualization tools for instance because those would typically reside in the front end and this is this is kind of purely a back end view of the system and of course you can do that in user space you can do that for end user applications but primarily that's going to live in whatever client is talking to orbit as a server inside of that you have the operating system layer itself which is this the standard library components the event handler and event log and the uh the the language which is designed to to build everything and work efficiently in that regard orbit has its own language called hoon that things are done in um it has some interesting features from a computer science perspective everything in it is a is built as a binary tree so everything in it is is a leaf or a pair of of uh nouns and we have a we have lots of materials on that if you find yourself needing to delve into the language it's kind of a refreshingly different way of thinking about language construction and i've found it to be a helpful and rewarding way of thinking about what's going on with this and then at the center of this we have the actual run time which is our vm layer which is doing you know it's actually evaluating and executing the noc specification however receives that from from the urban os layer that is above it as a hackathon participant you are primarily going to be interested in three components of the orbit system this is the file system the server vein and user space user space refers to apps or agents this runs in a system the the percent g refers to gaul that is the name of the agent uh basically every end user application behaves as a persistent daemon that may or may not currently have a front-end client session that's attached to it so that's running in the background sometimes you open up a browser or an app and you talk to it but you're not always doing that but the the identity the stateful identity of that application continues and you're able to work with it as i said these all have a standard form i find that this makes it relatively quick to spin up new instances when i need to write something new because i know exactly basically the handshake that it needs to make with the system to work with everything the vein itself gall will broker your access to all of the other arvo services it's kind of a message passing interface they pass discrete messages called cards to each other and this is how they communicate this is these are the events that are handled by the the event handler and event log in the middle and then it's going to communicate with whatever front end you have via http http api uh using the air server which is this one so it has http api endpoints um it pri it tends to prefer a data subscription model right so there's different ways you can think about constructing an api um urbit likes a data flow model wherein you open a channel and you listen for that channel you listen for events to come over that channel so there's a cookie-based authentication that sets up this channel and then you either emit json events down this channel or you receive them back from this channel so you can communicate entirely in json and get put requests so you know you don't have to know the internal language of the system to understand how to talk to the system and then mime types are arbor level primitives they're urban os level primitives and so it's relatively straightforward to do conversions from the json that you're sending in or whatever values you have stored inside of that to whatever internal representation you need stepping back for a moment to the file system this is an interesting one so clay is a typed file system um every bit of data on it has a type associated with it and so and it's also a version controlled file system which means that the file system doesn't treat everything as a binary blob it actually understands something about the structure of the files that are stored there right the same way that you know if we change the the extension on a file name um in mac os or windows or whatever else we may get a warning from the system but of course it doesn't change the data right changing txt to pdf doesn't do anything to the data it just confuses the system because you've labeled it wrong orbit actually attaches that to as metadata to the file uh data which means that you were only able to access it through a type which is called a mark and so you know once you have data in some form on the system you can access it through different equivalent marks right if there's more than one equivalent way to access it if it's a json uh maybe you can open as an xml because there's a well-defined mapping between json and xml and of course you can open it as a text file so you get this for free by working on this this operating system it's also referentially transparent um so it collapses everything down it's a global namespace which means that part of the prefix that is on the location the path for any file resource there is the cryptographic identity that refers to it and so in principle you're able to refer to anything on any system in practice of course there are permissions and everything that you have to worry about and then currently there's a two gigabyte limit on what can be stored in clay this isn't the fundamental limit it's basically just a in media res development limit uh while we work on that at some point that will be taken off but it also means that something like ipfs and filecoin is very desirable for us because we would like to be able to store uh larger decentralized files without filling up our nice typed version controlled file system for applications where that those guarantees may not be as important right if you're storing a a mp3 or an mpeg or something you don't really want it under version control so much probably because as a binary blob that doesn't matter right you're gonna you're just gonna your your diff is just gonna be deleted and re-uploaded it's not gonna be a sensible diff so those are the three components there uh before i field questions i want to make a few comments on uh four or four hackathon participants so you know you have three weeks right hackathons are accelerated and they require you to focus on the economy of what you can learn and get done in that time frame so my suggestion at this point is to focus on client-side applications um that build on your strengths coming into this so for instance a browser front end or an app front end that communicates with urban on the back end but doesn't require you to do as much server-side development for many of you that will probably be faster you can work from good code models so there's currently an s3 integration which can be found in the urban code base and i'm happy to point anyone at that that's the sort of thing that you would extend if you for instance wanted to do a file coin s3 integration layer that would be the the part you would look at so there's a lot of good code models in the current code base that you can use to reason analogically about how to build other things during your hackathon window and then we have some specific learning resources called home school and app school which are available at developers.urbit.org i myself will be present on the discord and you can find me on twitter at sigilante or on orbit at laggard of noc feb but i'll be i'll be on the discord as well and so this is my my lightning fast look at what is going on with orbit are there any questions okay so i will yes i can show you some example apps and what that would look like so let me let me show you what it actually looks like to run it really quick all right so what i have here i have my my regular um bash prompt so i'm going to go into orbit and i'm going to locate a particular orbit executable and then i have an urban instance already sitting here in a folder not currently running i'm going to go ahead and open it and then this takes me to a repel so it'll sit at a repel there's nothing really going on here that's that's particularly interesting but you will notice that it has web interface live on http localhost 8081. so if i come over here and i open that up it asks me for a login so to log in i have to get a code from the back end it's always the same code for the default uh example ship here so i'm not revealing any secrets by showing you the password in this case but i'm only running this locally so of course you'd want to be more more cautious with that i don't need to save that and when you open that up it will open to some window that looks like this so these are apps that are installed by default when you run such a ship for the first time and so there's a there's a web terminal which is similar to the the command line repl here there's groups which is the main social media application there's a bitcoin wallet that i can link to and then you can install other applications if you're on the live network this particular example ship is not but if i go to my my personal ship which is running on the network then i can show you the kinds of things that are available so there's a there's a lot of applications that other people have built and shared all of these um show up in this this single box all of these are peer-to-peer distributed so it's kind of like bittorrent where if someone decides to put something out there i can synchronize directly with it so if i click into one of these like the docs app it'll go ahead and open this up so all of this is talking to my server back-end process that in my case is running on a raspberry pi um people run them on hosting people run them on their own laptops you can run it on a digital host and ocean droplet you know wherever you want right like run it in whatever case and so then um well this this is adopt a docs application so it's not like the most uh flashing exciting one to look at but there are some other ones that uh people have built uh there's a few game ones like like a wordle clone here that you can you can go through so there are this this is what it looks like to to run an urban instance uh what the actual experience is like the the code itself or exit here and um let me briefly show you some system code all right hoon is a language that is not keyword based and so it looks more like something like apl everything that starts with a double colon is a comment so most of this is actually comments on the line that i'm looking here so it's like apl but it's but it's like an ascii based apl so these are defining um structural or syntactic relationships between the data that are that are part of the system um it's a very expressive language and there's a lot of equivalent structures that you can you can set up that would do something similar but give you different ways of thinking about or structuring or interacting with the underlying data it sometimes blows minds a little bit when you see it for the first time because it does look like this kind of very very terse language but there's a lot of power in that and it for instance most of this is defining uh instances of um these are these are data types that are going to be used down file in this particular instance here so this is this is the arvo definition file itself right so it's an open source operating system so of course you can take the you know open up the hood and look at anything that's going on inside of that all right other questions okay very quickly what would you tell someone to convince them to use orbit what's the value proposition who's the target user okay so if you need true decentralization peer-to-peer apps if you value end-to-end encryption and if you value data ownership then urban is a good application for that it has been shifting in the past year to focus more on web3 applications for doing things like maintaining off-chain state uh you know ideas where you could do attestations where you do a calculation and you post the hash of the calculation to the chain and then you know anyone else can go verify that you did the calculation it was actually involved um if you need cryptographic identity that's really valuable i think you'll if you use orbit you will find that not having to worry about re-implementing a lot of common primitives like password systems and handshakes and um you know web web server apis uh you know and api endpoints systems these sorts of things um not having to re-implement these things and not having to pull in tons of libraries to make them work is powerful and valuable to you so that's the quick version of my my value proposition then i think i'm about out of time on this yeah we're we're about at the end here in terms of the scheduled time um but if anyone has any other questions we could take a minute a minute or two more but if not just to remind everyone that the urban team is available in the discord and in their other channels also thank you very much neil for an excellent workshop today encourage everyone to keep the conversations going my pleasure and um thanks everyone for joining much appreciated all right see you around