Simple and Open Identity Management with IDX - Joel Thorstensson

good morning buffalo corns hopefully we ever we all had a really good night last night after the first day of youth denver i'm happy to announce that we had over almost 4 000 people uniquely sign in to watch eat denver on day one so pretty amazing i have to say i mean it's um just a testament to the community if you haven't had a chance yet go check out the art exhibits come to the gamer jive virtual event space today and tonight there's a free roll poker tournament with a 15 eath prize excuse me 20 eath prize so with the you know the numbers and where they go it seems to be a pretty interesting thing it could be fun tonight um i'm just we're just going to cut right into the beginning of our programming today so um i'm happy to introduce joel from ceramic three box labs so see if we can get joel in here joel how are you doing okay pretty good well it's actually afternoon here but what good morning here good afternoon there that's the beauty of doing an international event right we have actually people that it's the middle of the night too so um take it away it's all yours all right cool so uh hi everyone um uh so i'm a co-founder and of three box labs and recently we've been working on like the next upgrade of three box which is called idx and i'm gonna jump into kind of uh how you build applications with idx but first i'm gonna give you an overview of what id exists and kind of the structure of it and like why why building applications in this way is extremely powerful so idx is really a protocol for open applications where you can kind of let the users have control and the nice thing about it is that you can store user data without having to keep your own kind of database uh for the user so it's kind of like a decentralized user table and you can really store any type of data so you can store like public information about the user like their profile or their blockchain accounts or like accounts on other networks as twitter github or you can store private information that's encrypted to only the user or selectively disclose to other users you could store links to data that live elsewhere such as like textile uh databases or or btb databases live directly on ipfs or filecoin or point to something on rweave or some other blockchain or peer-to-peer database you can also like point to resources in and centralized database of course like old data that might not yet have been migrated to to the web 3 kind of modality of like user centric centricity so how does idx work well idx is basically an index where uh that's like a key value map from a definition to ricker and a definition kind of defines a data set and the record is actually the data for for a specific user so you can see it's a key value mapping and and this is kind of the abstract uh describers of uh the um definition and records so a record what's the record so for example a record could be a profile so here we have the profile of satoshi for example it could be um your crypto accounts so here we have like an ethereum address we have a polka dot address a falcon address a cosmos address it could be a list of blog posts and these blog posts doesn't need to live in the record itself they could be live here we have a blog post on ceramic here we have a blog post on ipfs and there's a blog post on media um and it could be a social graph so like a list of other identities that you're following and these this is just like an array of the ids uh so that's kind of what what the records could be like so so each individual user of course have their own record but the definition is more global than that so here is the definition for the basic profile so basically is the name a description and the schema so the schema is kind of interesting because it describes the data that is allowed to go into the record so in the case of a profile the schema is tells you like you can use a name a description image an emoji and maybe something else and is basically a definition and the definition is something that's created by developers um when they're creating their app and like defining the data model of their app and the api of idx is extremely simple you have to get data you use basically the idx dot get and you pass the id of the definition and you can pass the the id of the user you want to look up and then setting a setting some a record to idx is pretty much the same definition id record i just idx it said all right so let's jump a little bit deeper so uh i kind of talked about the the the index and definition and record and here's like a good visualization of this so we have definition it's a ceramic document i won't dive too much into ceramic but ceramic is essentially a way to create documents that you can update and change in a fully decentralized way so definition is a document created by the developer schema it's also another document created by the developer and then the user comes to the app they see that oh the app once used this definition and then this record for that user gets created and in the case of multiple users you have the same definition but each user has so user one user two they have their own indexes and the record they all of course have their own records which the record is also a ceramic document um okay so uh so let's let's like remembering these kind of definition ids is not like super easy so in order to make the development experience easier uh we have created something or like something called aliases and this is pretty straightforward it's just a map from a name to the definition id which is this random kind of looking number or set of characters so ali is a map from names to definition id and there are some default aliases and you can of course add your own so let's look at what kind of definitions could look like so with idx you can really store any data and you can create any number of definitions idx by default comes with some basic definition one is of course the basic profile it's a very simple profile definition that has a schema and the schema is basically name image description emoji and a few other things and an example record could look like this crypto accounts i think i mentioned this a little bit earlier but a way to link blockchain addresses like ethereum address pocket.address filecoin address to your uh decentralized identifier and the decentralized identifier the did is essentially the the main thing the identity that idex is the idx data index is associated with and then there's something called the 380 keychain and this is essentially what allows you to link multiple blockchain wallets or blockchain accounts to the same dad so you can like authenticate with different wallets but still have the same identity and you can of course like find existing definitions that other developers created right now you need to like browse go into their social source code probably and like figure out what their definition they're using uh in the future there will likely be a bunch of different tools to like browse existing data sets or like definitions that developers have already created and then of course you can create your own definitions and i think that's really why we're here today to explore like how can i create a data model for my application and when you do this you create a schema so here's like an example so this is a json schema this defines how json data is allowed to look like so you create a schema you you then create the definition i'm gonna go through this in just a moment and then you can add uh your own alias for that schema all right so i think that's it for the overview and now we're going to jump into the coding so we have this um repo that i've created um it's at ceramicstudio at denver 2021 and we're basically going to follow uh this this simple guide today and i'm going to use it as like an example of how [Music] uh you can use or build an app with um idx so the first thing we're gonna do is clone this repo i've already done that so i'm just gonna run npm start and let's start the application um all right so now we can go to here localhost and all right so the application is loaded and the first thing we want to do is we want to change the metamask account so i'm going to use an account they haven't used this app before so let's use account 7 and let's connect so the first thing happens i'm choosing wallet so metamask and now this is 380 connect so 380 connect is essentially a thing that allows you to create a d80 and tie that to your blockchain address and then 3d connect also allows you to tie multiple blockchain accounts to one identifier on the id and now it's asking me do i want to tie this this new account to my previous account and i want to create a new account for now so i'm going to say no first thing that happens is i create sign a message this is basically how i authenticate to my dad so now the the 3d id it's created in the background i'm requested to sign another message and this message actually links my ethereum account to my the dad that was just created so this is you see here the id dose is created and this creates like a link so we can find my data and my dad from my ethereum address and this this kind of works with the same for any nft right so right so we're logged in so let's just look at in the console i open the console here to kind of uh show show you some code stuff directly because that's that's the the most fun part all right so the first thing we're going to do is um we're going to await idx and get index so so the index is used to mapping from definitions to records and we can see here that we have two definitions and records in in our idx and just looking here we can't really see what they do that much but i know that we have one that's like crypto accounts because i just linked my theorem account to my the id so let's load that okay so it says basically my account here uh ethereum address 0x573 um 0x573 is linked and here's like the the proof and stored as like a ceramic document within here i'm not going to go super deep into that let's try to load the basic profile okay so nothing was set here um so we're actually just gonna add data to our basic profile and this application is already prepared with like a simple function that sets your basic profile um so let's do that and the code here is really simple and i can actually show this first um so i hope this isn't too small i might want to zoom in a little bit so let's look in the app code so we have a function here called update profile and really what it does it's very straightforward it takes the name and the description from the fields here and then does idx.set basic profile name and description all right and and that's that's it so coffee and spacebar is like messing with me so let's set this as our profile and once it's been set we can call idx that gets basic profile and now i can see oh okay i have added um uh i have a profile here cool all right so that's kind of how you use existing definitions um so now let's create our own definition so this this example app um is a note taking app but but it's secret notes so the notes are encrypted to myself but they can also be encrypted to one of my friends um and this this app will store data directly in an idx record and you can imagine this app being built where the data is actually stored on in like each individual note is stored on a separate idx document which would allow us to update individual notes but for simplicity's sake we're just going to create a definition that allows to store encrypted data directly in the record so let's look at the description here of how we do this so the first thing we need to do is install so the ceramic cli and the idx coi so i've already actually installed these so i'm just going to run the ceramic daemon so up until now this application this is running against a public ceramic node that you can try to use for development purposes but to make it easy to kind of publish schema and publish your definition we want to have those on our own node that we have on our own computer we're going to run a local ceramic node and this ceramic node connects to the to the clay test net of of ceramic so we can actually send and view documents across these notes so then now uh let's create the id locally in our development environment so this now i'm going to use the idx the idx cli and i'm gonna create a b80 uh created the ad as you can see here this is a key did and this is kind of less sophisticated than the the 380 did method and because here you can't like add new auth methods and and like update the id this is the static public key essentially um so now we want to create the schema so schema publish and we're gonna use the local pid that we just created and the schema that we're gonna use is here so this schema is pretty straightforward it's called secret notes it has one property which is called notes and that's an array of these encrypted blobs and these encrypted blobs are jwe um and this stands for json web encryption if you're interested in that stuff like how encryption works and how authentication works and how that works with ibfs and i'm going to have a talk tomorrow that can go through all of that in much more detail so i'm going to copy this schema and i'm going to go back to my console paste it and now the screen is getting published to my ceramic node and next thing i want to do is i'm going to do idx oops definition and so we created this command before but i'm going to remove and use the schema that was just created so copy that paste so what we do here is we set the schema of the definition we created to the schema we just created to the name of the definition and to secret notes and we set the description to secret notes for myself and others actually myself and friends okay let's see so okay the now you just created this definition so what's this definition actually like how does it look like well uh we can actually so now this this exists on my local mode so we can do um ceramic oh that's not good uh i need a space this is the content of it we can also actually load this directly from the clay gateway so okay way clay dot surrounding network slash api slash documents and let's paste the document id and now we loaded the document from the gateway and this contains a bit more information because this is the entire state of the document not only kind of the content anyway okay so yeah what happened there is like the gateway and loaded the documents from my local node on my computer so the gateway is actually run by three box labs infrastructure so that's the two ceramic nodes communicating with each other okay so now that we actually created this uh let's see where to go yeah here this this mission we can go into our idx setup here and we have this all asus array and we can just paste that here and now [Music] we call create create idx you create an instance of idx and pass ceramic and aliases and essentially just create an idx instance that has our definition included all right so let's look at what this application actually does and like how it does it um so let's connect here again so we have two different um functions here we can perform one is create a node and one is to view a node and creating a node we can either include a recipient or just have the node for ourselves so secret i'm just going to create this secret node [Music] and once it's been created so well well it's been created actually we can look at how this creation process works so create node is the function that's being called here and first we load the ceramic record from uh or the idx record from the ceramic network uh using idx to get secret notes this is the alias we have created here um and then we get the recipient and the note information from our form here um we set the recipients to be both our own the id so we have window.did and get the id that's the that's the string of the the identifier of the user and we also get the recipient that was included here if it was included then we encrypt the note to to both of these the id so we take the note data and the recipients and essentially what this does is resolves the dads it takes the public keys and encrypts the information to these public keys and then we do idx set secret notes and the record here that we just created and the interesting thing here is like if you try to set some data that does not conform with the schema that data will not actually be allowed to be set to this idx record which means that you can have certainty around like which type of data your application actually has so if you have for example have create a record that includes um some some like um skills or something like that you can enforce that needs to have like skill proficiency level and when you learned it for example um in in every like skill you add to to this record uh as an example all right so we saw that we created notes uh let's load the notes um and yeah we have this note that's called very secret and of course we can add more notes um right and loading the notes is kind of very similar uh in terms of like decryption and encryption so we get let's see so first we get the the either you can type in a user here so the d80 or you can just load your own notes which i just did here and basically we get the notes from idx so idx get secret notes and then we just take all notes that we get and decrypt them and then put them into the the web page here so let's do something more interesting so let's copy this d80 and let's switch to another account let's take account three here and reload the page so i'm going to connect my wallet and now we're authenticated with this other d80 so now if i if i try to view the notes from the ada copy i will not be able to see the secret note but i can also create note that is encrypted to the other user let's create this node [Music] and what's happening here is we take this the id we take uh my own the id encrypts the message the the nodes to both of them both public keys and then yeah and once they've been encrypted they're added to the ceramic record of my user like this user that's currently authenticated [Music] and super secret and we can create of course like something that's not to the other user so let's do this and let's see how it looks like so we don't actually want to load from the id of the other user we're going to load our own data all right so this is pretty straightforward the cool thing now is that we can see that we get the recipient here and we see the name of the recipient because they've filled out the we filled out the basic profile of this before uh so we have um hello hacker enjoying a thunder and this is a super secret note all right so let's copy the d80 of this user and then let's go back to the other user so i think this was account seven and let's reload let's connect again now let's paste the d80 of the user we just were authenticated with just before this let's try to load the nodes okay so now we see that i see these notes that the the other user created i don't see the super secret notes so you can kind of do this kind of selective disclosure kind of thing uh cool all right so one thing that's might be a little bit annoying is like now i have this dead the id uh that and this has this like long thing and it's like different from the ethereum account and maybe i just don't know the ethereum account of the other user how can i load their notes without that so let's remove that and can actually load it from the ethereum address let's try uh let's paste that here and load it so this is the ethereum address of of this the same user that he's created and yeah it works the same way and this is going to be built into idx but right now uh it it is not so we need to like have this helper function uh that's basically is called eighth address to the id and what this does is it takes blockchain address and converts it into a d80 and this of course only works if this blockchain address has actually created the d80 in the first place but this allows you to like uh find out find the public encryption key of that blockchain account and because they have the id so usually blockchain accounts don't have like public encryption keys they're used like a hash of a public signing key so what we do is we use the ceramic api we create a document this document is called type 10 link type 10 is like a standard for representing uh blockchain accounts in an agnostic way so we can represent ethereum [Music] bitcoin falcon polkadot cosmos of all near like all all the different chains um and yeah we just take the address that we got and convert into the d80 okay so that then that's that's this for this application uh as you can see like this was very focused on one specific use case but really you can use idx for any type of kind of data model that you want to have in your application where you usually want to store your data user data and [Music] it's really kind of straightforward to create these schemas that enforce the data to be formed specifically and you can store data like directly in the records as we did here like either public or encrypted or you can store references to data that's like on textile or some other peer-to-peer database or on some centralized data database all right well i hope you enjoyed this tutorial and and good luck on the hackathon and and yeah thanks thanks joel good to see you we really appreciate the time that you put into that it was very informative and i'm sure that everyone's going to get a lot out of it um come check out the ceramic in three box labs event space later on in gamerjab they'll be there to answer your questions or in discord or invertly uh so make sure you engage with them they've got some really interesting things happening