Ceramic: Building Web3 Apps with Sovereign Data

[Music] thank you so much everyone from for joining us today um and i hope that you're all having a lovely day we have joel here from ceramic um and he's about to host his workshop building web3 apps with so run silver and data and yeah i'll hand it off to joel now and i hope you all enjoy the presentation thanks hannah nice to be here uh all right so let me jump in okay so as i said um well maybe before my name is joe um i'm the co-founder of bluebox labs and we've been building ceramic and so today talk about how we can use ceramic to build uh web3 apps uh with sovereign data so maybe let's start with like exploring what actually is something data so if you think about various types of data in kind of web3 you have kind of different things you could you have like a file and ipfs that has some cryptographic verifiability but you basically have a hash of it and then you can retrieve the object but you can't like change that now you could have a contract on ethereum as states and changes over time but that is essentially requires you know to sync the state of the entire blockchain um and finally i guess you have like uh other types of like cryptographically verifiable data uh like a jwt which is essentially just like some data object that has an attached signature to it uh but all these kind of things are static or you need to like sync an entire blockchain to know the states what if we can have like singular objects that are mutable and changeable over time but they have like a persistent identifier that's you know based on the contents and we want these objects to have like cryptographically cryptographic access control so some public key kind of controls the axis and about like what which updates gets are allowed to make into this object i want to be scalable i don't want to have to you know download an entire blockchain used to sync the comments on a web page and then verify those and verify that the comments kind of came from the right source so if we can have this kind of primitive like a data object that's like scalable and have this cryptographic access control immutable that's really what software data is and if we can have that on kind of internet scale you can make pretty much the entire internet verifiable and so we can have this interlinked data data objects that kind of are independently verifiable and uh we can start building kind of a graph of knowledge of data without having problems we have day and days where links can die and because servers go offline so with ceramic we have created software data objects and that's basically the core primitives of ceramic is these objects so an object is is made up of a bunch of different data which is called stream and so stream is essentially a linked list of signed objects that contains like a pointer back to the previous object and this is kind of like a merkle dag as a hash link data structure and this data object has an immutable we call stream id and the stream id is based on the hash of the kind of genesis commits the first event in this event log and this stream id gives it kind of like a global namespace so um there's like one global network of all of these data objects and then here we can see that this commit was anchored into the blockchain so we get the timestamp when it happened then there's two signature commits after each other that means that there was two updates it was signed by the controller of this stream and then another anchor you know the times timestamp and not update and this so these updates can contain like a patch or some kind of mutation to the state of this object and in the genesis we we have to specify a controller and that's basically the the owner kind of of this object and this is specified as a did a decentralized identifier which is kind of like a standard to represent an identity and a decentralized identity in a kind of blockchain agnostic way and this allows us to have like full blocking agnostic system but also like the key revocation and rotate change keys over time then so once you have this event log how do you like sync that across the network so if your new node joins to network you have some stream id you want to see what the state of that stream is you basically ask the network like hey what's the latest update of the stream and some other nodes that have the stream reply to you and say like this is the latest update and you shrink the entire event log and you verify the signatures you verify the timestamps and basically process all of the events and derive the current state and so the state transition function used in this system can be different things that we there are different strain types in ceramic so that could be like a key value store or use the json object or an append list or something like that and all these this networking is out of a liquidity um yeah so what certain nodes do insurance keep track of the event log and keep track of the current state of objects and so each node can independently chooses which streams they pin and they choose if they how they back up the kind of state of these all things so once we have these this sovereign data primitive it's like a data object what can we build with it well so we've been really interested in kind of self-sovereign identity and like how you can build applications around user data so we've created a framework called self id and so sophie is essentially a framework which allows you to store data around a person or a human or a user um and and the user kind of remains the the full control of the data so you can think of it kind of like as a user table a hint like a traditional web 2 app but instead of that being locked into the server it's shared across all of the internet so here in this example we can see that we have dids there are different types of did hits here but i'm not going to go too much into detail 38d is a dad method we've created for ceramic there's also a nfd which basically allows you to have an nft as an identity then we can see here for each column there's a data model so a data model is essentially a special type of ceramic stream that defines what goes into this column so each column here gets its own kind of unique name that's based on the stream id of this data model object um so here we can see that there's a name in the first column there's some other properties in the second column and it looks like there's some encrypted data in the third column the cool thing about these data models is we can have basically like an infinite amount of uh of rows of columns here so um there's no limit to like how much data you store with the user and there's also no conflicts like if someone wants a name property that's just a string and another application wants the name property that's like for name and surname um then if they were stored in the same column there would be a conflict and the apps would kind of break if they tried to use each other's data now that there's a data model that enforces structure of the data and there is no conflict so who controls this data well um you could think it would be the data model owner but it's not really because it's actually the user so each user can have full sovereign control over their row in this table in this imaginary table and each cell here is like an individual ceramic object and um the user can kind of choose what to do with this data everyone update it you can delete some data what have you and the cool thing here is this the data model actually describes the content inside of the cell so the data model has maybe a name it has a description it might contain some other metadata but importantly it contains a data schema so this is normally a json schema that describes what format the data inside of the cell takes and the really nice thing about this is that if you try to put some data into this cell if it doesn't conform with the data schema it will be rejected it wouldn't be allowed to be written um so this means that if you're you're building an application and you actually see that oh there's a data model that kind of really is exactly what i need to store with my app then you can just use that data model in your application and you can know exactly what the format of the data will be because it's always guaranteed to be structured like this data schema and so you kind of get this composability of user data you don't need to think about you know rely on some application that might change their apis or change the data that they store or anything like that all right so uh that's essentially self id i'm going to go through two short code snippets of how you can kind of get started using it so here we have an example of how to just retrieve some data uh from the network so we import from the self id library there is this core component here an instance of this we don't include the details i don't think these are not included here but oops but you can look that up in the in documentation how how to instantiate it you basically need to configure it with like a ceramic instance uh which should be pretty straightforward and then we have an alice here that has an id so you can see this is the three id uh and then what we do is we just do uh core.get and then we get the basic profile and we get the crypto accounts uh so the basic profile and the crypto accounts are essentially data models um so by default the self-id framework comes with some some very initial primitive data models for a basic profile for which is basically like you have a name maybe you have an image maybe you have like an emoji or whatever and crypto accounts is essentially accounts other crypto addresses so it might be like an account of ethereum might be an account on polkadot or cosmos or filecoin or something else and they include like a cryptographic proof that ties to alice's id here um so actually so we look here we actually look up with alice's id which is this three i needed uh but you can actually use one of the cryptocurrency if you know alice's ethereum address we can put the l-system address here and look up the data similarly we can use the self-id framework to actually write to um into the user's data store and so here we import an ethereum provider because we're using ethereum we want to use an ethereum and we imported the self id framework and so this is for self id for for web so normally if you have metamask or another wallet they there's a window dot ethereum object uh so you can just get like an address from there then you can just create an instance of self-id um and you do that with the ethereum provider passing the window ethereum objects and the first address we got back from this call and here in this example we connect to a ceramic node that [Music] runs locally on this computer and we use the clay test network and now once we have this self object we can just do self.set and basic profile and here we set his name to alice and now this this self object can of course be used to read data as well from uh from alice's id or also from like anyone else's if you have their address or id um all right so that's just a short overview of like how to use the self-id sdk and to build apps on top of ceramic as you see see as you saw we have just kind of a very simple set and get functions and when you do this so this basic profile actually requires you to like have a name if you try to set something that doesn't conform to the schema that's set here then that would not be allowed as i mentioned previously so if you want to dive into this and like really build on top of it we have our documentation at developers.ceramic.network if you go in there you should click on the tools section and you will be able to find the self id framework and some examples there we have the software the website which is just like a very simple website that demonstrates kind of how you can create a basic profile um on top of ceramic um and if you want to get in contact with us and like have the questions or you're sharing on something i can go to shout outs around the network which is our discord server and hit us up all right that's it for the slides i'm happy to have this answer i see there are some questions in the chat um so let's see maybe i'll start with it oh that's fine maybe i'll start with the shot um so there's a question that's if ceramic is similar to fluence it's quite different actually so ceramic is really intended for these kind of mutable data objects that have this history of data and you kind of have a state that keeps you know evolving as the user or the the identity that changes it you know authors something so influence is more like of a distributed calculation network and so i think there could be some really cool things to build that kind of combines the two and stores the output of these computations in ceramic um but i i can't say that i have deep enough knowledge of exactly how fluents work to uh you know say how to best combine them uh but yeah they're they're quite different um there's another question on key rotation [Music] okay does it let you change wallet without losing your web3 identity so yeah so self id bundles something called 380 connect and 380 connect allows you to connect one or multiple addresses to the same identifier and so you can you know add addresses to this or like multiple accounts this identifier and then remove one uh so you can definitely do that you can like change that over time and choose like which which addresses right now are associated with this identifier um okay then there's two questions about like examples you can build for the hackathon um i think it would be really cool to see some different hacks that um creates new data models maybe for like um um different ways of uh okay so there's a few different ones uh something could be like you know various type of social functionality like you you make a comment or you uh attest to someone's skills so you know someone's address and you you collaborated with them before and you kind of attest to like oh this is this person is good at uh solidity or for javascript or whatever uh so that kind of like uh giving each other's at the stations could be interesting about skills and you can have kind of similar type of structures for other type of i guess reputational data as well and another interesting thing could be like if you have a dao actually build like some kind of comment system and a molotow or some some kind of thing like that where in a dial you always know all the addresses that are members so you could simply like query the self id data for all the users and then kind of build a a view of what the compound kind of messages from all the users are so those are some examples happy to like ideate some more if someone has some fun ideas um this is the last question in the chat could you speak on glaze versus self id the glaze is kind of just a set of tools that you can use to put together an application self id is like a higher level framework that bundles a bunch of these tools together in a way that's more simple to consume so self-id includes something called create connect which allows you to connect to the wallet and connect multiple ids uh and kind of makes the whole flow kind of easier if you have like more custom needs for your application it might make sense for you to use the glaze tools directly um okay there's a bunch of stuff in the chat actually uh if you want to read out your question up out loud i think that would be easier um i am if you want to uh start uh actually i i was uh first uh first i want to make something like an inbox like something for email but for pep tv but uh then i thought that i need a data structure that makes two people the sender and receiver and in some case multiple people to access to a single document and i was i was thinking that uh for for accomplishing this task i need a data structure that uses that have a key for the document that document will be encrypted with the private key and that private key will be encrypted with the public key of users that have access to it and [Music] both of them there is a document key that that is a encrypted document and a list of uh keys for users that have access to that document i was thinking that whether this exists already in ceramic or i i could go and make this as a project or is it even possible something like this because it makes a lot of a lot of web 2 application needs a data structure like this for things that uh even in social things i create a post i want just my family see it i must be able to have some access control over it yeah definitely so um it's definitely possible to build it's it might be a little bit tricky so in ceramic right now we have support for encryption on kind of the did level so we have a library called jstid and that allows you to okay i have a message i want to encrypt that to a did which i know so that might be my friend or my family member and so take the object and encrypt that to that the id and then i can just retrieve that back right [Music] the the more tricky part is like um so that data object had encrypted you could put that into a ceramic stream and that should kind of just work we want to make that simpler so that we have like higher level tools on top of of streams in ceramics we don't have to like think about you know encrypt the object put it in there you can just create a private stream that would be like you select the users but that's currently not um implemented so there's only this kind of lower level support but it's definitely possible to build various types of encryption strategies with that in general i would recommend if you want to build an encryption system where you want to be able to add more members over time to encrypt all of the data updates with a symmetric key and then you take the symmetric key and encrypt that to the public key of all the people that you want access so and if you move if you add someone you can just encrypt the same symmetric key to the new person but if you remove someone you should really kind of like if you want them to not have access to newer things you need to generate a new symmetric key and encrypt all newer things with that new symmetric key and also send that to all the people who want access thanks yep colin you had a question yeah my first question was on the self id website i uh played around with it and when i linked my did accounts there was like that big long hash i thought that was would be kind of confusing for an end user uh same when i had to sign a message in metamask so is there any better you uh ux for that um so i don't know what do you mean by big hash um there's usually two signatures you have to sign one is like okay you have this app access and then the other one is like okay tied this uh out if their email address to my um to my dad yeah all of that there was a bunch of unreadable uh or just it wasn't in english you know it was one a string of letters and numbers that wouldn't oh that sounds like it might be a bug because usually there's always like text in the message you sign but you can at least read part of it um that might if you if you saw that that might be something wrong with your wallet or with the self-id website all right um if you if you actually can replicate that and take a screenshot of it that would be super helpful but in general your question of like making these things more user friendly we are working together with spruce and a few other people on um the sign in with ethereum spec and so that that specification is primarily aimed at signing into backend uh but we can actually use it to to create something called an object capability which will allow us to have a session key in the browser and then sign something with your ethereum while it gives us session key permission to write to ceramic streams so you can actually use with ethereum to sign into like uh not a backend but actually decentralized uh you know sovereign data objects all right cool so that's that's like a medium-term goal for us to like implement that that's not something that's like available yet all right and then um i want something to listen to my smart contract for events that are emitted and then i want to write that data onto ceramic or onto ipfs and then access it through ceramic how do i do that um i mean it sounds like you need some kind of service that connects to an ethereum node and then like whenever something happens uh it it has like um ceramic the id and writes that to ceramic yeah so i guess yeah that would need to be like a node.

js app or like whatever language you prefer and then that just connects to that ethereum point and uh okay would that have to be hosted on aws or uh maybe fluence uh yeah maybe it could be done i don't know if you can like listen to blockchain events like that influence if that's possible that seems that would be awesome yeah yeah all right thank you cool thanks um blockchain creature do you wanna ask your question um all right maybe not i can read it do you think it's possible to build a safer and more uh structured referral program i don't know exactly what structured means uh can you elaborate actually blockchain creature yes can you hear me yep yeah um yeah my question was is it pos like there are currently different types of referral programs which let people who refer somebody get a part of um let's say of a revenue or just a bonus of referring somebody my question is is it even um is it possible to dig deeper and make like uh one person a buzzer door um lincoln which is like having a ambassador who has a self id and that self id is connected to all the referral uh referred person is it possible to have something like that or is it more like yeah what you said like with the comments and those kind of things that are already assistant existing um on the let's say on the web yeah i mean so on ceramic you can't really make payments inside of ceramic directly but what you could do is like if you uh establish these relationships as inside of ceramic and like make them kind of verifiable there so you have this this ambassador as you mentioned that like connects to a bunch of people in in their data stores in ceramic and then if you have like that and multiple people doing that and you could you could potentially like read that data and do like an airdrop to that later um so that kind of approach could definitely work like just like building like a graph of like the directions and then reading that later to establish like an aircraft i see that sounds good thank you um i think we're running up on time but happy to answer more questions if possible all right oh and thank you everyone all right awesome thank you so much joel for the awesome presentation um if any of you have any questions in the future uh joel can be reached on discord uh in the sponsored channel uh for ceramic um so feel free to ask any questions there and otherwise thank you so much everyone for tuning in the recording of this workshop will be posted onto the global youtube and joel thank you so much for taking your time to give a great presentation for everyone all right thanks everyone have a great rest of your day [Music] [Applause] [Music] you