Next Level Source Code Verification w: Sourcify

[Music] [Applause] [Music] next up uh we have frenzy and she's going to be talking about sourcify so a lot of you are interested in verifying your contracts and uh so that makes that a lot easier so without further ado let's welcome francine on screen hey hi guys yeah thank you for having me um i'm super excited to be here um let's get it started there we go okay looks as if i'm all good right [Music] okay cool uh yeah so thank you for having me once more um today i'm going to talk about next level source code verification with sourcify my name is franziska and i am part of the solidity team and i am also part of this little side project called socify and why sourcify is next level source code verification and what you can do with this and why it's important all of this i'm going to share with you later so yeah exactly what will i talk about today first of all i want to highlight again what's wrong with today's ux in web 3. what is source verification and why is it important what are metadata files and what's netsmack why is socify next level source verification and last but not least because this is a developer summit um you can learn how you can use socify with the common dev tools or also manually so let's get started um yeah first i really want to remind us all what's wrong with the today's ux and web3 i mean many many of you might have seen this before we have lots of trust issues and uninformed decisions these decisions that we currently take when we interact in web three they are based on hex data strings uh we don't know what they mean and we don't know if this interaction we do is actually what we intend to do and uh yeah that process um is also called euro signing and yeah this really shows us nothing we don't know what we are doing and many many many questions might arise from this uh some of them could be you know other function arguments correct which contract am i interacting with here is this thing really doing what i wanted to do what is the source code of this and has it been audited or is this the one that has been audited uh is this the right address that i'm interacting with um did anybody check the legitimacy of this do i call the correct function and last but not least can i consider this safe to use based on this hex data string you cannot answer any of these questions and yeah if you're not sure what i'm talking about yet i also brought a few little examples with me um here we have uni swap where i am trying to swap some ease for die and yeah the information i get on which i can base my assumption or a decision on is these beautiful 260 bytes of hex data same as here i'm trying to buy a super rare nft and again i see maybe a bit of function type information and then again hex data but really nothing else which could prove that i'm actually doing what i want to do um yeah a couple of more examples the same here with ens i'm trying to prolong my ens name and again i only get a hex data and a bit of a function type in this case renew which could give me some indication maybe it's the correct thing i'm doing but it is not really a proof and lastly here i even needed to sign a message before i could even see any information on the website and again that is some data that is basically meaningless to me and even though this is a developer summit here today i also want to remind you that our users are not necessarily developers and for them this is even more confusing than it is for us so how can we improve the current status quo luckily there are some ideas how we can even make this whole problem go away and achieve trust minimized web3 interactions i'm just throwing a couple of words at you right now and i don't expect you to understand what all of these mean and no worries because we will go into more detail on all of these later on but basically trust minimized web 3 interactions can be achieved by a nice collaboration between several aspects which include source code transparency and availability source verification metadata and netspec availability and lastly wallet and block explorer integrations so with that in mind i will go through some of the basics maybe some of you are new to the web3 space and uh you don't know what the the term source verification really means that's not a problem at all because i'm going to explain it um source verification um is what we refer to as basically or in other um areas this is known as binary reproducibility or byte code reproducibility and what it basically means is that the deployed byte code that you deploy on chain equals the byte code that is compiled from the source code so let's say a project is open source and it claims that their source code can be found at location x and then there is some byte code and that's deployed on chain and basically by recompiling the byte code that is open source and verifying that this matches the deployed the deployed via code on chain we have verified the source and that's the process of source verification you might know this for example through esa scan because here this is a very popular block explorer and via this block explorer you can do this verification process so many of you might have seen this before um and yeah i will today show you an alternative interface how you can also verify uh your source files and um we'll give a bit more context on why we think that the current verification how it's been done for example by ether scan is maybe not enough it's already very good but it's we can go much further than that and throughout this talk i'm going to use the easter deposit contract to show you what i mean exactly using an example so here you see the etherscan website and we are on the page that shows the deposit contract and if you scroll down and if you click on contract you see this little green um check box here and you can also see here contract source code verified um so that means somebody has done this verification process and the the source code got recompiled and matches the bytecode that is deployed on chain yay um next up what is a metadata file metadata files are actually a quite cool feature from solidity which got introduced many years ago but unfortunately not many people are aware of them or are using them properly but so basically a metadata file is a json file that is generated by the solidity compiler that contains a lot of crucial information about the compiled contract at the very end of the compiled bytecode you see a hex data and this hex data includes already a bit of information it includes the used compiler version and a swarm or ipfs hash and if you follow that hash to for example ipfs and the metadata has been published on ipfs then you can find much more information you can find the full compiler version you can find the complete api of the contract developer documentation user documentation compiler settings to recompile the code to the bytecode so all of this is very important information crucial and some of this should in the best case be displayed to the users directly and let's have a look again what this means at the example of the deposit contract here we see the metadata json of the deposit contract and here you see all the things that i've mentioned before the used compiler version the language here you could expand and see the entire api here currently expanded are the dev docs so the developer documentation here are the user documentation and so on so all the information that i've just mentioned okay but what is netspec which has also been already mentioned in the metadata file netspec is the ethereum natural language specification format and that is basically a spec that helps you to provide rich documentations for functions return variables and more in the form of comments so basically the developer documentation and the user documentation that we've seen before in the metadata file they come from this uh comments that you put in the code using the netspec format and yeah so they are basically categorized and developer focus and end user focused messages and there are also some other categories that you could use as well all of this by the way you can find in the solidity documentation in case you want to read more about it and those end user focused messages that you can use to describe the public-facing functions that you have in your code they are really crucial to users since they basically translate into human understandable words what the contract interaction will entail and uh so that they can decide on their own whether they want to do this contract interaction whether that is what they intended to do or not and so looking back at the deposit contract we see that this deposit contract has been actually commented using this net spec format and here for example we see at def which is the tag that you would use for the developer documentation and here we also see at notice which is the tag you would use for the user documentation and i also want to show you again how this would look like in the metadata file itself so all what we've seen in the source code you can find again in the metadata file it's nicely structured here and it's basically all there just waiting to be displayed for the right person so they know what they are actually doing so why is this important that is especially important because the two things we just described on the one hand the source modification and on the other hand the code commenting in form of net spec commons they together have the power to create greater transparency in contract interactions but if we only have one of them the equation is not complete so on the one hand if we have a verified code that is open source but there are no net spec comments or there is no um code commenting been done then the end user does not understand what is happening even though it has been verified so that is bad on the other hand if um the developer used net spec and the end user can understand uh what they are doing because there is some sort of display of this netspec happening for example in the wallet front end but the code isn't verified so then the description could could be inaccurate or even worse it could mean something entirely else so that is also bad but when the code is verified in open source and the end user can understand it because there is some meaningful information displayed to him in the wallet user interface then that's awesome so that is why we don't only need source verification and we don't only need code commenting but we need both and we need it together yeah so next question would be how do we bring all of these pieces together just looking at the time there somewhere but i think i'm still good meet cersify sourcify is first and foremost an initiative to raise awareness about the importance of source verification and is also an initiative to try to raise the standards and the availability of the verified source files sourcify consists of several products that want to make it easier for devs to verify the sources but first and foremost it's this decentralized metadata and source code repository which i'm going to show to you later but we also see ourselves as bridge builders in the ecosystem and we really try to help make safer web 3 interactions a reality so what we do on a daily basis is basically talking to all the stakeholders in the ecosystem that need to be involved to make our vision come reality what are our goals on a high level we want to enable trust minimized informed work three interactions and that also entails stuff like trending hex strings which we've seen in the beginning in wallets into human readable information and on a technical level um that means stuff like keeping the metadata and the source files available via ipfs that can otherwise be a big issue and also become an infrastructure or base layer which allows other tools to really build on top of it and to leverage what we are offering so here's another little example of how this could look like if we would turn hex strings into human readable information on the left hand we see what we've also seen in the beginning a contract interaction how it looks at the moment in wallets we have a bunch of hex data and we have the function type contract interaction but what we would like to see there would be a function type would be arguments but would also be a description that would display the net spec and then also showing the source and basically a little information whether this code has been verified somewhere or not and then with the option to click on it if you're a technical person to click on it and then basically see the source code for yourself that looks like really a nice improvement to the ux we have today but what do we need to get there from our side we think we need several components for this to make it work on the one hand we need automated verifications or really really easy verification processes that are not as cumbersome and as complex as the processes we have today we need an open contract repository of all verified contracts that also includes the metadata files because i've mentioned before the verification alone without the metadata files does not bring us the translation part so both of them are really needed together we need verification interfaces and plugins that also contribute to the very easy verification process in case the automated verification process fails and lastly of course we need integrations integrations and more integrations because this is a very complex ecosystem and in order to basically bring this benefits that we then have on the infrastructure level to the users we need to integrate with blog explorers and with wallets and maybe even with other stakeholders first of all i want to now describe you a little bit more how this automatic verification could work so sourcify at the moment currently already automatically verifies new deployments that's very awesome and that only has really one condition the metadata and the source files must be published to ipfs so when you are a developer and you let's say deploy a contract to um ethereum net then your next step must be um to auto either to use auto publishing or to publish the metadata json and the source files to ipfs so that our society monitoring service can actually pick it up so how does it work oops like again the deployer deploys a new contract to the blockchain and then our monitor listens to all the new blocks that are being created by the blockchain and basically looks in those new blocks for new deployments once the monitor finds new deployments it checks in this bytecode that i've mentioned before the very last bit this hash the ipfs hash and then checks at this address on ipfs whether it can find the metadata file and if it finds the metadata file and if it finds the source files then it runs the verifier and after the verifier has done its work the stuff is being indexed and pinned and we make sure that the metadata files and the source files don't get lost on ipfs so that would be the optimal automatic scenario and in that automatic scenario no manual verification from the developer is needed whatsoever so this is really different to the verification process you might know for example from etherscan at the moment um let's say you want to verify a contract that you deployed two years ago or it was you didn't publish it on time so the monitor couldn't find it on ipfs or some other scenarios where it needs manual input that's also not a problem because we also have a manual verification interface which i'm going to show you later but this automatic verification we think is a crucial point in making it just easy and seamless for new deployments to be verified so why is this next level source verification i've touched on many of these points already firstly we do no code flattening that means the metadata in a very is verified as well that does not mean that the content in the metadata especially the content and the netspec files is true we cannot judge whether what the dev translated to be happening in the contract is actually happening but because everything is transparent and everything is open source what you can do next is basically as a developer go and see the source code go and see the comments and see whether that reflects what is actually happening or not as a next point all contracts are independently verifiable that means you have all the information you need to do the recompilation yourself anybody could do it so that is not a one-off verification but anybody could at any point verify the integrity of this again and lastly tackling the availability issue so making the storage of source files and the processes around verification more decentralized independent and hence more resilient that means especially uh we shouldn't just trust one third party to verify everything for us and what happens if this third party goes down or uh the database is somehow corrupted or i don't know what can happen so we are friends of uh open source and we are friends of decentralization so we really try to make this process as decentralized and independent from one middleman as possible okay and since this is a developer summit let's now have a look at how you can use sourcify today to do this verification i've just mentioned yeah in the best case you don't need to do anything as i've explained with the automatic verification process and the only thing you need to do in that case is to publish your metadata files and source files on ipfs here's the example of remix where you have a very handy button just right under the compile button you have a publish on ipfs button and all you need to do in the best case is click publish on ipfs and you're done because our monitor will pick the stuff up and we'll verify it without any need for you to do anything so i click publish on ipfs and then i see here um sourcify it's that's how i called this project metadata has been published and i can see the addresses of um the files that i've published to ipfs here so that would be an easy way how to do it with remix there's even also another way how you can verify your sources in remix because there's also a sourcefire remix plugin which you can use as well secondly if you're using hard hat um there is a hard hat deploy plugin and that hardhead deploy plugin um also con includes the specifier verification so all you need to do if you want to verify on cersify with hardhead is do hardhat network mainnet sourcify or whatever network you are on and then it will get verified as well um for travel it's again a bit different um so in travel you find the metadata output in the artifact and in there you find the metadata as a string and not as the metadata.js so what you can do in the case if you use truffle is you can go to our manual verification interface and just drop your source files and that artifact file in our manual interface and it will verify everything so no need for you to extract the metadata or to somehow transform it into an its own json file we do that in our verification interface so you just have to dump this artifact file and lastly here is how our interface currently looks i am sorry that this is a bit blurry i know but i try to make gif out of this so let's see what is happening here um in the first instance you saw the socified.dev website where i added a let me play this from the beginning because this makes no sense for you guys or let's wait until it's over basically what you're seeing here is already the repository um the contract repository with all information but i want to see it from the start again so let's see what's happening here that's the metadata file great i hope this will go back to what i wanted to show you yes so basically you enter which network you have been using and the contract address and then you dump the files meaning the source file and the metadata file and then you click verify but because the deposit contract which i'm using for the examples has already been verified i get the notification has been verified and it leads me to the contract repository where i can have a look at the source file as well as the metadata file here you see the source file of the deposit contract which is neatly stored in the contract repository and then you can also go back and you can click on the metadata json which shows you all um the net spec comments but also the other information that is included in the metadata file and yeah all of this you can see when you go to specified.def and enter the contract address so also if you are interested in knowing whether a contract has been verified or not you can just go there dump the contract address there and it will show you whether this contract has been verified already or not okay so going back to the initial um slide that i had in the beginning how can we achieve trust minimized web 3 interactions i now want to translate these into what you as a developer can do so let's have a look what you can do to help this make uh make this a real reality is you have to open source your code and make it available via ipfs you can use sourcify and verify you verify your source and metadata files you should use uh code commenting and comment your code using netspec and lastly you should either build integrations or encourage others to do so what should you take home from this talk no matter whether you are a developer or user please always verify your contracts or check if they have been verified and especially in the case that they have not been verified demand verification and raise awareness for the safety risks that are there if stuff has not been verified and on the more technical side of things because we want to become more decentralized please help us pin the certified content on ipfs contribute code if you like and generally um all ideas are welcome so please join in our chat um we are on metrics and github so please join there and contribute ask questions um we also do user support there in case the verification should not work um we are on twitter at societ um we are at in github ethereum sourcify and for all the plugins it's github.

com and our website is sourcify.dev and in the d web is ease yeah wrapping up i want to say big thank you and credits to chris eddie fabian jakov leaky and also welcome out to new maintainers khan and johnson because all the people i've mentioned they are our developers and they are the brains behind this project so i'm only the messenger here today and uh trying to be able to answer all your questions if you have any now thank you so much francie this was uh awesome um there were a handful of questions which uh luckily chris just already answered on the chat but i'll still uh bring up some of these uh here i think the common discussion that's going on is how do you think about integrations and obviously there's multiple interpretations of what integrations are uh whether it's about this plugging into existing developer environment to adding more plug-ins to support for how this gets used outside of just a web interface what are kind of plans for that and how does that look like yeah that's a good point so in one of my other talks i also had a slide on all the cool integrations we already have but because the time was short and i wanted to this time highlight how to use it i didn't include this but actually um in terms of so let's go deep into whatever we can okay so in terms of uh developer tooling uh there are already some integrations out there that i mentioned so for hard hit it's the hardhat deploy for truffle i think there's no plug in yet i hope i'm not mistaken and for remix there is the cersified plugin already so on the tooling side of things we are quite good of course on the wallet side of things things could improve but there are open conversations already with some of the wallets which are great and then lastly on the block explorer side the autoscan block explorer recently released a new version which actually includes cersify support which is very cool so um yeah there also needs to be some more work on the blog explorer side of things especially with the big block explorers but yeah we hope that this will happen soon and we are open to conversations and with most of the people we already have an open conversation channel but of course we also know there are so many priorities and so many things to work on at the same time that uh it's understandable that nobody can only prioritize sourcify all right looks like other questions are coming in um let's see well other than everybody just being said about what's going on so um do you have a specific right so do you have any specific small projects a hacker or a hackathon could build to benefit source supply it's like small improvements i think is probably the better interpretation here yeah that is a very good question if i had more time i would definitely come up with some stuff maybe chris has some ideas if he's in the chat um all right chris is answering oh okay great yeah yeah no because i think there's a couple of things that we can do and as i also said we are currently onboarding two new maintainers to the project which is really exciting and i think they would also have some ideas how you could help for sure um but also on that side yeah just i think we have some ipfs issues for example so if somebody's an ipfs expert um or just pinning the repository but yeah in terms of hackathon projects there's probably some stuff that could be done awesome well if uh there are no more outstanding questions um we can end this but this is obviously great and congrats on making this a big initiative in a better public utility so hopefully yeah thank you so much for having me yeah thank you so much all right so with that we are ready to move on to our next talk [Music] [Applause] [Music]