Searching the World’s Computer with Ren Cryptofish and Gary Thung

hi um so today we're going to be talking about searching the world computer um I'm Ren I'm an engineer at Electric Capital we're a crypto native VC fund that is building public goods in the crypto space hey guys uh I'm Gary I'm also an engineer at Electric Capital today we're going to share some lessons from building code search as a public good for ethereum one of the views that we have is that ethereum is most likely on its way to become the largest open source ecosystem but that comes with increasing complexity and challenges what do I mean by this we're going to look at a few metrics that highlight the scope of this problem one of the interesting trends that we've seen over the past few years is that the quantity of code on ethereum in terms of the deployed smart contracts has increased exponentially for instance the number of verified contracts on ethereum has increased 5x since 2020. in terms of what users want most users want transparency because if you're custodying your assets in a D5 protocol or trading an nft it's very scary to do that if the contract is not verified and is a black box so one thing that we see is that around 70 of usage on ethereum goes through verified contracts and that's why being able to have transparency around verified contracts is important but in terms of the magnitude of this code base let's try and put it in context of some of the largest code bases that exist in the world so today ethereums deploy smart contracts has roughly 290 million lines of code making it comparable to some of the largest software Stacks in the world uh it's comparable to the software in your car uh dbn 5.0 and even larger than Facebook's stack but the quantity of the code is not the most is not the only thing to highlight one key aspect is that smart contract complexity is increasing so one way to take a look at this is to see what what the average transaction looks like in 2017 the average transaction was making roughly 1.5 other contract calls that number today has gone up to 5.5 that's a 3.

3 X increase since 2017. and another marker of code complexity is as programs become more complicated they demand more storage reads and storage rights so since 2017 we've seen that around storage reasons storage rights has increased 4.1 X in a similar Trend with increasing contract calls one of the things we've done over the past quarter has been talking to different developers in the crypto ecosystem and trying to get a sense of what their development life cycle looks like and where we can build a public good that adds some benefit and save some some time in their day-to-day process so the average developer life cycle looks something like this a developer initially starts researching a topic it could be a protocol that they want to implement it could be new mechanism design but there is some form of research process once that idea is kind of like solidified they go through and they start the development process a testing process and a deployment process one of the common pain points that developers in the crypto ecosystem have mentioned is the difficulty of researching and having visibility and transparency on the Corpus of work that developers have have already implemented over the past few years and one of the major issues with using GitHub search for developers is that the versions of code that they're finding are not actually the versions of the smart contracts that are deployed on chain and this creates a lot of complexity in terms of being able to tie out what code is actually being run in in production so over the past few months we've kind of realized that as the verify code base has been growing exponentially and there is now this massive Corpus of work that developers are able to build upon and to reference and to learn from and as we see the transaction and code complexity increasing we realize there needed to be increased transparency through a public good that users can use for free and that's why we decided to build code search as a public good for ethereum our app is called codes it's a bit of a meme on coda's Law and it's live today and free for everyone to use and in an ongoing manner so let's put ourselves in the shoes of a smart contract developer and think about what they might want to search for so here you might think of someone who's an integration engineer working on a protocol or a Mev Searcher and they're curious about learning about code references to a unit swap V3 pool in solidity developers commonly use interfaces to interact with other contracts that are deployed on chain so one common example that they might search for is whether all of the instances where I uniswap V3 pool is implemented and called on so we can kind of see that once we run the search the top result is more or less what we expected which is the uniswap universal router and say we keep scrolling down then we find a contract that kind of is pretty interesting for us and we can click in so this is a tornado router and we're kind of curious about how they're using uniswot V3 pool and when we clip in click in it jumps into where where the interface is being called in the code and in this instance tornado cache is using it for a Time weighted average price that they're using for their Oracle another powerful use case is a lot of developers as their learning solidity for handling numbers in solidity is very different than handling numbers in a lot of other languages so say you've heard about a library such as such as prb math and you're curious about how other developers are implementing it and using it in production you can search GitHub for this but you're not going to get any results that match with the smart contracts they're actually deployed on chain so again we can we can use a search such as Library prb math dot Asterix which denotes a wildcard search so we're going to find all cases where prb math is being used where the library prb math is being used so again we get some search results we find an interesting contract that we've heard about so Clipper for example is an exchange that um is providing users with cheap uh execution costs for trades under fifty thousand dollars and we're kind of curious about how they're using this prb math Library we click in and we see the library being imported and as we're able to scroll through the contracts we see that they're using prb math for their in 256. which if I'm a developer maybe maybe I might consider using it as well another great use case for code slaw is trying to understand the composability of some contracts on chain and so what I mean by that is you can find references uh that one contract will make directly to another contract so we can do that by searching directly just for an address and so uh you know maybe you're like a die Maxi or you're just curious about where dye is being integrated and so what we can do is search for the die address uh and then what we'll get is a bunch of results where the die address is hard coded and so we kind of got some interesting ones here um the first one is uh an Ave Monitor and so that's from D5 saver uh the second is like a Ave Improvement protocol to wind down Ave but let's click into the first one so same as before you know we're able to go straight to the reference and dig into the code and uh you know do some investigation and figure out what's going on here one thing to note is that you know at some point we want to integrate bytecode as well as uh storage data that is not um constants and so what that will give you is the ability to search over the storage values as well so and in that world this uh query for an address will actually pull up a ton more results um where users have manually inputted the address and the last use case that we think is super powerful is to explore the code base for vulnerable contracts and so let's say you're a security researcher and you just came across news about the Nomad Bridge hack um you know if you look at the breakdown it of what some of these firms put out um they break down into the code and point out the exact lines and so you know you have this function acceptable root which is the source of the vulnerability and on code saw what you can do is you can search uh for the function signature and that what that's going to do is going to pull up a bunch of contracts that also Implement that function signature and you can reasonably expect that they're implementing the same logic and so after searching that code slot gives us 38 results and so what we can do is kind of scroll through click into the results you know if you're a researcher dig into the logic and see if these other implementations are susceptible to the vulnerability and you know if you find something that's acceptable you probably want to notify the team that is responsible for that contract uh and so that's some of the use cases um for code saw it definitely has to be build out more features um you know the power of it will grow but what I want to get into next is a little bit about how we built it and so essentially code search is like a search engine over source code and so a naive implementation to solve this problem would just be to like run grep over every single file but that's like a pretty inefficient Brute Force solution and so to solve this we can employ a two-step process um and so that involves pre-processing our data and then running some sort of query algorithm on the data uh and so the state of the art for doing code search is um kind of pioneered by Google early 2000s and they use a chart Ram index and so essentially that's a data structure that Maps three letter strings back to some document ID and so what I mean by that is I can demonstrate that here with this tour example where you know we have these two these two documents they're kind of short lines here but essentially you go through each three-letter string sorry three every three letters in the text and you note that that three letter string appears in that document and so by the end of it you end up with this this big lookup table of three letter strings mapping to a list of documents and then in the querying process oh sorry before I go on um in the indexing process we also score the the contracts to cut down on you know searching for junkie contracts or just you know ranking the results to serve you better results and so some of the signals that we put into that scoring is the biggest one is is transaction activity um you know with blockchains um there's not a lot of spam and so if a contract has a high transaction activity you can sort of trust that it's high quality because people are willing to spend money to interact with it and there's some other signals we use such as like deployer score um if a deployer is known for deploying like good contracts um and then also if a contract is forked commonly uh then that's probably a pretty good contract and so for the querying process what that involves is you take your query and you you extract the trigrams from the query and then you go to your lookup table and you pull out each list of documents that corresponds to each of those trigrams and then you return the intersection of all of the documents and so you basically want to find the files that contain all the trigrams and then some of the other you know Parts boring Parts I guess uh of the app is just like the data processing Pipeline and then the app itself so the data pipeline kind of what we do is whenever an account goes active on ethereum we put into the pipeline and if it's a contract account then you know we'll put it into a bunch of stages and extract information from the blockchain in various sources about the contract and then the web app itself is like a pretty standard modern web stack we use next.js like react postgres and the last part we want to talk about is the road map for Coates law and some of the things that we're thinking about for the future uh and so the first one I want to talk about is peering into contract storage um and so um this is interesting because uh if you're familiar with how um solidity is built for example there's a notion of private variables and these variables are not easily exposed at the top level for users and so what you can do to extract the private variable values is reconstruct what the layout of the storage should be and you can do this by using the original source code and then you can query specifically for the data at those storage slots um and so we're hoping to get that out soon and the second thing that we're super excited about is um featuring annotations on contracts and so you know what I have here is a screenshot from this website called Rap Genius which is a website where you can look up song lyrics and the community can annotate the lyrics and explain what they mean uh in this example specifically is an annotation directly from the musician and so you can imagine on code slaw if you go to your favorite contract and your or a new contract that you're trying to figure out and learn you don't have to do a hundred percent of the work of learning it from scratch you can dig into the annotations that the community has provided and jump start your learning and it would be also really cool if you could you know see a protocols account that's signed off on an annotation much like this example over here um and then you know just also some other features and improvements is um just always trying to improve the search quality and the efficiency adding more powerful filtering and sorting and then also dark mode which is a highly requested feature um so yeah that wraps up our talk um you know we'd love for everyone here to check out code saw and give us some feedback and you know provide us with any feature requests you might have thank you