The Case for Smarter Use Specific Sidecars | Wil Wade - Frequency

[Music] all righty hello everyone and welcome I'm will Wade I'm one of the core contributors to the frequency blockchain and today I'm going to make a case to you for the use of smarter more case specific side card for blockchain now before we get into the details there I want to talk a little bit about side cars what are they all right there's a couple basic points sidecars for the most part are standardized pieces of code they're API servers that you can deploy they're usually gateways to other servers there's other code that's actually doing some of the work it's doing kind of a translation layer and it's usually run by the consumer of that API now that's not always true you can see that little Asis and we'll talk about that a little bit later but the key here is why like why do we care about sidecars for blockchain in particular and why might we actually use one so this is kind of a a story about complexity so in the case of frequency frequency is one of the core use cases is decentralized social media and decentralized social media is complex there's a ton of data working through the systems there's a ton of pieces and parts we can here's a few pieces of it right there's a decentralized identity piece there are network operations there's public Keys encryption keys that have to be managed that have to be handled there is the actual social graph itself the encrypted social graph in the case of frequency and how that's managed on chain and there's the content itself that's flowing through or references to references of content as you might have need and so as we were working through all of these details we were having a lot of trouble with different people integrating and we came back to that classic age-old software engineering tool which is instead of building social media infrastructure for users directly let's build infrastructure for applications and so this looks a little bit like this in the case of frequency we have users they use applications those applications talk to the blockchain in this way we've kind of protected users and isolated that complexity away from them there's still ways to ensure self- sovereignty through direct no Fe delegation revocation but a delegation system this allowed us to say okay instead of users having to worry about all the complexity of the blockchain and such we're going to push that complexity over to the applications applications that that are probably run by people who have software engineers and people who can interact with that complexity but as always goes we had a lot of other problems to solve right so how do we get these applications on boarded quickly some of them are Legacy web2 applications that means they're they're running Scala they're running all kinds of different languages that might not be as common in the web 3 ecosystem there's a lot of complexity in web 3 right that a web 2 developer might not be used to right if you're trying to integrate an application you're going to end up with wait you say I just send this off and I I don't get a response back yet I how do I find out my return value right classic problems as people get into the web 3 ecosystem there's also data access right where is the data coming from from besides the blockchain do I need to worry about ccip or ipfs or any of the other decentralized file systems how do I get all that together how do I verify that it's true there's queuing involved right if I'm going to send batches of information off to the blockchain I might need that problem I might need a secure keyspace if I'm a application that's pushing things to the blockchain actually interacting with paying fees and such I need to have a secure place to keep that key and to have those interactions so many more problems and this brings us back to the classic right complexity You Can't Always Get rid of complexity but you can choose where you put that complexity so as we work through the kind of classic problems right so here's classic web 3 right you have blockchain you have clients nice and simple and here complexity obviously has a cost on the blockchain side it costs money to do things on the blockchain and so we try to do as little on the blockchain as we can and that forces every tiny little piece of complexity over onto the client side uhoh now we've got a lot of code we've got a lot of bloat on these clients people oh I want to build on top of my blockchain protocol what do I do well you bring in that nice cool SDK and it handles all that complexity for you or you try to do that and we definitely went through this as we were working on the social media interactions how much can we put into an skk how much can it do and of that original list right it did help with our onboarding speed it didn't help really though with Legacy web 2 applications right primarily because of the language problem there we couldn't build five different languages all at the same time necessarily it didn't hide much of the web 3 complexity for those applications didn't help handling the interactions with ipfs or other decentralized social uh decentralized systems of file storage queuing didn't happen but we did have this was very good we we had a nice secure key space right if the application is doing the SDK they're kind of managing that key and making sure it's safe so we got a couple wins here but not all out so let's go to a slightly more comp right A lot of people use infura or other providers maybe pinata for ipfs and so you have this common infrastructure that you can connect up to and let's see again you know we'll apply again once again the our our fancy sdks but we haven't still solved so many of our problems right our onboarding speed definitely increased that's good we've got a little bit easier web to application integration we we've managed to get language independent with these third-party infrastructure pieces we've got data access kind of figured out but we still don't have queuing we don't have now we've lost our secure keyspace right it's on a third-party infrastructure server that's a problem so we were thinking well what if that infrastructure can be more interactive can be smarter more case specific to our own use cases right what can we put kind of in that original diagram what can we put in between those applications and the blockchain and that is kind of where the the sidecar pattern came to use for us so here's an example application infrastructure that someone might build obviously still simplified but using frequency and the frequency social system right you've got your kind of all classic backend pieces right that you need to do if you're trying to provide search on over over top of the systems of social media if you're trying to provide user specific details and streams you may very well need to have a lot of back-end data that you're using caching to provide good experiences for your own users but notice that this backend no longer has to connect directly to the frequency or to ipfs it doesn't need any of that access instead we have this kind of Sidecar slot that we can put in all of those pieces and remember we've got a lot of those pieces and this kind of brings us to everything's green Bliss right this has changed our ability to do onboarding it's made it really easy for legacy web 2 applications to really get up because guess what Legacy web2 applications speak they speak API they are really good at it doesn't matter matter what language they're using they're really good at it we've got our data access we've got our ability to do queuing with redus or other cues that can be added into the system and all of this including a secure key space because remember it is not shared infrastructure it's shared code right all of this is still running inside of the application the the company who's running this application it's running inside of their infrastructure their controlling that security model but with the ability to also easily integrate so this gives us all of this place to stuff all of that complexity that we have right remember we had a lot of complexity we have that decentralized identity pieces that interact with it we have those Network interuptions like encryption keys and such we have this encrypted social graph that also has to be compressed and paginated to get on chain we have content management that gets batched and pushed out all of that can get pushed into these little side cars in fact we've split it out into multiple side cars so that each one is as specific and independent as possible so if you want to sit there and say oh well I'm going to use the sidecar for the graph service graph works just like I want it to that's great social graph all good but I want to do some crazy things with content or I want to do something special there you can only you don't have to use the entire package this is again a great piece of this this kind of shared code that many different people can run they can choose which parts of it they want to run they could even Fork it and modify it to run themselves so when do you think a side guard when do I think you might want to start thinking about a sidecar as a solution right because SDK is a great solution infrastructure that's public infrastructure like ipfs and such great Solutions when specifically might you want to use a sidecar so one is services so if your users aren't people but they're computers if you're actually doing infrastructure like this is doing for where the applications are saying hey we're going to take care of this you don't have to worry about this complexity if you're trying to talk to servers sidecars might be a great solution for you right servers are great at talking apis last I checked I didn't talk to humans and apis at least uh that seemed to be a more natural language interaction but but this can also apply if you're interacting with EIP 4337 the alt Bim pool alternative ways to pay and such this might also be a solution for you so if you're trying to operate a bundler and you want a more specific bundler to your specific use cases this might also be something to go look at another thing is if you're dealing with a lot of offchain data right if you don't want the user to have to worry about validating data going and fetching I mean do you really want your user to have to be interacting with ipfs now I love ipfs it's a great technology that they built and the other decentralized file storage systems but it is not quick it's not easy it needs a cache in front of it if you want to build a great user experience you may not want your users directly interacting with that data this also matters if you're dealing with data that that might change over time right on the blockchain we don't do too much of I'm going to change the structure of data but we might very well do that with offchain data however with a nice API we can hide a lot of that purely data structure changes and instead put that complexity all the way back into those sidecars finally if you're trying to do web 2 Integrations this I think is the biggest piece tier that pushed us toward this solution because web 2 applications they already speak apis they don't talk most of the time in web 3 time scales they don't think in probabilistic they don't think about finalization nor should they have to think about many of those things if the if the integration doesn't require it so are you not convinced yet I mean I thought I made a pretty good case but maybe you're not convinced so let's take a look at something most people are pretty familiar with here ens right ens is a relatively simple setup and it's it's got a nice wonderful set of sdks already so congratulations to the NS crew and all of their contributors that's a long list and yet it still doesn't capture all of the languages that are out there so we've already ran into that problem of if you're trying to interact with someone who's in a ecosystem that's not listed here you're already potentially in trouble and having to Port an SDK just for that it's also you know just what are we trying to do with it so let's look at the simple resolve if you're trying to just simply resolve something you need a connection to ethereum now that might not sound like a big deal right but if you're a server you need to make sure hey is that is that connection to ethereum still up what about retries do I need a retry do I need a fail over all of those different interactions when you're working at the server level do you care about finalization does it do you care if your ens lookups are finalized or not maybe you do offchain resolutions right ens has wonderfully expanded out to l2s and using ccip do you need to be able to connect to those do you want that details in your application or do you want it in kind of a shared code base that you can use there's also reverse lookup do you want to be able to verify the canonical address is this the canonical address is it not do you want caching do you want an interface that matches DNS and actually serving this up over Port 53 hopefully you're using htps DNS but all of those pieces lead to a very set a large set of complex questions for something that is at the end of the day a two parameter readon only function right we didn't even get into the details of of putting in a new ens entry and making sure it's up to date and managing the resolvers and all of those pieces and and this is a great SDK I'm not saying we should make a sidecar for ens but I'm this is a great little example of how a sidecar can help answer a lot of these questions and make your infrastructure a lot easier okay there's also a secondary benefit and this is actually one of my favorites which is you get great API documentation API documentation has evolved over the years but now with swagger and open API I there is great UI you have a nice standardized way that almost any engineer can look at this leftand side and parse it and understand what's going on and then it plenty of great uis to interact with this this is a system that while not necessary right there's plenty of great API and great documentation for smart contracts and such this gives you a much more feature Rich right this is specific this is is versioned right so you can talk about hey hit this version of the API it's going to do something different oh the underlying protocol is changing we need to deprecate something we need to throw an err we need to return something all of those become much more possible when you start interacting with API documentation that is actually both interactive and has things like mock servers and other things can easily be built out of it so the final thought right is at the end of the day it's a question of where do you want to put your complexity your code your protocol that you're interacting with your blockchain all of it has complexity and your choice is kind of where do I put that and there's a lot of different pieces right that go into that where is the cost how easy is it for users what are my users look like are they humans are they computers how is that going to interact and so all of those choices kind of come into play and I want to make sure side cars are kind of one of the things that come to your mind as you're thinking about well what's another place I could put this complexity I'm I'm it hasn't worked to put it in an SDK it hasn't worked to put it in shared infrastructure and these kind of sidecars give you that option to do that so again uh I'm one of the contributors to frequency you can learn more about frequency at frequency. XYZ and read about kind of the side cars and how they actually work we have documentation and such you can read on there and kind of see how it works if you're interested in socials particularly I suggest go look at and decentralize social go take a look see how it's kind of implemented and if you're just interested in the sidecar piece go look at and kind of see oh here's how they made decisions maybe we made the right decision maybe not you can feel free to please go to the GitHub and put some issues in put some questions in and I'll also be around later with that I'll see do we have time for questions maybe one question anybody have one question if not I'll just stand up here for another two minutes and until this guy kicks me off all right there's one uh nope not a question I thought there was a question but there's not that's great hey you can catch me afterwards I'll be hanging around thank you again very much and I hope you enjoy the rest of the conference