Open Source Search: Two Decades of Bad Design Decisions & Legacy Software - Berkeley 2024

[Music] hey everybody you see the title it was originally intended to be 20 years of bed design decisions in open source search but now I decided to go even like Beyond search so a lot of the concepts that I'm going to mention they apply to a lot of software storage databases uh file systems operating systems browsers and so on so there's this uh weird thing that we see in software all the time when and the consumer comes in they want features uh the consumer can be anyone can be another developer then we have the maintainers of Open Source projects who would say okay we're going to add that and we're going to do that by stacking another layer of abstraction on top of it uh which kind of almost always results in collecting more Legacy but then uh the third part of the puzzle that we keep forgetting about is that the software is changing but the hardware is changing as well and the direction in which Hardware is changing or at least has been changing last 20 years is practically orthogonal to almost everything we do in software the newer devices with a lot more transistors are a lot slower so you type in a button on your keyboard on a $3,000 machine from last year and it takes more time for it to render on the screen than it took 40 years ago why is that search is one of those places uh where we kind of use search in every in every application we use it in different forms but it's one of those things that is even though it's like ubiquitous it's one of those features that's never present in the standard library of any programming language or any environment the first root cause is about virtualization the more obvious thing to Benchmark however than the memory consumption is like how well you utilize recent assembly features comparing assembly to python or like other high level languages would produce Gap that even for a single core execution would be in multiple orders of magnitude and I'm not even talking about like concurrency and parallelism another problem that happens a lot in modern machines is IO and in most cases when we think about IO it's either storage or networking in fewer cases it's like the broader memory if you start using those libraries directly in the C layer or forking them and trying to PCH them you realize that the interface of those libraries kind of forbids you from using them properly in a synchronous manner the problem that it incurs is that when we have a process that needs to do IO in between we are wasting hundreds of thousands of CPU Cycles uh that are spent just waiting for data to arrive from somewhere from disk or from Network the last but not the least still huge is how we deal with strings so this is actually going back further than 20 years a lot of the string processing that's happening in C is influenced by the language B and back then the registers in the modern CPUs were really scarse so we used just like one pointer to describe a string and then everything that went into liap C had string interfaces that didn't pass the string length as an argument So within every single function call we had to compute string length and only then start processing there are a few projects that kind of help avoid all of this Legacy um here are the ones that I love the most so to make virtualization cheaper for all of us as a community I strongly encourage you to look into web assembly um there is a company and a project that takes it further so if you want a hardware friendly programming language built on familiar compilers there is a big effort by Chris lutner and his team at modular designing the mod programming language there are new system languages if you don't want the legacy of L lvm and ml like Zig which comes pre-encapsulated with its own compiler 2 Chain but then there is also systems like spdk and dpdk for colel bypass and the probably simplest approach the one that we follow designing custom kernels and operations for the most data intensive workloads but probably most importantly of all efforts is this one which brings all of it together into one single open source uh search engine it's still a work in progress but at the time of writing it was already used by some of the projects at Google uh it was used by open source databases like click house and now close to a dozen other database Brands some of which are open source this is probably like one of the world's most deployed search engines it also runs on mobile devices so I'll be more than happy to see all of you participating in this effort and ping me if you have any questions thank you