EthCC 2: Marcin Benke

hello my name is marcin benka I work in the research and development group of the Golan project and one of the main interests of our group is economy of decentralized markets and today I want to talk about a recent work on pay as he use in Golan but first let me tell you what is Golan and how it works golem is lesson is a peer-to-peer that fully decentralized network of computing resources with any theorem based market place and how does it works well network golem network consists of nodes who can be providers requesters or even both who are than these requesters and providers well requesters post the computation tasks to be executed and they form the demand side of the market and providers supply the computing power to execute those tasks of course this doesn't happen exactly for free well the pro requesters are provided are prepared to pay for the tasks to be computed and providers supply the resources in the expectation of monetary remuneration of money and this is the this economical side of Golub is what I want to talk about today so there is a bit of math behind all that but I'm promise I'm not going to inundate you with big hair equations but I rather want to talk about some intuitions behind it so I want you to start with picturing a familiar situation you want to hire someone to do some work like writing a program designing a web page or something like that or on the other hand you are willing to to do this book now the important question is this situation how is how should we set the price and this is this is an issue both for the contractor and for the world now the two common models in this context are either fixed price for the whole work or an hourly rate also known as time and materials so if we look at this if we look at these pricing models then it seems that fixed price is saved for contractors on in in our case requesters but it's a big risk for workers when job size is unknown and if we post if someone posts just computation requests on a peer-to-peer network there is no way of knowing what the size of this job actually can be on the other hand if we look at time and materials well the roles are somehow reversed now it is more or less safe for workers because now the workers say how much they have worked and how much they they will charge for the job after the job right because fixed price says the price is fixed before the job and time and material says well we set some hourly fee but at the end only it out at the end of the day the contractor the worker says how many hours or how many units it has worked so it seems that from the requester point of view fixed price contracts should always be the choice but it's really so let us look closer at this models again so if we look at the fixed price model we can notice some challenges that come with this model so first of all the requester has to estimate the size of the job the value of the job and errors in this estimate can lead to adverse consequences also the problem also the workers well obviously the workers have the problem especially in the decentralized and remote settings if workers don't see cannot estimate the work at any rate they just have to be they tell the price without knowing the size of the jobs so how are they to beat on the task with unknown size well this is not entirely unfamiliar situation in the real world because we can do if we if we you put yourself in a position of a worker that is to do some job well you can you can make some estimates but these estimates are only only often only rough so what you do if you have to beat on a job of an unknown unknown size well you factor all the risks in your price so you set a higher price so that's the problem with that's the problem with well two big problems for requesters right one problem of this problem for for the workers but two big problems for requesters on the other hand if we look at the time and materials model well ii from the real life works it's considered generally good for jobs of unknown size if we cannot estimate the job beforehand it's generally good to use the time and materials model if the both sides are honest it's fair to both sides of course the obvious problem with primary model is the risk of overcharged by dishonest workers so what happens if the world charges too much well it seems it's a really big problem but it's the lesser problem then it seems to be an even less prop even lesser in the centralized work because the requester always remember it always has the option of not paying at all and in this decentralized world if we have no course well the only the main risks for the requester of not paying is losing reputation what does mean that it loses the reputation well the same worker will not work for the same requester again but if this worker in question is dishonest this isn't so bad at all right so I I posit that there is appeal in the time and materials model of course we have not we cannot forget about the risk right about the problem and this problem I'm going to talk about today so let me put this into context of Golan but even though the model I'm going to talk about today is applicable not only to Golan but to other situations as well but it arose in the context of Golan so so far used fixed price contracts for the rendering use case which has this this feature that you can somehow you can estimate the size of your rendering jobs well this estimates can be can be off but you have at least some idea you have the scene you know from the Xperia store at least you have know from the experience how much work is it to render to see but as we want to introduce new use cases such as general web assembly computing we have the need for no new billing models because the the fixed price model has a problem I told you beforehand that the they pose a problem not only for the worker for the provider but also for the requester who not necessarily can estimate the job correct okay so we introduce the new billing model called pay as you use which is a new time and materials billing model for Golan and in which providers charge for resource consumption for now we focus on CPU time but of course the same model can be applied to other kind of resources and multi-dimensional resources such as used in cloud computing okay so let's come back to the main challenge of time and materials billing what if you providers inflates your resource bill and you get a bill like this 1 million dollars ok well one thing that's also known from the brick and mortar let's say world of time and material contract is you set budget limits right so it's called time and materials not to exceed so you say ok as many hours as you I pay you as many hours as you work but no more than thousand dollars so if you send me a bill for 1 million dollars I'll tell you to get lost but this doesn't solve the problem at all what is well it solves part of the problem you will not pay 1 million dollars ok but this still creates a risk for the economy as a whole because there are still perverse incentives ok there is a budget limit of 1000 dollars let's say so there is a perverse incentives to always say ok I work for 900 to 90 dollars right so the problem is still not solved so if we want because our broader goal is to ensure sustainability of the economy in the long term so we need to if not to eliminate the cheaters because of course we can never eliminate them completely right that's how it goes in society but we can at least marginalize them throttling they revenue streams so it doesn't pay if it's not if it's not profitable for the cheater to cheat then the rush at least the rational cheaters will not cheat and they they will not they of course they will cheat but they will not earn money on their cheating or at least not much but now the question how to prevent fraud so how do you even know we get a bill right remember we are in this request area in the situation where we don't know an estimate of the job right we don't know how much we should pay or it is how much resources should be used to compute this task so how do we know that we even that we are cheated at this particular point of time well so it's a quest of information what is the job worth so it seems we were coming back to the initial question right how to estimate job but if if if we are doing this estimation with a priori or before the execution well we can use some human input the request or the user behind the requester tells his estimate of job or we can use some ways of algorithmic evaluation but then we can use fixed wrest contro if we can evaluate as to estimate the job a priori then we can use fixed price contracts but the trick here is that a posteriori or after the execution we have more information obviously right but of course we need more than one data point right if we have just one users report we that we have to evaluate we we have a hard time evaluating it this if this is too high or not but if we have moderate data points so if similar jobs are executed by many providers in parallel and this is exactly what happens in Golan right because the I'm not going to bore you with details but the task is actually split into many subtasks and these factors are executed in parallel so then we can use this as a source of information and compare resource usages from various providers yeah and what can we do next so we considered many scenarios I I'm going to talk about three of them today so one a very simple idea requiring hardly any map use of math is what my colleague called biblical methods so always you have a number of workers and the one who is worst you punish him right this kind of works but it's not entirely satisfying perversity it works the moment it works really bad is everybody's honest everybody's honest but you still just always punish someone because you always punish the worse one so it but we have if we as I said if we have similar path if we have tasks of similar size or even identical size performed by various workers we can do much better so the second scenario I want to talk is comparable tasks so we have tasks of similar but not identical size executed by many providers then we can we can collect the resource usage reports from these providers sorry before I go into this he tell you the details I want to tell you a bit about how the offers are compared right let's say that well provider offer prices for resource unit and we compare this offers but should we always choose the cheapest one right not necessarily right because imagine that we have provider a that says he will work for one token per unit and provider B is willing to work for two tokens per unit should I always choose a well of course not necessarily I need to take their efficiencies into account so what we do is we assign to the providers numbers called usage factors which you can think of as amount of resources needed to compute a reference tasks and initially this values can be set by a benchmarking or analysis of hardware or it even works the schema even works if you set initially you set this factor to 1 for everybody because the main point is not how they are set initially because they are not set in stone they are adjusted and refined every time we get new piece information every time we know more about how this particular provider works and offers are compared by multiplying times price so you get this adjusted price and you compare them using this adjusted person so now this this main this main question is how to adjust this usage factors so we can efficiently at one firstly we can efficiently compare the offers and we can also eliminate cheaters so if we have multiple jobs with the same complexity executing at the same time it's a great source of information so we can adjust according to the usage reports but remember that the tasks are only similar and not identical so we we are not going to bluntly adjust to the last usage reports but it turns out that a better approach is to adjust this usage factors towards the weighted geometrical mean of the previous value and the latest the value coming from the latest users report the next another opportunity presents itself if the same task identity call task is executed by many providers of course to force this just for usage control would be wasteful but as it turns out in many scenarios and especially in the web assembly computing general webassembly computing I mentioned in the beginning and that was the motivation for this work is that we crossed we validate the results by cross-checking so they the same task is executed by more than one two or more providers and then we because it's the identical task we can well honest providers should report similar effective usages in fact this module oh they use these factors these are factors but if we have a cheater then we can quickly spot him and penalize him so we don't need to be as circumspect in as in the previous case no need for the geometric mean we just set the arm to the appropriate value and if this this this users factor exceeds some number let's say the usage factor exceeds 2 then we blacklist this provider complete ok so we've we've run discrete event simulation of these scenarios well discrete event simulation of the market in in manage this and many other scenarios to verify that they really marginalize cheaters and it turns out that well using either the second or third method the first method as I said is not as effective so using either comparable tasks or cross-checking or redundancy as long as the cheaters are in minority because well we are of course susceptible to the 51% attack if the cheaters and in majority and we we are in in in a tight spot even though there are some some things we can try and still try but if we have something like 30% of the of the cheaters in the market then you can see from this graph that they they take only well they get only less than 1% of the tasks and they get to two and a half percent revenues so they are as I said they are not perhaps not eliminated completely because we don't want to secure every transaction but we focus on the stability of the market in the long term but they are marginalized and the revenue stream is throttled so we achieved what we set out to do this is of course not the end of the road we're looking at some extensions to these schemes one promising thing that we are going to evaluate netic store actually started doing our game theoretic approaches 1 of this I wanted to show you is this game when well provider doesn't submit the results provider submits just the price and hash of the results and requester decides whether he wants the results for this price or not if he wants the results the providers gives him the results in requester pays if not the requester does not get results of course this even though this seems a very promising thing remember that the requester still faces the same problem he has to decide whether the price is right or not so it seems seems useful to combine this game theoretic approach with this quantitative approaches I I've talked about earlier so concluding the main points I want you to take from the stocks are that we want your land to be a good fit for general computation such as webassembly computation not only technically we are all of course working on this technical side as well but also economically and this is a step in this direction an important step I think so to this end we propose a simple yet seemingly efficient billing method payers of use we plan to reintroduce this billing method on main that together with whether webassembly use case and and the works on that are ongoing as we speak and of course we don't rest our case just yet we keep improving our solution thank you and I look forward to your questions [Applause] well I don't think you should compare apples to oranges you should compare well similar tasks to similar tasks right because if you we try we try it even if the well the type of the task okay the type of the task doesn't matter so much right but the size of the tasks even if you compare rendering tasks let's say of various complexity and you try to apply these kind of methods you don't obviously you don't get useful results in results but for webassembly let me say on webassembly you can run well well it could run rendering perhaps it won't be very efficient to run rendering on webassembly but things like molecular modeling and so on and you can compare them and they are part of one in this case but it's the size in this case it's the size that matters sorry I had two questions the first one is you say you blacklist workers when they behave poorly how do you black list on atrium where it's as simple as creating a new a light to create a new identity and the second question is what about our requester this is an attacker and proposes a computation and then say oh I'm not paying for that and just wasted time on the worker side yeah okay so as the first question yeah okay you cannot black this it's black listing to this particular requester right because it's as I said it's completely decentralized so what I'm talking about here is black listing from this particular requester on the other hand one of the other avenues of of this research which I didn't want to talk well didn't really open this Avenue because I could talk for hours about this it's distributed reputation right but this is the definite in our radar so the requester when black listing someone could also tell the neighbors the other requester to say okay this guy is suspicious and the second question was sorry I forgot your second question yeah I think this some are inserts or the second question the thing is how do you prevent someone to the same requester to come with a new wallet you mean like a civil attack yes that's also another another interesting thing and we have a blog post coming about it I think tomorrow how can I provider even without distributed reputation how can I a provider adjust to the market conditions how much jobs are paid and not and whether he should or should not accept jobs from unknown requesters so ok so read our blog tomorrow yeah it's called the code name model zero because we have more models in store but this simplest model it cost model zero on the Golem blog any more questions how stable is your system if attacker is not monetary interested in the outcome what if the attacker is ready to lose some money to just heart maybe the system as a whole how stable is it how like do they need exponentially more money to throw just to hurt you or is it something which is unstable I have a nit very little if you mean the attacker as being the worker well losing money he just has to perform the job well and as long as he performs the job well all's well and good and actually the system reacts very quickly so it's you can picture an attack where someone works like on a distributed repeated on a reputation any reputation system right like someone works very well for a long time in order to build reputation and then in at one moment decides to cash in everything and take all the money well first of all here is nothing to cash it's one small job here so he can cheat you only for a small amount of money and has to put big amounts of money upfront and then again paradoxically the better he works up at first the faster he will be excluded when he misbehaves right because he has a very he has a very good this usage factor right and when the when the usage reported the users reported and this at this attack time is very different from what is expected from his usage factor he gets blacklisted immediately so now I don't think it's acceptable to this kind of attack but of course we can talk and we can analyze or other scenarios I'll be very good to do that any more questions now then thank you very much and I would like to thank take this opportunity to take my thank my colleagues as well [Applause]