Quynh Tran-Thanh - CryptoCompare - Ethereum London keynote - #ethldn

[Music] so hi my name is Quinn I work at to compare we are a market data provider for the crypto industry and today I want to talk about our main challenges as data providers in this space and by the way I gave the exact same presentation two weeks ago at the crypto compare summit so if someone heard it already then sorry yeah but this time I will be slower I'm gonna show you all the data I have cool charts and you can also have questions at the end yes so I used to trust market data and take it for granted but working in the crypto space actually told me that I have to question every single thing and today I'd like to talk to you about our three main challenges so just to understand where we are coming from and where we are right now I brought you a chart that shows the exciting boom of crypto coins and data of the last couple of years what you're gonna see is week by week how the space changed on the x-axis you're gonna see number of exchanges so you're going to see different coins being plotted and how many exchanges they were listed on and on the y-axis is the number of trading bears like trading to Bitcoin trading to USD so you're gonna see each coins are trading and you see how the space evolves over the last few years so let's watch first bit going and then old coins came afterwards you can see Assyrian popping up with a red dot 2017 the whole boom of ICO coins and here we are so as a data provider what this meant for us is that the amount of data and the complexity just grew exponentially and here are the three main challenges we are facing and I'm gonna talk about one by one how we are tackling them and crypto compare the first problem is data is a mess second challenge is data is big and complex and the third one maybe the most interesting one and the most current one is data it's fake cool let's start with the first one so working in a decentralized industry is super exciting but decentralized where a data company means that there's a lack of standardization lack of yeah standards so what we have to do at crypto compare is to start creating naming conventions start creating standardization of the data set just to tell you a few examples the same crypto coin is trading with totally different names in different exchanges sometimes even the same exchange trades Bitcoin under sweetie so we had to start standardizing this dataset and what we've done in the last couple of weeks was we open-source this dataset so if you were a trader and you wanted to trade or do arbitrage across multiple exchanges you need to know what bitcoin is called on different exchanges right so you can actually use our right now just to match the same going across different exchanges and of course there are a lot of other use cases oh sorry did you want to take a picture okay so as a data company we are not only collecting and cleaning in standardizing data but we are also aggregating it what you can see on this picture or chart actually is every yellow dot is a Bitcoin USB trade and this chart is from 2015 one week worth of data there were around 300,000 trades throughout this week actually and what you can see is that there's kind of a clear line that depicts the Bitcoin price and there are a few outliers but for us we have to take in all these data all the trades and calculate a single index price and back then our methodologies for aggregation was pretty simple we knew that where the bit comprise life and the outliers were very simple to as well oops no wrong direction oh yeah so since 2015 obviously there are a lot more training activity a lot more exchanges popping up so our data set now actually looks like this so this is still a Bitcoin USD trade one week worth of trading activity and now on this chart you can see 1.5 million trade so 5x trading activity and so what you can see or you cannot see is actually a clear path for Bitcoin so I had to ask a question what is the real Bitcoin price how are we going to calculate an aggregated Bitcoin index we have to start adjusting our methodologies and start looking at each exchange each data source and become more selective about what data we are going to use for aggregation what we are doing is called exchange review we are publishing a monthly review of what data sources can come into our aggregation methodology and we are publishing it to make the process completely transparent so one of the reasons why the dataset grew so much apart from the growth of the space is also that data is free it might sound like a good deal for data consumers that data is free you can access data for free but actually for us free data means that anyone can is free to become a data provider it means that any change with an API that returns some sort of data can enter into this data market and any data and that leads to the next big challenge which has been the most the loudest challenge as well I guess you heard a lot about fake data or fake volumes have you heard about it before yeah recently yeah so probably you heard something like experts and probably like 95% of the total trading volume in this space is fake so I'd like to take a step back here and really dig into this question where we are coming from how did we even get here and what does fake volume really mean cool so it happened that all crypto data evolves around topless and rankings and all these rankings use volume as a metric and why because volume is easy to calculate and very easy to understand the real problem here is that volume has been used as a proxy for liquidity and in a lot of times for proxy for quality and popularity as well so we started seeing very weird exchanges popping up on the top of exchange rankings and people started to question if this volume data even real is the data even real can we trust this data anymore so what happened next is well volume is very complex it's actually not as easy as you might think you need to dig deeper one thing to look at is fee structures there might be exchanges that reward their liquidity providers there might be other exchanges there very expensive and that's gonna affect their volumes as well or if you remember a couple of years ago Chinese exchanges used to have zero feel right and they used to dominate or have 90% of the back then reported volumes another thing to look at is incentive schemes so this chart shows exchanges plotted in the volume axis and average taker field axis the green dots are using trading incentives things like training competition and transfer mining and the red dots don't use them so you can see that on the very high end of the volume scale there are all green dots so those exchanges are clearly using trading incentives to boost their volumes so you need to look deeper into the volume numbers but then what happened is that we started seeing a growing demand for better metrics and people started looking at alternative ways to find bed angle changes one of the projects out there started looking at trading patterns so what they did was they traded sorry they charted the trade sizes in a histogram and they found very interesting patterns and what they claimed was a good exchange should look like this so what this shows is that a average trader is more likely to trade with lower volumes and less likely to trade with higher volumes with some spice around full numbers like one Bitcoin or two ligands yeah and all the other histograms are fairly weird how can it be that some exchange trades with such a perfect exponential whatever distribution so they say okay but looking at trading patterns trade besides histograms you can identify good and bad exchanges but guess what trading patterns change this is one of these changes that showed anomalous trading behavior had a weird shape right but this is its shape in April sorry in March and look what happens until April so this is everyday trading patterns and by April it looks like a good exchange so we found that it's actually very hard to differentiate between good and bad exchanges by looking at trading patterns but we wanted to look deeper and see why do people want better metrics and it's because they started losing their trust in the data and we thought that okay crypto compare as a data provider we actually have to try and find a solution for this we have to bring this trust back so our question was to ourselves which exchanges I trust for trustworthy and which are not so we started doing a research on each exchange we are collecting data from and we came up with a benchmark framework that we now shared with publish the methodology and I'm going to show you roughly how it looks like so what we did was we collected a lot of data from each exchange and they it the total scoring basically consists of seven different one thing to note here is that out of the seven components we are using six there are due diligence components basically so we are not using volume directly in our ranking and our market data tests using trade and order book only consists of thirty percent of the total scoring so we look at a lot of things the geography of the exchange where it's located was the regulatory environment there we look at the licenses on an exchange if it's doing kyc or m/l we look at the investment structure and funding of an exchange that's actually a very good indicator as probably someone else already did due diligence on the exchange if they are funded by a well-known VC we look at team and company structure CEO CEOs their background their experience we did an extensive API test API latencies rate limits order books and trades and we check if there is a trade serving system on exchange for the market quality side we came up with five different metrics and we wanted to avoid volume as is because we wanted to come up with a ranking without using poem directly so for market cost of trading we looked at spread for liquidity we look at the order book test for price stability we look at minute volatility and we created this new metric called the correlation between volume and volatility basically we defined two new terms market maker driven markets where the volume actually goes down when price volatility goes up and investor driven markets where price goes price volatility goes up and volume goes up at the same time yep and we also calculate standard deviation of volumes to penalize for trading behaviors like this yeah okay so we came up with a top list and we had scores for each exchange and I think the main idea here was just come out coming up with an alternative for volume rankings and here's a list the way it looks like is that every component gets a score we sum them up an exchange gets a total score and then it gets a relative ranking based on the score and it gets in the end a letter grading so you can see the full list in this link which you don't see sorry but I can send you the PDF afterwards and you can click on it but I think the most important part is actually not the result it's more like the framework and what you can do with it so we wanted to introduce a new notion which is trusted volume so instead of looking for bad behavior and bad exchanges we wanted to look at which are the actual transfer trustworthy exchanges trustworthy data sources that you can account for and the way you can use this framework is that okay you are an investor first of all you need to identify or calculate your risk appetite let's say you are a very low risk so very risk averse investor that can only take Double A rated exchanges in two then your market for trading is only around three percent of the total volume of the total reported volume if you are more risk-taking and willing to yeah risk or then obviously your market is going to be bigger but it also means that your market size is not guaranteed so to test the robustness of our framework we actually plotted our ranking scores against volume which we didn't include directly originally and guess what there is a correlation which means that naturally you actually expect better exchanges to have more volumes but what this chart also shows that there actually on the middle top of the chart center top there are exchanges that usually talk the volume rankings but actually they are not that didn't score that well in overall in terms of business and funding and yep trustworthiness another interesting finding was that we separated the static so due diligence part of the score and from the market quality score so using trading data and order books and the two scores also correlate which means that potentially if you just do proper due diligence on an exchange that also infers the quality of the market so very easy way to check if your exchange is good or not it's just you know look at very basic stuff like has it been funded who are the people even the founded the company or are they even public very easy stuff a few interesting results so most of the best rated exchanges obviously have external funding like a well-known VC and worse exchanges many have private interestingly ICO funding was present across the world another interesting thing yeah as you would expect we the better exchanges were licensed more and they were also registered as some sort of money services business scoring score averages across locations so the top locations are the US Japan South Korea Hong Kong the UK basically where regulation is a lot more like a lot stronger exchanges tend to be a lot better quality as well so we've seen a growth of bad quality exchanges in the last couple of years and their volume took over the market share as well and I think it's very important that we actually take a step back and see which data sources we can trust and we hope that with the exchange benchmarking we could actually create a tool for investors data consumers and traders that they can actually use for identifying the best trading values and as a data provider company we think that data should serve transparency which it doesn't do right now and we hope that we can bring this to trust back and data can serve transparency again thank you yes was it was it around the mining reward could be possibly miners immediately selling after mining because they expected the we actually didn't we actually didn't go that far but we found quite a few exchanges having this sort of change and it was very close to the publishing date of the bitwise report that actually showed this these anomalous trading behaviors so we guess that some of the exchanges just change their histograms basically because of the report coming out but I can't really confirm but we saw the change of quite a few exchanges around that time you mentioned 3% of the of the volume available in exchanges yeah if you actually try to apply some metrics of the percentage of the fake volume for the worse exchanges then do you think that actually this 3% is much more like the try to estimate like it's not 3% but it's actually 20 or 30% of the real volume and just P percent of the total volume of which a lot is fact you try to assign a metric of percentage of fake volume for exchange we actually did not want to go down that route so we don't want to adjust values that are coming from a data source because it's a lot harder to justify so we are providing feeding our data into indices and institutional clients and for them the main like most important thing is methodology so I think it's a lot harder to explain why I would find 3% to be fake and adjust with that and I would rather just say this exchange is good I take it in 100% this exchange is bad I don't take it in and at work so we take this in or out approach yes yes so should I go back to yes so API latency I think we found any website that checks the IP address of each of the API as we were using so we could check their location and then I think that service also did a big test for us taking out the location bias I can maybe you can actually read in the methodology which service exactly we used yeah but otherwise we also just curate each API ourselves and calculated the response time and also the rate limit we had to test it as well not trade its just requesting or the book data and requesting trade data we did not actually execute anything sure but fair question very fair question we did an equal weighting yes with the market quality with double waiting so that's actually just and trade surveillance was just a single score so that had half of the weighting it's not super elaborate but it worked out well in terms of the results like this so we were benchmarking it to the volume results if you have better ideas I'm happy to it's Dara Singh liquid is number five yeah I'm not sure but we can like this one right this one right I think here the weird part is that they're still like there's this shape right which is what people usually expect like more trades being placed at smaller amount so you would expect even when people are like you know want to trade more they maybe just put in more trades but and there are less straight on the higher volume side but then there's this hump there which shows like someone is just putting random values around 15 bitcoins which how do you even explain that it could be washed trading someone is just like putting in random values we don't know actually it depends which exchange really like I can't really tell but you can see that the patterns are really like Verizon OTT like the top this one can I do this this one we call Wolverine some people say this is a whale or a duck I don't know so there are really weird patterns yes sir other than KRW we will add a few more we actually talked to one of the exchanges that ranked pretty low and that's because they were offering more like emerging markets currencies so South African currency Malaysian currencies and they were actually market leaders in that segment but they ranked pretty low because we were looking at the major fiat and crypto currencies so yeah we will add more well to be honest as an investor you are pretty much tied to where you can get into the exchange right so it could be that you need a Korean bank account to open in the Korean exchange so you might not be able to do that cool I think that's it we can take it offline and you guys can talk to me after [Applause]