Guerrilla user research for dApps: Step 1 to mainstream adoption by Kevin Kim (Devcon4)

cool so how's it going my name is Kevin today I'm gonna talk about guerilla user research tactics to get mainstream adoption for your depth so the original version of this presentation actually focused a bit more in the beginning on some of the user research I've worked on globally at Facebook and courts and some lessons to be derived from that for blockchain and payments but after doing some user research of my own a lot of people said that the second half of the presentation just the actual tips and tactics were the most useful part so this presentation just gonna focus entirely on a bunch of things that you could do for your DAP starting tomorrow to help validate the idea for your DAP and the concept for your DAP and all the way down to launch so a little bit about me first I've touched user research and data from a few different perspectives I cut my teeth spending summers at Google as a product data analysts running a lot of Python data jobs and analyzing user data for the Geo commerce team I spent a summer at Microsoft managing user research for Cortana then spent a few years at Facebook working on growth products product marketing and helping drive user research initiatives both on the product and perception and sentiment side but also across a number of different continents and currently I'm a product manager quartz work focusing on memberships and subscriptions and doing a lot of scrappy scrappy startup research so I guess you know what is user research instead of a dictionary definition I think I'll just give you my own colloquial definition which is a bunch of ways that you could interact with users potential or current users that help you understand how they do things and so these ways or research methods you could kind of cut in a variety of different ways you could think of the methods in terms of what do they track how do people think or how do people actually behave you could do it by the phase of the development cycle that you're in and which tactics map best to that you could also do it by the context in which the study is actually taking place don't worry you don't have to memorize another main split is qualitative versus quantitative and for those you didn't know a lot of user research is considered quantitative for example for surveys especially when you do them at a really large scale you need to you you need to calculate the margin of error and make sure you have statistical significance and your findings and results so what is guerilla user research that it's basically just a term I made up and what it is is it's being a it's trying to do user research for you tap in 2018 it's scrapping and clawing to get any sort of validation for your DAP that probably if it has launched an on main yet main that probably doesn't have a ton of users while dealing with few resources and the dozens of other barriers that you face as a blockchain founder in 2018 so this is just a quick thing I drew where this is the preferences of a real kind of person with an actual problem to the right is the solution that they're using now they like it slightly to the left is your blockchain based solution farther to the right it is the more they want the solution so you've heard the saying that you need to be 10x better than your competitor in order to get over switching costs in order for the person to actually switch over I've always thought 10x was kind of a lot but let's charitably say 2x so you're already starting off at a huge disadvantage you're already twice as far away as you were before just by being a competitor then you introduce the other things that are happening you know people are skeptical of crypto and blockchain because the market crash people may not even know that you could build apps on block chains all the standard exchange and meta mask barriers all the UX and copy confusion that has been very well documented and talked about I'm sure covered very well by a lot of presentations here and also just you know if you're a platform that requires regular other people like businesses artists musicians you're gonna have to convince all of them to get on and that's going to be really really hard and if you're an enterprise product you have to convince at least ten different stakeholders within that company that this is real and valuable and secure which is gonna be very very hard and so you're basically off the map now that's kind of your starting point as someone who's building gaps today that are competing with actual centralized solutions and even developers and those who invest in them don't use DAP so this is from a great medium article I read I thought it was pretty funny because it seemed pretty accurate that develop developers who built them and the investors who invest in them they just don't even want to use them because it's so annoying and hard to do and they have lots of other busy things to do in their life that's what we're dealing with yeah so great so some initial questions you may have thus far so isn't user research only a thing that big companies and fancy design consulting companies can afford no I mean obviously it helps to have money as you could see I've been on both sides of it so I've seen firsthand but the problems are kind of different a small company you're just trying to validate your idea at the beginning you're just trying to find any time to do user research whereas a big companies you're more kind of launching new product lines kind of protecting existing products to its competitors etc how many of you guys work for a company that's above that maybe has more than one full-time user researcher by chance well I'm guessing most of you people are consensus or perhaps coinbase or status which is you know which is good it's like I think we should all aspire to do that to be honest but I also know that a lot of you guys are really small and just trying to hack and get something out the door so these tactics hopefully should be pretty helpful all right well we're too small we're short on funds we can't hire a researcher well so you don't you may not actually need to hire a full-time researcher so in terms of finding your researchers so one aetherium token means you don't have as many resources for means you have more resources so when you're pretty small five people or less it's gonna have to be you and that's okay because a lot of the tactics that you'll actually be doing like interviews surveys usability tests which are just you know kind of walking people through your dap and seeing how they respond and giving them direction those are fairly accessible to pick up with some study of the methodology also allow some of the best people in the world at user research actually started from a founder point of view so you're set up pretty well when you get to maybe like ten fifteen twenty twenty-five people probably want your designer or PM you hopefully have a designer to spend part of their time doing that it maps pretty well to their skill set and if you're if you're really in a crunch i think two other skill sets that could map well to user research are one maybe like a marketer that's really interested in marketing research and really cares about the user or perhaps someone on the business side that used to be a consultant that cares a lot usually the kind of research a lot and deliver actionable insights within a time constraint kind of method helps consultants transition over and if you have a few more people you might be ready to make your first full time hire so obviously it would be nice to get someone with both the academic background usually some grad degree and one of the experimental social sciences and also the practical background having some experience at a tech company but that's pretty hard to find my opinion on two kind of requirements for a full-time research hire that you're making if you're on a bit of a budget are one make sure they have some sort of UX portfolio you know similar to designers a lot of aspiring UX researchers often put together a collection of articles or they walk through the ideation process they walk through user interviews they did they document you know their thought process and that's helpful just because you could actually evaluate their aptitude and also because it's just proof of commitment as opposed to just some theoretical thing they're like hey i'm like a product manager and i might want to do research you know convince you in the interview and then also some professional work experience i think would help just because you need this person to execute for you now or your dap might be done and often people who have been in a professional work setting are just more familiar with the types of constraints that they have to work under it's not a hard rule but those are my personal opinions and then finally if you have the money for example if you have ico funds and hopefully they haven't been siphoned off through a smart contract by anonymous CEO you I think it'd be good to poach an experienced user researcher from a tech company or design consultancy especially if you're doing a lot of upfront idea validation and generative research because in terms of evaluating concepts from scratch it's really hard to beat a lot of actual practical experience all right so okay we could find someone but it's expensive still I don't have money for all these things that you need for research well you really don't need much so this is just a quick example of some of the tools and pretty much all of these are free a lot of these are even just Google sweet products that you could use for free we at courts use a good chunk of these and don't you don't have to like memorize these I'll find some way to chair slides after but I think the idea is that there's really nothing stopping you from right after this talk setting up your own user research stack it's just screen sharing video conferencing survey tools actually scheduling is the most is one of the most important ones here if you share your Google Calendar link the other person can kind of ask ask you for a meeting or propose a time and scheduling actually is a big barrier but there there should really be nothing stopping you in terms of actual tools from doing an a user research and this is just an example like when I sometimes when I do mobile remote usability tests I just plug in my I ask them to plug their phone into their Mac if they have a Mac to clicks and they're mirroring their phone and sent aeneas ly using QuickTime on their laptop it's in their screen sharing alright great so oh that's great but I'm also just I'm just a developer I'm just a founder I don't really know much about what to actually do do I just talk to people well that's what we have the rest of this presentation for a bunch of things that you could do at each stage of the journey and I might optimize a bit more just I might go a little fast just to make sure I cover everything but at the end I'll make sure to provide my contact information and we could kind of talk through anything I'm happy to answer any questions so first you just want to start small not big I want to make the distinction between protocols which I think is good to thank super super into the future top-down house it's gonna survive in 50 years governance that's great but if you're a DAP which is just an app with some smart contract component you should smart small just like regular tech companies pretty much every giant tech company started with a super small slice of a pie Airbnb literally was just air mattresses at just conferences then went to just air mattresses then went to you know renting people's apartments now they're doing luxury travel and experiences and concerts same with uber started with just luxury limo cars then all types of cars then autonomous vehicles and the reason why that is is because when you win a small slice you get a unique vantage point into the entire kind of pie of that industry as one of the winners to see the other opportunities in that pie that no one else can you've learned the best practices you've learned the industry better than everyone else you have a well-oiled team with great operations every one of your people in their field is an expert in their particular duty so you're just set up really well to dominate more and more of that pie and that's kind of the mentality that I think would be beneficial and just a quick example so this auction / - I know I don't know him very well honestly he actually was never really a crypto guy I think he he's in his mid-20s and it's old e-commerce startup - Ebates for a modest sum beforehand but it's a super simple experience where you just download a Chrome extension and if I want to buy something at one of these retailers and there's a lot of retailers covered a little pop-up comes up I click it great it's activated I go through my normal checkout floo and then two days later I just had Bitcoin in my wallet and there they have custodianship of the wallet and I believe you could withdraw after a certain amount to either a bank account or your like a like a Bitcoin wallet but I don't know I mean it's this is I think a good example of starting small and for what it's worth even though it doesn't matter too much they have a lot of strong investors on their side a good amount of funding and they seem to have good adoption so I feel like you know there's really nothing separating any of you from being able to just really tackle very real problems yeah you want to do lots and lots of interviews up front depth is more important than breath I think and you don't have to quite go so far but I did want to highlight a cool example that I saw of a stable coin company sello who went all the way to Kenya to do user research on the ground to understand how Kenyans use paisa which is mobile money that they could send really easily over your phone to other phones it's extremely extremely widespread and popular in Kenya and for example I thought an interesting insight was empezó kind of spread adoption wise not by people looking it up and you know kind of trying to get it or download it or whatnot but through other people in the village that people knew teaching them physically and showing them how to use in Pesa it was a very like local growth that way so yeah just sometimes you really need to just talk to people to understand the nuances that really define the ways that people think and act about certain products and the tough part of this stage especially for adapts is that at this point you probably have the least resources like even if you're funded this is probably a pre funding point but research is the most important stage at this point just because of the large amount of time it takes to create a set of secure and well tested smart contracts so it's kind of a lot of pressure but at the same time a lot of the tools that you're gonna use are relatively straightforward for you to pick up and as long as you focus and try I think that will be a big improvement in self so when we talk about design and prototyping I actually went to a pretty great workshop yesterday by these designers named Samara and Hanny who walked through a few great prototyping strategies but you basically want to use interactive prototypes or kind of a web mobile MVP no blockchain at all just use a centralized server with some dummy data just to get something in front of people that resembles what they would actually be using so if people didn't like your idea in the interviews you could show this to them and this should be a fairly representative sense if this is a real problem that they're having or not and you want to get as many users as possible on it so briefly walking through an example of what exactly usability test is for those of you who don't know so at quartz we launched this product which was a story series with 180 degree video called machines with brains and we decided to be pretty experimental in terms of launching it you know you kind of scrolled down in your article amongst different cards to read your article and swiped left and right to get to other stories we thought it was intuitive but we also had a big time crunch and we heard some bad anecdotes upon launch and so we conducted some quick usability tests we found that the large blank spaces at the end of cards made people think that the story had ended and so they just left we also found that scrolling card to card because they were different heights was not very intuitive and people felt like they were there scroll was getting hijacked and we also found that people were accidentally swiping horizontally when they might meant to be swiping vertically probably because they were swiping at a bit of a diagonal angle and so we quickly kind of enacted fixes to these solutions so usability fixes these types of things can really help you in terms of your product you get a lot of great information up front you want a design for the least blockchain savvy and technical version of your user possible and so I actually wrote an article about this a while before the use case we did a hackathon the problem was build a farm-to-table local supply chain DAP built it on hyper ledger and they mentioned that one of the users was this old 70 year old farmer that doesn't know how to use email so we kind of designed with that in mind made it super simple this applies to really everyone like if you're building prediction markets you probably want non blockchain experts to use it at some point so you should probably try to optimize and design a bit more for like a regular kind of gambler even if you're building developer products you probably want developers who don't know that much about blockchain now to kind of on board onto blockchain and use your products so you want to make it as block blockchain jargon free as possible at least in some of the main areas just so it's really easy for them to on board since actually a lot of develop there's a lot of really skeptical developers for blockchain and crypto out there and even crypto products honestly there's a lot of people who trade crypto and on crypto don't really know how it works so you just always want to optimize for a little less technical and a little less blockchain savvy than what you might expect and yeah so you you got a you got a test your copy both product copy and marketing copy for product copy Prada copy is always not as clear as we think and don't use lorem ipsum use real copy in the context of the product from marketing copy even for like developer products so you know these are some really complex developer products with some really really simple marketing messaging stripe watson Twilio I know we're all proud of the technology we built and want to show it and you also want to impress other engineers and investors but especially on your main homepage you know permissionless decentralized credit market algorithmic efficient money markets it's just kind of not that necessary you're just not optimizing for as many people as you could be even financial products you know fidelity it's not like they're fast-moving startup credit karma wealthfront you notice that they're focusing on value propositions like what the end results of what this product will do for you how you will benefit as opposed to here all the small tactical things that we have and you know for marketing to global audiences I'll just kind of let this slide speak for itself but please just you know even just through some through the crypto Network or just talk to a few people translations can go really wrong so yeah just to talk to people and I would highly recommend that you don't move past this phase unless you have users in the bank and what I mean by that is having a bunch of users that are kind of you you interview them you're do testing with them they're so into the idea they immediately see the value this would bring to them they're bugging you every week asking you when your products gonna launch they're talking about other people kind of in their field that meet their profile they really really want it and you kind of have to be honest with yourself here but once you get into the smart contract development phase you start to get to some you know really heavy upfront costs so you really want all of the work or most of the validation work to be in these first two steps and then when you get to the test net phase you know test that's mainly just to test out your smart contract functionality by this point your idea and your actual product experience should have been vetted in the previous steps you will want to make sure you just explain in a simple way what a test net is to all users even though it has a word test in it it's not that obvious exactly how the components work for example on meta mask on your test set it still shows a dollar value of the ether that you have in your test wallet which is kind of confusing because when you're looking at a wallet you usually and you see money here's like oh is that like money that I have just make sure that you explain what it is I would recommend that you have basic event tracking like the funnel major actions user sessions Google Analytics is free you could just use that there's a ton of documentation and if you really care about user privacy just don't associate sessions with any particular user identifier I just I'm not sure how you could realize if you've hit success or realize how close you are to success without tracking basic metrics and when I have is too and that we've you know I've seen a lot is to use support messaging products on your DAP we're actually in the middle of installing one at quartz right now you know I'm sure most of you have seen this this is intercom you see a little chat window you click it it pops up you could immediately start chatting with customer service or you could read a few different resources I think this is a great way of just setting kind of a set it and forget it automated way to catch people and who go to your DAP and get some feedback from them because you really remember this is guerilla user research you need to use every tactic imaginable at your fingertips to try to get any any feedback at all when you get to the main net launch another example of an automated feedback tool that you could use our in-app surveys so in-app surveys actually in our comm has a plug-in for in-app surveys there's some other vendors like Survey Monkey and CIPA et cetera just a quick sample that you could do you know hey this is really important how'd you find out about this step why are you interested in trying it provide some free forum feedback options and then collect their contact information for follow-ups just tell them that we won't associate it with your wallet your you'd like your product data but you need the ability to follow up with these people to really understand some of the use cases or pain points or problems this is probably like the most relevant set of users you could get information on so you really need their contact information obviously you could go really extreme so it's well-documented that Facebook would send in apps would show in-app surveys or web surveys asking people if they cared if they thought the Facebook cared about its users it is a bit of a blunt-force technique but I have to say I you know while is there and looked at a lot of that data and it mapped pretty well to how people felt about Facebook so it was valuable be desperate for feedback aggressively scour the web social for anyone who has used your products similar themes don't be too proud to pivot at this point if you realize that your idea isn't something that people want you've already learned a lot about how'd apps work how blockchain works be honest with yourself don't do that you know everything else is great but the throughput so I'll just wait two years for sharding or it's just meta mass that's a problem I'll just wait a few months for delegated transactions for more people to download wallet connect you should probably be solving a problem that in spite of everything that's happening right now keep some people at least won't really really want to use it so if it doesn't meet that bar I would just really really stop so that's that here's some of my contact information any questions I'm seriously happy to answer them there's also just a lot of talented people at this conference that could answer some of these questions better than me I mean yeah I could sense lights anyone or I could find a way to post them okay thanks everyone for Kevin [Applause] you