Stop Building S*** People Don't Need – Zach Herring & Katie Johnson

so let's get this started everybody first off thanks everyone so much for coming to our talk stop building poop emoji people do not need with a sub-headline a friendly introduction degenerative research and prototyping so first off quick introductions I am Zak this is Katie I'm a product strategist she's an innovation strategist behind his boba fett he's a bounty hunter he's pretty cool he's a nice guy when you see strategists what do you think it's it's really just a fancy way of saying that we work with teams to help them sort of like take a step back from their product think critically about it and basically like there are a thousand things that you could be doing especially in these early stages of building products there's a thousand things you could be doing there's usually only about one to five things that you really should be focusing on in any given moment and an outside perspective and outside sort of advisors sort of perspective helps sort of clear things out and really focus on what needs to get done so that's what we've done we have been doing this for four years together and then before that we were working in different product teams I separated up the logos to like before Cho Lubin and after gel Lubin as you can see like we are we're doing enterprise stuff before afterwards we haven't had an opportunity to work with some amazing web 3 teams and it's just been it's been great we've really been enjoying the space so the question is pretty much how do you get good at building stuff that people need right there's a lot that goes into that answer you know you can talk about working with the right people collaboration iterating quickly but one of the answers like kind of the simplest answers just like I've had a ton of practice building stuff that people don't need right like this is a this is a logo wall of a whole bunch of projects that I've done from like 2011 to 2019 these are at least these are at least 3 months worth of work some of them up to 18 months with pivots with with refining the the pitches etc some of them hit market traction some of them didn't but ultimately there's just there's been a ton of practice that kind of goes into it so I'm sort of distilling down about like nine years worth of work into this talk the TLDR and I think like what's really important to understand like whenever you look at like the projects and I'm sure you can't to write like as some people who are perpetually building you probably have a massive amount of projects that you've been working on that you can kind of point to and go like oh yeah I was working on this and I've been through that the think the important thing to were like remember is that it's really easy to build the wrong thing right it's extremely easy and it's extremely expensive too one of the problems with this especially in early stage teams that I've seen I've interacted with whenever I was embedded as a product designer my experience was that design UX and research is often perceived as like difficult it's perceived as expensive time-consuming and oftentimes the last step in the process we think like okay I have the API design now we need to build now we need to like sketch out the pane of glass in which people interact with it right that's unfortunate because a lot of times the UX and specifically the research is actually the silver ball bullet that product teams can actually use to kind of get to that product market fit faster specifically whenever resources are constrained right so so when we talk about strategic design the best way that we can frame it for you is to think about making good bets so what do we mean when we say that so we think about the design process has this kind of five five step process that starts with understanding goes to define id8 prototype and evaluate most of you in this room have probably heard of that before what we love about this though is that it's something that we do again and again the iterative part is the part that's the magic sauce right and when I say it's about making good bets what you need to know about your product is how certain do you need to be what is the level of good bet that you're willing to take if we think about something like Iowa last week to see use a prescient example if you are launching your product and it is going on the main stage to be seen by a million people in the Iowa caucus 20 election then your threshold for what a good bet is should be very very very high in this case it wasn't it wasn't high enough right they launched it it there were a lot of other things wrong but the app itself was not ready for primetime we know that now and so when you're thinking about what you're doing if you're just launching your app to a couple people you know or your dap or your product then maybe you're good bet is a lot lower if you're going on main net for the first time and that's your big big launch and you only have one chance to get it right your threshold should be a lot lot higher so when we talk about what makes a good strategic bet the first thing that we like to talk about is research which often when I hear people talk about research they say okay Kati I get it research I build something and then I talked to some people and the answer to that is yes yes and as we like to say in design it is about building things and talking to people but it is also about building things with users okay so what do I mean when I say that coming back to the this kind of iterative loop that we talked about we see research as a multi-faceted approach so we start with and then you'll hear researchers talk about these things different ways I personally like to break research up into three categories formative generative and evaluative sometimes you'll your research is kind of lump formative and generative together and that's totally fine they'll call it generative probably are foundational or discovery that's totally fine I personally like to break them up into three for a very specific reason formative is about discovering and learning generative is about making stuff and evaluative is about refining so what is formative research formative research is discovery pure and simple and so it can be about learning about a domain for the very first time and and usually this is a very time intensive type of research you're gonna go out in the field you're gonna do what we call contextual inquiry you're gonna watch people do stuff in their natural habitats this is the one that I feel like people mostly grissle at they're like oh okay that sounds expensive I have to go out and talk to people in three dimensions that's hard I don't know I'll tell you right now if you are trying to build a product in an industry you are not that familiar with let's imagine you're building a web 3 DAP in healthcare but you don't know anything about healthcare if you don't get out there and understand and how HIPAA works how electronic medical records work how getting information across state lines these are all American based examples of course but if you don't understand how those things work you will launch your DAP and doctors will laugh at you because you didn't do your homework so this is the kind of research that I want to see people doing we actually have affogato right here down front in the room we're gonna talk about them in a little bit they are champions of formative user research if you want to know about what formative research looks like if it's done well go talk to them they have spent so much time out in the field with coffee farmers they're also here selling coffee right now today so talk to them anyway because their coffees awesome but they know more about spending time with their users than anybody I've ever worked with so please go talk to them they have done phenomenal work in this area and I'm sure they will tell you that it is worth it to get to know what's going on in the minds of your users so let's move on to generative when people tell me they know their user or more often than not they tell me they are their user we talk about generative research so my next question to you is ok cool your your user that's super convenient you know what you need do you want to just be the only person though that uses your product or do you want more people using your product than just you hopefully if you want to be making money and having success you understand that you're not gonna be your only user so if you know your market if you understand if you've been in health care for a million years and you know all those things that we just talked about that's cool let's do generative research and this is where I see them as separate formative in generative generative to me is making with users so you can get them with you in the room co-creating your actual product or service so that when you're done you know you know for sure that you are not the only person who wants to use your product you also in the process of generative research get to build your kind of tribe of evangelists right if these people have stake in the game if they sat with you long hours making your actual product they're gonna be the first ones out on the street telling people about it getting people to use it caring about it and making sure it has a future they're gonna be advisers on your board they're gonna be SMEs that endorse you this is important and so for me this is why I separate generative informative and I tell you that even if you can tell me that you have done the formative work because you've lived it you must must do generative research if you want people using your product that aren't just you and last but not least is evaluative research so this is the idea of make and then talk to people right so this is what people mostly associate with when they talk about research is usability testing getting out in the field you can see me here not really in this picture behind this gentleman using a product on a laptop from a long long time ago back when I wore legit dresses out to sales calls which I don't do anymore so evaluative research is about seeing your product in the wild the things that I swear you you think you got right you think that you did it perfectly you get it out in the wild I tweeted about this yesterday if you'd find yourself telling a user they did something wrong you need to be looking in the mirror because you have done something wrong your product did something wrong if you are targeting the right user in the right use case and that person can't use your product evaluative research is here to save the day now the thing about a valued of research that I hear a lot is people saying okay Kati I need to see as many people as possible and I'll tell you this no you don't you need to see five five people at a time you will get to 80% of the product problems that you are trying to solve in five usability tests that can be literally 30 minutes long there are people in the room Alex votos in the back he can talk to you about me being a great breaker of get coin avocado can tell you I've broken everything they've ever made I am a perfect person to break your stuff but if you find yourself telling people that you they are breaking your thing or that they are not using it correctly you need to look in the mirror because again you are not going to have the traction you want and you're gonna find out in Iowa on the main stage instead of here in a quiet room with one person where you can actually go change it so instead of doing 15 usability tests promise me you'll do three rounds of five where you actually make those changes in between each one okay people ask me this all the time Katie what are the common research mistakes I will tell you all of these are huge everyone who's ever done research shouldn't know them if you tell me that you cannot do any research or not doing any research for your product that is the biggest mistake you can make so please talk to one person one person is better than zero this chart is here to show you that really literally you will find eighty percent of the problems of big problems in five usability tests but you will find 33% in one that's not to tell you that you should only do one please don't take that away from this talk but if you talk to one person you're gonna start seeing things in your product breakdown that you can fix and make better so one is better than zero five is better than one not talking to your broad market is huge if you guys watch Silicon Valley you know that this happened in Silicon Valley where they talk about oh you know we tried Pied Piper out with all of our friends and everyone loved it and then they launched it to the real world and everyone's like I have to take a master class to get through what Pied Piper is and why I care about it don't be like that okay talk to people that are actually supposed to be your users if they can't use it and you tell them they're they're not intelligent that's on you we talked about that spending too long in any one phase of research people tell me this all the time you're saying I have to spend six months to in research before I can code no I'm not I'm telling you you need to spend time switching between research phases while you're also coding so you shouldn't spend too long in any one phase and the last but certainly not least thing I hear all the time is research is not my job if you are I'm not a researcher I'm not I'm in product I'm a CEO I'm this person I'm a CTO I do this thing if you are invested if you are invested in your product being successful research is your job research is your job you must partake in these different things you must watch usability sessions you must talk to users and you must give a about what they care about because if you don't your product will not succeed and you will have only yourself to blame so if you say if you find yourself telling anybody that research is not your job and you care about your product being successful you best look in the mirror because I promise you the day of reckoning will come alright and what that sacks can take back over seriously seriously I thought you're about to start banging your shoes amazing so once you have done the research once you've started gathering these asymmetric insights right research is about finding things out that nobody else knows right because your product your competitive advantage is having secrets that you can embed into your product once you start doing that used to move into prototyping when we say prototyping I think it's a dangerous project I'm it's kind of a dangerous word especially whatever we kind of think about code because generally speaking whenever I think about coding a prototype it's still coding it's still building out something that more or less kind of works the important thing to do whenever you think about prototyping in this stage in this early early stage is that you're sketching out the experience as opposed to the UI all right so prototyping is not an MVP it's really important to understand that I mean like remember an MVP there's viable in the name right which means that it's something that you can give out to people in that it'll actually work that's not what you're doing with the prototyping in the early early stages we've all probably seen this graphic this is still one of my favorite graphics that have come out of sort of like the UI UX sort of seen over the last 10 years everyone is kind of thinking like well how do I build a skateboard skateboards what I want to do like eventually I'm gonna build in Tesla but right now it's just a skateboard there's actually step before this right there's a zero where you where there's a zero step where you're actually building out the prototype and it's important to think that like let's say that I've kind of like the idea of a scooter as a sort of example if we wanted to start building out a scooter the first thing we would do is not like start welding start bolting wheels on right the first thing we do when we build a scooter is actually find a pallet mover it's it's a it's effectively the same sort of experience you don't have to spend a ton of time building stuff out and you can simulate what the actual experience of using the product is without necessarily like spending a whole bunch of time building stuff and I think it's really important understand right like when we talk about prototyping in the same way that people are worried about research taking forever people are worried about prototyping taking forever I would strongly recommend if this is if this talk has been interesting to you if you want to continue to research check out Jake Knapp's books print they basically fit the entire process into a week it's a rollercoaster but it is a week long they spend about two or three days and it has really good examples of product not just like websites and not just like DAPs but like actual like really complex products that would take years to build they simulate the experience using smoke and mirrors so that's what it's really important to think this is prototyping this is prototyping out the sky the SCOOTER experience right you're figuring out do people feel safe on this is this the right is this sort of the right path etc so prototyping has four essential steps the fourth one is recursive up until a certain point remember that graph the iterative graph so step one is what is the concept step two what is the assumption about the idea you want to validate it as true and I think it's really important to understand that when we say assumption we mean an assumption that has to fit as core to your product all right so it has to be something that is true and if it is true you have a product that people want if it is false you're in trouble right it dies on the vine so it's really important that when you think of assumptions sometimes like whenever we get with teams we just asked them to like spend 30 minutes spend an hour writing out what has to be true in order for this to even be something that can actually function in the world as like a product meet a market find customers that's an important step and that's what prototyping is meant to either validate or invalidate step 3 is what do you need to prototype in order to evaluate that assumption and then step 4 is make test iterate repeat okay so it's kind of a recursive loop so what are you evaluating it's a spectrum you have concepts market interest customer behavior on the left side you've got specific solutions on the right side as a general rule earlier you are in the product the more you're probably to the left I also think that generally speaking most teams myself included myself especially is further on the left of the spectrum than I'd like to believe especially when you first get that idea you get that ad and you're like oh my god this is amazing I cannot wait to make this but you don't actually validate whether or not like I need to take two steps back and actually start validating alright okay people actually behave in this way people will actually pay money for this like there's like there's a there's a a lot that goes into it so as you evaluate concepts it narrows down to where essentially like as you evaluate concepts there's a ton of things that you can use to prototype concepts out without actually having to write a line of code right avocado in the front they actually use storyboards to map out the entire process and then use it to talk with coffee farmers and co-ops to say like this is kind of what we're thinking for our product does this make sense is this interesting they usually like it's just say it's a simplest process of like building out a storyboard another ways blogs and squeeze page pages so kind of as Katie was saying when you talk to people you get SMEs you actually start building your beta list early blocks and squeeze pages pages to me is like an excellent an excellent way to sort of double dip you can test people's interest and you can also start establishing yourself as kind of a leader in the field you can even use different blog posts or different sort of searching through different feed different sort of avenues to test out different features or different niches which is really really nice cheap marketing buys this is one of my favorite sort of split a be tests like we actually have teams that have had different ideas that they were like well I don't know if we should do X or we should do Y what we did is we took a hundred bucks we split it down the middle week it we did two different marketing buys and then basically like targeted the audience and said like which one do you want is this or this interesting and we let the two marketing campaigns compete the one that got the highest conversion is the one that we decided oh this is this is what the market is actually interested in and then games so especially useful or token economics when we're designing marketplaces or ways in which we are assuming people behave in kind of complex scenarios games is a great way to simulate that right like you don't have to get them to put a whole bunch of money down they can start playing immediately everybody understands the concept of games it's a really quick way for people to suspend reality and operate in sort of the reality that you're sort of projecting ccr party in my opinion is one of the like one of my favorite experiments that actually validated some of the behaviors that tokin curated registries was what's like kind of meant to in view so evaluating solutions so basically like as you prove more and more things and you get closer to those specific solutions they become a lot more concrete you're actually testing out the solutions right you can do wireframes clickable prototypes you can even start coding things finally but like as you start validating the solutions it becomes a much more concrete thing and you're actually starting to model out the the final solutions for the so again remember there we go how do I know when I'm done we already talked about how it's essentially it's a you can be done and you can do these in a week and continue to iterate remember this graphic certainty a good bet you all you're essentially doing you can never be a hundred percent certain that what you're building is the right thing but you want to get it close enough to where you know you're making a good bet Jim Collins choosing to be great fantastic book if you want to read something that's about someone who's done a lot of research and study around entering markets definitely read that he is a great phrase firing bullets and then firing cannonballs your cannonball is the product itself it's coding it requires a ton of time a ton of a ton of resources and it's an opportunity cost for your team that's the cannon ball these are the small bullets as you fire to get up to this to that certainty what makes a good bet so every step of every project has different metrics right like I can't tell you what your time is worth I can't tell you what you're shooting for what technologies you need to be like kind of building out but I tend to use Nets it's a pretty basic napkin math with my team's whenever we're working through it a napkin captain a napkin math bet is just basically you take a medium-sized feature you say like okay well this probably gonna take three months three I seize 303 individual contributors Devin Devin the design let's say you're paying market rate 8k monthly that's $100,000 these costs balloon out way quicker than we can we can imagine and it gets away from us so whenever we're thinking through these projects we basically stop and we ask ourselves ok are we at least a hundred grand worth of certain that this is a feature that we want to ship or this is a product that we need to build this is also really simplistic and not at all accurate so if you like like it's not taking into account opportunity cost like what you could be building as opposed to building this it doesn't take into account revenue comparing the two different features this is extremely simple but it's also extremely effective in gauging the time especially if you're a small team especially if you're a small team working on the nights and weekends right like that's how I basically gauge what projects I want to be working on in pursuing whenever I'm whatever I'm gonna doing that so how does consensus inspire and practices so if you hear these things you're like wow that sounds pretty cool I want to try it out we have a program it's consensus relays these are essentially decentralized hackathons that you can that you can attend in out of your own home out of your own garage if you're doing the Steve Jobs thing and participate there 15 days we have a whole bunch of curriculum and basically benchmarks at each of the 15 days for what you should be doing who you should be talking to how you should be working through it and this is like we've seen some pretty tremendous teams come out of it big fans of affogato actually like the three three teams under here were both relays as well as micro grants we also have a separate program called micro grants where we do gap analysis with teams that have fun that have won and we do a gap analysis and work with them on like tiered funding unlocks basically and get them to kind of work towards where we see like the accelerator sort of launching point is for these teams where they can start thinking about raising money and enable enable idol and affogato our alumni for that talk to them we really enjoy the time that we spent working with them and then finally we have frameworks in curriculum so if you want to do this asynchronously we have all this available for free on github open-source you can go check that out and then we do YouTube we essentially like do a whole bunch of interviews on YouTube so we talk through the research methodology prototyping we talk to a whole bunch of Midler's everyone from Austin Griffith to the CTO of the Washington Post ton of really interesting talks there as well so ok so what should you guys take away from this talk design and research should be happening all the time within tight constraints to save time and generate those asymmetric insights give you that advantage way opportunity cost with napkin math like Zack said don't do anything without thinking about how much you could be doing it how much it costs right research and prototyping there's a couple insights here please make sure you're doing at least generative research creating with users as you arrive at product market fit and then make sure you're doing a value t'v as well formative to as I mentioned if you need it prototyping like Zack said not not viable right don't need to be viable purpose of prototyping is to get to the degree of certainty that better bets if you can only remember one thing from this talk that's scribble with the good bets line remember that write it down on your sheet every day test concepts and users behaviors first and then specific solutions later and then this concept of cupcake versus wedding cake which is basically like you don't let someone try out wedding cake you don't give them a piece of sponge and be like the frosting will come later you just imagine that part right you build them a small cupcake that has all the pieces of the love the big thing so they can taste the whole experience in one so think about that when you're building your prototypes as well like we said if you want to do more come find us at consensus real acecomm or right after this talk we are the cove co-founders and co directors of the program so we are we're it come find us micro grants round three is opening in two weeks please apply if you're interested in getting funding unlocks and getting us to help you figure out what you need to do next and please find us on Twitter at Kaitlyn B Johnson my name is spelled super weird but if you find Zach you'll be able to find me I'm in Z herring and we will post a link to this deck with all of the rich resources including all of these resources as well as soon as we step off this stage so thank you guys so much we are right at time and I know we probably don't time for Q&A but you can come find us after we will be right here so thank you so much [Applause] [Music] you