Opting Into The Surveillance State: The UX of Informed Consent by Jonny Howle (Devcon5)

[Applause] we don't have a display so I have a lot of beautiful slides a lot of that starting to do on the 10-hour flight over here and maybe I'll post them online or something like that so that's longer waste so enough to kind of pull we'll do this more verbally and maybe I'll hit up this whiteboard if I if I you know have the courage to draw on the fly but anyway I'm Johnny and also I okay so we'll see what happens there so I'm Johnny I work it you've worked with these lovely people here and today I wanted to talk about the UN of privacy and informed consent and how exploiting users of rationality and things like that can cause a situation where we end up opting in to a surveillance state I mean you know all saying you don't want but we get all interrogating something different from a state-run surveillance apparatus like the Chinese Social Credit System and why this is more of the threats for most of our most countries in Europe and in America things that are run by you know I don't we've heard this term surveillance capitalism but it's the idea that you know the underlying principles of the advertising model of the internet cause us to get this you know worst of all outcomes situation where everybody is surveilled and they do it they do it voluntarily even though we say we don't actually know what is sly just so I follow the lot so I have a very clever intro to this where I showed you a fake application about a defy credit score and you connect your Facebook to it and then you that I go on to tell you how this is the exact flow in a different context that was exploited for a Cambridge analytical and what they did was not a failure of any privacy technology it was a failure on the part of the product designers to or maybe not failure maybe they got exactly what they're trying to design for which is try to get users to came up more data than they were comfortable with with using kind of what we would call in UX dark patterns so [Music] so yeah and if you you know there's also a slide here around what would it look like to try and convince a user to not opt into this to this surveillance and in the Cambridge analytic example they're not familiar with the users were taking an online personality quiz and they in that facebook login screen where you say connect to Facebook and then it says something like Facebook will get your profile information and it also said you Facebook will get access to your friends profile information but nobody nobody reads this it's kind of the fine print and it has done nothing to stem the flow of data into Facebook's system even though it's kind of more of a cover-your-ass type of UX right it's it's it's implied consent rather than informed consent so there's this one of the interesting properties of decentralized identity and self sovereign identity net kind of raises the states in terms of product design around these consent and disclosure choices is the fact that we inverted the model for a relationship with the control of our identities so you know in the Facebook example the Google example they own your identity and they take on a lot of responsibility for stewarding that I have and they don't always act in our best interest however when we moved into centralized identity we invert this model and your identity really in a paradoxical oxy that way your identity becomes centralized on you right you are in control of all of the data you are a single point of failure in IT security circles there's a common phrase that it was vulnerable parts of any security system is the human right they are able to be exploited in various ways and this this becomes an even bigger risk once you centralize that point of failure on the human by introducing decentralized identity so this puts product designers in a you know a precarious position well I guess more of a in a fighting position or more important position in these in these floats so a lot of damage now can be done by getting people to disclose things especially if you did this user to disclose something and they don't realize that it's going on chain what that even means so you really have to avoid that so there is this common term that we probably all heard about and it is called the privacy paradox and it's that users often say they one privacy they're really pissed when you violate their privacy and you know in Equifax or you know Cambridge analytics or any of these other high-profile situations and then you watch their behavior and then make a long they keep doing the exact same thing over and over again which is it's you know giving them a lot of their data there's a couple psychological things that cause this to happen one has been pretty studied extensively in behavioral economics it's the risk perception gap and it's this idea that users are really bad or not users humans are really bad and you know anticipating risk and measuring and one thing users do is they sit will never have happened to me or even if there is a huge data leak the idea that or the chance that I am targeted is isn't going to be this pretty look of just one small drop in the sea of data and it's also there's these compounding effects that happen when you do disclosures over and over again and lots of studies have shown that you can be non lies data but the spew is like four five data points and a whole seed of anonymized data correlate those things and it's trivially easy to demonize these types of things and it's really hard to sense the lies that for most people's that's something we have to be aware of and I'm sorry you don't get to see all my fancy animations so the other thing we hear a lot from our research at you for and you know anybody that has done anything related to privacy and talked to people about it is another thing which is I've got nothing to hide right so lots of people say this I don't know so there's this idea that I've got nothing to hide that people people say this all the time and that comes from a improper framing and an improper conceptualization of what privacy is privacy is often looked at as hiding bad things and it's also looked at at an individual level but I think one of the things you have to start thinking about oh and it also comes from people thinking of privacy of this binary this binary but something we've learned is that crime scene our female ebony are are these spectral things right and movements along these spectrums gosh you got a whole team privacy and anonymity are these spectral things and really what privacy is about is about some transparency and control and the ability to move along this spectrum in giving context with as little friction as possible and that's the type of thing we want to enable so people don't feel like they're either anonymous or they're either private or not right we want to contextualise privacy as much as possible presentation this is what I'm going to show you up top an application defining credit we've all seen this login flow log in with Facebook get a credit score that we can use for key file ending and then I reveal that this is a joke and you all go home that was like is this what we would have to do to get a user to stop from sharing Dark Passenger valence inverted this is a fancy word I skipped over because I'm able to explain it okay now we're back so this is a quote from our research most recently that me and Audia the other designer on the team did and we heard from one of the users that you know in facebook as our data I'm okay with it but if Facebook is selling it to someone else or some other third party right like I'm uncomfortable because I don't have control of it and this gets to this idea of this sense of control that we have to you know at the UX level try and give users and that's kind of what really throws them off is not necessarily some objective level of privacy exposure or risk it's about feeling helpless in terms of managing it and controlling it and that brings me to there's this idea of UX called dark patterns and these are UX patterns that designers use you know in an unethical way to try and get users to do things that aren't in their best interest one of the ones that is relevant to this whole topic is something called privacy suckering which is kind of what I showed you at the beginning which is a flow where a user doesn't realize how much privacy they're giving up how much data they're giving up and this is done you know not in their best interest and it's because of these dark patters because you have this decision of how to design these flows that design is an inherently moral there's no obvious there's no offloading the responsibility and just doing your job or whatever right so it's incumbent on designers at the forefront lines of this battle for privacy and against surveillance for us to have some ethics about how we design these things and if you think you carefully not about just using the norms of surveillance capitalism not just relying on terms of service and privacy policies that nobody breathes very obfuscated disclose your requests like the Facebook example and when we have to come up with a better model so yeah and this is this is about moving through that spectrum is one way we need to think about privacy the other is about something called contextual integrity which is awesome research by an academic name's Helena Simone and what she frames contextual integrity is for privacy stake is it's these five principles data subject you know sender of data you have the recipient of data information type of transmission principle and it's when one of these five things changes that's when a violation occurs so it's about flows of information and that is kind of what our user was getting to when they say I'm okay with Facebook having my data even though we might argue and try and say like even that's bad but there have been started work out when one of these things changes but it starts going to some party that they didn't know about so there's also some interesting research around kind of the difference between hypothetical and actual disclosures when you ask users about privacy and about their information and differences in framing privacy as relative or objective so the research showed that and the privacy paradox is this right here this conflict which is in hypothetical situations you ask users you tell them how much weather risk is for a given disclosure and they say I'm not going to do it I would never never call that daily and then you have them you know they say give up all this data and we'll let you do this personality you have right and then they do but what we could do is we can frame or what the research shows is that users behave in a more privacy-preserving way when the risks associated with their privacy are framed relatively and that means that when you frame a risk of a disclosure what you're giving up relative to what the norms are so if you say you tell the user doing this instead of telling you telling you doing this has this objective risk like you have I don't know some eighty five percent chance of getting actor or the information kept you say this thing you're about to do is going to make you more or less private based on the previous behavior or more or less private relative to your peers then users typically will engage in more privacy-preserving behavior so that's one thing that we can implement at the u.s. level so there's this model there's this model of consent that we have now and a lot of it is implied consent this is the Terms of Service and all of that where nobody actually really consents to any of it because nobody reads it but it's implied that you have consented to whatever they want to do with your data because it's buried in the terms of the service and the practice of policy these talks kind of like about moving to informed consent but informing people is hard it's hard to get them give all the information in the world and they won't read it they'll ignore it you can put the fine print to login with Facebook button and it's not going to help so at you port kind of operate on more progressive consents model and this is actually a common I guess this is like a twist on progressive disclosure which the common UX pattern which is Just In Time notices wanting to textual eyes all of your disclosures rather than giving them a full sale like a wholesale disclosure around your around what around your data you want to do it bite-sized chunks and in the moment when it makes sense so I'll show you a couple of things were thinking about at Newport at the interface level just if you've used your port before you know that one of the key interactions and in the youth port app is this selective disclosure and this is getting users to affirmatively disclose things just in time in a progressive way it by an app by app and interaction by interaction defaulting information not to be shared having users opt in actively in the moment giving users a sense of control over what they're doing addresses some of the things so we're able to this is a you know maybe you come to and you would be able to share your name and your ticket all right this is something that you could do we did something similar in Denver this past year in February this is an idea of some people working on thinking about around after you share things in the moment allowing you to lay your preferences and defaults and build a robust and complex set of privacy preferences over time rather than having a user try and set this all in one one go right up front that gives users a sense that they're here they're usually on their way to try and do something else and this type of thing putting all of that up front most apps like Facebook and Google and whatnot they know that users are just going to opt-in to everything because they're trying to get something else done so we didn't even have users set up these privacy-preserving preferences in smaller chunks or they're easier to digest this is kind of a load and thinking about displaying data inside the app so one thing you'd be able to do is see who you share any given piece of data with ability to revoke data under gdpr log you can automate these types of requests to get those details that you know using sign messages we did we did log the request if you haven't made a request and then the you know whoever you made the request to is to respond and fulfill it time's up okay so I'll go through the rest of this really quickly so giving you some control allowing you to set some these preferences so the recap progressive consent gives users sense of Patrol framing privacy decisions relatively privacy preserving defaults allow this allow users to build preferences over time finally we just launched a new demo today so you can go try out some of this stuff not all of those screens are live in the app right now they're still part of our ongoing research but this is live right now if you go to the ecosystems you port Davi you can try that out and see how you or can enable a lot of these new share and use cases and new applications and we'll be with our shirts and our swag here in a bit out at DEFCON Park so thank you all [Applause]