Beyond the Black Box Algorithm: Health Data and the Public Good – San Francisco 2025

Thanks very much to Penny in the comments. This is my first time at this conference. Uh I am also fairly new to web 3. So it has been um mindexpanding experience and uh I have a lot of ideas now that I probably didn't have when I was putting these slides together. So hoping to talk uh to more after.

Um just quick note on my program. I'm the new executive director of a new program that's at UCSF and Berkeley that's seeking to join the power of AI, machine learning, new statistical method and clinical and public health uh excellence. Obviously um I'm speaking to as someone who has multiple actual multiple interests in uh in the topic of the talk. Um I am also a patient. So that's me on the on the right uh doing what I do every day, which is check my blood sugar using an old school uh glucose meter because I'm a diabetic.

I'm someone who's had the privilege of of a lived experience of in-person but fairly radical decentralized participation in movements to change the expectations of what the public can do uh in in terms of health. And I'm also someone who has spent a lot of time in the pharmaceutical development space looking at how to increase data access and resources particularly in the global south. So all of those things are sort of rolled up into the question of how health data is used and for whose benefit. Um I'm sure I'm not alone in hating the metaphor of data as the new oil in its basic extractive uh implications. But if data is the new oil, which is how it's regarded by many in the marketplace, health data is particularly precious and valuable, not only because it contains information about how we might live or stop living, and not only because it's tied to extremely personal um aspects of our lives, but also because, for example, for me who has now entered into multiple databases as someone with a chronic condition or for those of you who've had your genome sequenced, you know, that will never go away.

that will be true for me for as long as I live. So unlike a credit card, yeah, you can't kind of cancel your digital identity. And finally, our health data is updated continuously and through multiple means. So as I've moved about the conference, I've seen many people with Fitbits or um I know that um some people like me have glucose meters. I've seen aura rings.

um at least those of us who get our healthcare in the US about 93% of health systems have electronic health records. So a huge amount of data constantly being generated with potentially amazing uh benefit for AI and machine learning to extract insights about us as people and patients. uh and yet um you know the LLMs have entered and I we haven't even gotten to agents in this slide but um there has an enormous explosion of interest in the health industry. This is just one very partial survey of of entry that uh Kaiser uh put together, but basically billions and billions of dollars being spent on uh sort of putting together the power of AI and health data for various purposes and uh to various ends and yet with all of that we are really falling deeply short. So electronic health records, I don't know how many of you I've been interested because I'm a fairly recent transplant to California.

So some people in Kaiser for example have a single medical record number for the course of their lifespan here. But for most of us we have seen multiple health providers, multiple health systems. Um for digital health again explosion in the industry but extremely fragmented. So for example you know my diabetes I also am managing some blood pressure issues. Um, I have a smartwatch that measures how much I sleep or how many steps I take.

I have a portable blood pressure cuff. None of those things talk to one another and none of them are integrated with my electronic health record. So, super fragmented. And then to the extent that they use algorithmic decision-m uh it's mostly for simple prediction uh based on the way that things have been including including all of the biases and uh inequities that have pervaded in healthcare and also to be honest not taking advantage of some of the methods that would answer the much more interesting questions like what if I had changed med um medications two weeks earlier or uh many many different things that sort of distinguish between um just um sort of uh simple elements of fact and actual inference that could really help us drive clinical decision-m and I I should say that um just for example in diabetes there is also a wealth of apps a wealth of algorithmic notifications no visibility into how strong they are or how good they might be or how they might be improved because in fact they are blackbox algorithms you just kind of have to take it on faith that the company that has generated these things knows what it is doing and you sort of get the alert but uh no visibility even if you're a computer scientist into how it might be improved or evaluated. So in in keeping with the spirit of the commons and I know this picture is a very 19th century version of the commons I I before I came now I would probably use a different image and I but representing the digital commons is a a complicated proposition but I think you probably get the idea that it would be amazing to have a place where researchers could I don't want to say graze but let's say work collaboratively on pulled data um if there are a way to stream all the data from my devices and your devices and others um in in some privacy preserving way to be able to stream it, collect it, to share notebooks and code and then to return that data and insights to patients and to clinicians.

Um for those of you who are working as developers or in computer science or data science, um there is you know I think how many people are familiar with Jupyter notebooks just yeah so a lot of people. So obviously the idea of nature called this one of 10 computer codes that transform science. But about 10 million scientists a month use Jupyter notebooks um to share in the cloud collab oh collaborative code um visualizations that other people have developed that you too could use etc. And so uh the question is whether we could imagine something similar for health. And I have colleagues who are working on something called Jupyter Health which would again be allowed uh be a sort of a platform middleware to be curatable for mixand match data sets to be vendor agnostic to bring that ethos of sharable code and notebooks to allow for comparative testing of algorithms and benchmarking um and of course then to be privacy preserving and return benefit to patients and clinicians.

Um this is uh the team that is working on it and it's being led by Fernando Perez who's one of the founders of project Jupiter and also by my faculty uh colleague and co-director Ida Sim who is both a leading informaticist who has focused on interoperability her whole life but also a practicing primary care physician. Um and uh this team is also working with common health. I don't know how many of you I know this is probably an Apple focused crowd. Um, Common Health is an Android-based app available in 55 countries. It allows you to uh basically to download your health data and hold it on your phone.

Um, you can decide who to share it with and for how long in terms of research. It has uh basically links to about 15,000 different health systems and providers and labs. So whether you get your care from the VA or from Kaiser or from uh a clinic in Kenya or your lab is Quest or uh you basically can aggregate this data on your phone and carry it with you. Um it is not sold it is um uh basically it is not shared without your consent and you can revoke your consent at any time. The bigger vision is not just um digital health but actually a number of missional aligned platforms which are sort of aggregating important health data to enable research which might be joined.

So night andgale open science is another one of these. It's not about digital health technology so much as aggregating from around the world tissue samples uh waveform data from uh ECGs, microscopy, mamogs, TV chest smears and making it available for free for researchers. Um all of these data are ground truth with at least one real medical outcome. So it's not just what a radiologist thought would happen, but it's actually um what happened to that patient. Um and the idea I think is that if we could create the will to share which is quite absent in the healthcare industry at the moment, we could begin to build um what the organizers of this conference have called the infrastructure of resilience um with health data.

Um this slide is the one that I probably would most need to redo after coming to this conference as to be honest before the conference I did not know how to dow. I was not aware of uh you know these kinds of certificates uh uh cybererts. Um I was not aware of the possible insights that chicken communities could give us on intentional gathering as we heard this morning. Um I I have drawn a lot on uh colleagues who are working on the question of what is public AI because as we know at least particularly in the US um AI has been largely the domain of certain large mega companies and the question of what is public AI and what uh collaborative mechanisms to decide the direction of AI tools is has been relatively slim. So uh I don't know how many people have looked at uh Gideon Lichfield who was here but I think had to leave um his musings on public AI in his Substack or uh have followed the model of CERN in Europe which is supported by multiple EU countries for basic scientific research but actually is available to all countries of the world with a commitment to transparent and open science.

Again, I realize that these are sort of old school governance models that are not as radical, participatory, and decentralized as much as what we've heard about here. And it would be interesting to think about um what some of the tools that you all are much more familiar with might bring to the health data ecosystem. Um the classic tragedy of the 19th century commons was that more people graze than replenished the pasture or more people fished than replenished the sea. Um and so very have been very interested here in thinking about how we might return uh value to patients who contributed was moved and interested by our open ledger colleagues uh proposal that we we heard just uh before this one. Um and I will say that in the world of health care even though the root the Latin root for the the word data comes from the same root as that which means to give there's much more attention to what is given than to the giver.

So very few mechanisms to return value to those who contribute data. um very interested not just in whether or not we could um give them a share of a token for that effort uh but also whether we could support people to have a different relation to the idea of data contribution so that they actually had some support for using the LLM and these other tools that are available themselves um as they u move through their journey to um seeking better health. Um, I will just end here by saying that I really think that we could do something together and transform what is a super fragmented, super siloed, super proprietary ecosystem um, of walled gardens with rich with health information while locking out others or what my colleague Ida Sim calls the peril of a thousand flowers blooming but with no coordination or ability to aggregate. And so really it's more inducing of allergies than it is of of benefit to something that's much more like a shared garden where you could um assemble and curate data that was matched to people with multiple conditions across the life course and an an approach to uh health research that recognized that um we would be stronger uh if we were open source and u sharing. Thank you.

[Music] [Applause]