Toward Universal Access to All Knowledge – San Francisco 2025

Hi, I'm Mark Graham. Uh, I direct the Wayback Machine and a few other things here at the Internet Archive. And on behalf of the 120ome plus folks who work for the Internet Archive, I welcome you here to our headquarters. And uh we're honored uh and uh and excited uh to be sharing time with you this weekend. I've got about 20 minutes uh uh and I've got a lot that I want to share.

I want to give you an overview of what we do here. Uh, welcome to our headquarters building. I want to also, uh, put a plug in for our, uh, headquarters of a mission aligned organization we work with called Internet Archive Canada. And if you look at those two buildings, maybe you see have a bit of a theme there. Uh, we like to say that uh, we bought this building because it matched our logo.

Yes, the logo came first. What do we do? What we do is we pursue the audacious objective of universal access to all knowledge. That's where we end with the superlatives. Basically, uh we uh work uh as engineers uh trying to get as much material as we can.

material that's analog that we digitize, that we preserve, organize, and then make available or born digital material like for example from the web where we get to skip the digitization process and we focus on preserving, organizing and making it available. And we do that at scale. for example, like oh about a million software titles uh Pac Pac-Man, Tetris, um Oregon Trail, Prince of Persia that you can run in browsers. Uh 8 million moving images, these are movies, uh Prilinger archives, many of you know, things like that. Uh 15 million audio recordings.

Yes, basically almost everything from the Grateful Dead. more than 13,000 concerts, and hundreds of thousands of other live concerts, more than 2.6 uh million television news programs. I'm going to focus a little bit on our television news archive project here, more than 7 million ebooks. Uh these are uh digitized versions of books that we physically own and a lot of web pages uh more than 900 billion of them.

this we are on track uh to surpass 1 trillion web pages archived this year and if you come back here in October I think October 22nd for our annual event we'll be talking more about that altogether this represents more than 50 pabyte of material and yes it is replicated uh in at least two places both in this country and out of this country so rest assured We are taking precautions. We are paranoid. Okay. The Wayback Machine. Uh probably everyone here has worked with the Wayback Machine.

I'm going to try to share some uh uh some characteristics of it about it that you might not be aware of. First of all, we add more than a billion URLs to it every day. Archived uh URLs. Actually, it's probably closer to 1.5 billion the last few months.

And we do that through partnerships with many many organizations uh that provide us signals. Signals of in many cases millions and millions of URLs a day. Entities like Cloudflare, Wikipedia, Automatic, the folks behind WordPress, Reddit gives us their entire uh feed every day. Uh we get a ton of material from Tik Tok. hundreds of millions of URLs that uh occur in uh exposts formerly known as Twitter etc.

So what we do is we we have established partnerships in total more than 15,000 processes that run on any given day around here. Each process has an algorithm or a story behind it for why uh each of those 15,000 processes are contributing to the stream of URLs which makes up the more than a billion a day that we archive. In addition, any of you can add to the wayback machine right now uh from your phone or from your laptop. Just go to web.archive.

org/save. There's a variety of interfaces uh web uh iOS Android apps uh browser extensions you can send in via email. There's an API if you want to build applications and through this interface that anyone here has access to. Millions and millions of of uh web pages are archived every single day uh that we have nothing to do with. It's just uh you know the community adding to the the corpus of archived material.

Something happened last year which is that Google uh decided they were going to uh deprecate their caching system, Google cache. Uh which seemed like a bit of a bummer. Uh but I I called them that day and we worked something out and then in a few months after we worked through some bisdev stuff that takes forever, the tech was really easy. Uh then the this particular headline appeared. Google replaced Google cache with internet archives way back machine.

That was a bit of a nice day. Yeah. Yeah. We've got a few of these uh things I'm going to share where you might like here's another one for example that is kind of a bit of a milestone. We've been working with the Wiki uh media foundation and Wikipedia uh editors for years and years and have written robotic software and uh just as URLs across the web go bad uh so too links on Wikipedia articles they they deprecate like maybe 10,000 a day and we work to fix them.

To date we have found and fixed more than 23 million broken links on 467 wikis around the world. In addition to that, we're working to add context to what people experience on the web. We have uh hundreds of thousands of records now where if you go to the Wayback Machine, there'll be a yellow banner at the top where you can get additional metadata information about what you're looking at. Here's an oldie but a goodie. It was a a tweet by the WH that was telling us, rest assured, don't worry, CO is not airborne.

They ran this for months. In fact, we know that COD is airborne and and if you click on the the link on the top there, you'd be taken right to this article explaining that. Here's another example. Um, hundreds of thousands of academic papers are retracted for a variety of reasons. How would you know that?

You're reading an academic paper on the web. Wouldn't it be nice to know that it was retracted? Well, here's an example of some context that could bring you right to an explanation of the retraction. Okay. We work a lot with another organization, another mission organization called Better World Books.

It's one of the world's largest bookstores. We bought it a few years ago and then we donated it to a nonprofit uh and uh we take about 4,000 books off the uh conveyor belts every single day. That's about a million books uh a year. And we digitize them. You see here in the picture, you see the on the right hand side lots of boxes.

Those are going to be shipped to the Philippines and digitized just like this. There's a machine downstairs you can all touch this a tabletop scribe. Because of that, we've been able to work once again with Wikipedia to add uh links to uh citations in Wikipedia articles to books and papers available from the internet archive. One of the other things we do is catalog academic papers, more than 35 million of them so far in scholar.archive.

org. And through a combination of this digitization of books and um aggregation of academic papers and writing robotic software, we've added more than 3 million links to Wikipedia articles to those books and papers available from archive.org. This is part of our effort to help weave our library into the fabric of the web. And kind of one little side project, more like a demonstration than anything else.

And I'm sharing some of these ideas with you today uh in because it might resonate with some uh project or dream that you have that we might be able to collaborate on together. So for example, when the Muller report came out, I noticed there were 2,300 footnotes in it, but only 15 of them were links. So we get we went to work and we added more than 747 links to the Muller report, republished it as a new EPUB and now when you read that EPUB on an iPad for example, you can click on those links and go right to primary source documents. As Wendy said, since 2004, we've been working in collaboration with the Library of Congress, the National Archives and Records Administration, US Government Publishing Office, Stanford Libraries, University of North Texas, and now Common Crawl Foundation and others to archive much of the federal government. We do this every four years at the beginning and and a of every presidential election and then after the inauguration.

We've been working on this project now um this latest round the 2024 2025 edition for many many months and we have archived more than 700 terabyte of material a couple hundred million pages and why is that important? Well, we've been learning it since a few hours after the inauguration. We have been paying attention to the unprecedented removal of material um from our from the records of of our government. This is our material that our tax dollars play for. Here's some examples.

This is a NASA web page before the inauguration. That's that same page right now. Here's a web page from the US Department of Justice about United Against Hate Crime. Page not found. Now, in both of these cases, the material is available from the Wayback Machine.

And currently uh the wayback machine is integrated natively with the Brave browser. Um and I also noted that integrated with with Google cache and we're now working to integrate ourselves more fully with other of the browsers and with search engines and other platforms. We built a full text search of that 700 terabyte of material and last night I typed in the word transgender and then further isolated it for um a particular um part of the US government and I found 11,000 results. The first result was transgender behavioral uh disparities fact sheet and this is what it looked like uh before the inauguration. You could actually see the the document there.

You can read the report and now you see page not found. So we're just beginning. We've been focusing on archiving this material in all cases like getting the material, preserving it. That's job number one. And we're shifting now more of our effort into trying to help understand what has changed, what has been deleted, and reaching out to and working with and supporting journalists and researchers and others to try to help um make sense of what has happened and make accessible and useful the material that we have archived.

So um finding things that have have gone away is relatively easy. Finding things that have changed is a bit harder. Uh but here's an example where a uh an a page uh dealing with addressing hate crimes has been completely re rewritten edited and you can see the changes in yellow and blue there. Another example, the the actual um the bios of the scientists have been removed probably because they have some of the scary words in them. And here's an interesting one.

Results not found. This will be really hard to detect, but uh we're using AIS to help us in this work on on the on the left hand side under teacher resources and this is from the diplomacy.state.gov. So this is from the US state the US state department teaching resources.

There were 41 results before the inauguration and now there is zero results. So somewhere in the way back machine there are 41 reports uh teacher resources relative to diplomacy and even government archives are not safe. This is uh a section of the uh USID's website about global climate change. It was an archive within their site and now you just get a page not found. So you don't even get a record that there was anything there.

It's just gone without a trace, but available from the Wayback Machine. So, as of last night, we have identified more than 110,000 web pages that have been removed since inauguration day. We think that number is much, much longer, and we're going to be working to identify and making those resources accessible. This morning, this is what I woke up to. an article in the New York Times um suggesting that the administration plans to dismantle these seven agencies.

I just want to highlight two of them. The US Agency for Global Media, they run Voice of America. A few months ago, we decided to uh double down on archiving Voice of America and we identified hundreds of thousands of videos from Voice of America that were available from YouTube. We archived all of them. And then uh in addition I note there the instit the institute for museum and library sciences.

It's especially relevant for this crowd. The IMLS provides funding for uh museums and libraries and similar institutions throughout the United States. Uh this work has not gone unnoticed. To date, we have uh cataloged more than 250 news stories that have written about the work of the Internet Archive and our partners uh andor that um that have referenced the work in some fashion. You can follow this at archive.

org/about/news-stories. I don't expect you to remember that. However, you can come up and talk with me. I'll give you my card. Okay.

We also archive television news around here and uh you know it's it's been said by some that truth is the first casualty of war. Uh when when uh Russia invaded Ukraine, I decided to start archiving Russian, Ukrainian, Bellarusian and related uh television. So we've been archiving four Russian television channels 24/7 and doing different things with it. Here's what our television news archive looks like in the US. We've got closed caption.

We've got text. It's searchable. It's discoverable. It's accessible. This is what the Russian television archives look like.

You notice a difference? There's no captions. So, Russian television doesn't require captions and transcripts like US television does. So, we worked with one of our partners and we built a visual interface into the material. Okay.

that made it a little more accessible, but still uh not easy to to uh search through. I'm going to try a little demo here. We'll see if this works. Okay. So, what you're looking at here is you're seeing English language close caption on the bottom.

So this is basically um we're doing a transcription from the Russian audio into Russian text, a translation from Russian text into English text. Um and then we're displaying it here as captions on the screen and it worked. Yay. Yeah, that that was an applause slide, right? Um we're using a variety of AIS to do that.

Actually, the full pipeline has two other steps. In addition, we'll take all the material from a given TV channel over a 24-hour time period and then we'll group the stories together and then summarize the stories uh and then uh create headlines. So, we have English language summaries for news from Iranian, Bellarusian, Russian, Ukrainian, and other television. And this year we're working to expand that out to a 100 channels and uh and to have all of that m material run through the pipeline. So there are a few collections that we're working on that we can maybe use some help with.

Here's some examples. 350,000 uh European dissertations. These were going to be destroyed. Someone put a notice up on a list serve a few months ago and said, "Hey, we've got these dissertations. We're going to just like get rid of them.

Anyone want them?" And uh Brewster uh heard that call and Brewster said what Brewster almost always says. Can you like anyone have an idea of what Brewster says when you ask a question to him or make an offer? He says yes. So uh we'll work out the details later.

He said yes. And that that meant packaging up this material, creating it, putting it in boxes, putting into shipping containers, and bringing it here to the United States. And we did all of that. Here's another example. We haveund we we have 160 million pages of medical books and and serals that are waiting to be digitized and 500 million pages of periodicals uh that are in um in microfish mostly.

Okay. So this uh this gathering here uh today and tomorrow is called funding the commons. So I thought it might be appropriate to say something about funding the internet archive. How do we fund this place? So last year we spent about $28 million.

And here's how the money came in. Three different buckets. Bucket one, more than 150,000 individual people who gave us money either monthly or yearly. Many of many of you are in this room. Someone just said, "Hi, I've never been in the building.

I'm a neighbor, but I've been donating money to you for years." Thank you very much. The the second bucket is highwealth uh individuals and foundations. And some of those high-wealth individuals are associated with this gathering. So thank you for them as well.

And the third bucket is a program related business activity. That's nonprofit speak for stuff you charge money for. And in this case it's we charge money to some 1300 museums and libraries and universities and governments that we do digitization work um on for their behalf. All right. How how can you help us?

I'm not going to ask you to give us money. No, I'm going to ask you to do three things. One, please use our service. Uh we we are a library and library success is when people use the libraries. So, please use us.

Thing two, please tell us how we can build and offer a better library more than anything else. That's what we want. We want we want feedback. We want suggestions. We want complaints.

We want all of it. Please give us your feedback. And the third thing is please connect us with individuals and organizations and projects that you think could benefit from what we have to offer and that we might be able to benefit as well. And I'm going to actually add uh one more fourth thing I wanted to say. I'm going to make an invitation for everyone here to try something.

Um step out of your comfort zone while you're here today and tomorrow. Walk up to at least five people that you don't know. Don't hang out with the people that that you know. That's not why you're here. That's kind of a waste of time.

Go to the people you don't know and say, "Hi, I am Mark and what's your name?" and talk to those people. But my suggestion is do that with some intentionality. Intentionality for two things I want to suggest. One, selfishly consider something in your own life in in your world that you're trying to advance and that you want to make a material difference in how you move that forward.

And secondly, look for opportunities where you can be of service to other people who are here and try to do that with five people. And that's all I have for you. I am Mark at archive.org. Thank you very much.

[Applause] [Music] [Applause]