Luke Drago: The Intelligence Curse | Intelligence at the Frontier 2026

[music]

Well, hey y'all. I'm Luke Drago. I am the co-founder of Workshop Labs. We are a public benefit corp and AI research company with a mission to make people irreplaceable and I can talk a bit at the end of what we mean by that. But, I think the thing I'm here to talk about today is an essay I wrote about it about a year ago now, which is a bit insane, um, called the intelligence curse.

And to get a sense of what I'm talking about, it might be helpful to think a little bit about the concept of the permanent underclass. Uh, meme something like a year ago and the idea was, all right, AI is going to make a few people extraordinarily wealthy. And what is it going to do to everyone else? And if you've read the news at all in the last six months or if you talked to people outside of Silicon Valley one of the things you've probably heard is that people aren't too excited about say AI things. People here are quite excited.

There's a ton of opportunity and a ton of progress we can make. But, when I go back to North Carolina where I grew up, um, people are worried about two things. Called my sister about this last night actually and I get asked it asked in a kind of kind of somewhat semi-serious tone, is AI going to kill everyone? And then I get asked in a much more serious tone, and is it going to take my job? And I think people are much more concerned about the likelihood of the latter in some cases, although I think the former is still absolutely worth addressing.

I think if you take a look at why people are worried about this, it's actually just pretty intuitive. It looks like there's kind of two paths that AI can take with AI right now. Uh, and you know, to really illustrate this I've got the classic beautiful map of my path as digital fiscal as a map. Uh, but there's two paths that I think AI can take. One, where we build technologies that are genuinely empowering.

You get this app what that means a bit. And one, where we build technologies that are a bit more of the opposite. I think the kind of Silicon Valley meme of the age guy that did builds a Singularity system that does all meaningful economic work. And maybe because that's tend to be something like, "Oh, we'll just give life nice AI pace for everything." A technological bonus does not.

I think this is a bit counterintuitive. When we talk about automation, I want to talk a bit about the idea of job loss. Uh job loss isn't just, "Oh, we do a little bit of displacement." The goal of the last couple of years of the automation has been, like I mentioned, AGI. The idea behind that being uh all meaningful economic work gets automated by a single system.

And so, that's not a world where we automate some tasks uh and then other tasks are left alone. It's not a world where we uh automate a little bit of work over time and then we automate some other work later. The goal of something like a trillion dollars in capital has been aimed at automating everything. And automating everything doesn't just mean you lose your job. The bedrock of the economy uh and of civil society is this relationship between labor and capital, uh where because people are useful, people get rewarded and paid for these things.

And if you look at the history of automation over the last, you know, couple hundred years, kind of pre-post Industrial Revolution, when technologies rewrite that social contract, the social contract can move in two different directions. It can move more favorably towards regular people or much worse for regular people. In the Industrial Revolution, we built technology that quite centralized. That centralized power in the hands of a couple of people. And then uh made that made the individual people there are quite interchangeable.

Um there are only so many people that uh lots of people who work in a factory. Only so many people can do specialized labor. But as the Industrial Revolution enabled additional decentralization, uh as the as electricity enabled factories to pop up everywhere, and as the kind of bedrock of the economy moved away from just being single uh production in the factories to specialized goods, specialized labor, people got powerful again. And that's the same reason why today, you know, a senior software engineer can go to Google and demand 18 cafes on site and demand a $300,000 salary because they're specialized labor, Google makes more money with them in the picture, and so Google really wants to compete for that worker. In a world where those tasks are off the table, where the labor that's left is interchangeable, that specialization is less important, and that people matter less, well, in that world, the incentive for these powerful people who have concentrated power, the people who own the frontier models, the people who own the compute, the people who own the data, to actually do the kind of grand UBI experiments that are always talked about are pretty low.

And we don't have to look to, you know, the far-off future to think about this. We can just look to economies today that already exist and have factors like this. The intelligence curse might sound like kind of an ominous name, but it's actually named after a cold and dry concept in development economics called the resource curse, where states that have a a resource as their predominant mode of production, you know, they make a lot of oil, um, they have a lot of oil in the ground, they don't invest heavily in education, they don't invest heavily in building more schools, they invest heavily instead in what it takes to get the oil out of the ground. And so, even if these states have plenty of money, have plenty of power, they don't return that power back to the people. And because their people aren't involved in the production of that value, because very few people are required to get oil out of the ground, the result ends up being that those people with the rest of the world do less.

Does actually be better off. Uh, they aren't taken care of by uh, their system of rank and file. You've run this experiment in places that are data impoverished after depression of war. You've run it in Congo, even run it in the US inside of the Rust Belt and Appalachia. When the jobs leave, the incentives go down to do something else.

And all we argue in the intelligence curse, uh, my co-author and I, is that in a world where you could literally automate all labor, you get the exact same effect from that development econ, but just a bit stronger. And the result of that is that you, the people who don't necessarily own the frontier model weights and don't own the compute that the is powering the frontier model weights, you matter a little bit less. Both, you know, you're probably not employed, and you're probably not nearly as powerful. Now, why does that matter? Well, it matters for a couple of reasons.

In this kind of status quo social contract that we have, there's a relationship between these kind of powerful actors, nation states, and governments, corporations, and regular people. And they need you for value, and this gives you power to displace them. Governments need people to get things done. Companies need people to get things done. And because they need people, there's people to negotiate.

And the more they need those people, the more bargaining power they have. And the result is that both people and powerful actors constrain and write the rules that constrain each other. It's a relationship that's shifted. If you society is much less dominated by powerful actors, post-industrialization is much more dominated by regular people. And it varies country to country, place to place.

But this is what we mean by the social contract. This thing is constantly renegotiated and rewritten. But if you have AI that can literally act, and that's the only way you make that, well, it's not as good. Uh it's not as good for a writer. Um powerful actors don't need those people for the production of value.

There's no reason to spend 150,000 or 300,000 or million dollars on an army of people uh when GPT-5 or GPT-7 or whatever it is can do it for, you know, 10x less and about 10x faster. There's no reason to. And so, because of that, your relationship there is harder to negotiate. I mean, think about what's happening, you know, even in like regular recessions where uh when the labor market gets bad, your ability to negotiate for a higher salary is just much worse. But imagine that at the extreme.

This wouldn't be a problem if we weren't actually concerned that something like total automation could be coming. But if you listen to the leaders of the frontier labs, you know, Dario is saying something like we could see 20% white-collar unemployment in the next five or six years. Economist uh at the uh economist at the Bureau of Economic Research had a paper out as his core that can saw 2024 where they argue that in an AGI-like scenario, the thing you get basically overnight is wage collapse. So surely someone's going to work on this, right? Surely.

But the problem is a lot of the incentives for those powerful actors go down. They don't need to worry about it as much. It's not as important for them to fix the problem because they're winning the benefits. We call this uh resource capture in the resource curse of a literature where, you know, the guys who are making a oil money don't really care what happens everywhere else. It's not their job to care.

And when I've spoken with, you know, everyone from members of Congress to you know, people at the regular companies, none of them seem to know what to do. And that's because fundamentally this isn't a policy problem. It's a technology problem. The question that we get as a room filled with among other things builders is what kind of world we're going to build. And are we going to aim for technologies that replace us and knock us out of the picture?

Are we going to aim for technologies that do something a little different? The fun part about this though is there are technologies that you can build. And I don't think any of these technologies in isolation solve the problem. But I think combined, the question for us today is can we build technologies that are competitive with full automation, but that also keep people relevant, that make people irreplaceable. And we argue in the paper that there's three ways to use uh kind of tiered lists.

You probably would have averted the biggest risks from AI. Those risks are both bad in and of themselves, and those risks make it harder to centralize technology. Uh and the kind of simple fruit through what they're doing in the entire strategy is I refuse to ask AI and not the kind of like pray to be with, you know, more assistance for humans who will loop in and not actually matter. But today, humans and AIs were generally working together in ways that are worth path for. People will call us the standing descent of our period.

And then finally, to build technology to make it easier for policy makers to democratize control of an AI. So first, I got to actually do a first of these major risks. Second, you put AI's power in hands. And third, you then democratize control over me easier for people to ask for a friendship prior to the beginning of the year of these. What does that look like concretely?

On the averting side, there's a whole lot you can do to make the world safe for advanced AI. So there's everything from reducing bio risk. Uh this is probably the single biggest mis- risk uh misuse issue from like extremely powerful AI. This is everything from, you know, bio security evals to KYC to DNA synthesis screening to vaccine production. As a side note, if you're a builder building near at the frontier of AI capabilities, getting evals to this kind of stuff is not that hard.

Uh as a company, we're undergoing a bio security eval right now, and we realized we need to do this a couple days before a launch, and we were able to get a hold of the best experts on this in literally 24 hours. So there's an ecosystem that already supporting order and assess. As those working in those other categories, and that's what we're working to reduce the amount of risks. I think it's all going to come pretty pretty often. There's work on AI alignment.

And I I think everyone in the room has heard about alignment. I won't dwell on it too much. But there's work on AI alignment and the related technology, everything from inter uh to uh you know, formal technical methods to keep models robustly under human control. There's cyber security, one of the biggest threat vectors that would enable people to uh stop the diffusion of AI and threaten people to keep it centralized. It's because AI does pose a massive cyber security risk.

There's lots of vulnerable code that's never been hacked into because you haven't had the easy abilities. Relatively surely all code of AI is available for an agent to go and steal, for an agent to go and break into. And so work here on formal verification of code, on refactoring and secure code, all of this is quite important. And finally on AI control, which is different than AI alignment. AI alignment says we align the models they own very well.

AI control what it does and they do with our barriers or what are we going to stop this? And if you can accomplish these and really reject this, you may be safe for mass diffusion. And so instead of couple of trusted actors who are they need to have all power, you have hundreds of or tens of thousands of actors. You more ease develop it powerful models all the energy available. As the future can be always just like ports.

So how do you diffuse it? Is a separate question. Because diffusing doesn't just look like giving everyone a subscription to chat GPT. I think this is kind of the way that the big labs are treating it. And the reason they're doing that is because it also just increases their Lauren.

What diffusion needs here is that the actual elements that go into the production and use the models are controlled by more and more people. As part of this looks like augmentation overall issue. You want to boost this experiment the central theory. And we'll have some more this is for burning out for a couple of days or a couple of weeks now I guess. To measure how a model is or is the human supposed to automate that.

We're working on that now I think we should do more of that. I think the kind of dream of the AI agents do everything. Both be assured and also just not for learning and side effects. And of course side effects models that will go on psyche. Decentralization that is the critical part.

And this is part of why securing your forehead so important. Because you want to decentralize basically every layer of the stack. You want to choose as many as possible. You want people to call as a filter data. You want to train to be socialized.

There's companies who are doing great work on this right now. Premier electric, for example, have gone a whole up to decentralized training mate. And you lay possible for anyone to have a friend she lies in their backyard. Uh our sky, he just uh we are getting ready to open source under grain goes for a local community which is the best of class model that show 50 as of cuz we'll speed up. Uh essentially they will you more people to train post break models that are rewarding while they are getting votes.

And the more holes the ghost rate to build the billions of the circles tens or hundreds, the more people are in control different models and the more they're likely to win. On a privacy side, data is the biggest bottleneck in our nation. In particular, high-quality task data. And there's lots of companies that are doing everything they can to gobble up that data as fast as possible. See that all the people who are already generating the promise of frontier privacy.

Training models around you sorry pulse priority exactly. People will put their data to work without handing over their data to big labs that use it against them. Who don't use that data to train the next model that it automates them too. Uh and this is true but at the enterprise level and at the individual user level, the more data is diffused, the more control of data is diffused, the more people still matter. And finally, user alignment here which is different than just the general concept of can we align models?

But can we align models to individual users? Can we have a model that is robustly acting in the interest of one person? That works like them, alongside them, that can represent them when they're not there. Uh this could significantly increase an individual's ability to both get things done, to be represented in the broader society, to negotiate deals and contracts that otherwise they might be getting screwed on. And fine, sure maybe the other guys have bigger more powerful AIs, but I'd rather go up with my own AI than without one.

So I think the promise of aligning models to users is probably one of the most exciting ones for genuinely diffusing the technology so that it actually matters. You want to know the model that you're using is actually in your best interest and working alongside you. And then finally, you probably want to talk about democratizing and putting AI fresh into regular people's hands. So, this both technology called policy problem. If you done everything right, step one and two, we don't know what went wrong with lots of AI and lots of people of what.

But this is very irritating. This is what I wanted to happen. We're still in a possible outcome than today. We're still in a world where we expand this in upper of hospitals with AI. A lot of people have It could be honest.

What are people aren't only They won't always be suppressed with this. They aren't going to have the day means trained as your arms. And so, we don't just want to bring the people who are already at the frontier You know, this is my expanding the circle of winners from a couple hundred to a couple million. We want to make sure everyone gets across the finish line in the AI revolution. And that requires making sure that people can robustly control it even when it's against the interest of everyone of model providers and creators.

The problem though is that the intelligence curse is probably most pernicious effect, and we see this in resource curse uh afflicted states, is that needs of our society essentially. Democracies are built on a foundation of diffuse capital. The reason why the Magna Carta occurred in England was the weak community of nobles and lots of lords and not in Spain where they're the stronger one. Uh When you have this diffuse capital, you have to settle disputes. In fact, they didn't need both courts in England to set up parliaments.

And if you concentrate diffuse capital, If you concentrate wealth in the hands of a few people, well, the interest of governments become serving those people who are paying the tax base and not in serving everyone else. So, robustifying democracy is one of the most important things here. I don't know if you've checked recently, but democracy isn't looking that strong this week or this month or this year. It's had a rough I think two decades according to Freedom House. It's been a been a rough ride for democracy.

And it is both because democracy has been distrusted and because technologies we built have weakened the trust that we have in democracy. We can strengthen that. We can do it in a couple ways. One, we can build pro-democracy tech. And what I mean by here is tech that makes it easier for democracies to work.

Things like preference aggregation, voting tools, even in some ways prediction markets that help make it clear to politicians what outcomes of decisions might be. All of these things can make it easier for democracies to function. You also probably want something like human verification as private as possible. I think world has done some really interesting work here. But you want for you able to prove that when you're in the voting booth in 20 years or when politicians are reacting to Twitter, something else happens.

And finally, you want to build coordination technology and that are information environments. And you know, social contract enforcement, AI out there, incentive tools, internet laws. You want to enable people to download the decisions they're making and can make those decisions together. The thing I'll leave you with is that we don't have to live in a world where people are sociable. You don't have to build technology that erases people.

We can build the technology that makes people irreplaceable. That's not a lofty it's not an impossible one. We've been doing pretty well for the last couple thousand years. I think we can keep doing it if we choose to build something differently. And I think the window is wide open for different opportunities and options.

So, as you build today and as you work on you know, whether you're doing the hackathon or going back into a company, I think you should work on making people irreplaceable. Thank you.

[music]