Evan Miyazono: Alignment Is Not Enough | Intelligence at the Frontier 2026

[music]

Thank you all for coming. Um I have my computer here because I am very sleep deprived and speaker notes will be useful. Um I'm I'm I'm not a The PhD means that I know what expertise is. Does not mean I'm an expert in anything I'll be talking about today. I do talk fairly quickly despite the fact that I am no I was seriously under caffeinated at the moment.

Um In terms of what Atlas Computing is, we're a nonprofit. In terms of what Atlas Computing does, you should model me as a research fixer, um professional commenter on Google Docs, and arguably derailer of careers. Um with consent, for what it's worth. Um and I want to start by saying that uh whoop All right. Oh, yeah.

This is Yeah, this is working. Or is Is this working or not? Okay. Um so, on this slide, I want to start off with a thought experiment of an alignment box. This is a box where a person walks in and there's an AI system in there terminal connecting to it, and they can spend some time and they can walk out and the AI system is aligned.

Anything that they would want, all of their values, all of their preferences, are now encoded into that AI system. Sounds fantastic, right? How many people think that this is like, "Great, we're done. Problem solved." My audience knows If you're here, then you probably uh you probably know where I'm going with this.

Um I think this gets even more interesting if you look at, on the next slide, this claim that maybe this could take uh this could have some real constraints on it. Like, what if it takes 80 hours with the person to do it, and it costs 300 or 30 million dollars in tokens to do it? I don't think that this is an insane possibility, even if we could develop this technology, which I think most people working on technical alignment would say would look at this and say, "This is This is the problem. This is the whole problem that they're focused on. Once that this is solved, they would say they're done they're done."

It's very unclear what you do after that in terms of how this goes well. Who gets to use the box? And of course, the following question of who decides who decides who goes in this box? The um like if you only put one person in, this seems like a very bad outcome. If you put multiple people in, then you have a structure where you need to be able to determine what happens if two people's values conflict.

Probably the best answer that I've heard so far is you put up all of the money and you have every single person go into the box. And you have all of the AIs deliberate with each other, which still doesn't have great properties. There is no right answer to these questions. I think there are a lot of technical questions where you could imagine having an objective feedback cycle where you could evaluate, oh yes, this worked or this didn't work. But there are a lot of things here where what you really have is a gap between what you would say is the case and what you think ought to be the case.

This is the is-ought chasm or Hume's guillotine back from political philosophy. And the argument there is that you cannot logically deduce statements about the what how the world should be solely from statements about how the world is. You need to take as an assumption a statement about what the world should be in order to deduce things like what one should do. And so uh I think that it's really important to highlight that there is such a thing as a stupid question. A compass rose is very useful if you are making maps in most places, but there are a lot of times, like if you're going to the North Pole, putting a compass rose on a map of the North Pole does not work because North is not the same direction everywhere on the map.

So, um there are questions like this in alignment and AI governance and things like this. I claim that alignment might not be Well, because And my piece of evidence is that I now, me, am aligned with the things that I will want in maybe like 2 hours, 5 years, a decade. Um, those are two different people. Who do you align with? Um, I think that adversarial robustness might also be impossible.

Unless you say something like lying under any circumstance is wrong or doing something that looks like killing is wrong. You could end up in this some like weird Ender's Game scenario where any sufficiently intel- If you can craft the world around an intelligence, you can create the scenario in which the thing you wanted to do, that it might not want to do, is the right thing for it to do given the context that it has. And so you've tricked it to you've like redirected its moral thinking onto the thing you want it to do because you've engineered the scenario. And I don't think that strict deontology is like a reasonable thing. Kant's criticized for saying like, you should not lie to the ax murderer about where their victim is when their victim is hiding in your basement because lying is wrong.

I think that's not a world that we want a That's not a philosophy that we want AI systems to adopt. Um, I would also claim that alignment isn't sufficient. And this goes back to the Is-Ought chasm, this idea that alignment kind of optimizes for a thing and you have this general concern or I have this general concern that we don't train people to be functional members of society. We don't have useful civilizations by saying, "This is the most moral human, be like them." Or like this is the best citizen.

We say, "Here are some ground rules. Just don't violate the ground rules and you can pursue whatever it is that you want." And so I, at this point, oh, and yeah, I'll There's also an interesting open question about whether you want alignment or corrigibility. Alignment would be this notion that the AI systems we build have a clear distinct set of values and preferences and they act in accordance with those. Corrigibility would be that they can This is the opposite of uncorrigible or incorrigible where a corrigible AI can be convinced to do things differently.

You can change its values through deliberation or evidence or something like this. I think that on the alignment side, you have a risk where the AI system will be incorrigible and if its values diverge from yours it will do what it wants and you can't make it do what you want. And on the corrigibility side, anyone who gets control over the AI system will go and make it do whatever they want and this leads to potentially insane concentration of power type risks. Um so you might ask what what kinds of solutions does one have for these things and I start looking back to great examples where a small group of people decided to spread their power to a lot more people and probably the best example of this would be the founding of America. And I particularly love this quote from James Madison in Federalist number 51 around how security against gradual concentration consists of giving constitutional means motives uh means and motives to resist the encroachment of power.

This is like the original posing of we should have different branches with checks and balances on each other. It may be a reflection of human nature that such devices should be necessary to control the abuses of government, but what is government itself but the greatest reflection of all human nature? Here he says government, I would say governance of these systems because government has started carrying a lot of weight, but this is the very famous, "If men were angels, no government would be necessary. If men If angels were to govern men, neither external nor internal controls on government would be necessary. In framing a government which is to be administered by men over men, the great difficulty lies in this.

You must first enable the government to control the governed and in the next place oblige the government to control itself. Trying to figure out what structure we could create that would enable something like this over the most powerful technology that we are likely to have ever witnessed or could have ever created as humans seems very very important. The general approach that I've taken is that there probably are no silver bullet solutions. There are probably a lot of lead bullet solutions. I've looked at um breaking down it would starting from the statement it would be very nice if AI systems followed human instructions and didn't cause catastrophes or risks and broke down what it would look like all the different categories of things one would want to look at and try to mitigate in order to create potential organizations, institutions, or roles or responsibilities that could be very explicit and narrowly scoped enough for a single person to take on.

In a list like this, I don't want to go into it too deeply, but there is a blog post um from Atlas Computing. On the next slide, I have an example of what something might like this might look like. There's a current debate of should it the Department of Defense, war, whatever um decide what AI systems do or should it be the CEOs of these lab companies and I I'm delighted that this is a question and I'm infuriated that no one is shouting very loudly. The answer is no, neither. We should have a system with checks and balances and I think that you could have one that looked roughly like a 501c6 nonprofit that would Oops, sorry.

That was the next slide. This slide is my process of generating one of these solutions is to go through what is breaking um who has the financial incentive to fix it? Is it possible to deploy this fix? Is it useful? Would the stakeholders actually care or would the stakeholders who would have to adopt it care enough to fix it.

I think often times with a lot of the problems that people in places like this talk about, there are like three people in the world, maybe somewhere between like one and five people where if they just decided, "Yeah, this is a real problem. I want to fix this problem." They could in 15 minutes put in motion a series of things that would actually fix the problem. And those people don't attend things like this because they're busy doing the next thing. And so figuring out who those people are and making this very appealing to them.

So they're like, "Oh yeah, this actually solves a problem for me. I'm so glad someone's coming to me with a solution that will take this off of my hands." And then go find the people who will build it and then present all of these to people who would build and fund and support these things. So on the next slide I have a a specific example of what that looks like. I think you should have a 501c6 where all of the frontier labs send some delegates to agree on a subset of the model specification or the AI constitution or whatever you want to call it that all of the labs then adopt.

This then becomes a reasonable target of governance. You could start encoding things in like there should not be an AI singleton that controls all of that take controls all of the economy. We should have something multipolar. You should have uh you could start having something like a bill of rights that gets put into all of the models that all of the experts on model specifications agree is actually a really good encoding of this. And it has the nice property that it's verifiable.

Every time a model gets released, the model spec gets jailbroken out of the model very shortly. Um and so this is an example I would claim of what it looks like to try to engineer one of these solutions and then the question becomes going and getting the buy-in and uh getting the people who could build this thing explaining why it solves this problem. I consistently claim that as intelligence gets cheap, agreement or consensus gets expensive. And so trying to figure out how to do that part of the work much faster is going to be very important. Um but this is what I do.

This is what I hope more of you will do. This is something that I think Funding the Commons meshes with very well. This is a QR code to these slides which includes my email on the bottom here. Um and if you want to specifically talk about funding for Atlas Computing and this field strategy approach or becoming a field strategist or um a bunch of the other things that I'm specifically looking at around hardware verification, formal verification, things like that, I'm also available. And I think I might have a minute for a question.

Thank you.

[applause]

Where is the model spec for human alignment? I mean, we even have had the opportunity to kill ourselves and each other for many, many years, and we haven't. So, at some point, humans are aligned. So, where who holds that spec? And where does that spec live?

And can we learn from how human society has aligned individual human actors to tackle this problem? I feel like the closest thing that humanity has to a model spec would probably be either like the golden rule or the UN Declaration of Human Rights. I think that the So, empirically, we haven't killed ourselves, right? I mean, there are certainly people who have tried. But we have checked it out.

Yeah. Why not My My question is actually was was phrased erratically, but the question is, empirically, we have an example in which individuals are not aligned, and yet, as a whole, society moves on and progresses. So, why not use that analogy of human society to tackle the problem of alignment? Why do we need to go to technology to a model spec which clearly we don't have as human society? I think that

[clears throat]

that assumes a that assumes that AI systems will be more like any other technology than I think it will. I think it is like human minds. I think that the risk of extinction or catastrophic downsides or things like this comes from the fact that humans I I liken the the development of AI to something like the development of tool use in humans or language, and I don't think that among the creatures that were alive when tool use or language emerged, that it was very good for the ones who were not the tool users or the verbal humans who it succeeded well, and there was nothing that prepared there's nothing that could have prepared the other species to it survive the rapid development of humanity and the self-improvement, the technological development, and the kinds of things I would I'd be curious to hear more of what you think it would look like to apply learnings from society and how humans have survived. I think that the as I talk to experts on specific things that AI systems could break what I see are a lot of very specific failure modes where they are all sociotechnical problems, but and I agree you cannot solve a sociotechnical problem with a technical solution, but you also can't solve it with a wholly social solution. And the so the approach that I've taken is to drive the technical solution as quickly as possible to close as much of the gap as possible while matching the interface of the social side as well as possible, because that seems like the fastest path to an effective solution.

All right. Thank you, Evan.

[music]