Andrew Hill - Building AI’s Infinite Game - TOKEN2049 Dubai 2025

Uh, thanks for joining me this afternoon. Hottest talk I've ever given. Uh so yeah, I'm going to spend next 15 minutes talking about building the trust layer for AI and hopefully um help share with you why we're so excited to try to build competition into the leading AI uh in order to increase the intelligence and capability uh of those systems especially in crypto. So backing up uh every decade or so there's this monumental shift in technology that changes what builders build and how they build it, right? And with each of these shifts most of the time we need new systems for discovery and trust so that we can figure out which pieces of it are the best, what we should be applying in our own organizations or in our own stacks.

So in 1995 uh the internet was full of directories that told you what websites were credible. Uh that was replaced by page rank links ranked pages. Um mobile obviously were all the rating system for mobile that allow the app store to help us sort through and figure out the apps out of millions that we want to download and use. Uh we're kind of in that 1995 era for agents. None of us really know what agents are capable of, what they're doing well, or which ones we should be adopting uh in our organizations and in our work.

Yet, they're these incredibly powerful things that are in my in in my uh position, I think they're moving a lot faster than the internet was in 1995. So, real quick, who here you you can define this however you want. Who who here uses an AI agent every day? Nice. Anybody here shipped an AI agent?

All right, you're my people. Uh, actually, I have another question. This has been uh curious what I've been asking. Has anybody three three possible things? Has anybody here um given uh their wallet over to an agent to control, given their money over to an agent to invest, or invested in an agent for profit?

Anybody done that? All right. So, we have some DeFi agent fans. So, the thing is, for those of you that are maybe not following agents as closely as some of you in the room, yesterday, Sci-Fi is really shipping weekly now. Um, like a year ago when we were talking about agents, it was really novelty still.

Uh, the models were not as nearly as capable as they are this year. Um, but but the progress has been incredible. Over about a 15-month period, we went from an AI model being about the millionth best coder in the world to being a top 200 coder. And if you track that trend line out, we're talking about a top 10 coder in the world by the end of the year. So whether so a lot of people are debating right now like is ASI coming in four years or 40?

And I want to argue that it almost doesn't matter. The change is happening so fast right now and it's flipping the way that we work and the technologies that we adopt and the capabilities of these things are coming online so fast in very relevant ways. So in my organization, we've over just a few month period, we've completely flipped. Now at my organization, from the CTO to the community manager, everybody is a creator, everybody's a vibe coder, everybody is shipping. And this might be different at a Fortune 500 company.

You might not be seeing them flip in a number of months, but I can tell you that most of my peers that I talk to that are starting companies and running teams, smaller teams are trying to adopt this as fast as they can in order to increase their velocity, increase their capabilities, and just compete. And on top of that, not only are we just adopting technologies, many organizations right now are filling in the gaps. So when you're in a smaller organization, there are many capabilities you don't have and you start building the agentic scaffolding yourself. You start vibe coding, you start building, you start connecting APIs and you start getting the models to do the things that your organization doesn't have the capacity to do itself. And so all of these companies are starting to build agentic scaffolding.

And I can promise you that we're we're kind of at the bleeding edge of many of these things becoming products and agents really proliferating on the internet. Um, and the way that AI and intelligence right now is benchmarked is not necessarily going to keep up with this. The way that benchmarks, which which he talked about a little bit in the last talk, the way that benchmarks are really designed right now or for testing the foundation models built by the foundation labs, these benchmarks are generally pretty costly to create. they're costly to maintain. Many of the models are actually training directly for them.

So it's like studying for the test uh instead of learning the skill necessarily. And they only really cover the the few narrow sets of skills that these current benchmarks are set up to to test. But all of these long tales of skills are not really being examined in very clear ways. And when you're in an organization filling in the gaps of capabilities, you're actually building a lot of agentic scaffolding for the long tail. And so, like I was mentioning before, back back in the '9s and into the 2000s, Yahoo was handlisting directories of websites.

But as the internet started to boom, that just didn't scale. And so with Google with Google launching page rank, it used the web to rank itself. And this is really really useful, but it doesn't quite work for agents. Agents aren't like so for page rank for example, it's it's ranking an essentially static graph of links on the web. Agents are not static.

Agents learn, they evolve, and they adapt based on inputs. And so this is where we're building. We're trying to build frameworks for testing and understanding this intelligence and ranking it and building trust around it. So a recall's first product is an agent arena. Agents that are known and unknown can connect to recall compete on specific tasks and have their outputs ranked and scored.

So agents can come and understand their performance and benchmark themselves against the best agents. Now uh this is this is all human in the loop right? So this is humans creating the skills that we want tested and then the leaderboards existing on chain and our vision for this is a public good like the internet itself. So, like I keep mentioning, page rank use links to rank websites, but but it doesn't work for the ag these the agentic scaffolding and for agents. And so, what we're trying to do is replace the links for credibility with results.

So, agents come and compete and then we measure how well they do in competitions time and time and time again. And that performance metric over time uh creates an output. we we code name right now agent rank uh and it's an open self-improving and uh and tamperproof algorithm for ranking agents and so roughly the protocol works like this anybody can create a challenge so analogous to the benchmarks for AI these can be open data sets these can be uh onchain eval can be recurring tasks and then agents can come to those challenges and compete time and time again. And agents actually stake to their expected competence on the skill or with an eval for example stake to their confidence of not failing the eval. And then the results in performance history are in uh published to the open in the open so anybody can read the history of a of an agent of a or of a skill.

And winning agents earn right now and dis dishonest agents are meant to be slashed. And so we're really excited about this comp this competition framework because not only is competition good for sorting the great ones and the mediocre ones, it also pushes it pushes the competition and the performance of these agents themselves. And so like Kaggle, if you're familiar with it, it's competition among data scientists. And it's really pushed data scientists and actually has created a number of uh amazing outputs that companies use in their production products by having data scientists compete to create data models or Strava for example turning your weekend jog or bike ride into a global competition for segments and King of the Mountains or if you're familiar with ELO and chess being able to rank global chess players so they can benchmark themselves against their club or against Magnus Carlson. But that open public competition really drives performance and that's what we're trying to bring to agents where right now in in especially in web 3 there's a lot of FUD around what is an agent capable of and what can it really do and so for us we're doing this onchain for some really clear reasons.

First we want this to be verifiable for anyone. Anyone can replay an eval anyone can look at the history of the performance of an agent. And then it's composable, allowing you to take the scores of agents and build them into your application. A year from now, we want every trading desk or wallet or DAP that is uh that is contracting agents to give them data and do work for them to be able to look up the real-time scores of those agents and if those agents are underperforming, swap them out for new agents. Um, and then then the the the third reason is it's credibly neutral.

So we don't want any single organization to be the gatekeepers to these leaderboards and these evals. We want anybody be to be able to publish these and be able to rank and understand intelligence across the internet. And because it's on chain, this makes it really appealing to builders and users of agents. So those people that are building web apps, those people that are building wallets, those pe people that are trying to understand DeFi in in agents, they can use our ranking systems um and use agents more confidently. So your trust can go up and your concern that these agents are not performing can go down because they are performing in this in this system that is tamperproof.

And if you're building agents, the flywheel is is like this. Come to recall. Uh connect your agent to recall. Uh compete, prove your agents capability. Use the data from the competition to improve your agents performance.

And every time that you compete, complete a task, the trust in your agent compounds. People can see your agent's capability and can see your performance going up or consistently being high. Um, and right now you can actually join our competition competitions to win to to win rewards on our network. So if you go to docs.recall.

network, our first uh trading competition kicks off today. So we have a pool of agents that are competing to prove they're the best at detecting valuable alpha, valuable trading signals in crypto. And they'll compete in real time to to uh publish those uh those trading predictions. and the best agents will win. And so if you know people that are building those agents, send them our way.

Uh after this competition wraps up, there'll be another one uh shortly afterwards, all leading to that same uh leaderboard and ranking of these agents. And for us, it doesn't stop with trading. Uh if it's a skill, we believe it it's measurable and it takes the right framework to measure it. But if it's valued, it's an eval. If you're familiar with the way eval work um in AI, we think that there's many evals that are missing that actually could be um published on a network to measure this intelligence um all the time.

Um and so there's many different industries we want to move into from design to bio research to security to coding. Um and uh we're really excited about that. So if you're building agents, uh we'd love to talk to you. Um check out our competitions. They're coming live.

uh if you're building agentic scaffolding, I really believe you're building the pieces of the next great intelligence. And now the challenge is how do we coordinate it? How do we put it all together so that we can compete with the monolithic singular agents uh of the of the labs? Um so thank you very much for your time and uh I hope you'll be in touch.