From Clicks to Agents: How AI Agents Are Changing the Internet | Andrea Muttoni - Story Protocol

[Music] all right hello everyone seems like the energy is still hanging out Saturday at eth Denver for this next talk if you're into the AI agent wave you're not going to want to miss it stick around we've got from clicks to agents how AI agents are changing the internet given to us by Andrea maton president of the story foundation let's hear it yeah amazing all right I'll get right into it because I already lost one minute thanks to this introduction um okay clicks the agents in this talk I not talking about any crypto project at all so no Shilling um I want to really just think about how the internet will change as agents become more and more The Operators and The Navigators of the internet and to do that let's take a step back first and think about clicks right uh which is what humans usually do and the first clicks were Punch Cards my dad started an IBM programming um main frames with Punch Cards and technically when you punch a hole that's kind of like a click uh but then you know terminals came about and clicks started to be around keyboards and then um Xerox also started to introduce the concept of a GUI and uh clicks suddenly started to become on screen and the concept of a mouse started to be introduced and then of course we all know uh that GUI evolved both on the Apple side on the Windows side uh into basically a um the way that we interact with technology today and um that's great but so far the internet was made for clicks right for humans and if we look at the internet and how it used to be especially you can see that things like buttons things like input Fields things like links were all built around links built around clicks um this is for example a Google site in the late '90s this is Believe It or Not how Netflix looked uh in the late 90s as well and things got a little bit out of hand when uh yeah uh that this is probably one of the darkest periods of the web um but things got better and now we have beautiful designs we have beautiful uis uh great best practices there's a whole industry of UI design ux design um on our team we have amazing designers and and this is amazing uh but this is still all built for humans right and if you think about it some of these components and we'll look at them in a bit more detail in just a second some of these components could potentially be a little confusing for nonhumans like iconography that is not clearly labeled or buttons that are unnecessarily animated or hidden calendar Pickers which are the bane of uh you know accessibility and the whole web right now is built around optimizing it for humans this is Google's page speed insights the score that you get when you analyze your website is is how good is it for a human so for example the first contentful paint right how quickly does something load um what's the first input delay um and and uh little consideration at least today is given to agents but AI agents today are starting to use the web too not just scrap it right to train their models but actually use it you can start to delegate actions to users uh to agents so this is an example of um this is using a library called browser use which is great if you haven't tried it and you can basically unleash an agent on a live website or multiple live websites and it can use a browser and it can navigate and you see this is like a representation of what the agen SE it highlights all the elements it tries to figure out what to click what to input how to navigate the website and right now it's kind of like forcing agents into a web made for humans because they have to adapt to the way uis have been created for us but that's probably going to change in the future and this is what I want to talk about let's talk about what elements of the web right now could be problematic for agents the first one is as I kind of mentioned earlier difficult user interfaces things like navigating this to us it seems so obvious like oh flights cars packages things to do cruises but for an agent all of that content is hidden unnecessarily right now it's just on stays so if I want to if an agent visits this website and uh wants to book a car it could get confused and think that the input fields are related to booking a car date Pickers as I mentioned earlier are especially confusing even for humans sometimes like the one on the right is absolute torture the one in the middle I'm not even going to get into and this one is kind of okay unless you have to change the year and then it starts to get complicated but you see even the one on the left has extra buttons and functions on the bottom um that will probably confuse agents that are just visually trying to navigate the site also things like custom selects and dropdowns uh Airbnb has created like a custom dropdown custom select with buttons where you can specify but it only shows if you hover over the guests um modals or modals are uh especially annoying to humans but also to agents because they basically cover content that would otherwise uh need to be accessed and it it prevents further action from being taken unless you take the actions that that the modal asks you to um carousels carousels hide UI unnecessarily right the agent will need to know that it has to click the arrows and wait for the carousel to finish and then browse the UI also um asynchronous content so being able or having to load additional content is especially um is is especially inaccessible because not all of the information is right away and context dialogues for us it's natural to rightclick and have more context uh but for agents they don't even you know need the concept of a click necessarily so you all of this information all of this context should not be hidden away for agents in order to for them to immediately take action the second let's say dark pattern for agents is hiding content on sites for for example uh interstitials like in the New York Times you can skip to content as you see in the top left for example I'm based in Europe and what you see on the right is what I is what I have to live with every single day every single website that I visit um and it's damn annoying because no one really cares about cookies anyways uh but things like um you know login popups they're all hiding very precious information behind them that an agent could use to to take action and especially in modern times there is a trend towards hiding information hiding text hiding um basically descriptive content right we see this both in uis but we also see this in advertising advertising is basically pictures of people doing stuff and a little logo of the brand somewhere um and and so an agent that visits the site and you've asked it to shop for something it it you know what's it going to choose from these three options that they both two of them show shoes and one of them shows a person kneeling down with a hat and another trend is collapsing content because we are simple you know simple creatures we can't take a lot of information at once and so designers have correctly decided that we shouldn't have too much information in one go we should have only the uh you know minimum necessary information needed to get something done and so things like collapsing content things like sidebars are kind of gatekeeping access to agents that could absorb everything at once thirdly I mean I know that humans hate this as well but this is this has a like an even more uh potentially dangerous um outcome for agent is advertisements and deceptive content so if you look at this website everything that's read has nothing to do with the website right it's basically F 10% of screen space is dedicated to the actual content and so if if a website if if if an agent like the one that you saw in the video earlier were to load this website and try to take action they would completely go off base right away also deceptive practice practices like when you book a a flight and the airline conveniently automatically adds a seat for you which is not free right um an agent would have to look out to deselect the seat otherwise you would probably not delegate this action to an agent because you would be spending more money or rhion Aire um that automatically opts you into to Insurance unless you go through a massive drop down and select in the middle of all the op options do not insure me and the insurance is obviously has a cost so if agents will be the main Navigators of the web how will the internet change right and this is where we have to start going from user experience to agent experience so ux to ax we had ux we had DX developer experience and now we have to start thinking about ax agent experience and in in in in a lot of ways experience really is synonymous with accessibility for example as I mentioned earlier humans are you know we tend to hide elements as much as possible we designers tend to make things look really cool for them to be uring and to give a you know the brand A vibe and eye candy like Transitions and glowing backgrounds on buttons and things things that are completely unnecessary but enhance the user experience for agents the best way to the best way to design websites is to show as many actions or intents as possible and a great example is Craigslist right you go on Craigslist and you can basically access more or less everything right from the homepage because everything is exposed um agents also don't really care about marketing or fancy images they want rational information they they have a task and they will execute on that task based on the instructions that they have and they will make choices based on the facts and they also are trained on you know the whole Corpus of the internet so they can fact check instantly so the website needs to show rational information of why you should use this website and and really I candy is completely unnecessary so as I said we're we're going from SEO right which is how do I rank as high as possible on Google on uh bing whoever uses it and two how can I make my an agent choose my website choose my Service choose my uh tool and agents have autonomy of choice so when you delegate an action to an agent an agent will go and probably evaluate a couple options and choose the right option based on you know how best it fits the requirements of the user so tools and platforms that are poorly suited for llms or that have complex UI are quickly going to fall behind and open ecosystems will likely outperform uh closed vertical Integrations as well right the agents will need to be able to lean on each other and so being able to have open ecosystem is going to be very very important and going back to Google page speed insights which I talked to you about um at the beginning of the presentation right which is a way that Google measures the human score of a website I think there's a missing piece there which I've added which is Agent task completion speed because when you start delegating actions to agents you are entrusting them to carry out things for you and the most important kpi for that is how quickly and how effectively can an agent do that right and so my prediction is in the future we will definitely start to have agent related uh metrics on things like Google page speed insights um that will probably be used as signals for agents on what website to go to and also I think it's really going to change advertising as well I mentioned earlier you know uh we as the marketing industry has evolved in the you know in the ' 50s it used to be all about information it used to be about describing the facts of a product and um recently in the last 50 years it's gone to be much more about emotion and Vibes and feeling and you know connecting emotionally with the potential consumer so on the left we have a you know a classic example of an advertising today and on the the right you see how much text that old advertisement has because it wasn't trying to sell you a lifestyle it was trying to literally inform you of all the specific details of why you should choose in this case Herz and I think we're going to see a trend back towards very descriptive advertisement because if humans are not the ones browsing then agents will need as much information as possible and in my hot take is that AI agents will do more for accessibility human accessibility than anything else to date right and this is a tweet that I was you know I have to admit I was a little late in preparing this presentation and so if you see the timestamp on this tweet um it was because I looked I looked it up a couple hours ago um I found this tweet randomly and I think it summarizes perfectly what I've been trying to convey so far which is website that have not changed in the last 15 years are the most agent friendly so in a way and ironically the new Modern Wave of the internet which is all agentic is actually going to trigger a um a you know an evolution backwards towards simpler websites simpler uis and simpler and more informative information rather than visually stimulating information another thing that is important to to to consider and I I don't think we've actually tapped into this yet but I just want to Inception it in your minds is agents are Dynamic humans adapt very slowly to things whereas agents don't have that burden so humans hate it when anything changes right if a website changes a single button people go crazy um and because we tend to be habit ual animals we like to do the same things we like we like routines um and imagine if a UI that you had to use every day completely changed every single day agents don't have that limitation potentially potentially um uh I don't know if the audio is working here but it's fine if it doesn't um in in this example two agents are are conversing I don't know if he but um they decided to switch to a more effective language so not only the UI they're on the- fly agreeing on a new language which is made up of beeps which is 80% more efficient than natural language so if on the Fly they can adapt to a new language think about how on the Fly they can adapt and potentially generate uis on the Fly because it's not just going to be agents interacting with human websites it's going to be agents interacting with agents so every website every service will have its own agent and so if depending on the exact need that the visiting agent will have right let's say I delegate it to do something and it has to visit a website to do that it can it can ask and say hey I need to do exactly these things and the websites agent or the services agent can dynamically generate uh generate an ad hoc UI specifically for that intent specifically for that need and we're already kind of doing it as developers using cursor or using v.d where we're you know asking with a prompt to generate a you know like a input field or elements of website agents to agents will do this on the Fly for specific tasks and what's going to like turbocharge this is humans have to do things serly agents can do this in parallel so they can visit you know a task like this which is to book like a holiday everything that this poor individual will have to think about can be delegated entirely to a collection of different Agents from a collection of different services and all of this will happen in parallel which will essentially trigger a ton of transactions and this is where I think crypto is the best product Market fit for agents don't need to have identity they can get an away and transact immediately uh you know plenty of work in the last years have been done on the infra side trying to uh make things as fast and as scalable as possible so I won't even get into that so I really think that crypto is going to be the best transacting layer for agents and lastly I'm out of time but I'm also out of slides agents will beat your new users so it's not going to happen overnight uh people still be able to visit the internet of course but um next time you build something think about how an AI agent is going to experience it uh think about how it's going to use it try to play around with tools like browser use which is um library and try to get an agent to visit that website that service that tool and use it and see where it fails see where it gets confused because the whole internet will change accordingly o o over time and that's it thank you