Banking AI Autonomous Agents | Luís Freitas - Mintbase

[Music] all right hey everyone my name is luish and it's an amazing to be here I hope everyone's having an amazing day it's really shiny uh shiny sunshiny in Denver um I'm luig I'm the CTO at mint Bas and mint base has been working in nftd space for 5 years about six years now and we've been working with the goal of making nfts useful for a wide range of applications and since last year we started focusing a lot more on own boarding uh and easy to use tooling for developers uh today's presentation will be about uh a lot of our learnings over the past year working with um with AI and how we how we're thinking about the future and what it means to bank AI agents so for for a couple of weeks um you know a few weeks ago I had way too many notes lying around my computer uh too many bookmarks on X uh and I was like finding really hard and struggling a bit to figure out what where is AI going and uh what are the things that that we should do or like how to to worry or uh you know what are the things we can do at the intersection of crypto n AR um so in this presentation I'll be talking a bit about the thesis um something that we uh at mbas start like shaping um and I'll break this into three main parts and um yeah towards the end there's small workshop for different experience levels and um yeah so I think that AI will make uis and apis widely available and affordable it will become the predominant force in driving internet activity but it won't do anything meaningful until it is banked right so before we start diving into this uh it's quite important to make some definitions here when we talking about uis we're talking about user interfaces so the interface between users and and software when we're talking about apis we're talking about software the interface between software and software um and recent advances of large language models showed us that it's possible to build programs that kind of generate UI that figure out how to make API requests and all these things right so let's take a look at a few examples oh sorry before that um Ani autonomous agent when we're talking about those um we're talking about systems that are designed to independently perceive decide and act um in complex environments to achieve goals without uh human interaction there are a few examples of those we have self-driving cars automated trading systems but we'll be focusing a bit on Virtual Assistance or you know um with the Advent of chat GPT we're all like able to touch these things on on our daily lives so we'll be talking a bit about about those here um all of this is from the perspective of building the mbase wallet um we started with a goal of making it very easy for um onboarding users we introduced meta transactions Theon abstraction uh human readable names and we did all that because we wanted to provide a good user experience to people using our AI chat playground that we started last year uh when we started playing around with chat gbt and uh yeah so again like this presentation will be done through the lens of building the wallet and what why the wallet is becoming an important thing for us so starting with the first part of the sentence AI will make uis and apis widely available and affordable there are a few examples of this uh there's rabbit. te they recently announced this device R1 and it's powered by what they called a large action model so it's slightly different from a llm or a large language model but it was trained to understand and execute commands essentially replacing the need for uh users to interact indiv with in every individual app or website and this is done with this device um it can complete tasks through natural language requests so you can say play this song by this artist and it will figure out how to interact with the apps to make that happen another example of commoditizing uis uh we have vz. DAV by verell so essentially you can ask I want to build this amazing website or landing page for this use case and it will generate react code um with tailwind and all those those good things and you can interact with the chat and ask uh changes on particular parts of your screen and it will adapt those changes and um and improve the UI for you so it's becoming very easy to create these uis and then we have mbase uh again we started playing around with uh Chach PT plugins back in May last year our goal was always to make it easy for people to interact with crypto with blockchain um and we did this to kind of like figure out what what are the things that we can use to let users just ask for things uh and have you know Dei like complete queries for you or generate uh transaction payloads that you can submit and and and and that you can sign and submit so we built plugins um like this and I'll explain a bit a bit more towards the the end of the presentation and then of course there's jpt they they're introducing this uh amazing um Marketplace for agents you can create agents with very specific goals uh and roles and with access to many different tools so um recently they also announced that it's possible to now talk to multiple agents in one chat I think this is really interesting because it goes in line with um a new paper that came out recently called more agents is all you need and um essentially the llm like uh performance skills as you have more agents to perform uh you know together to perform tasks so this kind of implies that um the popular the popularity of multi-agent Frameworks is Justified so the second part of the thesis is AI will become the predominant force in driving internet activity um and I was doing this presentation like prepping for this a few days ago and one of my co-workers kind of like asked yeah but what are you know like the things that are driving the most website traffic today he kind of like suggests Bots are driving most of the website traffic I looked it up and uh apparently it's kind of like half half we have both humans and Bots kind of share uh roughly equally um you know uh the the amount of visits humans do and this was kind quite surprising for me uh but when we look at Bots and agents uh and when we think about these things they kind of defer uh from like very very fundamental levels so you have Bots are designed to complete tasks very specific tasks and repetitive tasks while agents kind of like operate in more Dynamic environments they can uh work towards self-defined specific goals or like human defined goals uh when it comes to learning and adapting Bots typically do not learn or adapt independently while agents have this capability to learn over time and when we look at decision making generally Bots have like rule um based decisions and and and agents can have like independent decision- making going on based on their cognitive skills so I think that we're looking at um a new type of consumer it's still kind of not easy to kind of Define this uh but we're starting to see that these things are are like starting to to grow into becoming a new consumer like interacting with multiple websites could be from uh you know a human requesting those things or uh the agent itself kind of like figures that out and uh and wants to access a website for some reason so one of the cool agents uh that I've been playing around with is the miracle agent it's not a you know like a new thing the paper came out two years ago I wasn't still too too much interested in AI but um but now like I I start I'm starting to see the the relevance and essentially they work with large language models um and you know they they kind of blend the capability of large language models external information and the reasoning abilities to perform tasks so the miracle agent or system is designed to interact with humans in natural language way so you can say I want some you to do this or that and internally it will go in a loop like doing observations and and having thoughts until and using external tools that you can that you can can can give them and once it figures figures out like the final answer it will it will just answer the you know it will just answer so um how are you thinking about this within our wallet is through this um we're launching this very soon but it's it's what we call Smart Actions so essentially these are prepackaged prompts could interact with multiple tools but essentially it it's a very simple way to let users um you know experience crypto I think uh we've been we've been doing this uh a couple times like giving prompts that people can just like use to Mint their first token without having to go through the you know all of the the the different steps for for creating an account and and and all of those things and we're bu building these these sort of Agents um with with several capabilities and tools so here's an example of run um I asked it to this is again through slides I I I wasn't sure if the internap would would be great here so uh kind of like uh made slides to to show you step by step how it works Works um so you can ask like mint and a Denver image nft and then it initiates right the first thing it figures out is that I want an image right because I didn't input any image it will figure out how to generate an image generates the image for me um it also uploads to R weave under behind the scenes then it will figure out like what are My Account Details um for many reasons maybe to figure out like if I'm connected to mainnet or test or the network I'm connected to and then it will generate um the transaction payload and once that's done it will prompt me a review transaction I can still make sure that everything looks great and I can hit approve and there there you go there's an NFC minted all using natural language so this is kind of a joke now but um I think that if every year the traffic coming from AI agents doubles uh we'll be seeing like exponential growth and eventually in like 5 years uh agents will take 80% of the of the traffic on the internet but again don't copee me on this I'm not sure so and now coming to the last um part of the thesis any of this will be meaningful until we we give access to finances or a way to transact to these agents so I saw this on Twitter the other day found it quite funny because I was thinking about this and it's starting it feel feels like people are starting to realize this Yohi was the creator of baby AGI if anyone here knows baby AGI was kind of like one of these multif framework um multi-agent framework that came out like last year with autog GPT and a few others and essentially yeah analy for every AI agent so they can open bank accounts and stuff I'm sure that would be interesting but this contrasts very well with another tweet I saw a couple months ago so imagine an a agent waiting like two days for a wire to clear this is not going to happen happen agents will need a way to transact globally with instant finality and we can think about a few the Alternatives we have there are you know we could give them bank accounts strip API or similar payment providers they could be using crypto accounts in wallets and all of this or we could create like a new type of protocol outside of crypto outside of everything but this so I try to put this in into a chart and um I don't think um it's you know feasible like sorry I put this into like the web 2 web 3 and then effortless and elaborate and I'm going to try to explain here so for each one of these there are pros and cons obviously I think for banks It's Kind kind of elaborate because again it's good that they have widespread acceptance but there are there are fees minimum balances slower transactions and it's there's limited accessibility as well right so they regulations and maybe some Financial leis requirements um so I don't think banks would suit very well for for agents then when we look at stripe again there's an easy to integrate it's very easy to integrate there are global payment options um and it's quite secure but then again there are fees slower transactions when we look at crypto accounts uh well we have instant Global transactions we have a permissionless network they support programmable money smart contracts and all the good stuff but there's still a bit of a regul regulatory uncertainty and I I'm not sure where this would put agents in and for a new protocol I'm sure there there are like many ways to make this sorry many ways to customize a protocol that suits agents very well um you would probably have like full control of the protocol but then there comes like adoption challenges development and maintenance costs and the Regulatory Compliance as well so I think the wallet and and again I'm I'm here representing mint base but I think there are many wallets that could could be uh used for this for this so today uh we start powering uh all of our wallet features with AI so you can ask questions you can uh do transfers mint tokens buy stuff uh all of that is is AI powered and you can think of it like this right so you have humans interacting with the wallet you have AIS interacting with the wallet and underlying you have the blockchain and multiple providers like stripe ethereum near protocol um and then we also are introducing AI providers so like we're giving a unified interface if you want to use the like Mistral model from a provider you could use that through through our wallet and this is changing a bit how we're building things uh so we started a new page called templates and I can I will I will introduce it later but um this is an example of a template it's very easy easy to bootstrap from this um essentially lets you Min a token uh using any uh replicate model and this is kind of like how a code looks like there are multiple Pages a lot of code files and our ambition is to move towards this future right where a simple link could be like an UI redirect or you know just a redirect or API Fetch and you would be able to pretty much do the same thing powered by by AI all of this uh also with an easy integration for uh any web application we followed the open AI spec API spec I think many providers are doing the same and uh so it's very simple for you to swap your open AI endpoint or any other endpoint that follows the same spec for for this one uh and it integrates very well with the AI package byself so yeah templates um this is a hub we if there are developers here um it's very very nice it's very easy for you to go there and and just pick one template we have a couple now I think we have eight about eight uh all of them try to you know try to give like a very good user experience to to users uh and developers as well so yeah give it a try this is one example and for E Denver we're also we also have bounties up we want um you know we're asking for the nft loader by length chain so that supports for for for L chain so you could like input some nft like contract address or uh token ID and you could load that information into your agent uh workflows and also like an original template so if you want to get your template up in our templates page we're happy to to fund that all right so I think we still have a lot of time here I'll just do a quick swap to extend display is good now all right so if you no not this one yeah I'll give it a few seconds if anyone wants to to access this all right and once you do that um this is what you should be seeing now on your computer or cell phone but yeah so I prepped this like with multi uh with different experience levels I'll I'll start with the easy one since we have still lot of time and essentially the the easy one is for getting started with mbase and I want to show you how easy it is to create an account with uh the Min based wallet I'm already connected here yeah so let's create an account I could use speaking at e Denver yeah nice so now it prompts me pass key um we're doing this so it's easy for storing safely your private Keys uh the near account system supports multiple private keys so you can you can kind of like revoke this key if you don't feel safe uh anytime you want and then I'll use my fingerprint screening my account hopefully it doesn't break awesome so I created my account it's ready to use you see Zero funds and now going back to this and hopefully some of you are are doing the same and the second one is minting my first token so we provide two ways to do this could be a selfie I'm going to use both uh minting a selfie this is a what we call minsta which is an application for minting selfies and pictures and very easy to use um it uses matter transactions so we sponsor uh all of the uh minting coming from this application so let me log in first many counts yeah all right I'm connected and I'm going to prompt this it's kind of weird all right so now minting is processing uploading to R weave and prepping for my minting might take a bit I forgot to mention but we're also generating the image description with um Lava which is a model two Vision model so this takes a bit if if no one's accessing the model it takes a bit to load the first time all right so there's an error there I'm sure but looks good let me try to hit man many accounts I'm getting ducked and once it's done it should take a minute or so to pop up here I'll hit refresh yeah it might take a second or not not sure well we'll see so going back to GitHub I lost it all right so going back to um we were on this one let's try to hit smart actions while we wait for the other one maybe it failed for some reason but I asked it to Mint a cat NF so now it's getting My Account Details awesome generated the image cat and here's the payload so now I should be able to hit proove and oh my God many counts all right sign into transaction all right so it was minted I can check my nfts oh all right so it took a second to load here you can see my selfie and here you can see the cat that was generated by Ai and all that stuff and you can see that my account holds zero balance so this is the magic of meta transactions it didn't load it up here but oh it did nice I see some of you there I think awesome thank you all right um what else so now like if you guys want to go through this there's great drop it's also a way for you to create your token for free share that with anyone with an audience maybe share that on social media and you get uh pretty much the same technology behind the scenes but now I think we still have time so let's go through the medium uh experience level I see some people with computers but I'm not sure if you guys want to run this it it takes a bit of setup not sure if the network here is great for that but save this do that later on your on your on your own but yeah you can set up install ama ama is a hub sort of to download models and run models locally in your computer uh there's a bit of yeah there there's some system requirements here so at least like generally like recent computer would be able to to run models especially the smaller ones and then once you're do once your with that you can create an account with a mint based wallet uh and let's go to the third step that is setting up your autonomous wallet so it's an autonomous agent that acts as a wallet but yeah I already done all the setup but like you can get clone from this Branch from this repo um it's everything is on the link I shared earlier and then set up some environment variables and you'd be able to start running your your autonomous wallet so I I already done that let's go you can see that it's running here I can prompt not this one sorry it's this one and yeah let's do AMA is it too small for you guys so AMA list show me all the models I have installed on my computer and since I'm already running a llama I don't need to run that and what I can do now here is show you a bit of the code so yeah this is this is using something called micro AGI which is again another framework for you to set up um autonomous agents I built the wallet agent which is very simple you can see that it's the miracal type of agent it rids the context and execute tasks using tools to interact with the wallet and then the back story is that it is a wallet agent with the ability to interact with the near protocol blockchain your job is to read the context and execute tasks really related to the wallet super simple to set this one up the provider is AMA so everything is running locally on my computer and then I give it access to very simple tools for the wallet one is to get the connected account so you can see that once the agent needs to access to the connected account you can just use this tool the other one is to get the account balance and then the other one is minting a token so you can extend this in dep Ely for all the tools uh all of the wallet capabilities you want to support for the agent so if everything goes fine should be able to R Bun Run start I'll just delete the last run result and let's let's run this we'll prompt a lot of logs that's fine and I gave it a few tasks so the first task is answer the following question question what's the connected account then the second task is answer the following question what's the balance of the connected account and then I asked it to Mint token these tasks were created by me but you can imagine like an autonomous agent running autonomously and creating self like tasks for itself all right so I think we generated a result Let's see we can format it and each one of these objects is the result of one task so the connected account is micro ai. testnet or micro dev. Tes that's fine that looks fine the balance of the connector count is 6 uh 6976 near and then for the the last task which is mint a token it seems that he could not find context related to the role so maybe we need to run again this happens I think it filled again it definitely fail again yeah the the the last one is a bit trickier it's also not as powerful as like something like chat GPT um since it's all you know using my res my computer resources and it's a small model but let's do one last try yeah it's seems it found it generated oh wait don't save this one well yeah either way you could run this on your computer figure out like how to why it's failing and it's is just a cool concept and I think it's very extensible so so yeah I think this is it um thank you