Public Good Integration with web3 AI - ETHGlobal Singapore 2024

hello I'm Harish I'm a developer Advocate at Gia and this is your guide to getting started with Gia before we learn of what G is I'm going to take you through a quick Journey or a recap of what AI is what web 3 is and how Gia has to leverage both these and build AI agents so let's get started here is what we are going to cover today I'm going to cover why AI in web3 then we are going to look at a quick introduction to GAA and also what Gaia offers as a developer platform then we are also going to look at what to build with G what are some examples that you can use to build with GAA and at the end we want to see some quick demos of how you can launch your own AI agents using Gaia and then customize them and also some ideas that you can use for all the Gia hackathons going forward so let's jump right into it what is web3 web3 is decentralized web aiming to create more open secure and user Centric online experiences right web3 incorporates cryptocurrencies tokens for transactions governance and also incentivizing Network participants we all know this right data today is being taken from us right and AI is controlled by very small set of people which essentially is the problem ml needs to be monetized not us all the data right here are in detail some of the concerns of today one is censorship and bias with centralized a models they often reflect on the biases of their creators right and lead to unfair and inaccurate outcomes at the same time controll and governance centralized control essentially means that people make decision that affect every body who uses them right for example if an entity launches something new a new model it will behave like how the entity want it to be but not like how you as a user wanted to be then there is privacy and IP issues of course where we lose control the moment we start interacting with an AA model or provide it with data then there are of course operational issues where centralized infrastructure can be a single point of failure right if they go down everything we built on top of them also goes now or everything we built or created using their services will also go down and the last one is of course it is not cheap to run uh fine tuned lbs these are some of the primary concerns as of today right so why Ai and web 3 Let's get right into it decentralized AI what is decentralized AI in decentralized AI a models and data are distributed across a network and this reduces centralized control and single point of failure second one is enhanced data privacy which essentially means we three by default offers cryptographic protocols that protect user data while still allowing as systms to access them third one is improved AI transparency with blockchain obviously there is auditable trends for AI decision making then tokenized AI services and autonomous AI agents with tokenized AA Services AA capabilities can be monetized and access through token based systems potentially democratizing access for users then there is autonomous a agents which G lets you build today I'm going to show you how to build an a agent but autonomous a agents with web3 infrastructure could enable AA systems to operate more independently interacting with smart contracts and managing resources in an automated Manner and the last one is of course connecting Ai and web3 with a efficient decentralized Marketplace for data and a models these are some high level use cases of how web 3 and ai go together let's go to next the next one is an intro to Gaia what is Gaia Gaia is a decentralized Computing infrastructure platform that lets anyone users like you me to create Deo scale and monetize their own AI agents that reflect their style values knowledge base and also expertise with that high level overview let's see what GA has to offer right Gaia offers support for using any open source a model you can bring in fine-tuned llms or any custom model into your a agent there are opportunities to create AA agents that generate Revenue through API and of course decentralized and economic model using all these Gia transforms decentralized AI into a network of living knowledge integrating data knowledge and compute right here are some of the key features like I mentioned decentralized AI infrastructure right that enables individuals like given me to create and deploy AI agents and the node architecture specifically provides high performance run typ find your LMS and knowledge model with a default Vector database a PR manager an open API server and also a plug-in system for external tools you can use tool calling to call external tools Gia domains is a collection of G nodes organized to offer structured and reliable a agent services to the public while load balancing and incentivizing or node Builders web three integration in Gaia essentially utilizes blockchain technology for payments governance and also creation of tradable assets like nfts from Individual knowledge bases and components these are some of the key features of what Gia is here is a quick overview of Gia node's architecture in this the node operator is the one running the node who is bringing the proprietary knowledge base and also maintaining the node you can see each node has a node ID and an API server that lets you do functional calling tool usage customized llms has a vector database and multim mod Ting models that help eventually create the a agent right and of course through Marketplace is where the opportunities are you can create applications on top of your modes or also build applications and algorithms for optimizing Rag and also create custom workflows do dual calling to build tools to perform external tasks using nodes knowledge base when you launch a node on GAA it comes with a high performance run tank which is wasum based you can bring in find llms by default you get F 3 llm and also mamic embed as the embedding model along with that each node also has a vector DV and prompt manager where you can customize the prompts and also if you prefer using any other Vector database you can bring those in by default Gia offers quadrant as its Vector database each node when launched also comes with a open a compatible API server which works as a dropin replacement for any of your existing applications or also tools like vs code using Code gbt or cursor you can use your nodes to power your experience if you are a developer or you can also use the nodes in other tools which I'm just going to mention in the next slide along with this each node is also integrated with domains so that the nodes are structured into domains and there is better governance at the end last but not Le is there is plug-in system which allows each node to do tool calling for extel Tool Integrations these are some of the primary things that you should be aware of so you can get some ideas of what you can build with G all right so quick summary of what G is and what you can do like I said Gaia is a open source developable platform you can use pyot or Lama CPP to F tune your llm at the same time you can use Lang chain or llama index that can create a vector collection or the knowledge base that you want to bring to your LM and put these two together in the node so that Gia can run the node service for you right here are some tools that we already integrated with I will show you just in a minute where you can find the documentation for all of these tools and how to use these tools with a Gia node but this is a quick overview of what Gaia offers with all of this context in mind let's get into understanding some quick terminology that I'm going to use for the demo and also let's get into demos all right while the diagram looks a little complex here is simple English explanation of what embeddings are embeddings are essentially words from your knowledge base or data that you were providing to the llm converted into numbers right essentially a numerical representation of words or phrases into numbers what embeddings essentially do is capture semantic meaning and relationship with the phrases in your knowledge B so that when the llm is responding back to the user's question the context and the meaning Still Remains intact gu by default uses nomic embed as the embedding model but you can bring your own right the second concept that we have to understand is llms before we talk about llms it is important to understand that once embeddings are created they are stored in what is called a vector database Gia comes with quadrant Vector database by default but if you prefer using pine cone you can bring that Vector database and use in your AI agent imagine Vector database as a Shelf with multiple boxes like I said numerical representation of words or phrases with similar words group together is an embeding and imagine the boxes have similar words together so that so that llm finds it easier to pull these when it is looking to respond to the user llms are essentially smart storytellers they're responding to the user based on the vast knowledge B that they are provided with and like you can see on the screen these are all llm providers some in the application layer some in the infrastructure layer but you get the idea llms are smart story dealers that use the vector DV and the embeddings to create the response for users questions smartly here is how a vector DV works right you in G you can bring any knowledge based form it can be in the text format PDF format CSV or even a URL right you bring your knowledge Bas chunk it into different segments so that llm can easily understand the context because if you provide the llm data that is not chunk there it will lose meaning because if I continuously talk to you for 2 minutes without giving a break and not giving you enough context what I'm talking about you'll probably lose me right same with LMS right they you have to teach them properly you can use an algorithm to chunk your data or you can do it manually depending on how large your data is and then take this chunk data and provide it to the embedding model which creates embeddings in the case of GAA by default like I set there is noic embed but you can use any embedding model or provider to create embeddings for your knowledge base and then store it in a vector database right once this process is complete you will use this Vector database in your Gan node you can bring any Vector database and use it in your Gan node to power your a agents GAA also provides a tool that is currently in beta which generates embeddings for you by default the embedding model essentially is also an llm but it only is find to generate input sentences into a vector representation right instead of generating a completion sentence as an answering like an L regular llm that you've used like chat gbt where you are getting a response in text format here embedding models essentially are specially trained llms that turn input sentences into a vector representation right in the case of the web tool that we have that you can use to generate emings we use nomic embit text model version 1.5 specifically you can access this from tools. ganet doxyz I'll give the link to this in the video description below but this tool helps you bring your chunk data like you can see in the screenshot there is a chunk text file we give it a snapshot name and when you make rag it'll create the embeding and gives you a Json response this tool also has an API so you can Auto automate this process without having to access the web tool in the JSM response you get an embedding URL a snapshot URL the batch size and the context size this will come in handy when we customize your aent mode all right with all this information of what GAA does why a and web3 is relevant how what are some main problems in the current landscape of llms and llm providers we are here now to understand how to run and launch AI agents scan this QR code to get access to our documentation where you can also get started along with me to launch your node so let's begin once you scan this you should see a quick start guide page which I'm going to show you now and launch a node along with you before showing you the demo you can also scan these two QRS to get a plain text file to knowledge based conversion or if you have a markdown file to knowledge based conversion you can access the documentation using these two QR codes all right let's go to the demo this is G right this is the website of guia you can access it through ganet if you scroll down you can see that these are some of the active notes currently online across the globe and once you go into the menu go to products and agents here you can see some of our public agents that you can access and chat with right they there are different types of Agents some of them are digital twins which are essentially virtual representation of a physical entity or a person right we have one for vitalic there is one node on train on formula one that you can chat with these are some examples of what you can build right and the public nodes that are currently available you can click on these and chat with them directly as of today right along with this you can also find some of our most popular llms that we use that you can also chat with and all the nodes that are currently running for all these LMS you can click and go through and find nodes that are currently live using this specific L that's a quick overview of what the website has to offer feel free to explore but let's go into the documentation to access the documentation go straight to dos. G a once you go here under the node operator guide you will find a quick start with ganet node once you click this scroll down you will see a curl command this is how you install the node this also installed the ganet CLI right let's see how to install node remember that you can run multiple nodes in your machine to be able to run multiple nodes scroll down to how do I and go to install multiple nodes since I run multiple nodes on my machine I'm going to use this command I'm just these are the additional parameters that you have to pass which is the paths to where you want to install the node right so I'm going to copy this and I'm going to paste this command here and then run it all right let me just quickly update this to my path I'm going to call it what should I call it let's call it D best all right once you run it it'll install the ganet CLI for you first and then also we'll download the runtime that comes with each node along with the pr manager we use wasam Edge runti and once all of these are installed you should see a command that you to run this is the most important step you can also see this in the quick start guide which is the source command you get a command to run first so that the gyet CLI is available to you in this terminal otherwise this will not be available to you let me reduce the size of this so that you able to see and zoom in a little more okay looks good once this is done let's run the source command this is done now I'm going to copy this and run it now gyet CLI will be available to us in this terminal window right now that is done out of the way second step is to initialize your node by default GA comes with a specific knowledge waste and a specific LM I'm want to show that to you but the second step once you install the CLI is to initialize your note using Gia init Command right you can also see that in the documentation once you do gu in it since we are running this in a specific folder remember to always pass the paths to that folder okay when you do initialize it will download the model and the embedding model and also initializes the quadrant Vector database so that that it is ready to go for the next command we'll get to this once it is complete and remember depending on the llm size you are using this may take longer or shorter duration and also this depends on the internet speed of yours all right we are almost done within it and now you should see that this is complete and the last command for us to run is gyet start let me clear this first start let me provide the path again all right once you run the start command you should see that the quadrant instance is starting and also the Lama JBS server starts along with that you should get a response which is a URL to your nde so your machine or the node that you just installed Auto assign a node ID and this is mapped to a domain so that you can access the node directly in a browser and start talking to the llm like I said when you launch a node you get a open a compatible API which you can use to extend your existing applications or build applications on top of it along with that you get a chatbot UI where you can interact with your node directly and see how it responds so let's interact with this node that is a expert in Paris and acts as a tour guide for Paris when you open the URL that you just received for your model you should see that it shows the node ID along with the model name and to interact with the model you have to connect your wallet by default you have metamask and this is an opportunity for you to build Integrations to bring more wallets and once you click on start you can start interacting with the not you can ask it question and it'll start responding like I mentioned this is the default that you get with every node along with this there is a API URL which is your node URL followed by open a end points right like I said every guy node comes with opena compatible apis you can essentially build anything that you want to build or have built with open as apis and replace them with your custom node and custom knowledge B that is how you run and launch a modde with Gaia now let's quickly see how to customize your note now you would obviously want to bring in your custom model your knowledge base and customize this entire node to work as per your needs right to be able to do that here is a Quick Command that you should try which is let me clear this which is gu confli list these are all options that you have to configure your specific node you can provide a chat URL which is the llm URL guia today supports ggf based models only and the you can provide the partex size the bat size the embedding URL that you get from the text to act tool that I showed earlier where you can generate embeddings from your knowledge base which is a text file right if you are converting this through another algorithm automatically into an embedding you can bring those files also and provide the URL here you can also use llms from hugging face and bring them here right you can provide the system prompt and the rack prompt you can also provide the embedding context size and which Chrome template to use and all of these right these are all the options that you can use now when you go to the folder you should definitely check this out to get some ideas of what you should probably built using G these are some of the options that are available where if you see quadrant is the folder that is available and Gia by default runs on a quadrant Vector database and there is a dashboard the chat is loading from there and obviously conflict. Json is where you can configure and if you if your node doesn't respond or if you run into any issues the trick is to run stop the node first and do in it and then start it again and you can also go to the log folder and see the logs for all the tools that help run more now like I said we can customize anything to be able to understand how to customize if you go to our documentation you'll find that there is a section called customize your ganed node when you go in there you will find different options like I mentioned you can pass these parameters and customize what we are going to do is quickly stop our current node let's do gu stop and provide the base remember if you're installing multiple nodes always provide the parts of which node you want to stop once this is top here is an example of another knowledge base which is an expert in now in London earlier we saw that our node is an expert in Paris you can verify that by going to saying what is the system prompt that this node has which is your door guide in Paris France and please answer the question from Paris visitor accurately that's the default this is the default node setting right now we stop the mode the next step is to customize your modde I'm going to use the same command you can try using this command and then bring in your knowledge Bas chunk the data and then create an embeding for it get the snapshot URL embedding URL these are the two that you will need to provide along with the context size like you can see snapshot embedding URL context size and the custom system BR and R BR let me copy this command and run this here and at the end I have to also provide the pops because this is running a specific folder for me which is a different mode right once you pass this and click enter you should see that the default values are now updated right it is updated to new URLs right the snapshot is updated problems are updated and we are good to go now the next step is again the same you stop the node first then initialize and then start so I'm going to initialize now then it in it Cas you can also change the model to a new model if you wish to like you can see in our documentation you can see that you can also always pass chat hyphen URL parameter to a new model you can use lava 3.

1 which is the most recent one if you have uh at least 16 GB of RAM on your machine Lama 3 and 3.1 would need that capacity to be able to run on your machine right once this is done the last step is obviously then it start that's all that's how you customize a n you can check out knowledge ways and see how to customize a node and go through all the parameters that you are to pass to be able to customize a node now if you go to the node you should see that the system prompt is automatically updated to London and now any question that you asked this no it'll start responding about London because the knowledge base that this was trained on is now changed to London similarly you can use the tool that I mentioned which is tools. g. XYZ when we open that up you can always use this tool to drop your file here which is a text file chunk and then give it a snapshot name and then make rag so that you get a Jon response here that you can use to customize your node that's how you launch a node and customize a node on Gaia along with this here are some tips on what more you can do with G now we've only talked about bringing a knowledge base that is in text format now if you want to bring something that is in a PDF format or a question and answer format or a CSV or a UR how would you do that right for that we have the documentation that is available to you on docs. G.

A that you can go to and go to Creator guide and under Creator guide you will see something called knowledge bases and if you any go into knowledge bases you should see that there is a tutorial for PL text here is how you convert a plain text file into embeddings then we also have a markdown file to emings tutorial like I said there is also CSV or question and answer pairs knowledge based to be converted into an embedding and then a PDF file or a URL the tool is also available here that we just saw of how you can bring your Chan file and convert it into an embedding you can also see a tutorial of how to Chan available right here always leave a line break between segments so that the context is maintain for the llm that's how you can bring your knowledge base in any form into Gia customize your gu Lord and run your own a agents that are experts in specific field or behavior if if you want to run your nodes only locally then you will just have to pass this parameter when you run the start command so that the nodes run only locally in your local machine there are also tutorials on how to start a node with docker and our documentation is quite extensive I highly recommend you go check out dogs. gy. and find what your review for the other important thing that you can also try is the public notes that we have that you can straight up replace your existing open AI apis with and use our public nodes you can also do tool calling using the tool use laama endpoint and we also have five endpoint Gemma Endo Cod strol endpoint that you can use and the chemistry teacher demo and also you can see how this was built we also have the digital pin of vitalic that you can try these are some examples of what you can build using G along with de I also have some fun ideas of what you can do play around with for the hackathon that you are participating in to build on top of that remember all G nod come with open a compatible apis which means the API for your node can act as a dropin replacement for your existing opena based projects and also if you're familiar with open AI API structure you can simply just change the url and build applications anything that you want to build using open a apis you can build using G nod apis along with that you can also think about building daps that leverage both blockchain and also AI you can build AI power daps with enhanced privacy and control right leverage blockchain for transparency and also secure AI Services you can also build personalized AI agents which means you can train and find your llm on specific topics or specific programming languages or specific issues and respond back to the user on those issues and then build applications on top of it you can also customize or create wall AI based wallets like coin based wallets using the node apis you can create digital twins like I showed you using ganet nodes you can always think about building uh decentralized a market places where you can group similar nodes together or build AI applications that you can offer as a service to the end end user and also the last one but not least is to think about tool calling or building external AI capabilities where you can build a custom Plug-In or a custom integration that talks to an external tool or an SDK or is customized to a specific technology and responds back or does certain tasks like fetching data from an external tool these are some high level examples of what you can build for the hackathons that you're participating in remember always you can come join us on our telegram group where you can ask us for support or or any questions that you may have or any issues that you may run into when you're building with G with that we come to the end of this Workshop do join our telegram group that is where the guy team is and all the other builders who are creating AA agents are who are more than happy to help you and be part of the community on our telegram group come join us on Telegram and show us what you're building on guy thank you