2024 ETHTaipei Demo Day: Team 4 - Grok vs Llama

next one is Gro versus Yama please [Music] welcome um so hi everyone I'm Kathy from Aura so just a disclaimer before I even start the presentation uh I am a team member of Aura so this submission is purely for the community award I'm not applying for our own bounties so don't worry if you're building on Aura uh so yeah bro versus llama uh if you have realized around I think a week ago on the 17th X has open source its gross model it's huge it's 3 billion parameters they claim to be the best of open sour s llms out there but is it true right so first of all it's 300 billion parameters so most of you probably haven't tried it out uh so that's um where this battle comes in it's like how do we actually track these kind of like glitches or bucks in lln right glitches typically means that they're just not really responding to your prompt so this solution or this uh I guess like application is built upon oras on AI Oracle so right now we have Gro llama as well as stable diffusion being offered uh it will be open up to the mnet soon so if you're really eager to play with Gro you can do so but here there's a very simple prompt which is just like give me 10 dayt night ideas for me and my partner and if you use uh gr for it it actually glitches so let's see what it says it says something like basically it's do it like Co era right you ask for day night ideas and then you're like it's I'm not sure if you have heard but there is actually a pandemic going on and it actually say it dozen of times right so even with such an easy comic glitches and typically when glitches happen What do you do well on Twitter or on X you probably see a lot of screenshots of tragedy right people are like death do something so many glitches and they just post screenshot but this is not a really good like end to endend solution to it right a better solution is what if I have a verifiable way of saying hey I make this inference I give this prompt this is how it glitch and well pay me for finding this bug for you and H this is where llama comes in so I guess the title is sort of a misl cuz it's not really Gro versus llama but we're using llama as the kind of assessment of Gro so what I've done is I did a little of prompt engineering where I took llama to your only answer yes and no and I give him I give it the prompts and the response that I got from Brook right so everything is uh the prompts and the response is all already available on chain because that's how our on chain Oracle work and then I'm taking that appending with the prompt engineering and then fitting it to llama right so in reality this happens in the smart contract but I just want to show you that uh with llama with this prom engineering llama would give a very simple answer of no or yes right so here is uh the growth versus llama um contract which is really the main contribution of of this project so in here you can see that the uh the contract has like a one if Bounty right and if someone has found a bug or a glitch on Gro what they can do is they can actually call a function that's called request battle and type in the sort of request ID or the prompt that they found to be uh eress and what will happen is that we will actually look it up right so we will see what was the previous um AI Oracle result that was given for the pro and then we take that result we put it back into the smart contract append the pr engineering that I just did so that llama would only answer yes or no right and we are done also receiving this call back from llama with the answer yes or no and I'm just directly checking that end to end again uh against in the smart contract itself right so when llama when we receive the AI Oracle call back from llama um the smra is going to automatically check is the answer yes or no and then pay out the Bounty right so this is the sort of simple idea it's very short uh presentation but I think it's a very cool uh exhibition of what AI sorry what watching could do for AI because a lot of times we're kind of struggling with what applications you can build to to um to kind of help AI out so I guess this sort of like AI Improvement initiative where it would be great if there is a way that you don't have to reproduce these results all the time right because otherwise without this verifiability that Oni Oracle provides what you will have to do is you will actually have to take that prompt perhaps some random seats as well and reproduce it on your end with small models it's probably not that expensive but with things like bro where you need like a tons of like h100 gpus to do so it might not be economical for even X themselves to reproduce these results all the time in order to check whether it's an actual glitch right so here is a verifiable end to end solution where you have a glitch you submit it um you will receive a bounty and there's no like manual um uh manual uh interception involved so yeah so that's what I built and open up to any questions all right any [Applause] questions we're good actually so who won also no it's not really a battle as up right but to be honest just my personal opinion right so well why we use llama as the assessor because you could also use gr right to assess itself right so I can do the same problem engineering for that so first it's definitely economical and second of all at least from my experience right now llama is a little bit easier to prom engineer than Gro so Gro is definitely more glitchy at the moment all right thank you thank you [Applause] T next upop eil