Silvio Ventre - Hands on ChatGPT - Best Practices on Prompt Design

in this room and we're going to get started with a very interesting topic which is prompt engineering and Sylvia Ventre who's the head of innovation at mobile asset management as well as a startup mentor at startup jix will lead the workshop thank you very much

thank you so much so good morning everyone thank you for having me uh I don't know if the the audience is Italian or mixed audience because if you I think we can we can switch in Italian but uh I don't know English English is okay great so um just uh before start uh the let's say the workshop itself uh I would like to uh introduce also the technology I don't know uh how much are you confident about the technology itself so the generative AI uh but it's uh very useful to know how uh basically it works just because it's very complex and um it's not so smart as you can think uh actually it's quite dumb let's say uh so how does chajp work chpt generally speaking so uh I'm referring in particular to all the large language models not only chaptt but uh it is the first one uh going live uh just two or three years ago maybe. Um one of the main concept that you have to consider is the auto reggressive test text generation. Uh what does it means? Uh auto reggressive uh is one of the main characteristic of the LLM uh LLM uh on the web. Uh basically the architecture behind uh all LLM is auto reggressive.

So um in it has the ability to component word after word doing auto reggression. So going back to the first word analyze what is the best word uh the most efficient word efficient word going after the first word and this is like a loop a continuous loop uh once the the sentence is complete. So basically the model calculates the probability of every single word word after words. So the model itself doesn't know anything about the the words that is writing right and that's one of the important one of the most important things uh to know uh of the LLMs just because they don't know anything about their writing. The other concept really important uh about the the LLMs is the power of attention mechanism behind the architecture.

Um there are a lot of neural networks that are the that has uh that have the ability to um focus attention on the most important word in the in a sentence. So that uh the attention mechanism uh allows the model to examine uh other positions in the text, identify the most important word and uh process every kind of words to define uh the the last prompt uh the last uh sentence. uh only after uh that the attention uh is focused on the single term the right term that's very important because um actually LLM's work exactly like the human brain so when we talk with someone we are going to focus on the most important concept and we uh try to develop to uh to uh give an out in output a sentence which is probabilistic uh coherent with someone uh who you are speaking. So more or less those are the main concepts the main concepts behind the architecture. So attention and also the probabistic way.

LLM doesn't know anything about you are talking about they know everything but you uh have to ask in a better way uh how to um uh extrapolate uh the real value behind a question behind uh behind the architecture and uh the way you are going to um to have a real um a real result a real output is uh the prompt. So what is a prompt? A prompt is just a request. It's just a request uh or a question formulated by a user. And uh uh since the moment the the the only way to talk with large language models is with natural language.

Um during uh in inside of the academic context uh we're born uh two aspects two main topics prompt engineering and prompt design. from design is uh like the art of compon of compose or formulating a great question uh so a clear and effective question to the LLM uh in order to generate accurate response. So basically it's like the the art of uh craft a real request a a great request in natural language. the prompt engineering it's like the step the step on of the uh of crafting a good uh and effective uh request using also the code. So basically in Python which is a coding language um with prompt engineering you can have uh a level up let's say uh of the results of the output that you uh you can expect from a large language model.

Now we are going to focus on prom design just because it's uh the the most common way uh to to talk with a uh with a large language model and how we can craft an effective prompts uh in in Chach or um in another large language model like Gemini or perplexity or someone uh like the instruments the tools that maybe you you may uh are using right now. Uh first of all you have to be specific very specific. So you have to clearly uh define what you want. Uh that's because they don't know anything about what are you saying. So uh if the more you uh the more you are specific the more you you will have uh an accurate response in output.

Um of course you have to avoid uh bug instructions. Uh the second is provide context. You have to define all the context uh useful for the model in order to generate an accurate response. Just because uh if you want to for example uh craft an email to someone who are trying to sell you to sell you a product for example they don't know what kind of product they are going to sell you. they want to sell you.

Uh they don't know the large language model doesn't know uh what is the let's say uh the tone of the response that you want to to craft. Uh, it's a simple email, but it's not so um it's not so well to to make a prompt like uh just craft an email to respond to this email. You have to define the context. You have to define the subject. You have to define a lot of data uh useful for the model in order to generate an effective email.

That's a real use case. But uh going going on, we we're going to see some uh some examples uh some specific example. Uh once uh you provide context, you have also to specify the format because if you want to um obtain from the model for example uh a table, a spreadsheet or uh an exact output, you have to define it just because the model doesn't know what you want at the end of the prompt. If you don't like the result, you just have to iterate. So one of the characteristic that we um we didn't um we didn't talk about just before is that generative AI it's um a very interesting aspect of this technology uh because of the gener generative part.

So once you ask something to a large language model, you never you'll never obtain the same result. Never. So uh in this room we can make the same question and obtain like 20 or uh yeah 15 results. Every results will will be completely different. Not so much completely different.

I mean uh the the formulation at the end uh of the prompt will be different for sure. More or less the the uh meaning of the sentence will be more or less the same because the prompt is uh is equal for everyone. Uh but it's important to understand that the generative part is very um um how do I say um it's one of the the component that you you don't have to ignore because if you don't have the results that you expect you just have to iterate. So change the prompt uh I don't know one times, two times, three times uh whatever it takes to to have the right results that you want to expect in particular. I don't know if it's uh big enough to to read something but uh here are like nine uh rules to to write an effective prompt.

So uh you have to start with action verb. Um basically you you have to define uh as much as you can the the the request in a clearly way. So starts with action verb use clear actionoriented words like create write or explain define or something like that. You have to add context. So relevant background, relevant information to the large language model in order to personalize the response.

Uh you have to be clear and specific. So you have to ensure what are the um uh the request. So avoid bug answers. Use example because example examples help um the model to understand what is the request what what is the output that you you're going to have you want to have uh and also help uh helps the model to uh to understand what are you talking about. You have to specify the length.

For example, for the um for the example um what I was talking about the email just the email if you want to craft an email uh of 100 words for example which is very long very long email you have to define it. You you'll never have something like 100 words email if you don't write it. So you have to do it. You have to guide the the AI just because they don't know the LLMs doesn't know uh don't know anything. You have to give explicit instruction in order to have uh the results that you want to have.

You have to clarify in the tone. Uh for example, if you want to argue with some vendors or with someone else or if you want to craft a report for a general audience or a report for the sea level of your company or something like that and then the experimentation part. So once you don't have the result that you you want to expect, you just have to iterate. So continue to refine the prompt. So another uh really interesting thing to me uh about the large language model is that they are very similar to the you to to humans in general but not smart humans completely dumb humans.

uh not only because you have to um provide context, provide information uh define the output that you want just because they work exactly like the human brain. So basically uh you have to talk with them using patterns uh more or less something that we do every time when we talk with a person we we do it implicitly. Uh so with with the with the AI model we have to do it in in in a in an explicit way. There are a lot of patterns in order to talk uh in the right in the right way with a model. Uh one of them is decision pattern.

They help you uh they help you to make decision um I mean in your work in your daily routine uh in every kind of uh task you are perform um so you can use it as a support tool a decision support tool on specific topics for define pros and cons something like that u there's another one data processing patterns. Uh it's very uh useful for um manipulation and inter interpretation of data. So for example, if you have text documents or uh you know PDFs uh or spreadsheets uh documents in general with data you can process, analyze uh and interpret data. Another one is data extraction. So for the same format for example you have spreadsheet you have PDFs you have uh every kind uh even a JPEG so image you can extract information uh basically just asking to the LLM and another one is very interesting to me is the audience persona pattern uh which it works also in uh in two different ways you can simulate a role And you can uh make the LLM refers to you uh as a specific person or uh refers to you uh as you are like an engineer or a marketing manager or someone uh someone you you want to be in order to have a response uh completely personalized customized on what you are defining.

Uh, of course you can combine every kind of uh patterns. Uh, those are only four patterns. There are a lot of patterns uh behind uh the prom design uh infrastructure, the prom design uh topic. You you can combine all of them in order to have a uh an effective output. Okay.

Uh I don't know if you have laptop laptops with you. Uh I don't know if you have the free version of Chad GPT. It's uh perfect just because if we make it just a bit more um interactive uh it it would be better in order to understand how to craft a prompt. uh it's a very uh I mean we we are going into all the patterns that I uh explained just before um the first one is the audience personal pattern uh which is one of the maybe uh most useful pattern that you can use in your daily routine uh even at work even at home it depends uh for whatever you want uh but you uh the pattern format is very simple. You can make the LLM assume a specific role.

So assume I am an engineer and I want you to perform a task. So basically really four words to define uh this pattern. And uh the other way you can see this pattern is that you can make the LLM assume a specific role. So it's not you the subject but it's the LLM. So you can say you are a software engineer, you are a marketing manager, you have to perform an action whatever you want a report or an analysis or something like that.

in this prompt. Uh, of course you can copy and paste this prompt, but it's just an example. So, uh, feel free to, uh, to try another prompt similar in the structure. I'm going to read it. I'm going to read it.

So, in this on this prompt, you can say I am a finance professional business conference. I'm not a developer, not a developer or a tech guy or a tech guy. I'm intrigued by surprises that explains that explains what is why it matters why it matters and how it might experience. It's a coin back a coin back on what I was I was about before I I set said who I am. So curious around finance professionals professional uh I said also what I'm doing just to provide just a little more context context uh I also said what I'm not so I'm not a developer I don't know anything about coding or something like that uh I'm not a tech guy uh it's really important to to the model in this in this prompt to understand that I know anything about coding about uh technology in general.

I work in finance and I want to understand what is web 3. I want to understand what is web 3 technologically speaking, why it matters and how it might reshape the internet experience in a small report 500 words. In this prompt is completely different to another one. For example, what is web 3, which is a prompt, but it's a really small prompt without context, without um the information about what you are, uh who you are, uh what you are looking for. So this pattern is um is going to have an output really effective instead of the simple question, what is web three?

I If you tested uh something like uh this pattern uh so assume I have am someone. If you want to share if you want to share something I I can I mean if you want to explain the the example feel free to do it. I mean uh I'm happy to make the interaction Thank you. My prompt was a little bit different than yours. Assume I'm a DEP developer.

I'm a professional visiting a tech conference focusing on crypto and blockchain. I'm a tech guy, a coder, not a salesperson. Provide me a short pitch why it matters to start coding in this area right now. That's a great point, a great prompt uh also because I guess you had also an effective output. You can define also what is the format of the output.

In in our case, we said like 500 words. You can say also at the table that compare the I don't know some information. You can also stimulate the the pro uh the LLM to go on the web to find some information. So basically you can combine a lot of information, a lot of requests. The more you are specific, the more you have an accurate response.

That's a great prompt. Thank you for sharing it. The second personal pattern that we're going to uh we we're going to see. Um this is uh a bit longer respect the other one. So maybe uh I don't know if we have time to to read also this one but uh anyway the the prompt is very similar.

It's a role simulation. So in this case you are telling the LLM that he is a specific role. So you are a web3 marketing strategy strategist advising a consumer brand entering the decentralized ecosystem. I provide a lot of context and I said that he has to prepare a one-page executive brief outlining how a brand can use NFTTS and tokenbased communities to increase customer engagement and brand loyalty. Then I defined I defined also all the output.

So ballot point uh which is the what is the focus that I want to have but basically the pattern is the same. So you are going to uh interact with someone who has a role a specific role. In this case the LLM is going to be a web3 marketing strategy advisor. Um and he has specific skills. occurs.

Uh it's very important because once you um whenever you you are going to make the LLM assume a specific role, uh it's like uh he's going to have all the skills that a real marketing manager, a real web 3 marketing strategist has in the real life. Uh and of course you can do it for wherever you want uh whatever you want. So it depends by the use case. Now I'm I'm talking about the web three marketing strategies. You can use it also at home.

Uh if you don't know what are you going to to eat at lunch or at dinner. Uh but I'm going to skip this one uh just because you tested. Uh I'd like to um talk um you about the decision pattern that I mentioned before. This is the help to understand pattern. It's very useful when uh when you don't know something about a specific context, a specific car uh a specific topic.

So you want to understand something you have to take decision on something that you don't know you can use uh chach or any kind of LLM to understand a specific topic and you can use different patterns uh different patterns uh like the parto principle application I don't know if you know you already know the the parto principle uh which is like um a principle uh that identifies the 20% the the most important 20% of the knowledge of a specific topic that make you uh understand uh the 80% of the specific topic. That's really interesting uh as a as a pattern as a principle uh just because you can um combine this pattern and the previous one. So you can ask the LLM to impersonalize like the web 3 marketing advisor uh and make you understand what is an NFT from a marketing perspective using the parto principle. So you can ask tell me all the information that make me understand the 80% of the specific topic but the 20% of the information of course an example prompt is like break down corporate finance. If you want to know more about corporate finance into small easy to understand parts use real life analogies and examples to simplify the concept.

This is a small prompt, but it's way better than say just tell me something about corporate finance, which is good as well because maybe you want to know only uh small topics about the corporate finance. uh but you want to have some ex examples or a breakdown structure uh of the of the topic and another one is uh with the parto principle. So I want to learn about sustainable finance but could be also web three or uh tokenization or real world asset for example identify and share the most important 20% of the knowledge on this topic that that will help me understand 80% of it. That's a great uh that's a great prompt. I mean a a great way to understand uh topics that you don't know and you are curious to know more.

Another one um is the data structure pattern. Uh this is uh useful for a lot of use cases. I use this uh I used this one for example um like one week ago. Uh I had um a a small accident car accident uh last um last month. I have to define for example uh I have to craft an email to the insurance company the counterparty uh the counterparty I have to extract some information uh from a picture so it's not the the example uh that I'm uh uh that I'm showing here uh of course this is also something with an incident location this is an example uh an example from Microsoft.

But I used this example uh even in my life just one week ago for example to extract information from an image of my uh car that was uh of course damaged and uh I had to do uh a specific report in order to have um reimbursement from the the other from the insurance company. Uh but that's only uh an a use case. You can use it for example at work if you have a spreadsheet you have to uh I don't know find some information in a very long uh PDF or a very long uh massive uh spreadsheet. You can ask extract extract the following information from the spreadsheet that I attached here in the chat. Uh or for example in that case from the phone conversation below.

So you have a transcript of your phone conversation. You can extract reason for the call, cause of incident, names of all the drivers if uh during the phone conversation are mentioned something like that. You can also ensure the point uh the output of the points. So for example in this case I ask the LM to ensure fields from one to six uh are answered very briefly. For example just stating the location name for the incident location.

So basically you have to define the output. You can define everything in the prompt but if you don't do it you won't have uh an effective uh an effecting output. Uh actually we uh uh I mean the workshop it's finished uh because I didn't want to uh make a lot of um examples um passive examples. I'd like to understand also if you have question uh I'm happy to. Yeah, for sure.