Identity, Privacy, and Security in the New Age of AI Agents | Marton Csernai - Gigentic

[Music] hello everyone um my my name is Marton Cher and I'm very excited to be here um I will be talking about uh identity private Y and Security in the new age of AI agents um briefly a little bit of background about me um I come from a research background I did a PhD in telecommunications and network science then uh did a was a postdoc at the Neuroscience lab and in 2018 I joined the crypto scene and I was working at killed protocol in the padot ecosystem which was uh mainly about credentials and decentralized identity and since 2024 uh I started building my own project around AI agent AI agent platform and and how we can work together with AI agents as humans in a seamless way so the content of the talk will be I would like to give a brief overview of the current current identity Concepts what privacy implications those have um what are the security mechanisms that we can uh attach to these Concepts and then put them together into actionable items so what is an AI agent um like jokingly you could say that it's easier to build an a AI agent than to explain what it is but more concretely I would say States it's a autonomous software entity that acts on behalf of users uh and makes decisions and executes actions now you could say or you could see that how we go from simple automation to an AI based workflow and then to an AI agent and the main concept is that an AI agent is a fuzzy and autonomous system and the reliability is much less than in a the simple Automation and um so the identity uh of a AI agent it the the key the key thing to uh see here is that AI agents have distinct identity needs that account for their transient nature and their delegated authority um so human identities are quite static and AI agents need uh Dynamic content aare identities um also we can uh think of it as ephemeral identities that reduce attack surface but this also complicates auditing and um delegated identities must preserve chain of custody for compliance and and as a simple use case we can say that in a healthcare like a a diagnostic AI agents would be accessing patient records uh that both need eer ephemeral session identity and the persistent organizational identity so the key challenge is that AI agents are both autonomous but they need to be constrained and um one of the key problems is over permissioning which we can see there was a big uh agent breach in Microsoft due to a broad use of the graph graph API access the current Frameworks are moving towards accompanying or or accommodating AI agents but we don't really see the um readymade Frameworks for AI agent identities and also like monitoring is super important and traditional uh security information and event management tools Miss 73% of AI agent anomalies um so one key thing here is that oous 2 already handles 80% of agent authentication needs the main the key thing or like what what you need to be aware of is the proper scoping of these oous uh authorizations and um so the technology is there it's just we have to use it the right way and um so regarding decentralized identity systems we have two approaches that we can take here one is a simple peer-to-peer decentralized identity system which is based on this conflict free replicated data types this is the same technology that Google Docs uses basically you can edit documents with your colleague at the same time and there are no conflicts somehow there is a protocol to make this happen and and um and the other concept is the blockchain based decentralized identity where there is a central source of Truth where the identities are are anchored to and both have their uh con pros and cons one uh thing to note is that revocation authorization revocation is challenging in the peer-to-peer version easier to do in the blockchain based version uh but but the blockchain based version incurs a bit more delays and costs so it makes sense to use uh hybrid approaches like um for example for routine operations one would use the peer-to-peer approach and for um higher value operations you would uh use the the blockchain based part of the identity um so regarding privacy privacy and AI systems is a feature of identity management not just data protection so zero knowledge proofs enable proof of compliance without data exposure so um trusted execution environment can provide model conf confidentiality um differential privacy can help with um agents learning uh capabilities in a private way and um and for example Chad GPT already provides a way to opt out from uh uh from getting so that your data that you enter in to CH GPT and ends up in their training data you can opt out from this so it is also like this you have to find that little box in the configuration of CAD GPT and click it and then you're already in a much much safer more private way um regarding um security I would say that um AI security requires moving from perimeter perimeter defense to Identity bound defense so um that means that usually in human-based ident identities we we secured the resources here we have to think about what kind of authorization we give to each of our agents because in many cases we will work with a whole agentic framework with hundreds or thousands of Agents running um in that framework and they cannot have the same authorization that comes from my user identity and um there are ways uh for example agent to agent TLS um prevents many in the- Middle attacks for multi-agent workflows um behavioral attestation uh can use machine learning to detect agent drift and um and also one of the case studies um like a 50 million deep fake scam shows that we need to have continuous authentic authentication which means that it's not enough to log into a system once and then use that for um months but there is a there is a need need for AI agents to be authenticated continuously on a a per request basis um so a bit about regulations um regulations are converging on auditability uh as the Cornerstone of of AI uh so the one of the main thing in Europe is the new AI act that's uh becoming active this year uh fines up to 35 million or 7% of the of the company worldwide annual revenue uh so that's that's the amount that uh government can find a company so it's pretty steep um in California there are new regulations uh that say we need a meaningful human review in the um in the process of handling with AI and um and yeah so there just in general there is a a need for compliance strategy to implement immutable audit Trails um around a agentic systems um and also we have an emerging ISO standard for AI man uh AI management um yeah regarding implementation just a couple of things to note um we have to secure delegations chains which means the authorization and delegation so I delegate the right to my agent to tweet in my name but maybe it cannot read my tweets it can read other data so um that's that's that's what we mean by delegation chains um and context awareness context is super super important for AI and and we want to monitor how the context gets put together and how that can leak out um so a bit about current protocols um or or emergent emerging protocols that are built on current uh systems and protocols um all out uh extensions now so if you look into the open AI um op uh API controls there's a huge level of different Scopes that you can set up for your API keys and this is the way uh that we see that there's a very very high level frine grain control uh for oout Scopes um verifiable credentials uh is is a concept that's been around for a couple of years and and now there is a new uh action force around AI agent identity um also other policy based access Frameworks and conventional Kerberos systems start to integrate and and um handle a agentic systems um one thing to um uh regarding like what's coming in the future uh on on on device AI or on on device agent which means that we will be able to run an agent on our phone which is pretty cool because it can use the data that's locally available on my phone and then that agent will somehow authenticate get will get its delegation or authentication from my phone or from my accounts that I use on my phone and um and this is this is a really really interesting so I I will not I I wouldn't even need to go to a central place to get my credentials because I could have my credentials already on the phone and everything on the phone um the other thing crossplatform uh tools and analytics and um and also like system integration like that's that's one thing that's probably going to be a very very um not high but very important um part in the next years how do we integrate agentic workflows into current Legacy systems um okay so um just to W like put this into like a context or wrap it up um first point use oout Scopes and you are more or less uh SEC like more or less on a good way or secure with uh authorizing your agents um we don't need completely new authentication systems um the uh other point is privacy starts with the training data what goes into the training data what can leak out from the training data because all the other things are they don't they didn't really change the only thing that changes with AI agents what got into the llm that that we're using and if we fine-tune that llm with our private data then we don't want that to leak out so that's number two um and then uh the third Point regarding security I think that security in in regards to AI agents and agentic systems will be compliance heavy with human oversight so you could say that you like you could automate a lot of compliance things and and reduce the amount of human intervention in the last or or or or yeah in the last decade and now I think we will move we will move in the Direction Where We have already a lot of automation for compliance but now we will have to put back the human into the loop in very very crucial points of agentic work flows so that we can monitor and control the outputs and the operations of agentic systems and um I think this this this will be more or less what I wanted to uh talk about as a wrap up thank you for your time of course um if you want to to learn about what we do then please go to our website you can reach me uh on telegram or or on GitHub uh X so with gigantic my project we we are building a platform where we can seamlessly integrate human professionals Freelancers who can build their own AI agents deploy it on the platform and monetize them and they can do this they can build their agents and connect their own private data and uh and then the by using the platform they can uh share the revenue with the platform and then build a reputation for their agent and um and we are thinking about uh focusing on the compliance aspect how the how the human Freelancers human professionals can check the work that their AI agent is doing and sign off sign off on that work when they deliver that to the customer so that's one of the key Concepts in our uh platform all right well thank you very much and um looking forward to connect and if you have any questions let me know