[00:58.620 --> 01:00.860]  Since I know these folks, I will focus here.
[01:00.860 --> 01:03.160]  So this talk is very basic.
[01:04.400 --> 01:09.140]  So there's plenty of online stuff and other places to kind of really get in the guts of the techno.
[01:09.140 --> 01:11.240]  This is a little basic because it's a new emerging field.
[01:11.240 --> 01:15.360]  So a lot of companies seem to be doing kind of AI red teaming evaluations.
[01:15.360 --> 01:20.020]  Has anybody here had their company either get an AI red team in or do some kind of evaluation?
[01:20.020 --> 01:23.500]  Like have you seen AI security where you work?
[01:24.080 --> 01:26.460]  Okay, I want to make sure I'm not getting too basic.
[01:32.900 --> 01:34.140]  You're my audience.
[01:50.320 --> 01:51.020]  Oh, cool.
[01:51.020 --> 01:51.840]  Okay, perfect.
[01:52.080 --> 01:54.160]  Yeah, so I mean, this is an emerging field.
[01:54.160 --> 01:55.220]  So all these things are great.
[01:55.220 --> 01:58.260]  You know, ICS security students, you know, LLMs, all this stuff's great.
[01:58.260 --> 01:59.640]  So it's an emerging field.
[01:59.680 --> 02:02.480]  To say you've been in it for four or five years, that's a long time.
[02:02.840 --> 02:03.920]  So it's an emerging field.
[02:03.920 --> 02:05.440]  You know, what is AI security?
[02:05.440 --> 02:10.040]  So it's somewhere between kind of data science, application security, network security.
[02:10.340 --> 02:12.580]  I'm mostly a network security person.
[02:12.880 --> 02:15.060]  It's this convergence of all these things.
[02:15.060 --> 02:18.640]  And in the middle, what Rachel's expertise is in, is in risk management.
[02:18.640 --> 02:27.220]  Because a lot of the decisions, you know, if you're at a fintech company or other companies, for some of these tools and what you do around security is really a risk kind of business decision.
[02:27.220 --> 02:29.660]  I say more so than regular security.
[02:30.440 --> 02:38.280]  So some of the objectives in this field of AI security are, of course, securing your own internal AI systems from attacks and vulnerabilities.
[02:38.460 --> 02:41.340]  You want to avoid an AI incident, I mean, with your company.
[02:41.340 --> 02:44.880]  And to understand, you know, there's this tension.
[02:44.880 --> 02:47.020]  How is this different than regular security?
[02:47.140 --> 02:52.980]  So what are the new attack surfaces that we may not see with kind of the regular infosec security field?
[02:53.500 --> 02:56.540]  So there's a lot of reasons not to care about AI security.
[02:56.540 --> 02:57.760]  It's a new thing.
[02:58.400 --> 03:05.840]  Machine learning attacks or AI-specific attacks, they are still somewhat hard to identify in practice.
[03:06.380 --> 03:09.020]  Detecting, you know, is it a model that failed?
[03:09.120 --> 03:11.020]  What exactly was happening there?
[03:11.020 --> 03:19.660]  You know, the telemetry and detection to do that and to say, this is specifically an AI machine learning security issue, you know, that's still a little difficult.
[03:20.280 --> 03:30.460]  It's easier, as most of us know, to social engineer, DOS the network, attack a database, just do your traditional security attacks, don't worry about the fancy AI stuff.
[03:30.980 --> 03:34.460]  You know, there's not been, except for deepfakes, there's not been a lot of flashy threats.
[03:34.460 --> 03:37.900]  And there's a lot of other stuff to do, your documentation, your other stuff.
[03:37.900 --> 03:39.960]  So these are reasons we shouldn't care.
[03:41.840 --> 03:43.360]  I will tell you why we should care.
[03:43.360 --> 03:48.380]  And kind of, first of all, off the bat, compare traditional security, which, you know, most of us come from, and AI security.
[03:48.600 --> 04:02.300]  So the left, you know, your traditional security really focuses on, I'd say, if I were to bucket things in general, you know, defending the network, putting up firewalls, code vulnerabilities, access identity management, and a secure software development lifecycle.
[04:03.660 --> 04:07.680]  On the premise that you need to really understand the systems in use and the data flows.
[04:07.820 --> 04:09.880]  AI security, you know, we have a lot of that.
[04:09.880 --> 04:12.440]  Plus, you got to worry a little more about the data.
[04:12.720 --> 04:19.220]  The code vulnerabilities, just because models and that kind of code, it evolves very quickly.
[04:19.620 --> 04:21.440]  The models, of course, have security around that.
[04:21.440 --> 04:26.380]  The usages and applications of AI, but also the secure software development process.
[04:28.260 --> 04:34.040]  So the security focus, I guess, really depends on, you know, are you building the tool?
[04:34.500 --> 04:36.100]  Some companies do build their own.
[04:36.100 --> 04:37.500]  Many companies just buy a tool.
[04:37.500 --> 04:41.440]  So kind of the focus of maybe what I'll talk about, you know, are you just purchasing something from a vendor?
[04:41.440 --> 04:42.640]  Are you building it?
[04:42.860 --> 04:44.780]  Are you giving the tool your data?
[04:45.200 --> 04:46.700]  Are you using APIs?
[04:46.780 --> 04:51.160]  You know, which a lot of people, they may just have an API to open AI.
[04:51.460 --> 04:53.080]  Are users just doing whatever they want?
[04:53.080 --> 04:57.700]  Are they just, you know, you tell them don't use DeepSeek and they still decide to use it?
[04:57.700 --> 05:01.720]  So the focus on what you might be interested in depends on kind of the usage.
[05:02.280 --> 05:10.080]  Some of the trends since about 2020, which is I'd say when this field kind of started to take off, was these AI red teams have gained traction.
[05:10.580 --> 05:21.760]  Generally, I see these in large companies, you know, NVIDIA, Microsoft, some of our colleagues from the AI village are there, other big companies, and a whole lot of consulting in this area of AI red teaming.
[05:23.000 --> 05:32.200]  This whole rise and kind of disappearance and then kind of flattening out, I guess, of machine learning AI ethics, you know, what does that mean in terms of the usage of the AI system?
[05:32.240 --> 05:34.560]  These teams are kind of, they come and go.
[05:34.560 --> 05:38.600]  And, of course, this explosion of language models and jailbreaking them.
[05:38.600 --> 05:47.560]  And, of course, we have the Lakera example with Gandalf in the back where you try to jailbreak and get the LLM to reveal some stuff it's not supposed to.
[05:48.940 --> 05:53.520]  So, again, when will you face AI security, even if you're doing traditional security at your company?
[05:53.520 --> 06:04.860]  So what I've seen and probably some of you are seeing is LLM, the large language model, the crawlers, they will scrape your company sites and they do not obey the robots text you might have up.
[06:04.900 --> 06:13.720]  So they, especially in the past week, there's been some very interesting press around them scraping and almost taking down Wikipedia and some large websites.
[06:13.720 --> 06:16.660]  So they're going to scrape your company sites, be aware of that.
[06:16.660 --> 06:18.260]  That's a security issue.
[06:18.260 --> 06:26.440]  A lot of, you know, CIOs and CISOs, they'll buy, you know, a new AI tool to automate security by AI.
[06:26.440 --> 06:27.460]  So what does that mean?
[06:27.460 --> 06:30.880]  What should you be asking if they want to buy that tool?
[06:30.920 --> 06:40.940]  A lot of folks I see, especially in fintech or healthcare or law firms, they are building their own internal LLM, which is very easy and I'll talk a little bit about that.
[06:40.960 --> 06:48.960]  A lot of users, you tell them we don't want you using this to give your, you know, to ask a question about the company, they still do it.
[06:48.960 --> 06:49.920]  What does that mean?
[06:49.920 --> 06:51.180]  Can you stop it?
[06:51.180 --> 06:56.260]  And, of course, social engineering or applicants using cloned voice or images.
[06:56.280 --> 06:58.520]  And I see this an awful lot.
[06:59.840 --> 07:03.160]  So AI security, you know, it's part of a very easy attack.
[07:03.160 --> 07:06.280]  This is from, you know, gosh, nine years ago.
[07:06.280 --> 07:10.860]  If you don't remember this, you probably do remember this, but Microsoft had this chat bot called Tay.
[07:11.840 --> 07:18.800]  Microsoft foolishly had Tay learning in real time from Twitter, from the corpus of Twitter.
[07:18.800 --> 07:30.940]  So people very easily figured out, you know, Microsoft figured out training in real time from real time feedback was not smart to using Twitter as a real time feedback was not that smart.
[07:30.940 --> 07:39.060]  So within, I can't remember if it was ten minutes or nine minutes, it turned Tay, the chat bot, to, you know, racist, spewing, all sorts of hateful things.
[07:39.060 --> 07:40.740]  I think it was about ten minutes it took it.
[07:40.740 --> 07:45.880]  Because people figured out how to do an attack on it and they teamed up to post this stuff.
[07:46.900 --> 07:54.620]  Other attacks that emerged, you know, around 2017, 18, I bring these up because, you know, these are still around but in different formats now.
[07:54.680 --> 08:00.500]  So these stop signs and optical illusion stickers, these are called adversarial patches.
[08:00.620 --> 08:20.480]  And they were a very early way so that the little metallic looking sticker, it's a sticker that, again, they're called adversarial patches, and the way it fooled the AI is the AI kind of gets distracted at the time with certain pixels, certain portions of that shiny metallic image.
[08:20.480 --> 08:25.980]  So therefore, by focusing on that, the AI wasn't focusing on the rest of the picture.
[08:25.980 --> 08:28.740]  So they put the sticker next to a banana.
[08:28.740 --> 08:31.460]  The AI would not be able to classify the banana.
[08:31.460 --> 08:34.160]  It would be distracted by the optical sticker.
[08:34.240 --> 08:37.220]  That helped, of course, you know, refine and make better decisions.
[08:37.220 --> 08:40.820]  The stop sign stickers were used a lot in testing autonomous vehicles.
[08:40.820 --> 08:47.620]  And, indeed, in a closed environment, putting that shiny thing on the stop sign made the autonomous vehicle make bad decisions.
[08:47.620 --> 08:53.840]  So that's a way, you know, in hacking AI and looking at AI security, we helped to make the algorithms better.
[08:53.840 --> 09:04.560]  And, of course, the famous, this kind of makeup was able to fool facial recognition because the AI at the time was facing certain types of highlights on the face.
[09:04.560 --> 09:08.360]  This particular, you know, juggalo makeup fooled facial recognition.
[09:08.360 --> 09:11.400]  And, of course, that, you know, was pretty funny to watch.
[09:12.900 --> 09:19.140]  One caution is that, you know, we talk a lot about LLMs and guardrails and being mindful.
[09:19.160 --> 09:30.200]  In the past year or so, AI security is always being conflated with LLM security, prompt injection attacks, can you make, chat GPT and other things, you know, tell you how to make a bomb and whatnot.
[09:30.460 --> 09:34.940]  That's LLM security, but there's a whole lot more to AI security than just that.
[09:34.940 --> 09:37.140]  But that's kind of taking a lot of the air out of the room.
[09:38.940 --> 09:41.760]  So many of us are used to doing regular threat modeling.
[09:41.760 --> 09:44.300]  How is AI threat modeling different?
[09:44.820 --> 09:52.900]  These are the things I would add in that if you're looking at, you know, what specific things should you ask?
[09:52.900 --> 09:54.460]  And I'll have some examples of those questions.
[09:54.460 --> 09:57.680]  You know, what are the additional things you've got to think about AI security?
[09:57.760 --> 10:00.760]  Data storage, data collection, data access.
[10:00.960 --> 10:21.840]  For most of us, that's a problem anyway, but really thinking about the data usage and storage really is very different with AI security because, you know, your users are giving information or making what they think are just innocuous queries into an LLM.
[10:21.840 --> 10:25.680]  That's being fed straight into the companies who may get proprietary information.
[10:25.840 --> 10:29.840]  The network, of course, you need to think about whether it's on-prem or in the cloud.
[10:30.600 --> 10:43.460]  I'd say one of the big differences or things that are highlighted is a lot of these even proprietary tools that are supposed to be very well vetted tend to use open source libraries and models.
[10:43.460 --> 10:53.600]  And we are seeing, especially in Hugging Face, there was a big study, whether it was last year or the year before, with like hundreds of vulnerabilities in Hugging Face and more so all the time.
[10:54.640 --> 11:08.380]  So just to step back, you know, Ben had a great talk, but this is just a refresher, kind of what is the AI pipeline, even if you're not creating the tools, just kind of understanding in general, there's some training data fed into an algorithm, the model does some learning,
[11:08.380 --> 11:10.480]  there's now a trained model, you get results.
[11:10.480 --> 11:11.780]  Super high level.
[11:12.520 --> 11:17.300]  So kind of looking along, you know, if you were to take the threat model and just kind of look along the pipeline.
[11:17.300 --> 11:25.280]  So there's some data collection, the model's developed, the model's trained, you do some inferencing, and then you can use it.
[11:25.280 --> 11:28.620]  So it's kind of three places we want to have security.
[11:28.620 --> 11:40.740]  We want to secure the data, whether it's your user data being fed in, we want to secure the model, make sure nobody can see what you're doing, and you want to secure how it's used, if it's making decisions about, you know, credit card fraud or whatnot.
[11:40.900 --> 11:43.000]  So you want to secure those three things.
[11:43.200 --> 11:45.420]  Of course the infrastructure and the AI governance.
[11:46.440 --> 11:55.820]  So on the right I have, you know, what are some things to think about when people engage kind of AI red teams, or whether you're an AI security person thinking about this.
[11:56.180 --> 11:59.620]  Stakeholder access, of course, to the data as anywhere.
[11:59.820 --> 12:06.780]  The data classification, of course we think about this a lot, just in regular companies, is it confidential, is it open?
[12:06.780 --> 12:18.340]  But thinking even more so with an eye for if I'm going to use, if I have, you know, an API and I'm going to use an open source model, that data is going to be fed to that model.
[12:18.340 --> 12:32.800]  So being maybe extra careful about how you think about that data classification Some of the attacks, you know, as you're thinking about securing the model, particularly if it's a proprietary one for your company, you're thinking about a supply chain attack.
[12:33.900 --> 12:43.000]  Especially as people repurpose, you know, they grab off the internet, you know, pre-trained kind of open source models without really looking at whether those models themselves have vulnerabilities.
[12:43.340 --> 12:48.160]  And then of course the prompt injection attacks that we see with our example of Gandalf.
[12:48.960 --> 13:03.240]  In terms of securing the infrastructure, you're adding on kind of really thinking about as people buy, you know, through professional versions and they get the API access to these models, kind of really making sure the API is secure is very important.
[13:03.280 --> 13:18.220]  And then in terms of things like if you are using, you know, if your company buys a tool and you're kind of noticing the decisions are maybe a little wonky from that said tool, is that because the model itself is drifting and maybe the parameters need to be adjusted?
[13:18.540 --> 13:23.600]  Is there some fairness issue with the model where, you know, I didn't really think it was made on that?
[13:24.060 --> 13:27.220]  I'll give you some questions to be asking along those lines.
[13:27.980 --> 13:33.220]  So here's an example I see quite often, which is people building internal LLMs.
[13:33.760 --> 13:41.720]  We've done this, I've seen this across, you know, finance, again, law firms, places where they don't necessarily want to go out and even get a license.
[13:41.720 --> 13:44.780]  They want to just build their own because maybe there's some purpose-driven reasons.
[13:45.000 --> 13:57.620]  So, again, I work in finance, in blockchain, in cryptocurrency in particular, and we do a lot of tracing who was using the cryptocurrency maybe for not great purposes.
[13:58.120 --> 14:12.540]  So part of why we built an LLM is we wanted to have other functions of that, which would be to not only train the LLM on things we cared about, but also to interface with something like Neo4j, where it could also show me a relationship graph.
[14:12.540 --> 14:21.180]  So I use my internal LLM trained on what the company's looking for, and also that LLM helps train a graph that shows me visually relationships.
[14:21.780 --> 14:37.840]  So building your own internal LLM, whether it's an open-source model like DeepSeek, we've heard a lot about that in the press, or LLAMA, you can get a hold of this, and kind of the three traditional things to think about, again, the attack surface of the network and infrastructure,
[14:37.840 --> 14:39.460]  the data, the applications.
[14:39.620 --> 14:41.020]  So be mindful of these things.
[14:41.020 --> 14:43.080]  I want to build my own internal LLM.
[14:43.940 --> 14:46.460]  Kind of the red is things to really think about.
[14:46.460 --> 14:49.760]  Are you going to do it locally or use a corporate kind of cloud provider?
[14:49.840 --> 14:50.940]  A lot of people do it on-prem.
[14:50.940 --> 14:56.080]  You don't need a super beefy system to run your own internal LLM.
[14:56.080 --> 14:57.400]  So a lot of people just do it on-prem.
[14:57.400 --> 14:58.480]  They don't even go to the cloud.
[14:58.480 --> 15:01.340]  You can also do it in a secure VPC in the cloud.
[15:01.640 --> 15:05.520]  The data, I mean, one of the reasons you want to do this is the data then stays within the company.
[15:05.680 --> 15:14.760]  So talking to some lawyers yesterday, they want an internal LLM where they want to do all sorts of stuff with case studies and be able to research.
[15:14.760 --> 15:25.980]  They don't want to do it in public LLMs, with the addition that they want to tweak and make sure the LLM generates documents that sound like their law firm and they don't sound like generic chat GPT.
[15:26.060 --> 15:28.920]  That's one of the drivers for doing an internal LLM.
[15:28.920 --> 15:38.860]  If you're a certain type of company, you want it to be trained on the flavor and the tone of what your company's doing in addition to keeping the data within the company.
[15:38.860 --> 15:41.100]  So you can customize that for your own use.
[15:42.200 --> 15:45.440]  So some of the takeaways, not a whole lot different than regular security.
[15:45.440 --> 15:54.780]  Ensuring access controls are enforced, and in this case, access controls to who's able to tweak the model, who's able to get a hold of it, who's able to set permissions to it.
[15:54.780 --> 15:56.560]  You want to make sure it's kept offline.
[15:57.160 --> 15:59.940]  You want to make sure that any sensitive data is encrypted.
[16:00.380 --> 16:10.240]  And most importantly, if you're going to use an open source model, make sure it is patched, it's updated, it's retrained, because these vulnerabilities, you have to take control of them.
[16:11.960 --> 16:24.380]  Here's another example, say your CISO or CIO goes to RSA and gets sold this AI tool to do whatever, you know, agentic, automatic, SOC stuff, whatever.
[16:24.960 --> 16:32.640]  So here's some, you know, if you do come in and you come across this as the security person, some kind of basic questions to ask.
[16:32.640 --> 16:34.620]  You probably all know this, but what do you do with our data?
[16:34.620 --> 16:46.820]  So if you're going to buy one of these tools, oftentimes they will use that data, and regardless of the data usage agreement, they're using the data about your company and how you use it to help train their model.
[16:46.920 --> 16:49.000]  Pretty basic question, what do you do with our data?
[16:49.620 --> 16:55.640]  One that you don't have to be a math expert or even an AI expert, but what is your model architecture?
[16:56.420 --> 17:04.780]  So, and the question I say to ask here is because if you ask them that and the answer is it's a black box, we can't tell you, you need to call bullshit.
[17:04.880 --> 17:09.820]  If it is completely open source, okay, you know, I don't know if we want to buy a tool like this.
[17:09.820 --> 17:17.580]  If it's in the middle where they say, here's the general architecture, here's the general data flow, and where we use a model, and the type of model.
[17:17.900 --> 17:21.960]  As Ben pointed out in his talk, it's okay to ask, you know, what type of model do you use?
[17:21.960 --> 17:27.740]  You don't have to know a lot, but asking this helps kind of force them to tell you the data flow.
[17:27.980 --> 17:32.480]  Most importantly is kind of this fourth question, how does a model make decisions?
[17:32.480 --> 17:45.100]  So I see this a lot in security tools, where they say we make decisions about, you know, incidents or how to flag an anomaly, and we have like these, you know, I've heard them say we have 60 features we look at.
[17:45.220 --> 17:53.780]  When it boils down to it, the reality is many tools actually just use two to three key features, even when they have a ton of data they're training on.
[17:53.780 --> 18:03.080]  Some highly complex precision AI tools generally use two, three, four features, not 50 or 60.
[18:03.100 --> 18:12.120]  So that's a good way to kind of suss out from them something about the model, and it encourages them to tell you more as your CIO is buying it.
[18:13.220 --> 18:16.980]  So I'll talk a little bit about some trends that I'm seeing.
[18:17.580 --> 18:27.280]  So again, this is evolving very quickly, and what I've seen over the past couple years are tools, models, and research around multimodal AI.
[18:27.280 --> 18:28.680]  You've probably seen this.
[18:28.680 --> 18:32.300]  It's a way that combines images, voice, and text.
[18:32.360 --> 18:36.840]  So, you know, generating text description from an image.
[18:37.440 --> 18:39.140]  I mean, these tools are very popular.
[18:39.140 --> 18:42.500]  One of the big uses increasingly is medical diagnostics.
[18:42.500 --> 18:51.160]  So a doctor will feed it, you know, your CAT scan or your radiology scan and all your medical records and say, help me do some diagnostics.
[18:51.700 --> 18:56.240]  So fields like that where you can combine kind of things that are all relevant.
[18:56.780 --> 18:59.040]  There's just, there are more tools in this area.
[18:59.040 --> 19:01.540]  And these are some of the models that are used.
[19:01.540 --> 19:03.080]  GPT just came up, yeah.
[19:10.940 --> 19:12.700]  So I tried to grab a picture.
[19:12.700 --> 19:14.160]  So yeah, so those are the three.
[19:14.160 --> 19:19.580]  So let's say the modalities are text, video, and images.
[19:22.660 --> 19:26.380]  Yeah, so they can either, you know, there's a few ways to process it.
[19:26.380 --> 19:33.700]  They can process, in this case, they process each mode independently, and then it's combined, then it's fused together.
[19:33.700 --> 19:36.120]  That fusion can be done earlier on.
[19:36.140 --> 19:41.720]  There could be unimodal where it says, okay, these things are similar, so we're gonna process them all together.
[19:41.800 --> 19:47.340]  Generally, most will process text, data, video separately and then fuse them together.
[19:48.120 --> 19:50.720]  Yes, it's, yeah.
[19:54.080 --> 19:58.060]  Yeah, so that's in general what, how they work.
[19:58.060 --> 20:08.360]  So this is, you know, and again, this is another great place to ask about what data feeds it, you know, how do you get the data, just all these questions, but realize this, this is coming more and more with all sorts of applications.
[20:08.360 --> 20:13.800]  I'm very rarely seeing, you know, kind of people buying one kind of purpose-driven application.
[20:13.800 --> 20:18.120]  It generally tends to be text, video, or text, speech maybe.
[20:23.510 --> 20:24.570]  That's right.
[20:37.680 --> 20:48.920]  So I think what I see is as people try to do foundation, so there are people trying to do foundation models for multimodal, particularly like the medical people or, you know, biomedical community.
[20:49.120 --> 20:54.040]  And I think that the security issues I see are mostly around what I would call trustworthiness.
[20:54.140 --> 20:59.240]  So technically, they can take a bunch of data sets, process it independently, and have some results.
[20:59.240 --> 21:00.620]  The issues with that are twofold.
[21:00.620 --> 21:12.500]  One, kind of understanding if you don't have ground truth for the output of it, so you want to do, and I have a great slide about this, so you want to do something that combines these three and spits out a diagnosis about you.
[21:12.500 --> 21:19.020]  You don't know what the ground truth is, though, so it's very hard to say, is that a trustworthy decision, saying you have cancer or whatever.
[21:19.360 --> 21:21.760]  So that's one issue.
[21:21.840 --> 21:24.300]  The second, though, is with the training data.
[21:24.300 --> 21:29.500]  So understanding kind of the limits of the different training data and then the fusion.
[21:29.500 --> 21:41.140]  So at each level, that becomes more a question of, not that it's insecure, but how trustworthy is the data that's collected, the data sets, and then the algorithm that puts them together.
[21:41.140 --> 21:42.840]  Is it grabbing the right features?
[21:42.840 --> 21:50.620]  So that's another kind of whole line of inquiry, is like, of these features of this video, are those the right ones to combine with text?
[21:50.720 --> 21:52.300]  That's kind of what I've seen.
[21:53.700 --> 22:04.980]  And then there's some other kind of interesting things in that realm of, especially with some of the LLMs, is like, you know, if you want to detect certain types of...
[22:05.940 --> 22:14.980]  a certain type of doctor's diagnosis, you might also have in there a model about... something about stylometry, that, you know, it follows a certain kind of pattern.
[22:14.980 --> 22:16.440]  So I don't know if I'm going to trust that because Dr.
[22:16.440 --> 22:18.140]  So-and-so didn't make a great diagnosis.
[22:18.140 --> 22:21.100]  So building in more of those things in addition.
[22:23.400 --> 22:30.820]  Probably all... another big usage that probably most people have seen, at least, or heard about is synthetic media scams.
[22:30.820 --> 22:32.020]  So I see this an awful lot.
[22:32.020 --> 22:33.560]  And the reason I bring this up is...
[22:34.480 --> 22:36.040]  so I work in the deepfake space.
[22:36.040 --> 22:37.980]  I was one of the creators of Deepfake-O-Meter.
[22:38.640 --> 22:42.260]  People ask, you know, is there any foolproof kind of way to detect these deepfakes?
[22:42.260 --> 22:43.440]  There's really not.
[22:43.580 --> 22:53.100]  And so what I've seen, the issues are, again, working in finance is, it is very easy to evade, you know, what they do in finance, which is that you've got to know your customer.
[22:53.100 --> 23:01.580]  So you'll either submit a video or a picture, or, you know, it was only a couple years ago, you would say, here's today's paper, here's my ID, here's my picture.
[23:01.780 --> 23:06.320]  It is super easy to evade those, even the video ones.
[23:06.320 --> 23:23.320]  And there have been numerous reports you can look up one that occurred in Hong Kong just a year ago, where the scammers were impersonating the CFO, telling some of the finance department, I want you to transfer, it was the equivalent of 20 million US dollars.
[23:23.320 --> 23:25.760]  The person, of course, said, I think I'm being socially engineered.
[23:25.860 --> 23:28.060]  No way, I'm not going to respond to this.
[23:28.100 --> 23:29.920]  They said, well, let's hop on a video call.
[23:29.980 --> 23:40.080]  Had a video call with a deepfake that looked just like the CFO, a deepfake that looked just like the head of finance, a deepfake that looked like the head of HR.
[23:40.080 --> 23:44.260]  The person said, well, okay, I'm on video, they're chatting with me, that's the CFO.
[23:44.260 --> 23:46.120]  The CFO says transfer the money.
[23:46.120 --> 23:48.780]  20 million dollars transferred, gone.
[23:49.100 --> 23:56.460]  And that is one of many examples that are, and some of you just don't hear about, but the video's becoming so good.
[23:56.460 --> 24:00.560]  And probably you've seen where people are interviewing candidates that are deepfakes.
[24:00.560 --> 24:10.260]  And they are very realistic, because the processing power is such that it used to be you'd create a video, there'd be some delay, or it wouldn't look quite right.
[24:10.260 --> 24:17.720]  There's still some indicators that I'll talk about, but essentially you can do this in real time, and the tools are out there, where you can get scammed very easy.
[24:17.720 --> 24:22.540]  So that's where I say to security teams, some people ask, you know, are there great detectors for this?
[24:22.900 --> 24:28.720]  A secret between us here, and hopefully it won't be recorded to show elsewhere, is some of the commercial tools just don't work.
[24:28.720 --> 24:35.280]  So some of the very big name companies have tools out there, I won't mention their names, they just don't work.
[24:35.280 --> 24:39.000]  They cannot ever tell you 100%, and they're just not that good.
[24:39.460 --> 24:49.820]  So Deepfake-O-Meter, which ours is open source, it's not that it's better, but what we'll give you is a confidence level, to say, you know, if the CFO, you know, the CISO or somebody says, is that a deepfake candidate?
[24:50.100 --> 24:55.220]  You know, I don't know, but it's 98% confidence level that it's a fake.
[24:57.300 --> 25:06.580]  So again, yeah, and what the tools look for, and a lot of people probably know this by now, the tools in general look at videos, images for things like pixel changes.
[25:06.580 --> 25:09.960]  It used to be very easy to find errors.
[25:10.180 --> 25:12.440]  You know, the blood flow, does it look like there's blood here?
[25:12.440 --> 25:14.120]  The eyes, the hands.
[25:15.140 --> 25:23.340]  They will look in error level analysis, you'll detect kind of artifacts when you compress pixels, which is the edges are kind of distorted, you know, in fake images.
[25:23.340 --> 25:32.720]  So if you look at somebody who's a deepfake, the background might be crisp, but the person might be just like slightly distorted, because you can't quite smooth out the edges to look like a human.
[25:32.720 --> 25:37.320]  So those are kind of indicators that it's very hard for the human eye now.
[25:37.380 --> 25:40.820]  You might say something's off with my Spidey sense, I can't figure it out.
[25:40.840 --> 25:48.960]  But some of the tools will have at least a confidence that, you know, based on the pixels and how they look, I don't think that's a real person.
[25:49.740 --> 26:00.720]  And again, in text, these are becoming just much better, and some of the judges yesterday at that conference I was at were saying, you know, how do I know, is there a tool that shows me whether this text was generated by AI?
[26:00.720 --> 26:03.480]  And of course, teachers will look for this for a while.
[26:04.100 --> 26:13.320]  These tools have become very good where you create something with AI and then you put it back in a tool to say, make sure you hide the fact that it was generated by AI.
[26:13.340 --> 26:19.780]  So what they look at in those cases, again, the stylometry, are there certain words that AI uses?
[26:19.780 --> 26:27.540]  You know, therefore, you know, however, whatever the fancy words are that AI tends to use, is there a certain stylometry?
[26:27.540 --> 26:31.020]  The sentences are a certain length or a certain cadence or a certain tone?
[26:31.500 --> 26:34.700]  And those are some of the methods they'll use to detect it.
[26:36.140 --> 26:39.000]  So just, I guess, a little more about data protection.
[26:39.000 --> 26:50.000]  Again, always important in companies, but even more so when you are, if you're setting up your own LLM, you're going to the cloud, if you're doing something with an AI tool, kind of really think it hard about protecting your data.
[26:50.000 --> 26:53.940]  Some trends that are coming around.
[26:53.940 --> 26:58.580]  So this is a picture in the upper right of Grace Hopper chip, H100s.
[26:58.580 --> 27:11.740]  We have some of these where I am, they're in high demand, because it is a hardware-based way to do confidential computing, meaning the data's encrypted at rest, in transit, and it's at the CPU level that it is encrypted.
[27:12.100 --> 27:20.200]  So for some people, and these are getting cheaper as they become more popular, but a really great way for hardware protection for data.
[27:21.180 --> 27:31.280]  Trusted execution environments have been around for a while, Intel SGX, where even a cloud provider can't see the data, but now we're seeing with the chips, that eventually we hope will get cheaper.
[27:31.300 --> 27:34.920]  This is encrypted at the CPU level, which, you know, then you don't have to worry as much.
[27:35.180 --> 27:37.980]  The newest confidential computing is not hardware-based.
[27:38.100 --> 27:44.020]  I think AWS has, the name escapes me, AWS has a version of this.
[27:44.140 --> 27:48.220]  It is not hardware-based, it is very expensive, and kind of, there's a lot of overhead.
[27:49.500 --> 28:02.900]  Some of the use cases, you might say, for my company, this is kind of, you know, this is overblown, but some of the use cases, places like pharmaceutical companies, or even smaller companies that want to share data, even if you're sharing threat intelligence data,
[28:02.900 --> 28:12.420]  or sharing data with other companies, it used to be you'd have to use multi-party computation, or kind of really heavyweight applications, and cryptography.
[28:13.120 --> 28:21.260]  More and more, as people are doing things, in either competition with competitors, or other firms, even finance firms, they want to share things.
[28:21.500 --> 28:38.100]  Confidential computing is a great way to do it, because you don't have to worry about all the, kind of, rolling this out, rolling crypto, you can roll it out on these processors, which are all the H100s, and confidential computing is ubiquitous at all cloud providers.
[28:38.100 --> 28:45.680]  It's a little more expensive, but you can go to AWS and set up your VPC in a confidential computing environment, if you need that.
[28:46.640 --> 28:48.880]  I'll just talk about the last few slides.
[28:48.880 --> 28:50.640]  I work in academia.
[28:51.300 --> 29:08.040]  I've seen AI a lot in scientific discovery, so, of course, it's all sorts of things, like synthetic data for patients, which is hit or miss, but as people do drug trials, they will use AI to create synthetic patients.
[29:08.060 --> 29:17.640]  Again, talking to the lawyers yesterday, they will use AI to construct a potential jury, and say, like, here's my AI jury, now I'm going to ask them questions.
[29:17.640 --> 29:23.280]  So, like, here's your typical persona of these five people, let me ask you questions.
[29:23.280 --> 29:26.060]  But in scientific discovery, we're just seeing it an awful lot.
[29:26.060 --> 29:37.180]  For example, one of my collaborators at Kentucky, this middle one, there was this charred ancient scroll, and you could not unravel it, or else it would crumble.
[29:37.180 --> 29:53.900]  They used a CT scan to get pictures, and then there was some AI, along with historians and natural language processing experts, to be able to say, okay, the scroll's rolled up, and these words are in, you know, ancient Sumerian or whatever, but also they're folded upon each other.
[29:53.900 --> 30:01.540]  So using the same kind of techniques as default for protein folding, they were able to say, if I was to unravel this scroll, here's what it would say.
[30:01.540 --> 30:05.740]  So we're seeing just a lot more of this in science.
[30:05.820 --> 30:16.920]  Some of the challenges, you know, for us that might be challenges for you all in your work, is science instruments , workflows, have a lot of machine learning models.
[30:16.940 --> 30:20.840]  Some that people don't even know are machine learning, but it's just been around for so long.
[30:20.840 --> 30:23.800]  And mostly it tends to be open source software.
[30:24.020 --> 30:25.340]  So people create these.
[30:25.340 --> 30:26.700]  They grab libraries off the internet.
[30:26.700 --> 30:27.520]  They make their software.
[30:27.520 --> 30:28.800]  There's not a whole lot of controls.
[30:28.800 --> 30:29.520]  Do what you want.
[30:29.520 --> 30:30.940]  Do what the best thing is.
[30:31.180 --> 30:34.280]  Make sure you do your secure CID CD pipeline.
[30:34.280 --> 30:35.540]  We're not going to really check.
[30:35.580 --> 30:36.680]  You know, it's good enough.
[30:37.660 --> 30:41.100]  Now, as we have more gen AI and multimodal, it's more complex.
[30:41.100 --> 30:47.400]  You know, that kind of those three modes, it's more complex to secure the pipeline, bigger threat surface.
[30:47.500 --> 30:52.120]  And some of the worries are, you know, integrity, not so much security, but integrity of the results.
[30:52.120 --> 30:53.280]  And it's not glamorous.
[30:54.560 --> 30:55.620]  So why do we care?
[30:55.620 --> 30:59.740]  Here's some actual kind of, you know, when I talk to people and say, here's some tangible things.
[31:00.180 --> 31:01.660]  Poison training data.
[31:01.740 --> 31:05.240]  You know, if you think about the threat model, you know, oftentimes we do threat modeling.
[31:05.240 --> 31:06.740]  We hear, like, who cares about this?
[31:06.740 --> 31:16.140]  Well, you know, for some companies, poisoning the training data and making bad decisions, you know, benefits the attacker such that they are going to poison that training data.
[31:16.280 --> 31:22.380]  In the case of, you know, somebody studying climate change, you know, it can bias the model and bias the decision.
[31:22.760 --> 31:27.260]  Extracting the training data, particularly with things that are PII, can breach privacy.
[31:27.260 --> 31:34.740]  If I know whose information is in the model, I can therefore, that's a big privacy violation.
[31:34.880 --> 31:43.800]  Of course, misclassifying objects, jailbreaking, like we do with our Gandalf example, bypassing the safety guard rails, and then the prompt injection attacks.
[31:46.080 --> 32:00.580]  This, again, just because I think it's a somewhat frightening example that, you know, people like us in security have to kind of put the clamp on is, and you may see this in your fields, is there is a lot more, like, here's some cool AI, I'm going to feed it a bunch of information,
[32:00.580 --> 32:04.240]  and it's going to do science or it's going to make decisions for me.
[32:04.560 --> 32:09.560]  So, it enables LLMs to conduct scientific research autonomously.
[32:09.680 --> 32:12.600]  And I see this, not just in science, all over.
[32:12.840 --> 32:20.060]  Give me a bunch of data, give me some, you know, my multimodal model, give me data, text, video, you know, it's going to spit out something interesting.
[32:20.440 --> 32:36.040]  So, having the ability to kind of circle back, having the ability to not necessarily be an expert, but to say, what are the models you're using, what's the data you're basing this on so that if it spits out some decision autonomously, and you hear this a lot with tools that are on agentic AI,
[32:36.040 --> 32:38.120]  what is that decision based on?
[32:39.680 --> 32:45.480]  Kind of to wrap up the last couple slides, of course, like with security, everywhere there's tools, there's frameworks.
[32:45.480 --> 32:48.240]  These, I think, are useful tools and frameworks.
[32:48.560 --> 32:52.660]  Counterfeit is a tool that was open sourced by Microsoft.
[32:52.700 --> 32:53.960]  They can use that.
[32:54.080 --> 32:55.960]  It's fairly easy to use.
[32:56.540 --> 32:59.280]  Google has a secure AI framework.
[32:59.880 --> 33:16.020]  OWASP has now, I think Gavin was part of this, they have their Gen AI security project, so for better or for worse, they kind of sum up some of the attacks pretty nicely, and they have a nice blog with recent attacks on, you know, like a top 10 list for LLMs and Gen AI.
[33:16.560 --> 33:19.000]  They have some red teaming evaluation.
[33:19.000 --> 33:25.420]  They have a lot of really interesting kind of incidents to kind of sum up where the attack occurred and how it occurred.
[33:25.560 --> 33:28.180]  Their Gen AI project is quite nice.
[33:28.660 --> 33:41.560]  MITRE Atlas, if you're a fan, maybe your CISOs are a fan of MITRE, so there's MITRE Atlas, which is a particular spin on your traditional MITRE knowledge base of TTPs, particularly to AI.
[33:42.540 --> 33:45.340]  It's useful for categorizing things.
[33:45.840 --> 33:56.520]  They have particular exploits on machine learning model access, initial access, so if you're a fan of the MITRE framework, it's not a bad way to think about how AI security is different .
[33:56.980 --> 33:58.560]  Finally, some standards.
[33:59.460 --> 34:00.800]  Things to point out.
[34:00.800 --> 34:03.440]  NIST, of course, has an AI risk management framework.
[34:03.700 --> 34:12.400]  If you work in the medical area, there is still very much a lack of certification for the machine learning or AI, but this is still around.
[34:12.400 --> 34:16.700]  This FDI AI machine learning software is a medical device action plan.
[34:16.820 --> 34:19.620]  I have friends that work in the medical device industry.
[34:20.140 --> 34:31.600]  FDA has nothing that says you are a certified machine learning, you're not going to kill anybody, so they don't have a whole lot that's tangible, but this action plan at least guides them for what to think about.
[34:31.600 --> 34:43.180]  In general, ISO, some of these are pretty nice to kind of skim through because they give you some ideas on what it means in a very tangible way to do AI security and think about model security.
[34:44.800 --> 34:45.620]  That's it.
[34:45.620 --> 34:47.300]  This is my basic presentation.
[34:47.300 --> 34:57.880]  Happy to answer questions or my colleagues can answer some or you probably had enough AI security and I won't be offended if you go to lunch.
[34:57.880 --> 34:58.320]  Yeah.
[35:32.000 --> 35:32.880]  Sure.
[35:32.960 --> 35:43.140]  I mean, if you're going to do your own on-site, you can... I mean , I'll just use the example because I have a friend who just did DeepSeq on-site just because they wanted to see if that was a good one for them.
[35:43.140 --> 35:59.620]  So you can grab an open source model as it is now and you can run it internally, but you assume it's been trained to a certain point, so the way it's been trained is good enough for your usage now and you can download that and then you can add the functionality.
[35:59.620 --> 36:00.960]  What was your first question?
[36:10.560 --> 36:12.980]  Oh, like training your own LLM.
[36:12.980 --> 36:19.900]  No, it's taking... it's using DeepSeq or LLM or something and training it for...
[36:19.900 --> 36:25.300]  so you deploy it, it's not connected to the internet, but then you can train it on your own data.
[36:25.300 --> 36:36.800]  So in finance they do this a lot where they will train it on the data that's relevant to their company, their usage and their data sets and you are not then giving it to DeepSeq or LLM.
[36:36.840 --> 36:39.660]  So you take it and then you train it internally.
[36:42.770 --> 36:43.470]  Yeah.
[37:30.190 --> 37:36.890]  So rather than training your own from scratch, you essentially take it and then you modify it, you train it up to what you need it for.
[37:46.070 --> 37:50.070]  And I've seen LLM used as probably the most popular roll your own internal.
[37:50.070 --> 37:52.910]  I mean a lot of people it's fairly easy to do that internally.
[37:56.330 --> 37:57.130]  Any other...
[37:57.130 --> 37:59.270]  Thank you for your time and attention.
[38:10.210 --> 38:12.910]  Thank you for your time and attention.
[38:12.910 --> 38:13.950]  Thank you for your time and attention.