[Music]
thank you
[Music]
all right
foreign
[Applause]
holy [ __ ] two years two years ago
two years ago I'm talking to Michael
about doing exactly this coming down
here
just having a blast with all my hacker
buds here in the 4-14 where I grew up
where I bet most of you grew up I'm a
lifer here in Wisconsin it is super
exciting to be here in Milwaukee
I'm going
[ __ ] I've gone more countries I want to
think about to do talks and things and
this is the first time I'm doing one
here in Milwaukee I am so excited so are
we doing well you guys good yeah where
are we
so I mentioned two years ago two years
ago I was supposed to be here with all
of you we were supposed to be doing this
party we were supposed to be doing it
right
right before they shut everything down
I was out in San Francisco this was the
last trip I did
before they shut us all down
and I'm out in San Francisco and I'm
talking about the same thing we're going
to talk about today deep face
and you know I'm up there I'm telling
them I'm like let's not even talk about
election security screw that you know I
I everybody was talking about election
security it I'm talking to them about
hey here's how deep fakes are a threat
to your business here's how deep fakes
are a threat to people this is where
they're a problem
and
yeah
that hasn't changed but boy you know we
we saw a lot of things all these scary
headlines said oh my God deep face are
going to be the worst thing for the 2020
election oh gosh you know what will the
impact be to our elections deep fakes
threaten the 2020 election this was our
fear and it was legitimate right I mean
deep fakes are there's this crazy thing
that we can create these videos that
people look at and they're like
wait is is that really the president I
can't tell right it's not just videos
it's audio
you probably heard some stories we'll
talk about that in a little bit
it's
even just pictures
people are using images from sites like
this person doesn't exist
you know what I'm talking about right
this is what we're doing with deep
facing so it made sense that we were
worried
it made sense that we were scared one of
the most well-known deep fake videos was
that one that Jordan Peele did of Barack
Obama
so of course everybody thought about
politics right away
but what did we get
well I only got Mark Zuckerberg that was
a fun one we got a bunch of these tick
tocks of Tom Cruise like Wow way to make
an irrelevant do relevant again
okay even if you go out you'll find a
ton of them from the office I don't know
why that one is such a popular Target
but my God there's tons of them
but where's my election security problem
I was told wait we're we're not good our
elections are going to be overrun by
this
and it didn't happen
and so when Michael talked to me about
come in
decipher account 5.3
to do the talk that I was gonna do at
5.05.1 there's something wrong in the
math there but okay
whatever we're calling that thing that
didn't happen in 2020
I was like yeah let me let's do that but
if I do that talk now it's a whole
different ball game
because all of the things that people
were scared about we just haven't seen
so maybe we backtrack a little let's
look at this from a different
perspective
and so that's what we're gonna do we're
going to talk about deep face
for the anybody who's been living under
Iraq for the last you know know five
years and that isn't sure what depicts
really are
deep things are nothing more than ultra
realistic media that's created using
deep learning all the rhythms or machine
learning algorithms take your pick
whatever you want to call them
deep face itself was an attorney I was
coined just kind of by someone on Reddit
and we'll talk about that in a minute
but let's go back to the beginning of
this thing where did deep fake start
the technology for deep breaks is better
on for decades okay I mean it was 1997
that there was a paper released by three
people who now all apparently work for
Google shocking uh Kristoff gregler
Michelle Colville and yes I am reading
them because there's no way I was
remembering these names and Malcolm Stan
family so they did this paper in 1997
[Music]
just talking about how they could use
this
um
is there a tool to go to go and create
these videos that weren't real
and one of the examples you see up here
JFK why not right so they did a couple
they did like a Forrest Gump video with
him and a couple others and you know
they called their tool video bright
because that's what it was going to do
it was going to rewrite videos
but it didn't really go anywhere I think
part of the problem was we just didn't
really have other than in you know
really large academic research
institutions we just didn't have the
horsepower to do this we were not ready
the way we are now
so 2017
let's fast forward two decades
2017 is when deep face really started to
appear on the scene
so a user on Reddit named aptly deep
fake
starts posting these videos
and of course where does any good
technology always seem to take hold
first where tell me
porn of course it's always porn right
like where did VHS land okay it blew out
betamax why because porn
the internet what took off the most
first in the internet it was porn it's
always porn so what was he doing he was
creating porn
he was creating deep fake porn media of
you know Hollywood celebrities like Emma
Watson
or Gal Gadot
or Natalie Dormer
or even Natalie Portman
these were some of the earliest ones
now
I don't recommend you do this from your
work computer first and foremost
if you go out and you search like
Natalie Dormer deep fake
you're gonna find out they're still
making this stuff
so right there you can already start to
see like yeah deep fix they may not have
exploded on the scene when we were
talking election security but hey guess
what
they're still doing damage to people
now it's really ironic here is um
there's been some legislation in the
last you know three or four years funny
this when you know politicians were
getting scared suddenly they wanted
legislation to deal with deep things
prior to that these people were actually
using the very thing you would think you
would use to get these taken down
to defend keeping them back up
he it's amazing he used the dmca to
protect
the Deep fix that they created using
other people's likeness
so that's changing but let's talk about
how these things get created in the
first place because
it's going to give you some under
understanding of where we are today and
why maybe we didn't see what we thought
we would so how do you create a team
fate well deep fakes are simply created
with these things we call Gans
all right anybody who works in machine
learning and artificial intelligence you
know what again is
anyone who doesn't let me quickly
explain it and for those of you that do
work in machine learning in your your
heads down in tensorflow forgive me I'm
oversimplifying I know this is not a
hundred percent technically accurate but
it makes it way easier to understand so
again generative adversarial Network
listen to that name because it tells you
exactly what it does it's going to
create something using two networks that
work as adversaries so you have these
two networks the first is called the
generator
well that's an original name because
that's exactly what it does it's going
to just generate stuff the other side of
this thing is the discriminator the
discriminator's job is to look at what
the generator creates and decide
Is that real or fake
or how real is that
more to the point
so we train these things
we go out and this is traditional now
this is I I'm sitting here talking like
this was like a decade ago and this was
like four years this is 40 Years of
advancement right so four years ago this
is what we were doing with these things
you had a training set you got a bunch
of images of the person you were going
to try to put in your deep fake in this
case Barack Obama
sure why not
and you know we use these images so that
we can train those the discriminator and
the generator now there's this is one of
those areas where it's not totally
technically accurate how that actually
goes but
bear with me
so then we feed it some Target Media in
this case it's a it's an actor who sat
down and pretended to be Barack Obama in
the old wall office giving an address
we feed that through the generator digs
through it literally frame by frame
and it attempts to to recreate
that frame
only with the other face that it's been
trained to recreate
so just go in here looks at that says
hmm how realistic is that uh well you
know what not so much so it sends
feedback to the generator to say yeah
you're not there yet try again
and this cycle repeats and repeats and
repeats and repeats and repeats
so you can see why computing power is
important here right
so let's dig a Little
Pony deeper right I know they don't get
any better y'all I'm sorry jokes as as
I've been told many times by my partner
I am not funny
um and I'm okay with this so I actually
sat down a couple years ago and built a
rig to do this and I started playing
with it so we have this thing we call
the ground truth right that's kind of
that's where we start that's that's the
image of something that represents
something that the the generator wants
to recreate
so the way that we train these things is
we feed this ground truth into what we
call an encoder now the encoder is going
to use facial recognition to find all
the various points on your face and map
those out and you know other elements of
like colors and whatever
and it puts it into you know basically
this encoding that then the generator is
going to use to try to recreate it so we
have this decoder and the decoder's job
is to take that encoder and try to
recreate what was created by the encoder
we get some result okay cool yeah all
right that they look very similar
so now that goes to the discriminator
and as I said the discriminator takes a
look at the ground truth and what was
generated and says hey yes or no this is
good
and then it sends that feedback in the
form of what we call weights weights is
kind of how it adjusts again and way
over simplifying but we could spend I
could do a dissertation on this stuff
and you'd be bored to death so uh
ultimately that ends up in our model and
we Loop through this again now this is
where it gets kind of cool
so now I'm going to do this with myself
and Melissa Milano and this is I mean
all these images are actually from the
training I did and so I'm going to feed
both of those to the encoder so now I'm
teaching the encoder how to create not
one but two faces so it's looking at
them and it's deciding okay you know
it's doing all the encoding like it's
supposed to do for both of those faces
and then I'm going to hand those to two
different decoders decoder a is there to
try to make my face decoder B is there
to try to make Alyssa Milano's face we
go through that process we we teach that
thing we go through the learning and you
know our discriminator is constantly
updating that encoder getting it better
and better and better until we reach a
point where we're kind of like all right
good enough
that's where we want to be
and then all we do this is the
Brilliance in this thing
so I just take that
to convert this to the Deep fake video
all I'm going to do is I'm going to take
that ground truth feed it to the encoder
but now I'm going to switch things up
and I'm going to feed what was just
encoded to the decoder
and that's going to create my new frame
with the new face
now again so maybe this is this is where
we were four years ago right this is how
we did these things and the software I
used for this was one of the most
popular still around people are still
using it today because there's a lot of
research happening here let's face swap
but then we got more advanced you see
because that takes a lot of time it
takes a lot of horsepower yeah people
were doing it in gcp and whatever but it
but it still is it
so we started to researchers for
counseling organized like what if we
didn't try to like create these encoders
and train them on specific faces what if
our encoders were smarter what if they
could look at just the general
characteristics of a face and build a
model that could then be used to
generate another phase
and we got into what the the research
calls us talking heads and it's really
just more of a metal learning
environment where instead of trying to
you know find the specific points on
these two separate faces and teach a
model and and do all that learning it's
hey let's focus more on the The Meta
characteristics of someone's face the
color patterns and you know facial
points and whatever else and we can skip
a bunch of that learning now the
challenge is
you kind of have to have
a specific Target
so what you gain in speed you're kind of
given up in terms of who you can create
these for but it makes it repeatable so
now I could take a hundred different
faces in this room pass them through the
same model and very quickly after I've
trained the model one time create
multiple deep fakes of all of your faces
replaced with one person of my choosing
so what made things faster and this is
where we started seeing now all of these
apps right like I mean if you go you see
like face app does some of this not so
well there's a few of them out there
that you can download and then somewhere
on your phone because of course they're
ultimately feeding that all back to the
cloud and who knows whose Cloud that is
or where that's going
or what we're doing with a lot of facial
recognition data we just uploaded
so
consider that but that
they went and
and it continues to go it can keeps
getting better and better so okay
one of the more uh early ones that
adopted this was avatarify this one was
cool because it was a zoom
uh this gentleman in the upper right
here supposedly surprised A co-worker by
pretending to bring Elon Musk into a
call
yeah I know really
really a Elon Musk right
oh Twitter R.I.P Twitter
okay good point they're okay yeah
granted there's no popular time to bring
up Eli musk Fair
but where where were those election
depicts I was promised deep fakes I was
going to see candidates doing all sorts
of wild stuff I wanted to see Joe Biden
doing blow why didn't I get that video
see there we go see I told you I could
be funny once in a while
but seriously so why didn't this happen
everybody said this was going to be I
mean my colleagues I even bought into it
I figured it was going to happen I just
figured there were enough other people
talking about it I didn't need to
region why not
what we got instead of everybody was
like this by the way this is a really
cool video If you go look it up
um it was actually a get out and vote
video
so I mean I guess we used it for good it
impacted the 2020 election maybe a
little bit in some way but definitely
not what we were expecting and so the
question kind of comes up all right was
this a false alarm did like we do the
thing that we do in cyber security
really super excited and create all this
mess and get everybody really oh my god
look out it's gonna happen we're gonna
have deep face tomorrow
when there really wasn't much to worry
about
or
this is like Y2K
where we got everybody up in arms and we
we you know spent thousands and
thousands and thousands of millions of
dollars on consultants and oh my God
Cobalt programmers were cool again
hi now I know who is in
it and Y2K because you're laughing you
know what I'm talking about right we
Y2K there's there's I mean my God it
kills me if there's journalists out
there right now who are trying to say
Y2K was no big deal and then we over
blew it
no we just actually stirred up enough
excitement and got people enough
involved that we actually stopped it so
is this you know I started a political
thing someone's going to come after me
later for that but
the question remains with deep face
which is it
in the reality
it's kind of a little bit of both
so let's dig into that a little bit so
first of all
does anyone remember this video from
right around a little bit before the
2020 election
so Nancy Pelosi was at some event I
don't even remember what the event was
it's not important she gave some
interview
and you know some people on the opposite
side of the aisle if you will got their
hands on the video and what did they do
they slowed that video way down and it
may and by doing so it made her look
like she was drunk right because when
you do that with a video what happens
things distort
normal speech starts to sound really
Disturbed and slurred right I mean
that's what happens so who needs deep
face when you can just create a video
like that so they did that this video
millions and millions of re-shares even
after it was all over the news that this
video was faked it was a modified doctor
video people were still sharing it they
were still up in arms oh my God look at
how unprofessional she is she's an
alcoholic all this stuff
why do I need to spend time
and compute dollars if I'm using the
cloud to create deep fakes when all you
got to do is take a video and slow it
down
so the reality is here when it comes to
why didn't we see this why didn't we see
more of this well some of it is we just
didn't need it
I mean my God we had politicians
standing on stage screaming fake news at
the top of their lungs
it didn't even matter what the news was
it was oh yeah that's fake news or you
know I mean
if you can do that
why bother with this
so that definitely plays into it
so one of the things I talked about when
I said I was out there in 2020 talking
about this one thing I was talking about
was what about Market manipulation
like let's think in the business world
what if I created a deep fake of a high
level figure you know maybe the
aforementioned new owner of Twitter
whoop I jumped ahead here somehow let's
go back
um
spoiler uh yeah so you know that was my
thought like and I talked about ways you
could like conduct
a fight in cyber trading things like
that and of course we didn't see much of
that either we've seen a little bit
there has been some attempts but not
really anything we have people building
businesses
on the idea that they would detect deep
fakes targeting your company
all right I met some of them at black
hat two years ago
fruition I'm not sure if that company
still around it'd be actually
interesting to go see instead
we did get
some World opinion manipulation just
recently right I mean
who had this one
on their deep fake Springfield card
with a nation
using deep
for wartime propaganda
how
did we not see that coming
well the reality is we kind of did
um this is probably one of my favorite
books of the last 10 years
Theresa Payton she was
um I'm not even going to try to say it
now because I'm spacing on her exact
title
but she wrote this book in 2020. now I
was mostly focused on Election security
but honestly if you can't go out and
pick up this book because this book does
an amazing job talking about the
concepts of misinformation and how it
actually works
because it's important to understand
that as we look at Deep fakes and we
look at their implications and things
like the world markets and we think
about how they could impact our politics
or they could be used in other horrible
ways ultimately all they are at their
core is misinformation
that's it
and what makes any psyops misinformation
style campaign successful are the same
things that make deep fake videos go
viral
so I recommend it if you if you have the
opportunity grab a copy of this book it
is awesome for understanding some of
that
but what about audio tax no that was
another thing I talked about
two years ago when I was given the talk
on deep face and you know at that point
there had been one case where one
company claimed at least that there uh I
think it was like a divisional president
got duped into transferring a couple
hundred thousand Euros to attackers and
the claim was that you know well they
use deep fake audio to to fit the CEO's
voice
I say that kind of sarcastically because
there's a lot of skepticism and looking
into the facts of it I'm not sure I
really buy it
but more recently we do have one that
does actually seem to be legitimate
partially because one of the reasons
that makes me feel like this might
actually be the case because it was
combined with business email compromise
now that sounds like the attackers we
know and love
so
it is out there but again it's not like
this is something that is blown up on
the scenes because again why would I
spend a bunch of time crafting emails
to not only just push somebody but then
you know create this deep fake audio
that I'm going to use to then try to
further that attack
I'm only going to do is send one
phishing email to some poor admin
assistant who isn't you know so busy
that they're not totally paying
attention and they click on it
so you know it's not that the attack
vectors disappear
it's not that the attack Vector was
overblown
it's simply that there are just so many
easier attack vectors out there right
now
that work and work consistently that
maybe this isn't the one that we need
so
here's the last piece of this there is a
ton of research going on two years ago
when we thought this thing was going to
crush our elections you had many of the
biggest names in the media Google
Facebook Twitter
Amazon my Microsoft all put in money
to sponsor this million dollar challenge
for researchers to come up with deep
fake detections
but we got a lot of them like we started
really simple in the upper left there
you see our favorite villain once again
Mr
Twitter now
but it was really easy you used to
originally all you had to do to tell a d
pick was you could kind of look at it
you could see distortions you could tell
that like things just didn't fit right
so you know within a couple months of
that research being released well okay
now we've made cans better so then we
moved down below and we see one of our
other favorite villains from the
political Spectrum
and
you know
now it was you know they figured out hey
the Deep fake algorithms aren't
replicating eye blinks very well because
think about you're training these things
off of pictures that you have of these
public figures how often do you see
pictures of public figures with their
eyes closed
not very often
so you know it was tough I but it didn't
take long literally the paper that I
pulled that picture from was released
and two months later we were already
seeing updated versions of the facial
recognition and the Gans in face swap
that were already reproducing eye blinks
at an appropriate rate and with
appropriate speeds and everything else I
mean they had this really it's really
impressive how they dig into some of the
stuff right so all right well then
someone had a great idea this was kind
of cool they said well what if we don't
try to detect the media at all
what if we actually look at what the
person in the media is doing and try to
match that up to the context
talking about
so they started mapping behaviors so we
hear see yet another political figure
villain depending on your how you sit
um but
what do they do they took hours and
hours of Barack Obama videos
and they started to map out his
behaviors they look for very specific
facial patterns and movements and other
things you know little ticks and things
that were common
to various types of content that he was
talking about you know so when he was
talking about something very serious but
you know where he needed to be more of
that compassionate voice you could see
it there were things about his face that
made it very clear and other times when
he was like you know delivering a really
strongly worded speech to North Korea
well yeah suddenly you know his browse
for all things like that
and so for a while this was really
accurate like really accurate we're
talking like 90 percent
only issue is
first of all to do it you have to have
that big database of every you know all
this different video of that subject so
you can actually learn those models
which then in turn means there's kind of
a limited set of people that you're
going to be able to create those models
for so I you know it was cool but maybe
not the answer
now the last one
this was really impressive this came out
about a year and a half
and in this study what they figured out
was they could actually detect and this
is the part that's really cool they
could actually detect your heartbeat
through the video
and what they found is that what they
were doing and I'm not going to get into
all the specifics because I barely
understand it so there's no way I'm
going to be able
to be able to y'all understand it
exactly but what you were able to do and
then what they found out is that in the
Deep fake videos those same markers that
indicated a heartbeat just simply
weren't there
so this was like 94 95 96 but already
deep fake creators have figured out ways
to not necessarily bypass this but
they're basically frustrate those
algorithms altogether
and break them
that's some hacker [ __ ] right there eh
so I mean and there's people still
looking I've seen maybe you've seen
pictures of it there's people looking at
how can I do my makeup in a certain way
that thwarts the facial recognition
algorithms that are needed for this
people are doing that to avoid it in
countries where that's really common not
that we would do it here in the us and
our airports or anything
but here's the thing these are all
Technical Solutions
as I said
this is a misinformation situation and
when we talk about misinformation
misinformation is not a technical
problem it is a human problem
so when we sit down and we look at this
and we say okay why didn't this impact
the election why are deep fakes not more
of a problem did we did we solve it
yeah someone this is the other side
remember I said okay it's kind of a
false alarm it's also kind of like Y2K
because the one thing we did really
really well
who's got the news out there these
things existed
now they went both ways they also kind
of worked against us because you had
politicians who were legitimately caught
on tape doing things who were claiming
all in all that's a deep fake that
wasn't me
well you know I guess that was
predictable but
how do we deal with this misinformation
because this is the problem
is we've seen now in the last few years
is that this
like once someone's buying into this
crap trying to get that out of their
heads is
insanely difficult
so the reason is because what
misinformation does it first of all it
plays off of our existing prejudices our
existing biases all the things that we
believe all the things that we want to
believe
and so it's like you know people will
actually be willing to just ignore every
sign of oh that's way wrong and there's
no way that's credible because I want to
believe it and we've seen that we've
seen a political discourse it's you know
we see that all over the place we see it
with pandemic information right
what we do when we're going to launch a
misinformation campaign
is we attack what we call logic changes
in our brain so when I'm processing
information and I'm looking at a cause
and effect there's this logical chain of
events that leads from what I see as the
cause to what I see is the effect
what I see as the evidence to what I see
as the truth
misinformation just lodges itself in the
middle of that logic chain and that
makes it really difficult to get rid of
it
our brains don't have that big undo
button that sometimes I think we all
wish we had
so what do we do about this and why
did I say that we got the news out and
that was helpful so when you're going to
try to debunk misinformation
the very first key here is to get the
warning out that it exists
okay because by getting this warning out
people are expecting it
now people know to be more skeptical
at least you hope they will be and in
many cases they do and this works again
this is this is stuff militaries and you
know uh intelligence agencies have been
doing for hundreds and hundreds of years
this is not new
so what we need to do is start first of
all just get that warning out but then
it's all right I need to somehow and
this is the challenging part now
I need to take that piece of information
that got stuck in your logic team and
I've got to replace it with something
else
it sounds like a simple concept but oh
my God think about this for a minute
you've got this thing that's embedded
it's something that somebody wanted to
believe because they hated this person
this politician so much
they just they wanted to believe it
regardless you're trying to take that
out of that logic chain and you have to
replace it with something else that is
equally credible to that person
and that's where we struggle
but then the last piece of it is you
have to keep repeating that because now
even if you've successfully gotten into
that logic and you've you've wiped out
what there was a piece of misinformation
that they were leveraging and you've
replaced it with the actual truth
you're not done because there's tags at
that all the time that you're trying to
afford and so constantly repeating that
credible truth which God that almost
sounds like alternate truth I can't
actually just believe I said credible
truth
but that's the reality because when I
say credible in that case I'm talking
about credible in their heads right
which actually for you and I who might
disagree with whoever it is that we're
talking about maybe to us it's not
credible
but it's the credible thing to them
and that's where this becomes truly
challenging this is the battle that
we're fighting and our piece of it from
a cyber security perspective even though
I know we're talking Tech and deep fakes
we're one piece of that
but we're not alone in that
so
before I wrap things up I wanted to
leave you with this because I think this
is something that really drives it home
it's very bezmannoff for those of you
that don't know as a former KGB agent
who's on the talk circuit he wrote books
everything else
but it's funny because as we've been
going through the last few years of this
pandemic and the political Spectrum I
keep thinking of this quote
talking about
what how misinformation functions what
the goals are
the Soviets back in the 80s when Yuri
was doing this they called it
ideological subversion I won't read the
whole quote to you hopefully you can
take a picture of it or you can read it
from where you're sitting but the gist
is basically we want to overload you
with so much conflicting information and
so much you know BS that you'll make
decisions well first of all you'll be so
wrong you want to know how to make a
decision but the decisions you end up
making actually are against your own
best interests
now if you think of what we've seen in
the last few years I think probably
everyone in this room regardless of your
opinions on any of these topics you can
think of cases where you've seen that
happen
so this is the reality of what we're
dealing with
there's lots of really cool things that
have happened with deep fakes right
there's a lot of great things
technology is insanely cool we're seeing
it used in movies
we're seeing people take things like the
Luke Skywalker scene from Mandalorian
and redo it and make it more realistic
than the CGI that they used to
originally create it like that is cool
if you haven't seen that one yet look it
up it is impressive
foreign
stuff but it all comes back to what do
we say all the time in security it's the
human factor right
that's where we're at that's where we're
at with depicts they continue to get
more and more impressive
like I said every mousetrap we build
just creates stronger mice
so kind of like every other aspect of
cyber security this isn't something
we're going to win it's something we're
going to continue to fight back and
forth back and forth so I encourage you
all to get in touch with me it's been
great being here like again I'm so
excited first of all that cybercon's
back second that I'm actually here in my
hometown for the first time in my life
giving a talk in my hometown
um you all have been amazing I can't
wait to see you around the rest of the
conference do connect with me on social
media whichever of your preference I'll
be on Twitter for as long as it
continues to be a viable option
not that I'm biased against Elon Musk or
anything I promise
so awesome why everybody Take Care thank
you again so much
[Applause]
right here
[Music]