[Music]
good morning it is an honor to be here
with you today I am stunned by how many
of you are here it's actually really
exciting you know as somebody who's been
looking at privacy for so long to see so
many people deeply invested in it and my
goal today is to give you a twisted way
of thinking about your profession and
the challenges you face so work with me
promote so privacy is about control most
people in the business of privacy think
that privacy is about the control of
data who should access data and under
what circumstances and tech companies
build privacy policies to circumscribe
access engineers build access control
structures people try to communicate
what privacy might mean to users and
regulators try to drill into what the
implications might be when companies
violate this yet time and time again
people come back and say what is wrong
with people don't they understand
privacy and you know people will talk
about how much they care about privacy
but everybody else complains they don't
seem to be doing it right and it's a
really funny thing because we call it
the privacy paradox right people's
attitudes compared to their actions and
I was found this sort of fascinating
because my life is spending time with
people trying to understand what it is
that they do and so for 10 years I spent
time in the trenches talking to
teenagers around this country as social
media was becoming more and more popular
trying to understand what they were
doing and how it in fits in our lives
and one of the things I learned really
quickly is that they do care about
privacy but they're speaking a very
different language than you are and that
disconnect has caused tremendous
problems now teenagers talk about their
desire for privacy but what do they mean
by it they actually also talk
simultaneously the same voice about
wanting to have access to public life
and that's where the contradiction seems
to occur because they want to share so
that they're part of everyday life and
they want to have privacy so what does
that look like my colleague Alice
Marwick and I spent all this time trying
to untangle that and we came to
something really important young people
aren't focused on the control of
information they're not Kisan the
control of access
they're focused on the control of a
social situation now that phrase might
seem a little weird right because when
it comes down to is that they want to
know that their interpersonal dynamics
will be managed appropriately by the
systems as they work through them
they're not really trying to think about
it in terms of the information they're
trying to think about what will unfold
how people will interact with them how
they can predict and understand what is
going to happen and that actually has
phenomenal implications for a lot of
what's going on today in particular
current crises regarding things like
media manipulation but let me first
unpack what it means to control a social
situation because that's actually a
really weird thing to think about so I
want you know dial back to middle school
right I know is your favorite time ever
right so think about that time think
about your friend Carol and you told
Carol something you told her not to tell
anyone right it was a secret you really
wanted to share with her but you really
wanted her not to spread that
information and of course what did she
do she turned around she started
gossiping and she spread it to all sorts
of other people and you were embarrassed
and you were horrified you were mad at
Carol the floor fell out from you
because it's terrible to be in middle
school and be shamed it was awful and
you felt like privacy was invaded trust
was violated but you also learned a
really interesting lesson you learned
that if you wanted something to spread
far and wide
you should tell Carol not to tell anyone
all right and that's a really
interesting lesson because what you've
done is taken a moment of complete
horror and turned it around to a moment
where you have power where you can
control the situation by understanding
the chessboard there is nothing more
empowering than being able to have that
control of a social situation to
understand how information will flow to
understand who will say what what
systems will work the difficulty is is
that in a land of technology having that
flow that understanding is extremely
difficult and for many people social
media is up ended the way in which they
can manage us control of a social
situation so everyone seems to complain
teeth like you'll hear these complaints
oh my gosh Facebook
violated my privacy and you're like what
happened and they're like they brought
this thing I posted 60 years ago back to
the present and Facebook's like you gave
us permission and people are like I
don't care that was really creepy
and that's the really important word in
all of this creepy well you hear over
and over again when people feel like
their privacy has been invaded in some
way is creepy and that will have nothing
to do with an access violation that will
have to do with how something doesn't
feel right your parents did something
that they shouldn't have done the social
norms were violated something went wrong
because somebody lost control of a
social situation so I wrote a book it's
complicated and in an ID tale in gory
detail along with other works from my
colleague Alice Marek what it took and
the strategies that young people took
during the rise of social media in order
to achieve privacy and it's a really
fascinating set of conversations and if
you really want to dive in I'm speaking
again on that topic at eleven o'clock
and there's a book where I can find it
for you and we can talk about it but I
actually want to connect the dots for
you of what that work is that I've been
doing for so long with a lot of the work
that I'm doing now because there was a
really interesting shift that happened
because the fact that we didn't take
teenagers privacy seriously has really
hurt us now and I want to explain what
that looks like because it started at
the margins and it is coming to hurt us
in significant ways right because what
happens when you don't give people
control over the social situation they
find a way to get control so I watched a
group of teenagers sort of on the
margins starting around 2004 who were
really frustrated by the fact that they
didn't have control over social
situations they couldn't stand social
media it made them feel really
destabilized and they wanted control and
like those of you who realized that the
key was to share information with Carol
so it would spread these teenagers
realize that the information landscape
created by social media gave them a new
opportunity to control a social
situation in a much more mischievous way
right and they started by creating memes
it involved Lowell Katz it was fun but
then they started hacking the attention
economy and it was funny at first I mean
after years of listening to Oprah
Winfrey and other TV people talking
obsessively about how the internet was
dangerous for teenagers the idea of
getting Oprah to share your message was
really funny let me explain what
happened she had been doing a series of
TV shows about how the internet was
filled with online sexual predators and
these teenagers were annoyed because
their parents were actually going after
them and telling them the Internet was
unsafe
so they started writing things on her
various bulletin boards trying to
contact very various people who work on
her show and basically they engaged in a
game that we would now call trolling
right because they trolled Oprah's folks
so much so that she spent a half an hour
on TV dedicated to a whole culture of
online predators that had an icon known
as pedobear and that she talked about in
on TV about all of these guys who were
talking in her message boards about how
they were coming to harm children and
she specifically quoted one who says he
doesn't forgive he doesn't forget his
group has over 9000 penises and they're
raping children now if you know anything
about internet subcultures well
doesn't forgive doesn't forget that's
anonymous over 9,000 is a meme reference
to a Japanese a video game that autumn a
in other words it was a massive in joke
within these early environments a site
called 4chan and she did it she spoke
loud and you can imagine the internet
erupted with laughter right it's funny
to get Oprah to say ridiculous things
but that kind of useful youthful media
manipulation shifted over time a media
manipulation it may not seem like it has
anything to do with privacy but actually
it's fundamentally rooted in that same
dynamic of the ability to control a
social situation media manipulation
allows disenfranchised members of
society to come up with alternative ways
of controlling ways of having power
after they feel disempowered and that's
a really important component because
people think about 4chan as a space of
anonymity and people hiding especially
back in those days but
many ways it was about purposeful
disruption of the status quo it was
about achieving a form of chaos when the
status quo was actually undermining any
form of agency and so what happened is
this coordinated group of teenagers
started to learn how to manipulate media
they learned how to alter structures of
information play SEO games figure out
the very architectures of the system so
that they could actually control a
social situation in their way it was
supposed to be fun and games a kind of
teen subculture a joke unfortunately it
stopped being funny a couple years ago
so there are many different aspects of
media manipulation that I can't go into
in 25 minutes but I want you to realize
that this aspect of media manipulation
is the control of a social situation and
a lot has changed since what I was
seeing in the early aughts so we've seen
sort of this evolution right jokesters
messing with Scientology the rise of
anonymous the sharing of information
from WikiLeaks we saw people hacking
into systems people getting doxxed the
swarms of people mocking people after
someone had died by 2014 we were seeing
the development of sophisticated and
coordinated harassment campaigns
implemented as media manipulation
implemented as ironic play but 2015 we
were seeing large coordinated efforts to
manipulate media for political purposes
in addition to economic purposes and
what ended up happening is we saw all
this unfold is the rise of another thing
that we saw from early days of Usenet
something that's known as Poe's law you
can read what Poe's law is here but the
really short version of Poe's law is
that online it becomes increasingly
difficult to tell the difference between
somebody who is joking around and
pretending to be a Nazi and someone who
is an actual Nazi and that dynamic is
something that we are living with right
now
Charlottesville was a wake-up call for
many but not a that much for the tech
industry not a lot changed but the
violence keeps coming you know I was
hoping that I wouldn't have to reference
another violent incident this weekend
and yet we did I want you to understand
that the extreme of media manipulation
is really about this
on this transition over trying to have
control because that shifting nature has
actually made this turn into something
that's far more violent and the flipside
of is when we talk about privacy is
realizing that the technical
architectures we're working with are in
the business of amplification and that
the news media is also in the business
of amplification and when we have these
two systems of amplification that are
intertwined they become something that
people can manipulate at scale so let's
talk about Christchurch
one month ago a terrorist walked into a
mosque in Christchurch New Zealand and
brutally murdered 50 worshippers in a
mosque okay and I want you to understand
that this incident has three parts to it
the first part is that absolute horrific
violent attack that the people of New
Zealand had to face and the consequence
of the media and people wanting to cover
that and doing what we talked about
whenever there is a terrorist attack but
there are two other technical components
to this that are important the first is
what that particular terrorist did in
setting up this attack this terrorist
understood the vulnerabilities of social
media the vulnerabilities of the news
media he knew how to play both of those
strategically so let me explain some of
what that looked like if you read his
original 8 sham post you would have
noticed something fascinating he
provided live links to absolutely racist
manifestos all sorts of content that was
horrific but the reference to the live
stream on Facebook was not a live link
it had spaces between it and that was
intentional because that meant that
people put the URL into their browser
and Facebook couldn't see that all of
the traffic was coming from a champ that
meant also that he played on the news
media by making certain that there were
all sorts of references to things that
the news media didn't understand so they
would immediately report things but he
had also staged that because if you
search for the title of his manifesto
what you got was a stream of horrific
content that had been purposefully se
owed so you could radicalize a whole set
of different people he did a whole set
of a
tactics that were very much attempting
to undermine any of the detection
mechanisms of any major platforms and
that's what occurred before he walked
into that mosque signaled and physically
from Peppard portraying himself as a
first-person shooter to get past a lot
of original expectations so much so that
the people that I know that do analysis
of Isis videos who watched it live
didn't realize what was going on
now the third act to it is also
problematic because the third act was
what happened afterwards people took
that video that livestream it quickly
went into a variety of different for it
went to tour in matter of minutes and
they kept modifying it and uploading it
Facebook publicly announced 1.5 million
uploads many serve it and that was in
the first 24 hours many services saw
many more uploads and each of those
uploads was a modification meant to
figure out how to get past all of the
content detection
what is this this is about manipulating
vulnerabilities in the information
landscape and you may recognize that
that language is a language of security
right exploits vulnerabilities hacking
in privacy circles we've long talked
about the relationship between privacy
and security talking about what it means
to think about the ability to access
private information about controlling
and building the secure structures to
control data but I want you to rethink
what it means about the relationship
between security and privacy when we
talk about an information landscape when
we talk about the ability to control a
social situation in order to secure an
information landscape where people are
trying to manipulate the social context
we need to recognize that data becomes
infrastructure and the idea that you end
up having here is not simply that you
want to exploit it by actually accessing
the data but by corrupting the data
right actually making it difficult for
any system to operate because you can do
a lot of things that we've often thought
of as separate search engine
optimization algorithmic gameplay all of
the different ways of contorting
recommendation engines and search but
what happens is that because the data
that people put in
through the front door and the kinds of
models that get built off of the variety
of different sources of data becomes so
powerful the way in which you control a
situation is that you exploit the an
underlying infrastructure you exploit
the data that builds the models and
that's what we see people doing on a
regular basis you know as we think about
our current obsession with data we're
not just thinking about how we can
actually follow compliance rules and I'm
so grateful for those of you what worked
so hard in this room both to make
certain that GD P R is a reality and to
make certain that it could be
implemented in a variety of different
ways because we want to give people
control over their information but it's
not enough because you can give them
control over their personal information
but if you don't actually secure and
protect the data that actually builds
the models the data that builds our
information landscape you end up with a
new kind of vulnerability amongst those
who wants to control a social situation
you know I don't know how many of you
have ever read Snow Crash so those are
the geeks in the room say hi to them
Snow Crash is a really interesting book
for those who've you have not read it I
strongly recommend it because it was
basically the Bible of tech culture in
the late 90s and when I was in you know
in the end in the industry as an
engineer I was intrigued by how many
people you couldn't go into a meeting
without somebody referencing what they
wanted to build from the world and Snow
Crash and they were so excited they were
like this is what we're gonna build the
Metaverse is the solution to everything
the Metaverse is basically like what
would you imagine when you fully came
into the internet right all of the
immersive environment conversations the
problem with it is is that Neal
Stephenson's book is a dystopian novel
it's not a utopia he argued what it
would mean to build these complex
information and landscapes many of the
things that our engineers have designed
for but the whole book the title is
rooted in the idea that there is a virus
that will spread that will fragment
society by making it impossible for them
to communicate the collapse of the Tower
of Babel right and the whole point of
this is that if you connect everybody
there will be new exploits and those
exploits will actually go into the
vulnerability of the human social fabric
and I want you to read this book when
you go home it's fun
there's even a good audiobook so you can
actually understand where we designed
for and what the implications are for
now but I want to talk about what that
is right now because it's not just an
exploit of a terrace is an exploit
that's actually happening on a more
everyday basis
so epistemology is a concept that says
how do we know what we know right how do
we build a set of knowledge how do we
construct it socially how do we have
competing in different epistemologies
and in those frames the idea is that
ignorance is what we don't yet know
right but there's another way to
actually think about ignorant ignorance
can be strategically manufactured and in
fact a 1995 Procter & Bowl point II term
called AG Natali G and AG Natali G is
the idea that you can strategically
enable ignorant you can make in
ignorance happen by making it difficult
to tell what is reality by seeding doubt
by just asking questions about whether
or not you know there is a really a
climate problem whether or not there's
any link between tobacco and cancer
whether or not as CNN put out a racist
epitaph saying something like are Jews
really people those frames are ways in
which you concede ignorant you concede
ideas of horror and they've been used
politically for quite some time Russia
today has been engaged in a coordinated
campaign for many years trying to say
asking questions and the whole program
is called question more so for those who
can't read this it says just how
reliable is the evidence that suggests
human activity impacts on climate change
the answer isn't always clear-cut it's
only possible to make a balanced
judgment if you are better informed by
challenging the accepted view we reveal
the side of the news that you wouldn't
normally see because we believe the more
you question the more you know that's a
way of seeding agna ecology seeding
ignorance creating fractures within
society and people who are engaged in
doing so feel power they feel as though
they can control information by actually
in many ways corrupting the information
landscape many of the reasons that
people get into privacy is because they
recognize
privacy is connected to user trust we
realize when we build systems that we
want our users to trust what we're
building to have confidence in us and we
are in a situation in this industry
where the companies that have been
struggling around this are struggling
with ability to communicate and engage
people on trust because they don't know
how to actually help people understand
that they can have control over social
situation I'm really proud to be at
Microsoft because Microsoft has actually
put privacy and trust at the center of
things and I'm not saying the company
has always done it right but really
making that commitment and making that
part of your values is really essential
to me and it's one of the things we need
to be doing collectively because one of
the things we're dealing with is it's
not just about whether or not we can
fill the compliance but can we actually
find a way through this industry to
rebuild trust and that's gonna require
us to really deal with the issue of
creepiness and right now I would argue
we're not doing it because we're not
actually starting from a place of values
we're not starting from a saying like we
are gonna put the user Center we're
still struggling to understand what we
all mean and we're thinking that we can
do this in a technical form and what I'm
saying is that there isn't a technical
solution through this this is a set of
processes and practices and
understanding like with security that
the vulnerabilities will keep coming so
fundamentally for me it's unacceptable
that the technical skills and economic
power allow people to exploit systems
and we need to recognize that those
exploitation --zz around privacy are
connected to the other forms of security
and that the very nature of security is
evolving but fundamentally it comes back
to this question of what are we trying
to protect in the tech industry we often
talk about the right to speak but it's
actually kind of evolved to do a weird
way of the right to be amplified and we
don't know how to deal with that tension
at all
we don't know how to deal with what it
means to reclaim control over social
situations because we don't have a
framework of power well actually some of
our favorite people in the history of
privacy do have a theory of power at a
pathway through this I have to imagine
that most of you know and perhaps love
Brandeis I certainly do well I'm
Brandeis and Warren pens their famous
article on the right to privacy they
built an argument around involuntarily
disclosure and how it harmed people but
the thing is I've also
the other things throughout Brandeis is
career so you may also know his essay
work he argues that sunlight is the best
disinfectant because the point of that
was you actually have to purposefully
expose systems of power in order to make
a difference and it's at some level they
seem contradictory but they're not
because they actually can be understood
as a way of seeing how he started to
model the role of power across the
information landscape a hundred years
ago and that's the case that I actually
find the most interesting is one that is
not usually discussed in privacy
conversations is the 1932 case known as
Packer versus Utah and in his opinion in
this he laid out a true theory of power
that allows us to unlock some of the
challenges and connect the dots between
privacy in these kinds of manipulations
so the case was that Utah outlawed
smoking ads tobacco ads on billboards
and packer corporations sued under the
First Amendment saying that they should
have the right to post on this why could
they not put anything up and Brandeis
you know wrote the the opinion as a
member of the SCOTUS and he said that
actually the Billboard is different than
a magazine a billboard is part of a
public sphere and a magazine you can
choose the buyer not buy and then if you
put it in the public sphere you will
force people to have to divert their
eyes and to do so would be creating
captive audiences and captive audiences
for him was a way of actually looking at
the power differential and if you start
to build this through you see that
there's a way through legally socially
and culturally through understanding the
power dynamics in other words the
commitment to speech the commitment to
First Amendment is about giving people
the right to speak but not necessarily
the right to be amplified certainly not
when that amplification can harm people
so as we think about privacy going
forward we want to actually converge
privacy through that theory of power we
want by focusing simply on the technical
access issues we lose visibility by
focusing on the individual we lose the
structure privacy needs to actually
understand that privacy is going to be
networked going forward it
part of a broader social fabric and we
need a framework that allows us to see
that the manipulation of systems is a
way of exploiting privacy at the
interstitials between people
privacy can't simply be about the
individual because it won't give us the
tools to actually deal with the power
dynamics and when speakers and
amplifiers have the unchecked power to
turn a blind eye to the cost of harm
what's at stake for all of us is privacy
dignity democracy and social cohesion so
I am deeply grateful for all of you
doing privacy work and I'm excited that
you're gonna get to spend some time
together really thinking through this I
might hope and offering this up to you
today aside from going into a dark place
is to actually give you a way of
thinking about and complicating your own
views of what's going forward and I look
forward to talking with you to more more
across today and thank you for your time
[Applause]