[Music]
hello everyone it's a pleasure to be
here I want to start off the way I do
every year when I teach information
privacy law on the very first day before
we've really gotten into anything I
always ask my students to define privacy
for me we go and write it on the board
and many of the definitions and
conceptualizations of privacy you would
no doubt be very familiar with and so
it's the standard articulation such as
the right to be left alone or privacy is
intimacy or privacy is dignity but by
far the most common articulation of what
privacy is is control
well not that control this control the
kind of control the ability to determine
who does what with your personal
information and although it seems to be
a little bit of an accepted truism in
our world that the concept of privacy
eludes definition and I've argued this
many times looking around it actually
appears as though we've already come to
some kind of consensus at least with
respect to the protection of our
personal information the name of the
game is control let's look at the
evidence Mark Zuckerberg has long
insisted that the way facebook seeks to
protect user privacy is by putting users
in control of their personal data but
it's not just Facebook looking around
the entire tech industry you will see
that the control is the espoused
fundamental ethos that drives industry
data protection efforts
the list goes on usually this notion of
control is often linked to the ways in
which systems are engineered and the and
the it refers to the tools that are
given to users that purportedly let them
determine their own fates it's also the
defining concept for the Internet of
Things as well and actually it turns out
it's not just industry academics and
advocates also adopt the frame of
control as the dominant goal for data
protection efforts in fact I must admit
I am in this boat I have advocated
strongly for control I yearn for it for
data subjects I think that it's it's
it's a worthy and and critical sort of
thing for people to have however it's
not just us it's not just advocates and
academics but where it matters arguably
the most is with regulators and when it
comes to regulators they seem to be all
on board with the idea of control as the
way to effectuate data protection the
GDP are in fact makes it quite explicit
but it's actually not just Europe right
if you look around the list goes on and
on among those who value the control of
personal information in fact control
seems to be the one data protection
principle that everybody can agree on in
particular I think the control is
relevant for the design of information
technologies it is the espoused dominant
privacy goal for privacy by design
efforts
the regulator's seem to have embraced it
informed consent regimes like the e
privacy directive are in essence control
regime's
privacy by design as a concept seems to
embrace control at almost every turn and
in fact in the United States as well
even the pioneer of privacy by design in
Kabuki and has embraced the idea of
privacy as control now the arguments
that I would like to make in this
keynotes is this I have come to believe
that control is the wrong goal for
privacy by design it's actually I think
perhaps the wrong goal for data
protection in general the idealization
of control in modern data protection
regimes ends up being actually adverse I
think to safe and sustainable data
practices too much zeal for control risk
deluding people and regulators about the
potential efficacy of protection
initiatives if we are to be we are not
more careful about the the goals and
values to be reflected in the design of
information technologies we risk
building an entire data protection
Empire at the crumbling edifice of
control now building this out a little
I'd like to make two distinct points in
this talk one I think control is a
compromised concept for personal data
and design second instead of using
control sort of as a proxy which is the
way in which we seem to be using it
privacy by design framework should
embrace more direct values like trust
obscurity and autonomy
now at this point you may be saying to
yourself well sure if control is
conditional if it's vague uninformed and
contingent upon action and of course
it's no good
but our industry / framework / goal is
different we advocate for real control
now I think that this kind of idealized
control is probably impossible in
mediated environments and here's why
first I think that control is illusory
it's a little bit of a shell game in
practice because the control we are
given online is mediated which means it
cannot help but be engineered to produce
a particular results when it comes to
control design is everything the
realities of Technology at scale mean
that the services we use must
necessarily be built in a way that
constrains our choices can you imagine a
world where everyone got to dictate
their own terms in an Open Text box with
this the services with its within which
they interact instead of some sort of
boilerplate term or a standard set of
tools our mediated perception of control
obscures the fact that design funnels
behavior and people can only pick on the
options that are provided to them we
give this sort of fake control to our
kids very often so for example when I
give my kids a choice between going to
the park or going to the movies they
feel empowered and I get to avoid a trip
to the pet store so that I can stave off
a conversation about getting that new
puppy and so I think that it's important
to consider that design also nudges us
by sending signals and making tasks
easier or making things harder
encouraging us to act in predictable
ways companies deployed dark patterns to
exploit our built-in tendencies to
prefer shiny colorful buttons and ignore
the dull gray ones are shameless into
feeling bad about withholding data or
declining options are simply making
exercising control possible but costly
through things like forced work subtle
misdirection and incentive tethering how
many people have seen this ad pop up on
your laptop and tried to click decline
only to realize there's no decline
button there
have to open up the app store in order
to get this thing to disappear here's an
example of what Harry brignall calls
confirmed shaming on his really
excellent dark patterns website
sometimes we get wheedled into over
sharing through the design of services
not only a sharing meant to make us feel
good such as keeping a street going like
here on snapchat or nudges to share old
posts on social media or congratulate
someone else on Facebook or LinkedIn but
sometimes they just make sharing so damn
easy the key to keeping the data spigot
flowing is to get people to want to say
yes and to require the minimum amount of
efforts to have them impulsively do so
design has a powerful tendency to dampen
skepticism if we want if users want it
bad enough then they can justify and
rationalize lots of different decisions
now of course we could create rules to
mitigate against this sort of thing but
of course the essence of design is to
nudge people into a particular direction
now of course these choices are not
always harmful for us many of them in
fact might be quite useful but asking
companies to engineer user control paves
the way for abuse and self dealing at
the margins and these margins matter
even among those acting in good faith we
are left with the problem of relying on
the notion of control and choice to do
more work for us than it is capable of
and we risk looking around at the robust
new frameworks that we have for data
protection the rules built to encourage
meaningful control over personal
information patting ourselves on the
back and saying mission accomplished
but it wasn't even the right mission the
second reason that control is a
compromised goal for design is that it
is overwhelming at scale control over
personal information is highly
attractive in isolation who wouldn't
want more power over the things that
affect our law
but within this power often comes a
practical obligation sometimes when we
do not exercise that control we are put
at risk companies can take our our
inaction as acquiescence the problem
with thinking about privacy is control
is that if users are given our wish for
more privacy it means we are given so
much control that we can choke on it
some recent designs some recent strides
and design rules such as data protection
by defaults actually does help mitigate
some of this problem however even if the
default works demands are still being
made of us so that we relents anyone
that has turned off notification for
apps like Facebook's messenger can
attest to the relentless grindin
requests of the user to turn on
notifications almost every single time
the app is launched many can relate to
the experience of a child asking for
candy over and over until the requests
become too much to ignore and we finally
give in simply to quiet them willpower
can feel like a finite vulnerable and
highly subjective resource and systems
are designed to deplete any rodents once
willpower and the ability to make
choices become compromised the control
that users have been given will largely
be meaningless
even if industry figures out the
Platonic ideal of how a company can give
users and data subjects control it still
wouldn't solve another dilemma which is
the bandwidth dilemma people only have
24 hours in a day and so many resources
to dedicate to making decisions even if
one company or all companies perfected a
control interface people would still be
faced with the barrage of larger
decisions because they use multiple apps
and services control will not change the
fact that the data ecosystem is
mind-bogglingly
complex with many different kinds of
information collected in many different
ways
stored in many different places
processed from many different functions
and shared with many different parties
and even if every single techcamp
merged together to form some sort of
giant Google APIs on book'll then the
tension between simplicity and nuance
inherent in the most complex and fraud
environments imaginable would seem
irresolvable this is because nuance
often gets glossed over when companies
tried to simplify information for our
benefits risk is either hidden away
through abstraction or made so
voluminous and explicit that we don't
even know where to begin because of
course the way in which control gets
expressed in design is through things
like this but it's never just things
like this right it's one toggle switch
that exists among a nested layer of
toggle switches that then exists among
an entire environment of apps and we
look back and we feel overwhelmed at the
number of choices privacy by default
again helps with this but if but the
nudging will continue until morale
improves now finally control is a myopic
value for privacy which i think is the
third reason why control is compromised
as a goal for privacy by design notions
of individual control really don't sit
well with privacy as a collective value
when privacy is thought of an
individualistic transactional terms
people's privacy is always being
negotiated against what others value
control should be tempered against other
values because no one privacy ideology
should reign in a diverse society where
other people's autonomy matters what
makes us think that the collective
result of atomized decisions will be
best for privacy anyway scholars have
noted that a large body of research
shows that people's privacy preferences
are uncertain they are contextually
dependent and they are highly malleable
the availability of knowledge doesn't
necessarily translate meaningfully into
good privacy decisions lawmakers may
favour mandated disclosures like the
warnings on cigarette boxes because they
are cheap and they counterbalance the
information disparity the reality the
companies often know
if that they know much more than
consumers regarding the wisdom of a
decision but in this context users are
being asked to consider the privacy
implications of every single disclosure
that they make an impossibly complex
calculation to make the future of risks
and consequences very difficult to
ascertain now another problem with
control and this is getting a little
closer to it is that it leaves us
sometimes lacking a more firm moral
anchor control presumptively gives
people choices and serves autonomy but
idealizing control can leave designers
and data subjects without much of a
sense of which values are really truly
at stake to be preserved by the system's
purpose or design the lack of an ethical
mooring encourages ideological drift and
abuse when control is the Northstar
company leaders and policy makers aren't
given much to work with when tapped with
improvements in the absence of more
articulate values CEOs and lawmakers say
what is needed is perhaps more control
if something goes wrong well then we
must not have had enough control as
though that would solve the problem but
it's not clear that more control and
more choices are actually going to
really help us maybe what we need are
actually fewer better choices driven by
different moral values but it's hard to
rank which choices are good without a
more concrete theory of exactly what
we're protecting against other than just
control this is where additional value
such as trust obscurity and autonomy
become important they give us a better
roadmap on how to allocate the resources
of control which is finite and precious
we need an approach that incorporates
multiple values and it's capable of
recognizing privacy is a more social
good now to my second point we seem to
be using control as a proxy when control
is expressed as choice in a way it sort
of undercuts its own mission and given
the pathologies that are inherent in
mediating choice people should have a
more fundamental base
level of protection regardless of what
they choose what we have have now in
certain pockets is a system that allows
companies to offload the risks of data
on to the data subjects justifying
control measures on privacy grounds
requires so much justification sometimes
so much stretching and bending and tying
ourselves into knots that it feels like
we're using control as a proxy for some
other protection goal that's just out of
reach for us control feels intuitively
right and it has selling power so we use
it but but what is the result that
policy makers industry advocates and
data subjects are really hoping for
surely it's not control for control zone
sake control ostensibly serves autonomy
but in mediating environments involved
in personal data idealizing control
actually seems corrosive to autonomy its
control valuable because people have
such different privacy preferences or is
it just does it just appear that people
have different privacy preferences
because personal data risks are
virtually impossible for people to
assess consistently across a broad
spectrum if data processing is so
dangerous that it requires formal
permission and meaningful in such
meaningful control consent in choice can
only exist in elusive demanding and
bounded environments with pre conditions
such as freely given specific informed
retractible and unambiguous why are we
allowing controllers to proceed in the
first place is it just a contorted and
indirect way to pressure companies to
lay off risky data practices if so why
not dispense with the pretense of
demanding a form of control that seems
destined and misdirect industry efforts
towards a more formalistic compliance
and a less meaningful change in
processor behavior
this brings me to the point about design
for design I think what matters are
relationships
and transaction costs in other words
what is the nature of the relationship
between the data subject and the person
who's collecting or using the
information and what ways does the
design of an information technology make
things easier reducing the transaction
cost of any particular action or make
things harder by raising the transaction
cost of any particular action and when
you take those two things in
consideration and you look at what's
truly I think important the values that
I think tend to rise to the top are not
necessarily control rather I encourage
an embrace of the values of trust
obscurity and autonomy now trust I think
is the key value for a data protection
moving forward trust the willingness to
make yourself vulnerable to the actions
of others is a key component in so many
different aspects of our lives it
underlies commerce it unders underlies
our our relationships and intimacy and
underlies self-exploration and I think
that's if the rules that we have about
the design of technologies are meant to
make sure that the the trust that is
given is respected then that's a much
more concrete and and in the end
effective way to channel the design of
information technologies and and Trust
contains several different components so
I think that that Trust contains a
responsibility to be discreet sometimes
confidential but maybe just anonymizing
or sadhana maizing data a requirement to
protect information by having good data
security practices a requirement to be
honest which i think may be technically
different than transparency honesty
requirements within trustworthy
relationships mean that that those that
are entrusted with data are actually
forthright about things that they might
rather not disclose we're actually
important to the data subjects so in
other words we don't we don't bear
everything we bear the things that are
most relevant to the data subject and
then finally in the
the key points loyal companies and loyal
and designed the design of information
technologies are those that that indeed
reflect a loyalty on behalf of the data
processor that they will not
unreasonably elevate their own interests
above those of the data subjects the
next value that I advocate for its
obscurity which is a little bit less
popular in our existing data protection
language but obscurity is is the idea
that when information is hard or
unlikely to be understood is to a
relative degree safe
and we rely upon this obscurity all the
time
we walk around cities being technically
in public but realizing that most people
won't see or remember things that happen
gossip that we we engage in at
restaurants we freely do so even though
people are around could hear us but
chances are and most people want we
don't even remember the people that sat
next to us probably the last time we ate
out and obscurity strikes me as a
fundamental components of the ways in
which we should channel the design of
information technologies when we have
obscurity when we have trust and when we
have autonomy which I would argue as as
the value that control ostensibly serves
then we can have better rules for how
our information technologies are created
we can have rules that that may ensure
that design is honest that it does what
it presents itself to do we could have
rules that prevent what I call abusive
design this is the triple negative right
I don't not disagree to something like
this most people do not affirmatively
choose this option it's when design
leverages our own limitations to
understand against us and and control
doesn't always tend to get us to the
place where the technologies would
indeed avoid being abusive and then
finally these values can help protect us
against dangerous design design that
surveil things that puts us unduly at
risk so in the end I think that
lawmakers have more direct options than
embracing control as the value for the
design of in firm
technologies prohibit collection out
rights mandate deletion get serious with
purpose limitations and legitimate
interests change the nature of the
relationship between users and companies
entrusted with their data so to one that
is more fiduciary like in nature mandate
non-delegable duties of care duties of
loyalty duties of honesty and duties of
discretion make rules to ensure that
companies cannot unreasonably favor
their own interests at the expense of
the data subjects the case for privacy
the case against privacy is control is
actually an appeal to more substantive
and effective privacy related values by
expanding beyond the notion of privacy
as control lawmakers and industry would
be free to create rules and practices to
ensure that designers data processors
and controllers are trustworthy
regardless of the control that we are
given and how we exercise it thank you
very much I appreciate it
[Applause]