So this talk will be a little different.
It's an idea that I've been kind of that I've been milling around my brain for the past six months or so.
So this is the first time I'm testing it out with an audience.
You're gonna help me crowdsource it.
So my name is Anita Nikolic.
I just finished up a tour at the National Science Foundation.
I was a program officer for cybersecurity.
NSF, National Science Foundation, funds most of the basic research in the U.S.
and they fund almost 90% of computer science research.
So security is underneath the kind of guise of computer science.
Specifically, I was a program officer for networking security, social networking, and anti-censorship.
So just to give you some background.
And prior to that, my background is mostly in networking security.
So I've been reading this book about kind of amateur scientists, professional scientists, and how the twain sometimes meets, sometimes don't.
I saw this cool quote from Arthur C.
Clarke.
He's the author of 2001, A Space Odyssey.
And I thought this was really interesting.
I definitely encountered this a lot kind of in my time at NSF because there's a lot of distinguished scientists that that come back and forth through the halls.
So I thought this was very interesting.
So here's kind of the proposition.
And as you'll see, I kind of ran this past a bunch of people.
And the semantics change depending on what your preference is.
But the proposition is there, you know, the past four years that I've spent in the government at NSF.
And prior to that, I was at a university.
Prior to that, I was just in IT operations.
There is a lot of money that's spent doing academic security work.
A lot of this work never gets any place.
And we'll talk a little bit about that.
It kind of lingers in papers, that's behind paywalls.
At the same time, there's a lot of cool stuff I hear at these conferences.
That sometimes doesn't make it, you know, the total feedback to whether it's improving security in a product or getting it out to people who could use it.
That doesn't happen either.
Sometimes it does.
But the proposition is how can we get kind of, I'll call them non-professional, you know, us basically non-professional researchers, operators, and official academic researchers to work together somehow.
So some things to consider.
Is this really even a distinction between these two?
Is this becoming meaningless, particularly for security and privacy?
Should it become meaningless or are there roles for each?
Should we have academics who, you know, sometimes can't push the limits as much as we can, not being in academia?
Or is research just research?
Before, about 1970, academics could use money very freely.
You saw it from World War II, through the 50s, 60s, they could use money very freely.
They were given money.
They could explore things.
They didn't have to so much worry about publishing and tenure as they do now.
Modern day, there's something called peer review.
And if you're not familiar, kind of talk a little bit about it.
But peer review means these ideas come to a panel.
People like me, we invite distinguished experts to come and listen and be thoughtful about ranking these ideas.
And peer review is kind of designed to prevent major shifts.
Everything is very incremental in that when you bring up a novel idea or something really transformative, your fellow peers often say, well, you know, that's not safe.
That may not work.
And science has become more like a business.
So what prompted me to think about this?
This isn't really anything new.
I'll give you some examples of where it has been happening.
So I remember back in high school, one of my teachers talking about that, and I don't have the correct name of it, but this great books program in New York City.
Basically, it was kind of ordinary people talking with Columbia professors about, you know, the great books.
So they would bring people in, you know, it's more common now you see meetups, but this is back in the 60s or 70s.
And the insight was, wow, all these kind of average people who read these classic works have really, you know, insights that are pretty thoughtful.
So that always kind of stuck with me.
The past four years I've been at NSF, NSF spends, and this is just one agency, there are others, DARPA and others, NSF spends 80 million dollars a year on basic privacy and security research.
Many other agencies spend millions of it on applied work, on basic work.
Why are there still so many security problems?
One of the programs that I started there was this transition of practice program.
I stole the name from Homeland Security, but basically what I wanted to do was, you know, that with all this money being spent, and this program that I was on was around for ten years, so that's what, eight hundred million dollars, and some of the same issues still are there year after year.
So what what can we do to transition some academic work and get it into companies, get it in the hands of people who can use it?
Many academics were very upset about this.
They said, we want money to just do basic security work and that's gonna take away from it.
Getting it more practical, we can't do that.
So it's kind of a mixed bag and it evolved over, you know, the four years I was there to people kind of accepted that we're not taking away your right to do basic crypto work, but there's a lot of work that's sitting there that never gets into the hands of people who can use it.
Another thing I noticed as I go to these a lot of these conferences, just I was lucky for my job, I got to go to a lot of these, if you look at the agendas and I didn't have time to do this, but kind of in the past I'd say, you know, five to six years,
you look at the agendas for these kind of cons and academic conferences, they're very similar.
The topics are very similar, the approaches are very similar, but neither kind of wants to be seen at each other's conferences.
So when I'd say, you know, I'm gonna put in my travel, let's go to DEF CON or any of the, we call them principal investigators, are any of them going to academics going, they, many just will not go there.
Can that be overcome that, you know, people attend each other's events.
Some talks that inspired me at ThoughtCon last year, maybe the year before, the cyber squirrel, Chris Thomas talked about, you know, the premise being squirrels are a bigger threat to the ICS systems and actual hackers.
I thought that was really interesting, so I brought him into NSF and we usually get like Nobel Prize winners and serious talks and he gave this talk.
And it's interesting, half the room walked out and half the room just thought it was awesome.
I thought, you know, there's something to that, you know, this is a really, this project is kind of tongue-in-cheek, but there's a lot of kind of quantitative metrics and approach to it that's very interesting.
So that kind of struck a nerve with me, there's something to it.
I'm sure many of you know, you know, Jay Radcliffe, this hacker researcher, talked about his project a few years ago about hacking his own insulin pump and going to Johnson & Johnson and getting them to fix it, but he had a nice talk last year and that kind of really inspired me to crowdsource this about,
you know, can we inspire kind of this community to maybe use more of a scientific method to get some of our things that are not academic research to get to get traction with it.
Maybe take things a little more seriously, maybe use a little more methodology and do we want to.
So at NSF we were asked a lot, you know, when an idea came by our desks or something, what does the community think?
What does the community say about this?
You know, and me not being an academic, I kind of landed by circumstance at NSF.
I'd say, well, who is the community?
What do you mean by that?
I mean, I came from security operations.
When you say the community, who are you talking about?
If it's the same people, you know, writing the same papers for the same conferences, is that the community?
Is that operators?
So, you know, what does that really mean?
So this isn't a new problem.
It got me to thinking, you know, are there other disciplines, other areas besides security where this might have occurred?
So I'm in one of these meetups for science books.
So one of the books they had us read was about Mendel.
And I'm not a biologist, not familiar with it, but it's a very interesting book.
And I learned about Mendel.
This is if you recall your high school biology, he's the father of Mendel's Law of Inheritance, which explains heredity and how traits are passed down.
Well, he grew up on a farm .
He worked as a gardener and a beekeeper, a quiet guy.
He ended up as a monk in this Abbey in Brno.
And what he did for his many years was grow pea plants .
He literally just had pea plants in the garden and he did these experiments .
And he kept very meticulous notes.
He wanted to develop new color variants and examine hybridization.
And this had never been done before.
So he did this for many, many years and published a paper which he sent to these proceedings.
He sent them to the Royal Society, the Linnaean Society, the Smithsonian.
And they just, because he wasn't a professional scientist, the paper wasn't taken seriously.
It was kind of put into this very obscure journal and forgotten about.
Well, when this Dutch botanist, a friend of his who was cleaning out his stuff prior to moving, sent him this paper.
He said, this is amazing.
So it took many years.
It wasn't until 1909 where all these things were finally pinned to the discoveries Mendel first described in that paper, which just by happenstance, this paper happened to be found.
So that got me thinking, you know, there's, it seems like there's a similar thing going on in our community.
Another example, which I love because I live in Chicago, this is Sioux, the T.
rex Sioux at the Field Museum.
It's the biggest, best preserved, if you never seen it, it's this amazing looking T.
rex.
Well, Sue Hendricks, it was named after her, she found it.
And she was a high school dropout.
She moved to Florida to go diving.
She lived with her uncle and she was an explorer, adventurer.
She caught these fish to sell to aquariums, rare ones she would just sell at cost to museums.
And she hooked up with some explorers and said, I'll help you dig up fossils.
And she ended up, their tire was flat so they had to spend the night there.
She said, you know, there's this ridge we haven't explored yet.
And it's kind of on her instinct of doing this for many years that she actually, not the paleontologist, not the geologist, an untrained explorer had the tenacity and she found this fossil.
So it's another example, you know, I think astronomy is another great example.
Although in astronomy, you know, typically we have the tools at our disposal in computer science and security versus like astronomy.
But there's a couple interesting examples.
I never knew about the Shoemaker-Levy comet, which of course many of us have heard about.
I didn't realize that Carol Shoemaker, who is one of the discoverers of it, at one point she had found them the greatest number of these asteroids and comets of any person.
I think that's been superseded.
But she was not formally trained.
She was a housewife.
She didn't start her observations until she was 51 years old.
She just kind of was a fan.
Her husband was an astronomer.
She had no training in this at all, but she kind of made a really big difference in this field.
So I thought this was interesting.
I'm sorry my slides are a little all over the place.
But I put together, side by side, I just randomly grabbed a few years ago, just randomly some talks from DEFCON and some from USENIX security.
USENIX security is a big kind of big event people like to publish in.
So I kind of put them next to each other and, you know, I wonder if people can really tell which is which.
Maybe some of you have been, but if you take a look at it, kind of look at the topics.
I don't know if anybody has a guess or if you maybe know this already.
So the one on the left is USENIX.
The one on the right is DEFCON.
You kind of look through the titles and the topics.
You could almost switch the two around and you could be at one or the other conference.
So some things we're not talking about.
Citizen science.
There's this really good article, which I know is a little hard to read on the bottom, but I could send you the link.
Really good article on citizen science.
Citizen science is where kind of scientists set up experiments and people help contribute to it.
And mostly the scientists get the benefits.
They're the ones that get kind of the fame.
The same with SETI.
That's where you donate your kind of compute power to look for extraterrestrial intelligence.
Bug bounties.
I think that's a whole topic in and of itself that has kind of an ecosystem set up already.
Same as these, I'll call them professional hackers, such as this HackerOne, which helps facilitate the bug bounty programs.
I'm also not really talking about kind of, you know, research being done on your own accord.
What I really would like to get to is how can these communities share better across the boundaries.
So a lot of people call this this valley of death from, you know, a really cool idea to making something happen or getting into a product.
This is kind of gap between research, whether it's academia or elsewhere, and how is it translated into either better products, marketable products, something that can be useful.
This isn't new.
There's a lot of these incubators.
There's these industry-university collaborations to, you know, generate commercial products.
But a lot of companies are eliminating or seriously scaling back their research arm.
I mean companies like, you know, Microsoft and RSA and Dell and these kind of places.
They're scaling back and what they're doing is just giving $50,000 grants to academics and saying, you know, we don't want to keep people full-time on staff.
It's a kind of discovery front, which many times was done by the Bell Labs and these big companies.
That's not the case so much anymore.
So I ran some of these ideas just past four different groups.
Academics, industry, I'll say underground, kind of, you know, hacker community, and government.
My informal crowdsourcing, about 70% of people were kind of excited, like that's a good idea, and about 30% mostly academics were like, no way, just it would ruin my credibility, you know, I don't see it that I would get anything out of it.
But I did have quite a number of people who thought, and I'll go into some examples that have been done in the past, that, you know, as long as you can assure there's no damage done, and as long as you can assure this is done in a pretty ethical way, that it's a cheap way to get good research,
and it's a good way to get people involved in something official.
So if you're, you know, many of us have day jobs, but we do kind of interesting researchy stuff on the side, it's a great way if you want to get promoted or get something, you know, on your CV, it's a great way if you just have altruistic motives, you really want to make a difference .
And it's a great way if you just want, you know, five grand to get some stuff to tinker around with and help somebody.
I mean, I think there's a lot of different needs that can be satisfied.
So just a short, kind of, if you don't know how academic security and privacy studies are funded, it's just, you know, I learned a lot about going to NSF, but before, as I mentioned before, about 1970, the funding was just, you know, you went to your university,
you're a professor there, you got money.
One example in the UK is, it's called the Royal Commission for the Exhibition of 1851.
It's a granting agency.
They used to allow awardees to just pursue research, they give money and say, pursue whatever research you want, wherever it goes, that's fine, it comes up, you'll come up with something useful.
Now you need to have a proposal be judged.
So say we go to NSF, you come in, and that's kind of the machine of proposal, right?
And you come in with an idea, you get a committee of people, they evaluate it, they give you the money.
And private foundations also give a lot of grants.
MacArthur, Gates give a lot of grants.
And they have very little obligations on the part of academic researchers.
It's almost like a gift, as long as you can get the money.
But the metrics of success, by and large, are papers.
I'm gonna skip this just because it's late in the day.
So some obstacles, and I'm sure there are many, these are some obstacles I have found to engaging kind of the academic side.
A lot of the workshops, academic workshops, are invite-only.
You don't hear about them.
Right before I left, I sponsored a really cool one, which is great.
It was great timing on trustworthy algorithms, you know, fake news.
And it was fascinating.
But very explicitly, my boss, kind of the government folks, wanted just academics.
And of course, if you do that, you're not going to get necessarily interesting opinions and ideas.
Failure is considered bad.
You think in science that failure is encouraged in experimentation.
Failure is considered bad because then you don't have a paper that says you did something interesting.
Whereas if it's really, really incremental, you know, we call it just a hack of something, that's praised because it's gone well.
Conferences, if you haven't been to an academic conference, it's mostly people reading papers that were already put online.
There's not a lot of socializing, except for maybe finding a collaborator you by and large knew at grad school or some other conference.
The incentive in that world, and I think we have to be very mindful, what's the incentive?
It's tenure, if you're an academic, which requires papers at top conferences, publishing often.
At many, you know, MIT, Stanford's nowadays, in the past about five years, they want you to show that you have some form of startup or entrepreneurial experience.
And of course, getting grants from funding agencies, it's all about how much money you can get in there.
So the driving factor, you know, if you look at this in a negative way, is not necessarily making security and privacy better in a tangible way.
You know, we can look at that and say, well, that is what they're doing, but in a very tangible way, the goal is to get tenure, and that's by doing these things.
Some obstacles to engaging, I'll call non-academic researchers.
Cons like this are culturally a little hard to navigate if you don't know people, or you're introverted, or you know, maybe you don't have something to do with the village.
So I think for a lot of people, they're culturally hard to kind of get into.
A lot of people don't attend these events, so their ideas or their work is a little hard to track down.
And people attend, many of them don't give speeches, they don't, you know, broadcast their ideas.
Of course, we know not all motives are altruistic.
A lot of the work, however, you know, I think is done by many people for personal satisfaction, for the challenge.
It's not necessarily, you don't really necessarily want, you know, car companies to fix things, but for your own challenge, to show that you had an idea and it came to fruition.
And I think a lot of people lack the interest in time and collaborating with any kind of official entity.
I won't read all of this, but so one, you know, one thing here is, well, can I just read the research and get inspired?
And it's interesting, I've noticed the past four to five years, and I've heard it already today, and I've only been here since mid-afternoon, is a lot of people do read academic papers or, you know, want to be able to get to the original source.
So academic conferences might publish the full papers online.
Workshops occasionally have the white papers online.
I find those two things, actually, I find them easier to digest oftentimes because they're shorter.
Since 2013, publicly funded grants or agencies with that grant over 100 million dollars in R&D, research and development, have to make the data publicly accessible to search, retrieve, and analyze.
They're still figuring out how, but it's via this memo.
Journalists, however, are always behind a paywall, almost always.
It's a big problem with a lot of debate.
There's one publishing company that you have to pay them to publish, you have to pay them to read it.
So there's a workaround called Sci-Hub, which has 64 million articles up there, and they claim to have 85% of the paywalled scholarly articles up there for you to read.
So a lot of people do that as a workaround through the paywall.
And I just found this really interesting quote that people have, you know, time to become experts on quackery and pop science.
Wouldn't it be nice to start seeing them take up actual science as a hobby and be able to read these papers?
There's a really cool paper in the Atlantic, or article in the Atlantic, a few weeks ago along this line saying the more kind of sophisticated science becomes, the harder it is to communicate results.
So if you try to read some of these onerous papers, the concept's often very simple.
The communication of it can tend to be very onerous.
So it's a very, it's a fascinating article.
It's kind of a long read, but very interesting.
I just noted, you know, if you're not familiar with these folks, I wasn't, but there's some really interesting work that's been produced by, just like plucked out a handful of people with some cool projects that I think would be kind of interesting takers on this idea.
Yoshi Kono, who's University of Washington, he had this article on the top about encoding malware into a strand of DNA.
Damon McCoy does all this cool stuff on looking at the Silk Road and the dark web and analyzing Craigslist rental scams and Nigerian gangs and things like that.
Reshipping and mule scams, it's how to use your Visa card to do reshipping scams.
Stefan Savage has this awesome project on measuring cybercrime and actually how much money is made off the dark web and cybercrime.
People talk about this, you know, but he's actually measuring how much money is made there.
He also did this remote car hacking in 2010.
So what if we brought the two sides together?
One of the, you know, we see this time gap kind of between academic stuff and stuff you see at these cons.
So, you know, if you went to DEF CON last year, this voting village had tons of press attention saying it was a first-ever look at voting security and they had all these voting machines for the first time.
People have hacked into them, but people have actually been doing this work for years.
It's just not published.
It's not publicized very well.
There are tons of, you know, kind of obscure journal stuff and different workshops on how people have hacked voting machines.
But again, you know, were we to get that out sooner, were we to kind of marry it up with people who could make a splash in the press, you know, perhaps that can make a difference.
This has nothing to do with security, but I thought this was a super cool project.
This guy came to talk to us.
He's a computer scientist at Brandeis.
It's a project called Digital Ahmadi.
And he is a computer scientist, a mathematician, and for whatever reason, even though he doesn't play an instrument, wanted to create the perfect cello.
So he spent like five years tracking down these artists and people who make cellos.
They're more into the art of it, not so much the kind of equations of it.
Well, he's really into the computational thinking.
So he thought, what if he could bring together his computational thinking and Euclidean geometry and the art of this cello and kind of the science of instrument design.
So he did this thing called a geometry engine, and he wanted to design the perfect cello to sound like these, you know, ones that were made in the 18th century.
And it took him years before people even... this famous one, he wanted to do a CT scan to see, you know, what does it really look like?
And it took him like three or four years.
He had to fly to Italy in his vacation time and convince them that he was worthy of what he did.
They said, you don't even play the cello.
Why do you want to do this?
But it was his fascination with kind of the perfect shapes and angles.
And as it turns out, they have a wonderful collaboration now of kind of arts and music and in computation.
So I thought that was just kind of an inspiring example.
Some prior efforts at DARPA.
When Mudge was at DARPA, there was a cyber fast track where they gave micro grants to just kind of average people.
He noticed that, you know, the cyber incidents kept increasing from, you know, kind of the five years prior to him going to DARPA.
But also spending on security was increasing by the federal government.
So what can we do?
So one of his answers was, you know, kind of give out these micro grants as an alternative to traditional funding.
And they averaged a week from proposal to giving the funds.
One of the projects that came out of it was the car hacking research that was presented in 2015.
Other efforts, I just wanted to put these kind of down.
There's things like experience tracks at academic conferences where you're not an academic but you have experience as an operator.
Case studies to show, you know, for human computer interaction and in AI conferences.
So there's kind of like other thinking but nothing really formal.
Some more efforts, bug bounties.
I'm the Cavalry is kind of a nascent or I guess not so nascent effort on promoting public safety research.
There's a lot of public private threat sharing, the ISACs and kind of different places who share threat indicators.
I thought this was cool, this workshop in London, you know, where they were trying to get together academics and kind of business people.
They had some ideas, you know, a number where you can kind of pair up a business person to bounce ideas off of and an academic.
Hackathons extended beyond just kind of the coding phase but, you know, how can you, you know, have an opportunity to learn and pitch and talk to each other.
So things like that to kind of make it more sustainable than just a hackathon weekend.
So what can we do with the current model?
Some adventurous faculty have said, you know, they'd be happy to sponsor kind of non academics to participate, to give them micro grants, encouraging particularly undergrads who don't have a lot of these boundaries in the way they think.
A lot of the government agencies could easily kind of include this underground component and I think inviting non academics to workshops.
So in this kind of underground area, what, you know, what can be done?
Some of these crazy academics who are interested in, you know, who have been pushing the boundaries explicitly involve them.
They cannot do offensive research with these grants.
They're banned from doing that.
So this is an opportunity, many of them that I spoke to said, this is a great way for us to not do it, but give you a micro grant and you can try it and tell us what happened.
So it's, you know, from their thinking this is like a win-win for them.
Maybe there's some matchmaking that can be made at these different cons.
It's interesting, many academic, like more than more than a handful who I've run across who are, and some of them are crypto experts, say only do they not use PGP.
They can't figure out many of the open source tools that many of us use.
You know, simple things like LastPass and other kind of tools because they're not used to the practical hands-on experience.
So can we, you know, school them on the just the practical design of usability for some of these tools.
So I think a part of the challenge of how we is kind of non-professional, you know, community is how do we broadcast what we know and find out.
Many of you, many in this world are kind of these media savvy people.
Some are not.
They don't want to be broadcasting things.
Is there a way, is there a kind of middle ground to help broadcast some of the results?
Are there things that haven't been tried?
You know, a journal.
Journals, the challenges, who would peer review?
It takes a lot of time.
Who put the time into organizing it, getting reviews?
It's a great place to show off your work.
It takes a lot of work to write, to read, to peer review.
Maybe a virtual journal.
Maybe for some of these CTFs, bringing academics and hackers together as equals.
You know, setting up, participating, doing the actual grunt work is something, you know, that we learn a whole lot.
Maybe there's a new joint neutral venue or conference, physical meetings and virtual meetings, kind of bring people together.
Does the model work elsewhere?
I just want to bring this up.
So I was involved in the bodybuilding world and it's very prevalent in sports performance, at least bodybuilding.
I put up there kind of the picture of Mr.
Olympia contenders and supplements.
They're called gurus, which makes you think of kind of the traditional guru.
The gurus are like these guys with the green check.
They're like guys who are really good at figuring out supplements, training, timing, all this stuff.
So that was a very interesting, you know, kind of convergence of you have people trained in this stuff.
They're not necessarily the ones that athletes are using.
I think the biohacking type stuff, quantified self, this guy Larry Smarr, he's an academic computer science professor, does a lot of this quantified self stuff, but he's got the tools at hand.
He's got a lot of expensive tools to be able to measure different things about his body.
And you have people who go to meetups and they're doing a lot of the same kind of thing, just kind of on a different level.
Could we get a feedback loop between the two?
So that's that's my idea.
Thank you for sticking with me to listen through it.
If you have some comments or thoughts, or think it's stupid or great, or want to make something happen, let me know.