[00:55.310 --> 00:58.210]  Hey, if you're coming in, I'm going to start talking soon.
[00:58.210 --> 01:00.010]  You don't have to start listening.
[01:00.010 --> 01:01.870]  I'm going to start talking.
[01:02.810 --> 01:07.150]  So, what are we doing here?
[01:07.950 --> 01:14.630]  If you look at the title and this ain't the talk you thought you were going to be in, you ain't going to hurt my feelings if you leave.
[01:14.810 --> 01:19.430]  We've already discussed how nervous everybody in here is, starting with me.
[01:19.730 --> 01:22.730]  So, let's get after it, okay?
[01:22.890 --> 01:26.790]  I hate standing up at a podium.
[01:27.830 --> 01:30.630]  I like walking around.
[01:31.290 --> 01:34.330]  So, there's pros and cons to that.
[01:34.330 --> 01:39.010]  So, this is infiltration and betrayal.
[01:39.250 --> 01:45.270]  This is an opportunity to watch whether or not I can actually do this.
[01:45.610 --> 01:46.530]  Oh, there we go.
[01:46.530 --> 01:47.230]  Okay, good.
[01:47.230 --> 01:50.810]  So, you guys are lucky I don't get to go all the way down there.
[01:50.990 --> 01:51.850]  All right.
[01:52.010 --> 01:53.570]  So, what are we doing?
[01:53.570 --> 01:54.750]  This is what we're doing.
[01:56.810 --> 02:03.850]  Is there anybody in here that thought they were going to come see Josh Knox talk?
[02:04.470 --> 02:05.310]  Okay.
[02:05.310 --> 02:07.090]  I'm not Josh Knox.
[02:07.090 --> 02:10.310]  I actually replaced Josh Knox at our company.
[02:10.650 --> 02:12.590]  He's a really smart guy.
[02:12.590 --> 02:14.490]  You guys are stuck with me.
[02:14.490 --> 02:17.150]  I am a born-again United States Marine.
[02:17.150 --> 02:24.510]  So, if you feel like somebody dropped a revival tent into boot camp graduation, you are in the right place.
[02:24.730 --> 02:27.190]  And I did a lot in OWASP.
[02:27.190 --> 02:29.590]  Is there anybody who doesn't know what OWASP is?
[02:30.310 --> 02:31.650]  It's cool if you don't.
[02:31.810 --> 02:33.390]  So, I founded a couple chapters.
[02:33.390 --> 02:34.450]  I started a conference.
[02:34.450 --> 02:36.090]  I did a whole bunch of stuff.
[02:37.590 --> 02:43.550]  Once upon a time, I was in charge of a lot of people doing a lot of security activities.
[02:44.130 --> 02:45.370]  And we were effective.
[02:45.370 --> 02:46.390]  It was good.
[02:46.490 --> 02:55.890]  But today, I'm actually a honeybee wrangler and solutions architect, technical marketing manager, some freaking guy working for Reversing Labs.
[02:55.890 --> 02:57.390]  It's a great company.
[02:57.670 --> 03:00.270]  So, what does Reversing Labs do?
[03:00.270 --> 03:05.590]  Somebody who has stopped by the booth, how many petabytes of samples?
[03:05.830 --> 03:06.950]  27.
[03:07.930 --> 03:15.050]  There's a big freaking data lake of malware that's been analyzed since 2009, capturing behaviors.
[03:15.050 --> 03:16.410]  So, that's Reversing Labs.
[03:16.410 --> 03:21.090]  We're pretty dang good at helping you understand how to trust software.
[03:21.470 --> 03:26.690]  Most people, when you look at something, it kind of looks like a black box, right?
[03:27.250 --> 03:31.530]  Hey, if I'm going to install software, I double click on something and it installs.
[03:31.530 --> 03:35.230]  Or, you know, I run a single line and it installs.
[03:35.510 --> 03:39.850]  But there are things to worry about.
[03:39.850 --> 03:48.770]  And when you start to look at that box, is there anybody that's a developer in here?
[03:49.150 --> 03:50.150]  Okay.
[03:50.190 --> 03:54.270]  How much of your release is first-party code?
[03:56.840 --> 03:57.600]  50?
[03:57.600 --> 03:58.120]  Okay.
[03:58.120 --> 04:04.560]  So, 50% of your release is first-party code written by guys in your shop.
[04:04.560 --> 04:14.720]  50% is coming down from someplace else, whether it's a library, probably on the internet, and it ain't your code, right?
[04:14.840 --> 04:16.580]  And what does it come with?
[04:16.580 --> 04:22.460]  Well, it comes with all kinds of stuff, and that's what we're trying to show here, right?
[04:22.580 --> 04:24.920]  So, who's this talk for?
[04:24.940 --> 04:26.420]  Remember I told you I was a beekeeper?
[04:26.420 --> 04:27.720]  Those are my bees.
[04:27.720 --> 04:32.900]  So, if you don't want to get stung by software supply chain attacks, hey, you're here.
[04:33.600 --> 04:35.240]  How we doing on the jokes?
[04:35.240 --> 04:36.060]  I'm laughing.
[04:36.060 --> 04:36.780]  We okay?
[04:36.780 --> 04:37.220]  We're good?
[04:37.220 --> 04:38.060]  Okay, all right .
[04:38.520 --> 04:41.480]  Hey, here's your first pop quiz.
[04:42.360 --> 04:43.940]  What's worse than Vones?
[04:43.940 --> 04:46.000]  Raise your hand if you want to answer this.
[04:48.100 --> 04:49.140]  Yeah.
[04:50.280 --> 04:52.020]  Exploits are worse than Vones.
[04:52.160 --> 04:57.500]  How do exploits look when they're in an open source library?
[04:57.580 --> 04:59.140]  They look like that.
[04:59.780 --> 05:12.060]  Is there anybody who doesn't realize that people have figured out that if they can backdoor an upstream library, they own your release?
[05:12.780 --> 05:14.180]  What'd you say?
[05:16.580 --> 05:17.620]  SolarWinds.
[05:17.620 --> 05:19.020]  We'll talk about them.
[05:19.020 --> 05:24.260]  So, SolarWinds is actually, they weren't a problem with an upstream library.
[05:25.000 --> 05:29.040]  They did get owned and they are bouncing back hard.
[05:29.120 --> 05:34.740]  But I'll show you, is there anybody in the room that doesn't have a browser on their phone?
[05:35.400 --> 05:39.600]  Okay, I'll show you something that I think will help everybody in here.
[05:40.140 --> 05:42.100]  So, what's an XE?
[05:42.100 --> 05:44.400]  Is there anybody who doesn't know what an XE is?
[05:44.400 --> 05:48.040]  Anybody who doesn't realize that XE utils got breached?
[05:48.240 --> 05:49.140]  Anyone?
[05:49.680 --> 05:52.060]  Hey, let's start with what is the internet?
[05:54.200 --> 05:55.600]  What's the internet?
[05:55.820 --> 06:00.580]  Hey, you know, people think about the internet and it's all this stuff, right?
[06:01.200 --> 06:13.260]  And somewhere underpinning it is a single project maintained by a single guy or gal, right?
[06:13.320 --> 06:14.500]  Am I making this up?
[06:14.500 --> 06:15.640]  Is this news to anybody?
[06:15.640 --> 06:16.860]  This is real.
[06:17.040 --> 06:19.940]  So, where were you in 2009?
[06:21.080 --> 06:22.640]  Anybody remember?
[06:22.840 --> 06:25.940]  Hey, miracle on the Hudson, remember that, right?
[06:25.940 --> 06:32.520]  Hey, there's things going on and there's a guy named Lassie Collin who got a new hobby, right?
[06:32.520 --> 06:33.860]  So, who's Lassie Collin?
[06:33.860 --> 06:35.200]  You remember this?
[06:38.520 --> 06:44.680]  Lassie Collin was the sole maintainer of a library called XE utils.
[06:44.680 --> 06:47.120]  What rides on XE utils?
[06:47.220 --> 06:53.840]  Everything that matters in terms of Linux compression and SSH.
[06:54.240 --> 06:56.280]  What could go wrong?
[06:57.820 --> 07:00.400]  That is the world, gang.
[07:00.420 --> 07:02.460]  So, what happened?
[07:02.460 --> 07:04.680]  So, think about it like this.
[07:05.940 --> 07:08.640]  It's party time and we're all pinatas.
[07:10.320 --> 07:16.740]  So, does anybody want to tell who Chen Zhuang is?
[07:16.740 --> 07:17.560]  Anybody?
[07:18.460 --> 07:20.200]  Okay, I'm gonna walk around now.
[07:20.460 --> 07:26.100]  So, Chen, is there anybody who hadn't heard of log4j?
[07:27.700 --> 07:31.540]  Raise your hand if you don't have any idea what log4j is.
[07:31.540 --> 07:33.000]  You don't, okay.
[07:33.080 --> 07:39.580]  Everybody in here that didn't raise your hand, one of you wants to explain what it was, raise your hand.
[07:40.720 --> 07:42.120]  Oh, you guys made it up.
[07:42.120 --> 07:43.100]  You don't know.
[07:43.100 --> 07:44.360]  Okay, great.
[07:44.360 --> 07:45.420]  All right.
[07:45.420 --> 07:49.080]  All right, log4j was a Java library.
[07:49.080 --> 07:51.420]  It's used all over the world.
[07:51.700 --> 07:58.360]  One day, the internet melted to the ground because why?
[07:58.360 --> 08:05.840]  Because log4j has got a remote command execution vulnerability and nobody can tell you where it is.
[08:05.840 --> 08:06.940]  Where's log4j?
[08:06.940 --> 08:08.760]  Oh, I don't know, it's a really good library.
[08:08.760 --> 08:10.380]  Yeah, but is it in your stuff?
[08:10.500 --> 08:11.580]  Well, I don't know.
[08:12.260 --> 08:14.280]  All right, so who's this guy?
[08:15.520 --> 08:23.440]  This is a Chinese guy who found the problem and he was told by his government not to tell anybody.
[08:23.920 --> 08:29.780]  So, if you think you had a bad week when log4j was coming, here's a bad week.
[08:29.780 --> 08:30.980]  So, that's Chen.
[08:30.980 --> 08:34.040]  So, who's the Chen for XZ?
[08:34.060 --> 08:37.840]  Well, does anybody know this story?
[08:38.100 --> 08:41.920]  Has anybody ever had their CPU glow red?
[08:43.220 --> 08:51.900]  So, this guy, Andres, hey, I accidentally found a security issue.
[08:56.560 --> 08:58.500]  So, what happened?
[08:58.500 --> 09:03.660]  So, this guy is a beta tester for Linux distros.
[09:03.660 --> 09:08.100]  He's got an interest in Postgres, right?
[09:08.360 --> 09:10.120]  And he's a maintainer for that.
[09:10.120 --> 09:14.020]  And all of a sudden, updates to the beta code and what happens?
[09:14.460 --> 09:20.840]  Does anybody else hear their fan go into overdrive when there's bad code on the system?
[09:20.840 --> 09:21.560]  Yeah.
[09:21.560 --> 09:25.500]  Hey, something's going on and it's a security issue.
[09:25.500 --> 09:26.180]  Oops.
[09:26.640 --> 09:30.300]  So, Andres is the Chen for XZ.
[09:30.300 --> 09:33.740]  So, let's talk about when zombies attack, all right?
[09:33.940 --> 09:40.040]  Hey, it's always a normal day when zombie movies start.
[09:40.760 --> 09:43.120]  But, you know what?
[09:43.500 --> 09:47.240]  Lassie Collin, remember the guy who started his hobby in 2009?
[09:47.240 --> 09:48.760]  Was having weird days.
[09:48.760 --> 09:50.380]  What's weird mean?
[09:53.240 --> 09:56.420]  Long-term mental health issues?
[09:58.520 --> 09:59.560]  Okay.
[09:59.560 --> 10:05.640]  So, is there anybody in the room that is actually a maintainer for an open source library?
[10:06.060 --> 10:06.980]  Anyone?
[10:07.520 --> 10:08.420]  Okay.
[10:08.420 --> 10:10.520]  There aren't a lot of those people around.
[10:10.520 --> 10:13.960]  This guy's been working it since 2009.
[10:14.740 --> 10:18.600]  And it ain't helping his mental health, right?
[10:18.600 --> 10:21.060]  So, that's what a bad day looks like.
[10:21.320 --> 10:24.600]  So, what do you do if you're an opportunist?
[10:24.600 --> 10:26.320]  There's an alias.
[10:26.320 --> 10:27.680]  There's a guy.
[10:27.820 --> 10:29.500]  There's a handle.
[10:29.940 --> 10:31.740]  Geotan, right?
[10:31.740 --> 10:35.560]  So, kind of hard to see this, but here's Lassie.
[10:35.840 --> 10:41.360]  Lassie is doing commits, updating the mailing list, XZ utils.
[10:41.500 --> 10:44.200]  Hey, we're doing what we've been doing since 2009.
[10:44.200 --> 10:45.340]  We're writing code.
[10:45.340 --> 10:46.700]  We're pushing it.
[10:46.700 --> 10:53.940]  We're trying not to let the world know that we're slowly going crazy at our little project.
[10:54.360 --> 11:02.660]  And we met a really nice guy who drops into the mailing list and says, hey bud, I can take care of you.
[11:02.660 --> 11:03.880]  We're gonna be okay.
[11:04.400 --> 11:06.720]  So, if you're Lassie Kong, what do you say?
[11:06.720 --> 11:09.020]  Oh, hallelujah, right?
[11:10.720 --> 11:12.780]  So, what really happens?
[11:12.800 --> 11:15.160]  So, how do we know this guy's a bad guy?
[11:15.160 --> 11:27.040]  Well, there's a couple things, but one of the things that was orchestrated is, hey, here's Gia, says, hey, I got a fix for you.
[11:27.040 --> 11:29.460]  I got help, right?
[11:29.880 --> 11:37.780]  At the same time, more or less, there are a couple of clowns that have never been seen before or since that are leaning on the project hard.
[11:37.960 --> 11:39.920]  Hey, this needs to get changed.
[11:39.920 --> 11:41.760]  Hey, this commit needs to happen.
[11:41.760 --> 11:43.320]  Hey, you need help.
[11:44.440 --> 11:46.000]  Where'd they come from?
[11:46.000 --> 11:48.040]  Well, they just came from zombie land.
[11:48.260 --> 11:53.220]  So, we think they're... and gang, this is one of the things we don't know.
[11:53.540 --> 11:54.880]  We don't know.
[11:54.880 --> 11:56.480]  That's part of the talk.
[11:56.480 --> 11:58.320]  How many more of them are out there?
[11:58.520 --> 11:59.620]  I don't know.
[11:59.620 --> 12:01.460]  You guys don't even know what log4j is.
[12:01.460 --> 12:02.540]  How would you know?
[12:03.620 --> 12:07.060]  So, anyway, so talk about spreading infection.
[12:07.060 --> 12:10.920]  So, here's Gia, right?
[12:11.660 --> 12:15.480]  Hey, so he's making... he's making updates to the mailing list.
[12:15.480 --> 12:21.260]  So, the maintainers of Debian, Kali, and Red Hat, those are some minor Linux distributions.
[12:21.260 --> 12:23.280]  You may have heard of them, right?
[12:23.780 --> 12:27.780]  Those guys are like, oh, hey, this Gia must be for real.
[12:27.780 --> 12:34.930]  And when he pushes code, that's okay.
[12:34.930 --> 12:36.470]  That's cool.
[12:36.470 --> 12:38.950]  Hey, let's add a backdoor when we're pushing code.
[12:38.950 --> 12:50.470]  So, we're gonna talk about this very briefly, but there was one red flag before the push.
[12:50.470 --> 12:51.810]  And only one.
[12:51.950 --> 13:02.730]  If you are looking at an insider threat, which is what this became, you're gonna look for different signals than adopters.
[13:03.010 --> 13:06.470]  The signal is very faint and we'll talk about that in a second.
[13:06.470 --> 13:09.110]  Alright, so how close was it?
[13:09.170 --> 13:11.050]  Hey, you know what?
[13:11.650 --> 13:13.930]  This is how close it was.
[13:18.560 --> 13:29.820]  You remember how this is an underpinning library for basically the entire Linux infrastructure to rely on for security and compression?
[13:31.180 --> 13:33.420]  That library made it to beta.
[13:36.670 --> 13:45.030]  So, if there wasn't one guy to save the Internet, then we're all owned.
[13:45.030 --> 13:46.230]  Everybody running Linux.
[13:46.230 --> 13:47.810]  That's what that's about, right?
[13:47.810 --> 13:49.590]  So, we need zombie detection.
[13:49.590 --> 13:50.890]  What's that look like?
[13:51.370 --> 13:53.830]  Hey, that's a clever slide.
[13:54.330 --> 13:55.710]  Let's try this one.
[13:55.710 --> 13:57.150]  I like this one better.
[14:01.850 --> 14:04.650]  Is there anybody in here who's like, yeah, that was a slam dunk.
[14:04.650 --> 14:06.010]  That was so stupid and easy.
[14:06.010 --> 14:07.290]  I would have nailed that.
[14:07.330 --> 14:08.970]  I would have nailed that in my sleep.
[14:08.970 --> 14:10.290]  This is a big deal.
[14:10.290 --> 14:11.930]  And this is... there's more common.
[14:11.930 --> 14:13.890]  This button doesn't exist.
[14:14.670 --> 14:16.830]  I wish it did, but it doesn't.
[14:16.830 --> 14:30.550]  So, on the day that that broke, the guy I replaced, Knox, was actually giving a webinar, and one of the questions was, hey, would your stuff detect the XE-Utils breach?
[14:30.550 --> 14:39.770]  Our architect, who was on the phone and is one of the smartest guys on the planet, said no.
[14:41.230 --> 14:43.630]  Inside job, very difficult.
[14:43.630 --> 14:57.340]  There were signals, but nobody, none of our customers, would have blackballed or black-holed that update based on that signal, right?
[14:57.340 --> 14:59.660]  But let's show you the signals, all right?
[14:59.660 --> 15:05.980]  So, in zombie movies, y'all know, like, it starts with a twitch, right?
[15:06.080 --> 15:13.240]  There's just this little, something's gonna go wrong here, and it's gonna be very wrong, and it's gonna happen very quick, you know?
[15:13.240 --> 15:19.600]  And in this case, you know, you've got one thing that's red and one thing that's green, right?
[15:19.760 --> 15:21.960]  So, what does this mean?
[15:21.960 --> 15:37.680]  So, in order for Gia to push his change, he had to back off on some of the built-in hardware hardening functions for the code.
[15:38.200 --> 15:42.720]  How many of you are testing to see if that ever happens?
[15:42.720 --> 15:44.660]  So, let me give an example.
[15:44.660 --> 15:52.600]  So, in Microsoft Windows, since Windows XP, in what, 2003, 2004?
[15:52.800 --> 16:00.260]  There's been a built-in operating system hook called data execution prevention.
[16:00.760 --> 16:05.740]  You hit a compiler switch, your code compiles to take advantage of that.
[16:05.740 --> 16:16.800]  If you don't hit that switch, your code may still be susceptible to treating data as though it were data, as though it were something that could be executed.
[16:16.800 --> 16:18.940]  That's an example of hardening.
[16:18.940 --> 16:27.320]  That's an example of what's missing here, is that, hey, we went from one level of protection to a lesser level of protection.
[16:27.320 --> 16:30.000]  And that was the whole story, right?
[16:30.000 --> 16:31.920]  So, what's that look like?
[16:31.920 --> 16:33.840]  Hey, you know what?
[16:33.980 --> 16:36.540]  That's not a very big twitch.
[16:37.180 --> 16:45.940]  So, one of the things that we do, remember, my chief architect was running with this thing and said, nobody is going to look for that.
[16:47.000 --> 16:51.660]  Nobody is going to look for one red, one green, one hardening error.
[16:52.460 --> 16:59.300]  So, let's build a rule like this that says, hey, man, I don't know what just happened, but it looks a lot like XE-utils compromised.
[16:59.300 --> 17:00.260]  It's a problem.
[17:00.420 --> 17:08.560]  So, when you see that twitch, if you've never been in a zombie movie before, do you know a zombie outbreak is coming?
[17:08.700 --> 17:10.460]  No, right?
[17:10.460 --> 17:18.700]  If you haven't spent a lot of time trying to understand that hardening itself can be a clue, do you know a zombie outbreak is coming?
[17:18.700 --> 17:19.560]  No, you don't.
[17:19.560 --> 17:20.540]  So, that's what this is.
[17:20.540 --> 17:22.500]  This is a rule, right?
[17:22.500 --> 17:24.740]  So, it's fairly verbose.
[17:24.740 --> 17:26.580]  It's cool, though, right?
[17:26.680 --> 17:39.120]  So, we go from this subtle thing, which is this, to, hey, you can't miss this, it's in your face, there's a problem, and it looks a lot like XE-utils, right?
[17:39.120 --> 17:41.380]  So, that's what you can do.
[17:41.380 --> 17:45.000]  So, what do we do to build an army of zombie hunters?
[17:45.580 --> 17:54.760]  So, we've published... if you all... is there anybody in the room who didn't think this was going to be about software supply chain security?
[17:55.640 --> 17:59.420]  The guy that's leaving didn't think it was about software supply chain security.
[17:59.420 --> 18:00.320]  That's cool.
[18:00.400 --> 18:01.920]  Hey, there's a really good report.
[18:01.920 --> 18:02.900]  It's available for free.
[18:02.900 --> 18:05.060]  It'll tell you what's going on.
[18:05.160 --> 18:07.600]  What's going on is that the problem is accelerating.
[18:07.600 --> 18:08.560]  Go figure.
[18:08.920 --> 18:19.220]  So, if you want to join an army of zombie fighters, open SSF, right?
[18:20.040 --> 18:24.540]  If you want to give them the tools, though, and this is where I need you to bring up your phone.
[18:25.420 --> 18:26.580]  Hang on.
[18:26.580 --> 18:27.700]  Come on.
[18:27.700 --> 18:28.720]  There we go.
[18:29.800 --> 18:30.920]  All right, guys.
[18:30.920 --> 18:37.440]  So, bring up your phone to secure.software, not secure.software.com.
[18:38.080 --> 18:39.420]  And this is free.
[18:39.420 --> 18:41.780]  This is our contribution to the community.
[18:48.110 --> 18:49.650]  You guys still hear me?
[18:50.150 --> 18:51.070]  All right.
[18:51.370 --> 18:54.810]  So, get up to secure.software.
[18:55.110 --> 18:57.050]  Does anybody need more time?
[18:57.910 --> 19:02.370]  Okay, when you get up there, you'll see there's a field you can enter stuff in.
[19:02.410 --> 19:09.510]  There's a drop-down that'll let you pick which repo or language that you want to check.
[19:09.510 --> 19:13.450]  So, in my case, what I've got... so, I've picked PyPy.
[19:13.450 --> 19:16.110]  I've put in a string cryptography.
[19:16.930 --> 19:25.670]  And I don't know if you all can see this real well, but the number one hit and a top 100 package is a library called what?
[19:26.070 --> 19:27.030]  Cryptography.
[19:27.490 --> 19:28.410]  Hey, cool.
[19:28.410 --> 19:34.030]  It ain't perfect, but it's not evil.
[19:34.330 --> 19:36.230]  Scroll down a little bit.
[19:37.870 --> 19:39.890]  Hey, what's cryptography?
[19:39.890 --> 19:40.950]  Why?
[19:45.820 --> 19:49.620]  That thing's backdoored and it has been for two years.
[19:50.880 --> 20:05.700]  So, if you don't believe that there are people right now today that are posting libraries that are deliberately designed for somebody to pull and backdoor the release, go to Secure.software.
[20:05.700 --> 20:06.680]  It's free.
[20:06.740 --> 20:08.080]  Show your developers.
[20:08.080 --> 20:14.460]  Whatever language they're developing in, just throw in a random string and look at what the results are.
[20:14.960 --> 20:15.760]  It's happening.
[20:15.760 --> 20:16.960]  It's happening today.
[20:17.240 --> 20:19.600]  Any questions about that?
[20:19.620 --> 20:22.500]  About Secure.software and that as a tool ?
[20:23.660 --> 20:25.580]  Dan, did I forget anything?
[20:26.120 --> 20:27.060]  Okay.
[20:27.060 --> 20:28.000]  All right, guys.
[20:28.000 --> 20:28.700]  Well, cool.
[20:28.700 --> 20:33.240]  In that case, back to the slide where and we'll wrap up pretty quick.
[20:33.300 --> 20:35.500]  So, somebody said SolarWinds.
[20:35.500 --> 20:36.640]  This is Tim Brown.
[20:36.640 --> 20:39.100]  He was on the ground when SolarWinds got burned.
[20:39.240 --> 20:44.880]  This dude has been working his ass off to make SolarWinds' release better and it is.
[20:44.880 --> 20:48.860]  And if you don't believe me, I will go toe-to-toe with anybody.
[20:49.140 --> 21:09.120]  What the Open Source Foundation, Open Source Software Foundation says to do is, okay, if you know that there are ugly libraries out there and you know you're only writing 50% first-party code, make yourself a secure repo.
[21:09.660 --> 21:12.640]  Have your developers pull from that.
[21:12.860 --> 21:17.980]  And furthermore, check things before it goes in, right?
[21:17.980 --> 21:21.480]  So, this is the guidance from ESF.
[21:21.480 --> 21:26.340]  You can find it and the bottom line is, what are you going to do?
[21:26.840 --> 21:28.720]  So, you're going to do a couple of things, right?
[21:28.720 --> 21:34.740]  You're still going to pull down libraries, but you're going to stage them in some place where they get inspected.
[21:35.180 --> 21:43.220]  Even if that inspection is just you manually going to secure.software and going, hey, is this library backdoored?
[21:43.220 --> 21:44.520]  Maybe so.
[21:44.520 --> 21:46.000]  And then what?
[21:46.000 --> 21:51.040]  And then don't post it to your secure component library.
[21:51.180 --> 21:52.860]  Everybody still with me?
[21:52.860 --> 21:54.280]  I lose anybody?
[21:54.440 --> 21:56.140]  All right, good.
[21:56.140 --> 21:58.860]  All right, what's changed?
[21:58.860 --> 22:12.200]  So, it's gone from a single contributor plus a special helper to there are a dozen contributors now on the Xeutils project.
[22:12.240 --> 22:14.060]  What does that mean to us?
[22:14.060 --> 22:15.540]  Anybody know?
[22:17.120 --> 22:20.220]  Well, we've got about 12 times the mystery.
[22:20.320 --> 22:22.280]  Who's the next Gia Chan?
[22:22.280 --> 22:23.720]  Who's the next zombie?
[22:23.720 --> 22:25.120]  Is it this guy?
[22:25.560 --> 22:27.860]  Don't know, right?
[22:27.860 --> 22:29.920]  So, that's where we are.
[22:30.220 --> 22:32.320]  What are the questions?
[22:35.640 --> 22:36.600]  Anyone?
[22:37.640 --> 22:38.440]  Yeah.
[22:41.670 --> 22:45.010]  Okay, so the question, Brad, I'm doing this.
[22:45.010 --> 22:49.410]  The question was, is this an argument against open source software?
[22:49.550 --> 22:53.410]  I think you could take it that way.
[22:53.710 --> 23:00.570]  The thing with open source software is we've trusted it, right?
[23:00.570 --> 23:03.590]  It's a benevolent force in the universe.
[23:04.430 --> 23:06.410]  Sometimes it's not.
[23:06.410 --> 23:10.630]  So, is it an argument to drop open source?
[23:10.630 --> 23:15.670]  I don't think so, because then you'd be writing 50% more code, right?
[23:15.670 --> 23:22.270]  And there aren't a lot of development teams that are like, yeah, I can write log4j from scratch, right?
[23:22.270 --> 23:23.530]  Just not happening.
[23:23.610 --> 23:42.210]  So, what it is, is if you're a developer and you're aware of this problem, and you're aware that some of these sole maintainers are having freaking mental health problems, and they're still soldiering on, hey, you can help.
[23:42.210 --> 23:43.190]  Help them.
[23:43.710 --> 23:45.290]  That's my take.
[23:45.290 --> 23:46.410]  What do you think?
[23:46.630 --> 23:47.790]  That work?
[23:51.090 --> 23:52.450]  There you go.
[23:52.450 --> 23:53.070]  All right.
[23:53.070 --> 23:55.250]  Anybody else got an opinion on that?
[23:55.250 --> 23:55.970]  Yes, sir.
[24:09.440 --> 24:10.580]  There you go.
[24:10.580 --> 24:24.280]  So, if you couldn't hear, basically, hey, we don't have the resources to rewrite everything we're using from open source, nor do we have the resources to not check what we're using.
[24:24.280 --> 24:25.740]  Did I get that about right?
[24:27.060 --> 24:27.660]  Yeah, yeah.
[24:27.660 --> 24:28.860]  What's your question?
[24:36.710 --> 24:38.210]  Yes, dependencies.
[24:39.430 --> 24:40.630]  Yes.
[24:41.390 --> 24:47.090]  So, the question was, hey, are transitive dependencies on the report?
[24:47.090 --> 24:55.450]  So, if you don't know what a transitive dependency is, I think log4j is actually a really good example, right?
[24:56.690 --> 24:58.510]  There's a library.
[24:59.330 --> 25:02.170]  It's not all first-party code.
[25:02.610 --> 25:05.010]  There's third-party code in there, too.
[25:05.010 --> 25:08.190]  It's called a transitive dependency, right?
[25:08.190 --> 25:16.930]  And so, the question, as I understand it, was, hey, man, if one of those transitive dependencies is backdoored, would you detect that, too?
[25:16.930 --> 25:20.250]  And the answer is, you got to look for it.
[25:20.430 --> 25:23.890]  But if you're asking me if we're looking for it, we're looking for it.
[25:23.890 --> 25:24.610]  So, okay.
[25:28.900 --> 25:29.620]  Yeah.
[25:29.840 --> 25:39.140]  So, on secure.software, you won't get the level of detail that says, hey, it's this dependency or this dependent library, right?
[25:39.420 --> 25:42.660]  In the commercial product, yeah, all the way down the rabbit hole.
[25:42.660 --> 25:43.680]  Where's the problem?
[25:43.680 --> 25:44.920]  What's the problem?
[25:45.080 --> 25:45.680]  Yeah.
[25:45.680 --> 25:46.400]  Okay.
[25:46.460 --> 25:47.420]  Cool.
[25:47.620 --> 25:48.240]  All right.
[25:48.240 --> 25:49.320]  Other questions?
[25:49.320 --> 25:50.160]  Go on once.
[25:50.160 --> 25:51.180]  Yes, sir.
[25:56.840 --> 25:57.380]  Okay.
[25:57.380 --> 26:02.180]  Your question was, do I have any thoughts on analysis of the build process?
[26:18.490 --> 26:38.810]  One of the signs it was a smaller signal was the, I guess, binary blobs in the test process for the build that were... they had no... there was no description of where they came from or how they came to be.
[26:38.810 --> 26:41.810]  There was no, this is how you create this artifact.
[26:41.870 --> 26:43.750]  It's just here.
[26:43.890 --> 26:46.270]  This is something to test it against.
[26:46.290 --> 26:46.310]  Yeah.
[26:46.910 --> 26:47.950]  Yeah.
[26:48.570 --> 26:49.330]  Okay.
[26:49.450 --> 26:53.690]  Any thoughts on that or analysis of... Okay.
[26:53.690 --> 26:59.570]  So, do I have any thoughts on the other indicators, the blobs, the trust me artifacts, right?
[26:59.570 --> 27:00.650]  That's the question.
[27:00.650 --> 27:01.150]  Yeah.
[27:01.150 --> 27:02.910]  I haven't thought about that.
[27:03.470 --> 27:04.390]  Sorry.
[27:04.890 --> 27:05.350]  Yeah.
[27:05.350 --> 27:05.670]  Yeah.
[27:05.670 --> 27:06.270]  Yeah.
[27:06.270 --> 27:08.610]  I, you know, come on, man.
[27:09.630 --> 27:14.050]  I don't want to stay up all night going, yeah, yeah.
[27:15.190 --> 27:22.210]  But I think if you're paying attention to your development, that ought to be a red flag too.
[27:23.010 --> 27:27.650]  So the trick we run into the SolarWinds conundrum is what?
[27:27.730 --> 27:32.510]  Hey, SolarWinds doesn't want to release backdoored code.
[27:32.770 --> 27:37.030]  SolarWinds customers damn sure don't want to deploy backdoored code.
[27:37.150 --> 27:43.210]  The binary artifacts that you mentioned are only visible to one side of that transaction.
[27:43.510 --> 27:43.790]  Right?
[27:43.790 --> 27:48.890]  So if you're, if you're mining the store, oh yeah, you should be able to explain that.
[27:49.110 --> 27:49.910]  Right?
[27:49.910 --> 27:59.450]  So I think that's a really good tip that relies more on diligence than instrumentation, I think.
[27:59.850 --> 28:01.570]  But yeah, it's a good tip.
[28:01.710 --> 28:03.050]  Yeah, absolutely.
[28:03.050 --> 28:03.830]  Great question.
[28:03.830 --> 28:04.690]  Thank you.
[28:04.830 --> 28:06.150]  Anybody else?
[28:07.710 --> 28:09.150]  Anybody else?
[28:09.470 --> 28:10.190]  Okay.
[28:10.190 --> 28:12.550]  Now I got to get back in range of the clicker.
[28:13.530 --> 28:14.530]  All right.
[28:15.070 --> 28:20.470]  So, oh, hey, the appendix.
[28:20.470 --> 28:22.710]  I'm not going to show you guys the appendix.
[28:22.710 --> 28:25.130]  Can we give a big hand to Brad, please?
[28:25.130 --> 28:27.730]  Who's been very patient with me.
[28:28.510 --> 28:29.570]  Thanks, buddy.
[28:30.030 --> 28:30.950]  All right, gang.
[28:31.010 --> 28:32.170]  That's all I got.
[28:32.170 --> 28:34.750]  If you got anything, I'll be at the table tomorrow.
[28:34.750 --> 28:36.430]  Me and Dan are going to go drinking now.
[28:36.430 --> 28:37.450]  Have a great night.