So, hello, welcome to my talk, the electromagnetic fault in our capacitors.
A little bit about me.
My name is Will McCardle, I'm an offensive security engineer in the IoT team at Praetorian.
We do have some interesting blog posts if you want to check it out, including taking over GitHub's CodeQL system for a few hours.
Used to be application security.
Then before that I was a programmer at cloud, et cetera, a little bit too confident in what projects I take on, such as this one.
And also I am a board member at DC608 over in Madison.
So a great little group over there.
Also I'll be giving a hardware hacking training course at DEF CON this year, in case anyone wants to come see my company's training course.
So also clarify up front, I don't know much about electrical engineering.
You think that would be helpful for this project and you'd be absolutely right.
So a few things first, this is primarily going to be talking about microcontrollers.
These are tiny computers that are usually purpose-built or generally built for lower powered devices.
An example would be this microphone or this clicker right here, this could be a microcontroller versus a computer that runs a full operating system.
Whoops.
Also, this stuff is dangerous.
It's pretty easy to get 240 volts going wherever you point it with this.
So try not to lick the probe and stay away from anything touching it that's bare.
So at high level, fault injection, let's go over that.
Fault injection is making computers misbehave using physics.
The canonical example of this is skipping instructions.
So, for example, if there is an if statement in place that has security functionality, usually this could bypass that and then you just skip whatever that is.
It could be checking to see if the firmware is valid.
It could be checking to see if a password is valid, et cetera.
Other ways to do glitching, other goals of glitching is modifying data.
You can also glitch memory to be different values than what it is stored.
Or modify instructions.
So at a base level, computers use instructions in assembly to talk or execute things.
And you can change which ones are actually being executed on the fly that way.
So the types of fault injection, probably the main one these days is voltage injection.
This one is kind of dangerous to the device you're testing because it might fry it.
There's also clock glitching or clock fault injection, where basically you very briefly swap out the clock or the crystals that the microcontroller is using to have a different frequency and that can cause it to mess up in interesting ways.
There's laser fault injection where you shoot laser beams at computers until they behave or misbehave.
EMFI, so electromagnetic fault injection.
I'll be talking more about that one here.
And very recently, there is now ion cannon-based fault injection where you shoot ion beams at computers to make them misbehave.
So there are a few problems with fault injection.
The primary one is that they do require fairly precise timing.
And that can be very hard because you're dealing with physics.
And they also take a very long time to hone in and nail down.
This whole project probably took me about three months to do.
Towards the end, probably one and a half of those months was actually just trying to figure out the right parameters for various test firmers along the way.
We'll explain that later.
And then low success rates.
EMFI has a very, very low success rate.
I am at about 0.01% success rate.
And they take a very long time, in my case, to run.
But it's awesome to do, and it's a lot of fun to imagine shooting lightning at computers.
So that's why we do it.
Electromagnetic fault injection is electromagnetic pulses precisely placed.
Broadly speaking, how it works is there is a capacitor that has a huge charge in it.
And then once it gets the trigger to actually dump the charge, it dumps the charge across the probe.
And magnetic coils of wire that go around the probe.
And that is what creates a magnetic field around the probe.
And that magnetic field is what gives computers a bad time.
This is hyperlocalized.
You can do things like have a bigger probe to affect the whole chip, but generally success is much better if you have it very local to a specific place on the chip.
So again, this is high voltage, but it's actually low power.
The device that I use is generating microwatts of power at any given time.
So the current is actually pretty low here.
You still probably don't want to touch a 240-volt live probe, but it's probably not going to kill you.
I'm not confident in that.
Here's a probe.
So you see three examples of the probes here.
The important things on the left-hand side, you can see that one has the coils around it.
That is going to generate a certain shape pulse.
Generally, the size of the core as well as the coils is going to affect the size of the pulse, et cetera.
It's kind of extremely hard to visualize a magnetic pulse.
I haven't seen any research of people who succeeded at it.
So a lot of this is just guessing and relying on magic.
Technical definition that I'm stealing from a book is a changing magnetic field through a wire loop causes a voltage difference to appear across the loop terminals.
Yeah.
So here's my kit.
Here's my kit.
So this is a 3D printer I got from a friend a while ago.
That is such a bad-ass job that I've turned it into a device to torture computers.
The faulty cat is the top-left device over there.
This is a remix of the ChipWhisperer Pico EMP, which is the original low-cost EMP-generated device.
I should probably clarify, the range of these EMPs is about this much.
Not much at all.
Lower right is a precariously placed faultier, but that's where the wires needed to be.
Faultier is a Raspberry Pi Pico-based solution where it's able to do things like time the glitches, be a delay generator.
We'll talk more about the importance of that later on, as well as talking to the device over both UART, so talking to its counsel, or SWD to check to see if you've actually unlocked readout protections.
It is primarily, it's also a voltage glitcher by itself, but a lot of the innards there are really good for the fault inject, the EMFI as well.
Raspberry Pi 4B, back when I bought it, was 45 bucks, so I'm going to put that there.
Targets, you generally need something to shoot it at.
You can get dev boards relatively cheap of whatever you want to do, or you can take apart an IoT device and target that, and that's what I did.
3D printer, I checked about a month ago, and 3D printers are about $150 at Micro Center right now for something that will work just fine for this.
Let's see here, hours, way too many hours.
And then also, all these prices were from about Tuesday this week, so no idea what they are right now.
And FaultyCat and Pico EMP are also out of...
Sorry, Faultier, Pico EMP, and FaultyCat are recently out of stock, so that was great.
And finally, you will need an oscilloscope.
I hope you already have one, because they're pricey, probably 400 bucks.
For the FaultyCat and the Pico EMP, they are open-source hardware, so you could technically, if you're a sadist, print it off yourself by shipping off the files to a manufacturer.
So, software-wise, this has a Raspberry Pi that runs an interface, EMFI Pi, and it controls the 3D printer over USB, so it talks to it using G-code.
This thing is coordinating all the other components of this, as well as recording the results, so I have a Raspberry Pi SD card full of logs, because there's so many attempts, connected to the Raspberry Pi directly on the 3D printer is the Faultier and the FaultyCat.
And, yeah.
And the Faultier, a little bit more in detail, it controls power to the device under test, the DUT.
That's important, because sometimes you need to hard-reset it after you soft-fry the chip, so this is a way to actually just cleanly cut all power to it, and that's why the Faultier is powering the device I'm testing.
It can also be configured to wait for a trigger on a certain pin.
So it's essentially saying to, once you see a signal go across this, tell the rest of the thing to kick off.
Super important.
It also, for my case, is reading out UART, which is the console shell from the device, and then connected to this is the FaultyCat, so the Faultier can send the glitch signal out, and then the device under test.
FaultyCat's the fun part.
It waits for a glitch signal from the Faultier, and then it dumps 240 volts into the target at a precise position, and hoping that we hit the correct instructions.
So here's the overall process.
Very first thing we have to do is, and I'm going to go into details on these later on, identify a target you want to glitch, specifically if there's any security functionality there.
For the most part, people focus on glitching the readout protection on these chips, so you can actually get the firmware if they've been locked down.
That is pretty effective.
I did not go that route.
Then you have to identify the vulnerable spots on the chip, because certain parts of the chips will not be vulnerable to any glitching, just because there's not much material there.
Identify the trigger.
You have to narrow down the possibility space to a very tight location, and then use that as a trigger.
And then identify the parameters.
Parameters are going to include things like...
including the vulnerable spots on the chip, how long of a pulse you want to be firing, how wide of it, and what kind of delay you're looking at here.
Then you have to automate it, because you are not going to be fast enough to fire off a glitch whenever you want.
You are fast enough to reset the microcontroller, but not anything faster than that.
Then you wait.
Then you iterate once you fail.
And you just wait a rate until you get it, eventually.
So, I'm going to walk through the process again, this time in more detail with my targets in mind.
So, indiscriminate pulses will never work for anything other than just firing, resetting the thing.
So, generally, you want to identify security-relevant functionality.
You want to bypass.
It could be on a microchip.
It could be on a device.
Then you want to buy a lot of dev boards for it.
You are shoving a lot of electricity into this very briefly.
There's a high chance of frying it.
So, you want to have a bunch of sacrifices that you can learn all the parameters on first.
Well, as many as you can.
Some people actually generate custom PCBs to do this.
I believe Aaron over here, coworker and friend, has a bunch of those.
That essentially allows you to get places, certain pins on the board exactly where you need them.
Makes it a little bit easier.
And then you're going to be creating a lot of test firmwares to kind of calibrate where all the parameters are going to be going.
And you want to get as close to your final target as possible.
So, let's talk about crabs.
The target I wanted to do was a smart plug I've been looking at for a while now.
The board on the right, the green one, is a smart plug that you plug in, you can control over Wi-Fi, turn off any appliances that are on it.
It uses a chip called the RTL8720CM.
And once I figured out I wanted this target, I wanted to buy a bunch of dev boards.
And you can see that on the left.
After peeling off the metal Wi-Fi shield, we see the exposed chip kind of near the bottom left of it.
So, that was the target.
These are kind of neat little boards.
They do both Wi-Fi and BLE.
They're called the BW15, and they're basically positioned as an ESP32 competitor.
And at this point in the talk, I should mention that all of the next parts is absolutely useless because the real product has the debug interfaces enabled, so I can just modify memory I want.
And also, they don't check the signature of the firmware, so I can just modify it to check away.
But that's no fun.
So, I've been reverse engineering the firmware for this plug for a while, and eventually came across a very interesting scenario where commands I was running on it, there's a console shell to log on to, were not working, but other people were able to do it.
And so, it sounds like, well, what they did was they added a password functionality to unlock most of the commands for these devices, where you have to enter the correct password and then, once you do that, they will allow you to run all the other commands,
all the debug commands and diagnostics, not at an application level, but like a root level.
So, that's cool.
So, we have this function over here.
It looks like a bunch of gobbledygook, but it's probably the most legible of any of the code you'll see on the screens.
So, what they do is they take in the user input, they hash it with SHA-256, and then where it says load hash, they are loading a hard-coded hash in the firmware and then comparing the two.
So, if you get the correct hash signature, you will be able to get into the password functionality.
This was kind of an ideal scenario for fault and glitching, I mean, sorry, fault injection of the application code, as opposed to the microcontroller itself, because it's based on user input, which means figuring out the trigger is very nice, very easy.
Here's the assembly, you don't have to read this, but the arrows point out a CBNZ instruction, and that is the single instruction I was trying to hit with these pulses across the entire firmware.
CBNZ is compare and branch if not zero.
Essentially, if the comparison, which is mem compare right above it, see if the password and the hash are the same, if it isn't zero, then it's just going to keep on going down.
And what it does there, if you remember, here's the C code at the very bottom, it sets factory mode byte equal to one whenever you see factory byte, factory mode or anything factory or factory.
That means that it's enhanced functionality that's meant only for the factory to use when it's programming these things the first time.
It's a way to, like, program the devices before they go on the manufacturing line or head out to people.
Very common.
So that was my goal.
I wanted to skip the single check that's responsible for seeing if the password was correct or not, and then go straight into saying it was.
So I figured out where in code I wanted to do, but I still need to figure out where actually in the chip.
So this is a graph of the various places I had successes with on a test firmware.
You can imagine this overlaid over a chip picture, probably should have done that.
But you see the dots, and that's where I saw successes for various parameters within that search space.
It is very clearly in the bottom right corner, too, and everywhere else might have some successes, but not many.
It's also more clustered in that corner.
And when mapped out onto a chip, find out that the danger zone is the clause.
The Realtek chips right here have a little crab on them for some reason, all of them do.
It's quite whimsical, but those clause are exactly where the pulses are most effective.
So figured out where in code and where on the chip to go.
That took probably a week of testing.
I had to actually prep the target so I could actually talk to it and validate that if my injections were correct or not.
Please don't judge on the shorting capabilities of this.
I did not know what I was doing.
But basically, I had to create a test harness, 3D printed a tiny one, is held in place with wires, and then soldered to four pins right beneath the blue light on the far hand side.
A second 3D printer is very helpful when you're doing this because you will be printing a lot of test jigs or test harnesses just to hold the thing in place because you do not want, when you're dealing with millimeters of difference between pulses, you don't want it to slide a little bit and just even value all your past three days of results.
That would be very unfortunate.
If you are just attacking the chip itself, you can make your own custom PCBs and that's pretty good.
So at this point, the process that I was following, just so you can keep track, is wait for the DUT to boot.
This actually takes about 15 seconds for mine just because I'm targeting application code here, a password prompt versus the actual chip.
I send in a single quotation mark that's just basically to force the prompt to show up, send the password command with something that was unlikely to be the password, immediately glitch after that, and then I send the privilege command in.
If I got a response saying the privilege command worked, that means the glitch worked.
Otherwise, just start over again.
I'll show you.
So the trigger.
Many cases for EMFI, you're targeting, again, the chip itself and you can trigger off of the reset line.
So basically, once the device powers up after being given power, that's your trigger.
Usually, if you're trying to enable readout protection bypass, it is very early on in the process, so you have like a few nano, like 100 nanosecond spread to hit.
Success rate is way too low to guess, so you want some kind of instrumentation in there to figure out what's going on.
Some people use side channel analysis to figure out like what the power usage is for like AES encryption or decryption or hashing, and then triggering off of that might be helpful just because you know whatever is going to be interesting, those will be interesting functions,
et cetera.
In my case, I triggered off of sending the new line character across the console.
So then you have to identify parameters now that you have a trigger.
X, Y, Z have been covered that one, although Z makes things complicated because Z also affects the power that is actually provided to the device or injected into the device.
The closer, the more powerful.
You might overload the chip, but the further away it is, you might hit more things.
So you might also hit it.
So that's usually you want to play with that as much as possible.
Delay, once you figure out the trigger, you don't usually want to fire immediately.
You want to wait a certain amount of time and then fire.
So you iterate through the possible delays until you get more successes with your test firmwares.
That doesn't actually help specifically with the actual application code I was testing, but it was good like just practice that technique there.
Pulse width is another factor you want to play with here.
There's all sorts of reasons why you'd want a thin pulse or a wide pulse.
The important thing to know is that for these pulses that you'll see an example of really soon, the only time they can actually affect the device is at the very start of the pulse and very end.
So you want to find just the right Goldilocks width, I guess.
Power is how much power is shot into the device.
This is kind of hard to understand or explain and it's not a linear thing.
I have reference in case anyone wants to learn more about that later on, but I'm happy to accept that it's all magic.
And then polarity is the direction of the magnetic field.
I talked briefly about that earlier.
The coils here, the way they're going is a huge indicator of performance.
So the left hand here I think is a clockwise probe.
And for clockwise probes, what they do is the magnetic field they generate pulls the voltage into the core, the ferrite core.
You can see it popping through there.
Whereas counterclockwise ones kind of push the voltage out.
For my test firmware here, I had a lot more successes with the counterclockwise.
Aaron here had no successes with the counterclockwise.
All successes with the clockwise ones.
So it just depends on the chip.
And as far as we know, no one knows how to determine.
You have to do both.
So let's look at the pulses a little bit.
This is a picture of my oscilloscope.
The line on top is the trigger line.
This is during a test firmware where I specifically am going to send a signal out to trigger off of versus finding one to use.
And you can see the kind of pale yellow line below it spikes up.
And they kind of overlap with the fall of that as well as the start of it.
So this is a pulse that has a little too early, not long enough delay in play.
So the only pulse that's really doing anything because the code that runs after the trigger in this case is in that gap between the two lines.
So that was not too effective.
This pulse actually filled the entire gap.
So it basically started immediately at the end of the first trigger and then finished at the start of the next trigger.
So that was useless.
You can kind of see the interesting little wibble at the bottom that's blue is the power fluctuation on the console line because of the electromagnetic pulse.
This one is just right.
It's right after the end of the first trigger.
And it's early enough that it's going to hit any instructions before the second trigger goes high.
So that one worked out pretty well.
This one, very similar.
I think it's slightly thinner pulse.
But you notice there's a complete lack of the second pulse because wherever the probe was pointed here with this pulse was enough to overpower the chip.
So it crashed.
It never actually got to the code where the second trigger went high.
We're going to show a quick video here, probably multiple times, of kind of the process of iteratively increasing various variables until we can find the right parameters.
It's not the right orientation, but whatever.
You can kind of see here it's glitching as it's moving, making the thing more wide.
And you can also see that at the very end, I believe, yeah, the delay increases a little bit more.
This is where I was trying to figure out where inside the gap there to hit.
And that's why you need to automate.
You saw in that video about seven pulses, I think.
That would be a real pain to do it yourself.
So a lot of people have now been using 3D printers now to precisely place, hook up to Raspberry Pi or their own computer, and fire it out there while they record the results and then analyze them later on.
Um, and then there's so much waiting.
Um, I think I said I started really testing some of the test firmwares a month and a half in on the project.
Um, and then past actually two weeks been working on the final product.
I wanted to test and it's just running nonstop, constantly trying different changes out, see what works.
Um, wait, but, uh, since I had a lot of time on my hands and wanted to figure out if I'm getting close, uh , there's a lot of debugging and whatnot, but thankfully this chip, for some reason, has a really helpful feature where they dump the stack trace and,
um, the stack memory and the registers and a whole bunch of really great information.
Um, I forget exactly what this one is.
Uh, but not really relevant.
It just shows an example of this.
So I'm going to talk through a few of the really interesting, uh, glitches that I found.
So this one right here, um, is a example of what the stack frame, uh, data looked like.
Up top, you can see actually the application stack and then down low underneath the back trace is the stack trace itself listing all the memory addresses at which functions were called.
Um, generally I was looking in the 9b address space because that was where application code was running.
So in this case, uh, checked out this.
Since I had already spent two months reverse engineering the firmware, I was able to look that up and it had a very interesting function name called OSAL AES CBC decrypt.
Uh, so looking a little bit deeper into it, uh, this was the actual instruction that I glitched.
It was the one right after the set key decryption for AES.
Um, I didn't fully dive into what this was doing.
Uh, this was all part of embed TLS, but, um, right, but I was able to get some of the AES intermediary values, uh, in the stack trace going back here.
There's a few of these were actually part of the key material, um, just in, uh, transformed in some way.
So in theory with enough glitches and as long as they, for some reason, don't disable this, uh, you could recover an AES key for encryption.
So this is another interesting glitch.
Um, well, more interesting to me, uh, where I see the 9b address down there and then it goes into a bunch of 0, 0, 0 ones, which usually means it's going into a hard-coded ROM code.
So this is code that the manufacturer of the chip made, not the people who wrote the application on top of it.
Um, looking at the top arrow function right there, a whole bunch of math, don't bother reading it.
I didn't, uh, this was, uh, the SHA-256 processing functions.
This was in the middle of running SHA-256.
Uh, then as I was slowly working my way up the, uh, back trace, I discovered that this was actually coming from the exact function that I wanted to glitch.
So you can see right here, this is just a reminder of what I was aiming for.
The SHA-256 hash wrapper was the hashing of the user input and then seven instructions down is the comparing branch, if not zero.
So this was the very first, uh, stack trace I saw that showed that I was in the correct location.
Uh, and I think at this point I stopped the scan and then just like tightened it a lot just so I can keep on targeting this area.
Um, oh yeah, that was just pointing out what I just said.
Great.
First time I thought I had a chance of actually succeeding here.
Uh, so this is another one just showing the possibilities here.
Again, this is the very similar address, but it's a much closer to the top of the stack.
Um, if we look over here, we can kind of see the full stack on the left and then on the right.
And it turns out that the two lines that have been blurred slightly, um, that is the hard-coded password hash that we are comparing against.
So from this, we are able to extract the hash even if they have locked down the firmware so you could not download it, which would allow you to, um, very slowly crack it.
So that was, that was really cool.
And then, um, some of the weirder ones, I got a lot of dead beef.
Um, dead beef is just a, uh, term used, or a lot of hex digits used, uh, as filler or like debug or diagnostic stuff.
Um, I have no idea what happened here that caused everything to be dead beef, but it was, it was catastrophic, I guess.
So, uh, a few other ones I'm not going to go too deep into, but, uh, there's application level for the secure boot functionality.
This is what they use in theory.
This is what, sorry, this is what they could use to, um, implement secure boot so you can't just modify the firmware.
A, got a glitch there, uh, right before it was zeroing out, uh, a lot of the hashing and decrypting functions.
So if I had to hit right then, I probably would have been able to get those out.
Uh, weirdly, it was also able to glitch the radio power control registers.
There was a register, uh, to control how strong the wifi was and hit that.
Um, and then also one of my favorites was the fault handler functions.
All of this stuff right here is the fault handler.
So at some point, apparently I glitched the fault handler or it's something.
And then the second pulse glitched the fault handler.
Uh, so I was able to see some information there, but found that kind of ironic.
Um, so at this point, you know, this is about, I don't know, three days ago where I still hadn't actually succeeded at my goal.
Um, but, uh, two days ago, I, uh, every night before going to bed, I would kick off an eight hour scan just to see if I got any luck.
Um, two days ago, I woke up, finally came and saw that there was in fact a privileged command that was printed, which meant I was able to precisely hit that single instruction and then bypass the authentication process completely.
So you can see here, um, the ATS question mark is the command that was previously not working, but, um, because of the password, but now it is.
So this was a success.
So that meant I wanted to figure out exactly what I could do with this functionality.
I hadn't quite looked into that part.
Interestingly, I could have made it more secure.
Uh, there is the secure boot functionality that you can command using this.
Uh, here are all the hard-coded keys they have.
Um, so I could have actually enabled secure boot.
And there was actually a possibility I did, um, while exploring.
Uh, other information, you can dump a whole bunch of information about the device itself.
Here's an example of a bunch of it.
Um, the main security relevant information here is the device ID because the device ID is how they generate, uh, determine, or sorry, how they generate the encryption keys used for, uh, cloud communication.
Uh, they basically just run that device ID through a SHA-256, split it in half.
The top half is the encryption key and the bottom half is the IV.
And that's how they get the password.
So good luck, uh, decrypting my cloud comms for a device that's currently being zapped.
Um, you can dump memory.
You can also write memory.
This is basically JTAG access over, uh, UART, one of my favorite things.
So in this case, uh, dumping a whole bunch of the memory for, um, some of the registers.
In the middle there, it's dumping, I believe, some of the peripheral information.
Or, um, no, that was the security, that was the security fuses, uh, like metadata.
And then down there was attempting to, uh, dump the actual security fuses themselves.
So I can actually see what's in there, see if they just for some reason do that.
Um, but, you know, perhaps my favorite use of this was, uh, typo a command and then cause a stack trace or a stack dump, which, um, invalidated about eight hours of work because it reset the device and I lost, uh, level password access until I can glitch it a second time.
Uh, which I eventually did.
That was great.
But, um, yeah, that is my talk.
Uh, huge thanks to my wife, Aubrey, uh, Aaron Wasserman, who I don't think I could do this without, uh, Cody Hein for helping reverse engineer the firmware and a bunch of the throwing ideas at walls and then Vector247 for the very first implementation of this I was trying.
Any questions?
Yeah.
As far as, uh, test firmware was essentially, uh, you want to be able to find ways to succeed, uh, without testing your main one.
So you create contrived examples, uh, that you want to hit.
So a example is, um, made a bunch of ones that had very similar assembly to the final target, but had a huge like empty space in between it.
So that if I did hit that, nothing would really break, um, and allow me to hone in on like, uh, what is the behavior of the chip when it's doing this?
What are the parameters for like height?
Well, there's a big one there as well as positioning of the chip.
Something that is likely to succeed, um, pretty often.
So you just hone in the parameters of the glitch.
So my test firmware, I was getting easily 200 successes in a hour long run.
Um, that's how much easier it is compared to, uh, like easily 32 hours for two successes.
Um, but it's essentially a way to have dummy targets that you can just hit and then, uh, hone in from.
Yeah.
So the question was about, um, if it's a way to increase the reliability with gear.
The answer is yes.
Um, this was the DIY version.
There are two main levels of professional version.
One is called the chip shouter from new AE and one is something so expensive.
I don't even know the name of it from, uh, riskier.
Uh, but they're automated platforms that automate a lot of this.
They had a lot more success with like ESP 32s with those, that gear.
Um, actually do have a chip shutter now halfway through this, but there's not enough time to learn that quickly.
Um, those handle a lot of this, uh, manipulation of parameters for you.
And I think we think it would be possible something like that to get this if we knew more about the chip, but like just the learning of all the parameters necessary, um, makes it too weak, a little tight for this.
Uh, yeah.
Oh, the layout is always going to be on the, the chip itself, which should be pretty similar between how many slides do I have?
Uh, yeah, it's always on the chip itself.
Uh, so the layout doesn't matter about the pulse position so much, but it does matter for like, maybe there's a giant capacitor in between it.
You do not want to hit that with 240 volts, um, or like other obstacles.
But does that kind of answer it?
Okay.
Um, I think so.
I didn't try that.
I was briefly looking into underclocking this thing, uh, so that I could slow down the execution of everything and then have a little bit wider of a window to hit a single instruction in the firmware.
Uh, cause presumably it would take a little bit longer to run, but a crystal would be similar approach.
Just, I have not looked at anything in default injection with crystals and learning each fault injection takes a very long time to wrap up on.
Oh, the question was over there.
Yep.
Uh, it was not single-threaded.
It was a, well, it was a, it was a real-time operating system.
Um, that brings up a great point, uh, while honing it in.
Um, I don't have a picture of it here.
Um, but since I was doing UART, eventually what I did for the trigger was I sent in most of the password command, except the final new line character.
And that new line character is how it actually knew to process it.
So then, uh, I knew what the baud rate was.
So I calculated how many milliseconds that baud rate would be.
And I compared it to the unit of time that the faulty cat used, which was cycles.
Um, and then figured out how many cycles were necessary to delay it.
And then once I actually put that number in, I immediately started getting glitches in that same function.
So that worked out pretty well.
Maybe if it's a popular chip, yes.
This, um, I'm not gonna say it's hard, but it wasn't easy.
There was a single manufacturer who makes dev boards of this.
And the, uh, official manufacturer only makes a giant board of this.
$80.
And when I went in thinking I need five dev boards in case I fry things, I was not going to spend that much money on the presentation.
Um, so thankfully I did find, uh, a few of the dev boards for this platform.
Much cheaper.
I would honestly want to keep on working on this a little bit more and refining it.
I did mention I have the chip shrouders.
So eventually I want to start trying out that, um, seeing what capabilities it actually has versus just this mythological thing that would make everything better.
Um, other things I would do is there was a slight, slight of hand.
Um, I would have done the wiring better.
Let me find a picture of that.
Uh, there are four pads there.
They are precariously close to each other and I did not snip the end to the magnet wires.
So, uh, actually the very first success I had was, uh, I could not do anything with it because the movement back of the 3d printer, um, shorted out the communication lines here, preventing me from actually sending commands in.
Um , so I would have done that better.
But, um, those are probably the main things.
The only other thing is I would have, I wish I had swapped over to using this actual chip first or earlier on.
I spent about a month working on Raspberry Pi 2040 as the test firmware, because that's what I had and I knew it worked.
Um, but this was pretty easy to get going in a manner.
And it also had the super helpful property of just dumping all the debug information whenever I glitched.
That was great.
Not, uh, so the first, not very, uh, first time happened at 545.
I started another run or about 730 when I noticed that it succeeded.
And I noticed that the wires were bodged.
Um, I didn't get a success until like eight at night, uh, there.
And then once I got a success there and messed up the command, which kicks me out, um, I ran it for about 12 hours and didn't get any successes, even though same exact spot, same height, uh, everything.
It's a lot of this EMFI stuff is highly probabilistic.
Uh, so you have to get everything just right.
You could have things affecting it.
Like if it didn't fully power down in between tests, like individual runs, uh, could there be some leftover, um, current or I don't know, I'm not like an engineer, some kind of leftover capacitors being full, a little too high when they're too high.
It doesn't do this anymore as an example.
So that's why it's magic.
Yep.
So there's a lot of actual, uh, various research into the correct probe shape.
Um, some cases, uh, I started out with a four millimeter probe, which has a pretty wide pulse.
And then I went to a one millimeter pro, which wasn't all that effective.
And Aaron made me a two millimeter probe and that was very effective.
Um, but even just the size of the core and the shape of the core.
So some people take an angle grinder to the ferrite core to cause a angle and that will create a very sharp pulse.
Um, that's very targeted or people will do it very, very, very precise, you know, micrometers, uh, probe cores with this and then find a device that can shave off a bigger one down to that size.
Um, there are some glitches using toroids, uh, for this.
Um, I don't know how they work.
Uh, and then I think the other one that is kind of interesting is, uh, the faulty cat itself comes with a pre-made probe, which is kind of nice.
Um, but it has a hollow core down the center and that makes glitching less effective.
But as compared to the Pico EMP, you have to make your own core, meaning your own probe.
And I am not very good at that.
So I was appreciative of having like a pre-made one, um, that worked.
Yep.
It's a little half donut with wires around it.
Aaron.
Yeah, they just have it.
Yeah.
Lop off the top half.
I'm not an electrical engineer, so I don't know.
Um, I think generally, uh, since there's two ends to the pro, uh, actual coils, I think it just goes up and then back down.
It's my understanding.
Have you heard of anything like that?
No.
The main source of wisdom on it right now, but...
Yeah , I can, I can give you a research paper that talks about it.
Um, a lot of people who do this stuff are not electrical engineers.
They're attackers.
So we know just enough to cause magic.
Yeah.
Any other questions?
Uh, easier or more difficult?
Okay.
Uh, yeah.
So the question was, how do the people defend against this?
There are a few semi-automated solutions that don't work all that great.
But the main thing that people recommend is just do the check twice or three times.
So in this case, make the comparison three times.
Make sure optimizer does not disable that.
And that means that you now have to glitch three times very quickly in a row and hit the precise moment in all of them to bypass this.
And, um, that's not possible, probably not possible with this DIY version.
Might be possible with like a super powerful version.
Uh, but the practice of just keep on, uh, checking multiple times everywhere is helpful.
Or if you're reading data from memory, read it three times.
Well, it'll kind of throw something off like this.
Um, you see here the glitch kind of shifts over time.
Um, since I was doing a, against an RTOS based system, it's a real time operating system.
Uh, what happens is they, uh, my trigger was not very accurate.
My delay was not very accurate.
It was waiting for the system to come back online and go through all its tasks, uh, in between the idle tasks.
And so even though I would have the same exact delay impulse with all those things, I was getting much different placements of the glitch because the RTOS was throwing off the deterministic ability of this.
So that's pretty similar to what you were saying right there.
Uh, but eventually I succeeded twice in 36 hours.
All you need is one.
Anything else?
No?
Wait.
Um, oh man, yeah.
Voltage is a little bit more accessible.
You don't need a 3d printer, um, for one, and there's a lot of pre-made tools out there.
Um, I can actually show at the very end of the slides that, um, but, um, when would you do voltage?
All right.
Yeah.
So, um, usually if be good for, if you're able to have like multiple power lines, you can work through and decouple the capacitors for, uh, if you only have a single line in, uh, might be more difficult.
Uh, one of the major downsides of voltage glitching, even though it's more repeatable than EMFI is that you generally have to modify the board.
Uh, and that can be a pain to do solder things and, um, stuff like that.
Before, before I get kicked out.
Sweet Jesus.
Um, I have a bunch of links up here if you want to take a photo of it.
Um, hex free IO is a site by, uh, YouTubers, uh, stack smashing and, uh, live overflow.
Uh, they have a voltage glitching course using the faultier.
Um, this is the tool I used.
Um, and they have a free floor.
First level of the course is free, uh, for voltage glitching.
The second one that's more interesting requires payment for the subscription model, but generally it's pretty good website.
They have a great Android, Android security course.
A faulty cat and Pico EMP are the two tools that can generate EMPs.
Um, right now, as far as I know, they are both out of stock.
I don't know when Pico EMP is going to be made.
However, the faulty cat, there is movement in their GitHub for a V2.
So that might be in place.
A curious bolt here is a kind of a really interesting tool that has a built-in challenge board as well as voltage glitching, uh, differential power analysis, and a few other glitching or, uh, similar related tools.
Um, I thought that was pretty good.
And then there's two blog posts down here with very long names.
But , uh, one is about glitching the one-time programmable transfer in ESP32s.
And the other one is all about the weird physics of this and how no one really understands how any of it works.
All right.
Thank you very much.
Thank you.
Thank you.
Thank you.