welcome back for those who joined us uh
yesterday and welcome for those joining
us for the uh first time today
we have a great selection of our
presentations yesterday and today is
going to be the same
from the market leading Professionals in
your industry
you can choose between the security Dome
here or the main hall from the agenda
you've been given
and take the chance to network check out
the stands in the common areas of the
building and there may be some places in
the workshops if you're interested in
that sit back relax enjoy your day our
first Speaker today welcome to the stand
Mr Martin
thanks very much thanks for having me uh
and good morning everybody
and welcome every early riser
uh today I would like to talk about
firmware's and basically talk about how
we can extract firmers uh but before we
begin I'm just curious is everybody ever
worked with firmers like reverse
engineering or or doing any security
analysis
all right it's good all right so then
hopefully I can show you some
interesting insights into that so um uh
let me just quickly uh jump on it um I'm
working for a company called wonky and
basically what we are doing is automated
firmware analysis and um what I would
like to present uh to you today is a new
open source tool which is called
envelope which was basically born from
the problem that we were facing by
working with firmers in any firmware
analysis or any any analysis the first
step is always like I have a firmer
image I don't really know what it is and
I somehow need to unpack if it's a Linux
system or maybe another system I need to
somehow extract it and the problem we
were facing that there are so many tools
out there specific to one firmware or
one format and we have to always like
figure out like all right this firmware
is by this vendor and I need to use this
script and if it's another one I need to
go to GitHub I need to pull another one
and I need to figure out how to extract
the firmware so basically what we were
looking for uh as a tool that we can use
in an automated way or a semi-automated
way at least which will do all this work
for us in an efficient way
so basically uh we wanted to create a
framework to extract firmware and we
created envelope and open sourced it it
currently has support for over 30
formats built in it's counting and
basically it can handle different
archive formats like the traditional one
starting from zip or tar files it can do
different types of compressions as well
and a bunch of file system format that
you typically see in embedit or iot or
OT or iiot or whatever buzzword you want
to choose in in the live
when we uh started to um to work on
envelope we wanted to basically find
something that can deal with arbitrary
formats in many cases the difficulty is
like figuring out like what the format
itself is and dealing with a problem
that uh many firmware images have uh
what I would call like a Matryoshka
format where there is different layers
and layers and layers and you might
unpack one and then you face another one
and we wanted something that we can just
work on it work on in with arbitrary
formats and work also with untrusted
format it's very bad and we have seen
cases where there are some malicious
thing in the in the uh in the packing
just think about like Zip bombs and and
stuff like that
so basically uh when we set out on this
journey and I will show you today in in
details we defined few objectives we
wanted to achieve so we wanted to have a
tool that's very accurate uh in any
analysis that you might have contacted
or from our experiences that if we are
not accurate then basically we are
facing with this garbage in garbage out
problem if you have a firmware we
extract it and it produces a lot of
garbage then any further analysis I do
on top of that might it be some manual
analysis some automated analysis or an
emulation for that matter but uh if I
get a lot of junk it makes my life very
very uh hard the second objective
obviously we are dealing in the field of
security we wanted something that not
introducing too much of an attack
service so you want to make sure that
the extraction process is as secure as
possible against things like like Zip
bombs or past traversal attacks
also it's it was very important that we
envisioned a tool that's very easy to
extend
um a lot of us in the in the reverse
engineering or in the security Community
when we Face a new format we just write
quickly a script and then maybe push it
on GitHub if they are cool or we just
forget it somewhere around our hard
drive or God knows where and then the
next time we Face the same problem we
might remember that oh I had a script
maybe I just rewrite it but we wanted to
create something that I can as a
researcher very easily extend and add a
new format so at the end of the day I
will have a library of all the formats
and in one tool which can unpack all
this all this file and last but not
least sometimes we have to deal with
large server a lot of firmware so we
wanted something that can actually
perform well and not too slow at the end
of the day it's not like we are talking
about milliseconds or seconds but it's
good if you can uh get results in
minutes so these were basically our
objective when we set out it was very
funny when we announced that basically
we get some some results we were using
also bimbok and basically we wanted to
something uh like this um like Dave
mentioned it this is something like Bill
mocked wants to be when it grows up so
we it's also uh like a replacement for
that
with that said what I would like to talk
about today is basically just give you a
very quick introduction to the tool
envelope getting started uh jump into
some details and show you how the
extraction and Analysis is done uh we
have some cool reporting features built
in I would like to Showcase to that and
what I'm planning to do today to show
the extensibility and also welcome
everybody to contribute to the tool and
show you how to create a new format
Handler which can deal with a format
that has not been supported before and
also show you how that format can be
extracted so that's the plan today I
don't have too many slides and I will
jump on the demos
but uh first thing first let's see how
to get started with envelope even before
that a big kudos to the team who
actually had to create it and again I
would encourage everybody contribute I'm
very happy to put any of you on the
slides next year it would be so great
all right so basically
um there is three ways
um to install envelope and we wanted to
make it super simple one is that we
created a pre-packaged Docker image that
you can just pull it and use it so there
is no need to install any dependencies
and whatnot I think this is the easiest
way to use the second one is to use NYX
I'm just curious anybody's using NYX
here
okay anybody okay anybody knows Nicks so
if you haven't used NYX check out it's
an awesome cool uh dependency management
system uh basically I would say on
package management the best thing since
sliced bread so it's really cool
um uh it's it's a good thing and of
course the third way uh to deal with
um envelope or just to get it get the
source it's on GitHub and just install
all the dependencies yourself
so um I said with Docker it's super
simple
um we are publishing basically always
from the master Branch a latest Docker
image on GitHub and all you need to do
is just run it uh volume Mount uh the
source directory the destination
directory and everything else is taken
care
uh also if you are using NYX
um you just need to install NYX and then
all the dependencies are automatically
installed and pulled in so you don't
need to figure out one by one like oh
how do I install my lzip compress the
compressor or tar or whatever tools we
are using uh to do the actual extraction
and of course
um if you want to go to the source just
get clone it from GitHub we are using
poetry for dependency management so
basically all you need to do is just run
poetry install you'll get the base but
it also means that you need to manually
install all the extraction
so
um with that said
um let's extract some stuff
so
all right so um I would show you I'm
obviously in my laptop I have everything
installed I hope you can let me make it
a bit bigger so everybody even in the
last row can read it using it is very
very simple
um
we have a extensive help page describing
it you can read it if you get lost but
nobody reads documentation as you know
so running it is super simple you just
need to Define what is the extraction
directory I will just pick a directory
and then you need to specify a firmware
image so I prepared a few
um image let's start with an open VRT
one that's fairly simple
and basically what it does it goes
through the file and does the extraction
and if I go I'm very old school so I
would use midnight Commander
um then all the files are extracted so
basically you can see that here is the
open VRT extract and it got extracted it
basically tries to find some chunks in
it and then goes through all the
extraction process so first it detected
that there is some gzip compression that
it extracted it then it went forward
with the content what's inside gzip and
extracted it it detected some file
system in this case some x file systems
then I can go there and all the files
are there and uh I can I can work with
all the all the files I need so it's
it's super super simple
the power of of envelope that it's not
just trying to blindly unpack everything
and try to find everything there but it
tries to be very very precise one of the
gold as I stated in the beginning is
accuracy which means also that there are
certain cases in files where we
encounter a unknown chunks sometimes we
detect that hey here is a file system
this is how long it is and there is some
data after that so we are trying to also
flag these unknown chunks and also carve
out from the files so maybe it can be
analyzed later it might be some padding
it might be some digital signature in
some cases or it might be just some
format that envelope is currently not
supporting so we give the chance and try
to be very explicitly what we can deal
with that we what we cannot deal with so
if I just go there you can see it's
actually a group boot record which we
don't handle right now but you can also
go to the end and see that it's probably
some padding at the end we are planning
to add obviously more support for these
but we are trying to be very explicit on
what can be extracted and what cannot be
extracted
another cool feature of envelope is not
just that it can extract the data and
extract the the files but it can also
report on what it did find and what it
did not find
so let me just override a previous file
and add a report to it
so basically everything we detected can
be written into a report so let me just
quickly rerun it
and then I have a Json report
I'm very bad with
report
so basically I get a very detailed
understanding of like what we uh
extracted if there are any files that we
identified we are also reporting on the
size whether it's a file uh we are also
reporting on every file if we detect
that we detected the hash information
and also we are reporting the unknown
chunks stating what are the start offset
and offset size so we try to give all
the information we also created a very
simple tool let me just quickly switch
there oops I'm just using Jupiter
where we can basically load this report
and generate the chart which can
visualize how the firmware is actually
built so this stun burst is actually
representing the different layers of the
firmware if I move there I can read the
details I can see what was the file size
where the actual file is and I have the
ability to zoom in so uh if I'm
interested I can see different layers
let's say I'm interested in the big one
so I just double click on it and I can
precisely see like what is included in
the file so this is just a simple one to
get me an overview of what we identified
uh it's also very cool let me show you
because some people might prefer other
ways
instead of a sunburst
you can also get a tree map and see like
what the what the contents are so it's
very very easy to very quickly
identified uh what's lurking within a
firmware also you can very quickly see
like if we identified a unknown chunk
where it is located in the file what is
the size uh I I did not put it here but
you also calculate the entropy of it so
you can get an understanding whether
it's maybe compressed or encrypted
um or maybe just a padding if the if the
uh
entropy is very very low so hopefully
this gives you a tool that
helps with the extraction process
and that's it let me just jump back to
the presentation and talk a bit about
how unblock Works under the hood so
basically the way how envelope is is
constructed that it is scanning for
certain format patterns every format
Handler can define a pattern which is
usually some some magic or some pattern
that we can search for we are using
hyperscan which is a very cool library
from Intel which is a super fast uh reg
X Factor so we are using it to find uh
patterns within the file once you find a
pattern then basically the handlers
which are specific to a format
they are calculating the size of the
chunk so if I have one big binary files
which might have let's say multiple file
system packed in it may be packed after
each other maybe there are some pannings
between that then the handlers through
the pattern can pick up those chunks and
then based on the chunk content we can
calculate how big the chunk is so we
exactly know where the start of the
chunk and the end of the chunk is so we
can carve it out so we are basically
carving out all the chunks that we
identified and we are also carving out
all the leftovers which are basically
the unknown chunks uh in the game
excuse me now one the once the extract
once the carving gout is done we can use
the extractor the reason why we do this
carving out that we in some cases you
are using some external uh extractor for
example if it's a zip file that you're
just using for example 7-Zip to do the
actual extraction but seven zip does not
know how to unzip from a file and an
offset within a file so we need one file
which is itself a zip file in this case
so uh we have the carbs we are
extracting it
and we are calculating the entropy for
the unknown chunks and then basically
recursively iterates through all the
extracted files to handle this material
problem to deal with uh any embedded or
other layer of operation so this is the
the high level
now the architecture of of um and just a
few words on the architecture the
architecture of fumble be super super
simple we're basically providing you a
core
um again everything is open source here
the core provided uh as a CLI so it's
the command line tool which I just shown
you but it's also available as a library
so if you want to embed it into another
python project because envelope is
implemented in Python then you can do it
as well also the core provides the
pattern matching and the chunk carving
functionality so you don't need to
implement it and it also provides a
plug-in management basically you have
the option to Define plugins and plugin
contains the Handler for different
formats and if you are developing again
just create your own repository put into
a plugin say my handlers and then this
plugin is automatically loaded all the
handlers from the plugins are loaded and
are used throughout the detection and
the extraction process
also the core provides logging and
Reporting capabilities just to show you
before and it provides a ton of useful
utility functions so when you are
dealing with different formats I will
show you right away then we are trying
to give you small tools to help you with
the parsing or conversion and other
utilities that might come handy which is
usually re-implemented all the time if
you are just writing your own script
now the envelope handlers are I would
say this is the core so amloop handlers
are the brain of the core that makes
envelope super powerful
uh basically handlers can be defined as
I said before in plugins so it makes it
super super easy to extend it you don't
necessarily need to open source it or
you don't necessarily need to Upstream
your handlers to keep it easily
manageable of course you are very happy
if you do so but you can keep them
private if you want and the goal of the
format handlers that they are
responsible for the chunk calculation
and the chunk detection and also the
extraction so all the format specific
log jig stays within the handlers so
that's on the high level the
architecture uh just a word on the extra
core because that's also a very tricky
thing from our experience it's usually
easy to identify a certain chunk but
extracting it is sometimes much more
difficult just to give you an example
for example
um squasha Fest which is very commonly
used in different firmers actually I'm
finding a squacifist and understanding
the lengths of it so the basic chunk is
not super difficult but extracting it is
not that trivial so for example we are
using a Sasquatch to do the job now
unfortunately Sasquatch was not very
well maintained so we decided with a
bunch of extractor to Fork them and
actually start to improve on that so if
we have a Sasquatch Fork we have a
Jefferson for QB reader Jeff Chief uh
fork and we are trying to also
fix box and add support for other
formats because some of the vendors get
very creative on dealing with formats
and they are extending in a way they
want to deal with so uh I would also
um recommend to check out some of the
forks we try to maintain it if there are
any problems try to fix it you're also
heavy users of all these Forks
in case of some easier format which is
easy to implement or there is no tool
available we just decide to build it as
an extractor but it also means that you
have the chance either to use some tools
that readily available or you can create
your own extractor as well
so let's jump into that and let's try to
create our first uh extractor
before doing so I just picked up a
firmware
and I this time I decided to pick
something simple so I just
picked up a Netgear firmware and our
7000 whatever I just downloaded from the
internet
and if I run an extraction on it
then it does the extraction if I go to
the results let me just quickly go there
I can see that here is the result and
generally it's not extracted I see there
is some squash effects in it which we
detected there is some lcma file I can
go into the details see what it is uh oh
it finds a Dev console root so it's
probably a a Linux kernel that we
detected here but generally there are a
lot of unknowns and I just picked this
very very simple one if I look at it
you can see that it's actually a chk
format which is specific to Netgear so
for this purpose I just want to
understand this Header information and
see how we can deal with it
so in order to conserve some time I
actually pre-coded it yesterday
and um
I just created a
Handler for it so here is envelope here
you can see all the handlers
and I created hopefully I will able to
find it
a Netgear Handler so this is basically
how a x structure looks like let me try
to zoom into it so you can actually see
it
um
okay
um okay let's go back to the command
line it always better
so this is how a let's use VI that maybe
they'll do some color
enhancement yes much better so
um actually doing a a handle is super
super simple this is basically how a
Handler looks like one of the important
thing that you need to Define is you can
define a pattern that you want to search
for so this is basically the pattern
that will be searched this is the magic
uh in at the beginning of the file if
you recall this two eight two three two
four five E this is basically the magic
for this fire format so by defining the
support for this format I just Define
like what is the pattern I'm looking for
and the other thing that I need to
implement is basically the chunk
calculation so that it's a callback once
a pattern was identified then this
callback is called and it's passed the
file and the start offset and all it
needs to return is basically a wallet
chunk type where I need to specify like
what is the start offset which is
usually simple and also what is the end
offset in order to do so what I need to
do is actually parse the header and what
we are using is also a struct a parsing
it's very very common that some
structures are available in the source
code so in this case what I did I just
copied a structure definition
uh which basically Define this format as
you can see there is some magic header
length some reverse Earth sync which
might make sense I don't care because
it's not required for the unpacking and
uh there is a kernel and the root file
system lengths in it there are also some
checks some so if you want to be super
precise with the extraction you can also
do some check some calculation
and then basically I can Define that I'm
using this uh as a definition
and then I'm just parsing it so we are
trying to provide all these utility
functions so actually creating the
handle is super super simple in this
case I know it's a big engine we can
also do and have some utility functions
to do some Indian as detection based on
some logic and then I do some
calculation just to be sure I'm logging
the header and then I'm returning the
identified
content so let's quickly run it again
and I'm just adding that I don't want to
go very deep
maybe I want to go a bit more verbose
so in this case I can see that on debug
level we identified the header they also
have some utility function to make it
super nice color coded if he parser
header so you can very quickly see if
the header is correct like what are the
details in the header so in this case I
can see the board ID and all the other
fields as well
that's basically how I can identify the
chunk there is also the log on like what
the chunk that I identified we also have
some speed improvements that for example
if we identify that the chunk covers the
whole files we start searching for other
chunks so it's it's super super simple
in that way now this is the first step
the next step is that I need to actually
extract it now in this particular case
the format is very simple so let me jump
to the next step
um
oh
where I have added an extractor so
basically every Handler I can Define
like what the extractor is where the
extractor is responsible for actually
extracting the carved out uh chunk
uh in this particular case there is no
external tool so I just decided to write
my own extractor
so let's see how complicated the next
structure is that's pretty much the
extractor again
logic is very very similar we are using
this track processor we can use the same
definition that we used before
and basically the extractor gets the
impulse the carved out chunk and gets a
directory where the extraction should
happen so all we need to do is just
parse again the header
um we wanted to separate the uh tongue
detection and the extraction so we are
trying not to pass information to keep
the API Simple and Clean and all we need
to do is just seek to the end of the
header and we know that there is a
kernel and a root of s i just created a
super simple
function which will dump it into a file
we have this function readily available
to carve a chunk to a file so I'm just
defining that okay here is a chunk which
in this case will be the kernel or the
router fast with the defined lengths and
then it will do all the extraction let's
see how it works
so in this case
um also going forward this is where I
can see that the charge was gone out and
as I defined a maximum depth it does not
go in uh deeper so it stops there so
let's um
let's quickly see in this particular
case it identified only the kernel
partition there and it extracted into a
file called kernel because this is what
I uh defined to be used as a file name
obviously in a more complicated format
you might dynamically create the file
name you also have a bunch of utility
function to make sure that there are no
pass drivers also if you are using the
carving out API then you don't need to
take care of all those problems now
let's um go a bit deeper I just remove
these steps limit
yeah I just left debugging on
so in this part UPS in this particle
case we picked up and we extracted the
kernel and this finds all the other
thing but there is still another header
if you worked with Netgear then this
hdr0 might be a tell this is another
format that
um Netgear is using so what we need to
do is create another Handler if we want
to be very precise
so I just go to step three in my journey
let's quickly check this out and in this
particle case I also added a TRX header
this is the format name for this hdr0
and I added
next to my
chk enter the TRX Handler as you can see
the logic is very much the same just
Define the magic I'd already defined the
headers I parse it and I just defined a
chunk so obviously you can add a bit
more checks to make sure checksums are
correct or of those sizes make sense in
order to avoid picking up some false
positive
and just to jump there I also created a
an extractor for this format it's very
similar the only difference that the
format defines actually three partitions
so what I'm doing I'm just uh
um
checking out part one A Part Zero part
one and part two getting the offset from
the header and then in this case
basically dumping three files let's
quickly see how it works
so it does the extraction
and if I look at the the results let me
quickly go outside
and then here I can see that even the
partitions were identified correctly so
again it helps me to have a better
understanding not just finding some
random uh squash a fash somewhere down
the line
but uh precisely identify uh what the
files are also if I go back to the
previous one
oops
let's go to the Sunburst maybe that's
easier to see
then I can see all the partitions and
all the details so I can jump to a
partition understand what's there so all
the visualization just get more accurate
I think with this uh obviously in this
particle case like there is no super
high need to actually understand these
formats
um as we could even without this format
handlers identify the kernel or identify
the file system but in some case uh it's
good to know like how these are
structured especially if there are some
other things lurking there and obviously
if it's something unsupported then the
logic gets a bit more complicated I just
choose this as an example uh sorry I
just choose this because the file system
format is fairly simple and I just
wanted to Showcase how easy it is to
actually create a format
so we actually created our first Handler
and first extractor
so um just a few words on where we are
heading we download
um so envelope is being very very
actively developed
um one of the things that we are working
on is creating an even cleaner uh
reporting of the accession process and
gather basically a result of a summary
of like what type of file systems were
identified what type of information what
type of files what are the unknown
chunks so we'd like to work on that also
right now we do not extract too many
metadata so for example we extract the
files what we would like to also report
on permissions which is sometimes lost
during the extraction process file
ownership stuff like that also we have
some plan and we are working on
extending the unknown chunks with some
Auto identification so for example if
it's a zero byte padding to pick it up
as a zero pipe padding so it's not an
unknown chunk if it's an FF padding pick
it up as an FF padding we also see cases
where there are some data data in it so
if we can identify some strings in it or
we can identify some logic to
automatically pick those up and give you
a hint so it's easier to deal with the
unknown chunk maybe create a new format
Handler as well and last but definitely
not not least we are continuously
working adding additional formats that
we encounter and we also encourage
everybody else to do so so basically
feel free to contribute test your
weirdest files report bugs everything is
on on githrop if uh if you have any
problem let us know we are super happy
to help you and we would love to see a
pull request coming from the community
so if you find a bug or have a cool idea
for a feature report it but even better
if you just
um
send a
pull request for that
so with that said
um thank you very much for your
attention and I believe we have a few
minutes for questions or if you want me
to double click into any of the details
would be super happy to do so
any questions from the audience
good
uh thank you do you somehow deal with
encrypted firmware like obviously if you
have credentials but you know can you
can you deal with the encryption in some
way so um yes and no I mean it very much
depends on if that could be decrypted so
um if if the uh for example if there is
a a publicly known uh decryption key or
a password then it's very simple to
create a a Handler for that we are
experimenting with some formats and
building in some some
encrypted format support it's a bit
different we have some of those servers
but we are not always in the position to
actually open source it uh due to
confidentiality agreements we're also
toying with some ideas where we can do
some simple brute forcing on on certain
formats so for example uh we think cases
where um it's just a zip file which is
which is encrypted and in many cases
it's just a very simple password so
there is that it could be feasible to
actually try to brute force it there are
also many projects out on GitHub which
is dealing with encrypted format where
the decryption scheme is is well get
public let's put it like that some
reverse engineered it so one of our
dream if I want to say so is actually
get those scripts which is lurking
around on GitHub and God knows which
Repro and incorporate into uh and blob
it would be really cool if all you need
to do if you are dealing with the
firmware just run on blob pull all those
format support I would say right now we
are much more limited by our time but I
would say that if there is a particular
format encrypted or not encrypted
especially if there is a already
available like open source or reverse
engine or script it usually takes like a
day or maybe half a day a couple of
hours to actually Port it to envelope
and then
um really you can have uh sorry
a unified Library uh to to deal with
sorry I think the the power of fan blob
is that you don't need to have many many
scripts one script for this one script
for that but you can unify them into one
one tool and let it take care so we will
just move it generally we are very open
uh it's either limited with some
confidentiality issue or our time but
otherwise yes okay awesome thanks thanks
anyone else
thank you Martin thank you very much
[Applause]