In this lesson,
I'm going to show you how to use a couple
of the open source intelligence tools.
Specifically, we're going to take a quick look at Recon-ng.
Now, recon-ng is a great tool,
but it is a little bit more
complicated than the other tools.
So to start up recon-ng,
you just type in recon-ng and hit enter.
When you do that, it's going to load up
and now you're in a special command prompt
within the recon-ng environment.
Notice here we're in recon-ng
and we are in the default workspace.
Now, recon-ng is a wonderful web reconnaissance framework
and it works a lot like Metasploit does for exploits,
and the social engineer toolkit works
for social engineering.
It brings a lot of tools
and a lot of capabilities into one place,
but because of that it is a little bit more complicated.
Now also, the other thing to note is a lot
of the things you're going to find online for recon-ng
are going to be wrong and they're not going to work.
The reason for that is there was a switch in a lot
of the syntax going between version four and version five.
You could see here I'm operating with version 5.1.2,
which is the latest at the time of this recording.
So all the things I'm going to show you now
are based on that version.
These will not work in the older version four,
and the old version four commands will not always work
in the newer version five.
So keep that in mind, if you're looking at any tutorials
online or YouTube videos, if they're older than about 2020,
you may run into some issues there.
Now to use this,
we're going to launch into recon-ng just like we did here
and now I like to personally set up my own workspace
instead of using the default workspace.
This gives me almost a place to store
and save all my different commands and tools
and information that I find in one area.
And so to do this,
we are going to use the workspaces command.
Now, if you don't know any of the commands in recon-ng yet,
which you probably don't,
you might want to first type in help.
When you type in help,
it'll list out all the commands you can use
and you'll see there that we have one called workspaces
at the bottom of the list, and this is used
to manage workspaces.
A workspace is just a defined area
to keep your different information from different
engagements into different buckets
so they're not mixing between clients.
So what I'm going to do is I'm just going to type
in the word
workspaces and then from workspaces, if I hit enter,
it's going to tell me I didn't give enough syntax
because we didn't know what to do with it yet.
Here we can do a create list load or remove.
In our case, we can see if there's any workspaces already
and there shouldn't be
because this is a brand new installation.
But by doing that we do workspaces space list and hit enter.
And you see we only have the default one.
I'm going to go ahead and create one called Dion,
and to do that we're just going to type in workspaces,
create, and then I'm going to use the folder name of Dion.
Now if I go ahead and do workspaces list,
you're going to see two Default and Dion.
All right, now that we've done that, we can go ahead
and select that workspace to work in it.
And in this case, because I just created that workspace,
it automatically put me into the Dion Workspace.
But let's say I had another one
because I was going to do workspaces create Udemy.
Now I'm going to have three different workspaces.
Oh, I actually typed that wrong,
so make sure you type it correctly.
Everything in Linux is case sensitive
and obviously spelling sensitive.
I forgot the S on workspace and that's why I got that error.
So here we go. I listed it out
and you see there are three workspace now.
Dion, Udemy, and default.
When you create a new workspace,
by default it moves you into that workspace.
So you'll notice when I created Dion right next to recon-ng,
there was this thing that said [Dion],
and that told me that I was in the Dion space,
and you can see that just below the first table
where I entered workspaces create Dion,
it moved me into that Dion Workspace.
Then when I created the Udemy workspace,
it then changed me from Dion into Udemy.
But let's say I wanted to go back into Dion,
how would I do that?
Well, to do that, we're just going to type in workspaces
load, and then the name of the workspace.
In my case, it's Dion.
Now you can see I am back into recon-ng
inside the Workspace Dion.
All right, the next thing we have to do
is we have to have some modules installed
to be able to do some functionality inside of recon-ng.
Now by default, there is no modules installed
when you first get recon-ng.
This is a brand new install
and if you just loaded up your virtual machine,
you have a brand new install as well.
And so these are things we have to work through.
So what we're going to do is we're actually going to go
into the module section,
which is another one of those key words.
Again, if you get lost at any time, just type in help
and you'll notice there we have a modules command
and it says that it interfaces with installed modules.
So to check if there's any installed modules,
I'm just going to type in modules
and then I would use the command search.
Now if you don't know any of the commands for modules,
again, just type in modules and hit enter.
There is your different options.
You can load, reload, or search.
Search is essentially like listing,
but you can list all of them.
If you just type in search and hit enter,
or you can actually search for a keyword
because there are many modules
and maybe you just want a specific module
for a specific use case.
If I go ahead and hit search and hit enter,
you're going to see that I have no modules
found and it has a red arrow there.
That's because I haven't installed anything yet.
And so now we need to go and find some modules.
How do you do that?
Well, looking back up at the help area,
you'll see a command called Marketplace.
This interfaces with the module marketplace,
and this is where you can search for
and download a single module or all the modules.
Now, some modules you're going to find are going to
require you to get an AP key key to be able to associate
with the service.
For example, there's a module to be able to search Twitter,
and Twitter requires you to register with them
as a developer to get an API key.
So you can then make that API connection from recon-ng
into Twitter and start searching their stuff.
Now to make things easy for our demonstration,
we're not going to go into that
and I'm going to pick a module that does not require
API keys.
If you're going to use recon-ng for real, go online,
look at the recon-ng manual, you'll be able to walk through
how to do all of those things
because there's a lot of capability in this tool.
For the exam, you do not need to know how to use recon-ng.
I just wanted to show you so you can get comfortable
with it, so you can use it in the real world
a little bit and then take it from there.
Now, what we want to do first is we want to find
a module that we can use.
The one I'm going to use is known
as recon/domains-contacts/whois_pocs.
I know this one doesn't require an API.
And the purpose of this is for us to be able to use this
to look up the Whois data for different domain names.
Now, to do this, we're just going to use the command
marketplace and then we're going to use the term install.
Now, just like before, if you don't know how
to do something, you can simply type in marketplace
and enter and it will give you the syntax,
and then you're going to put the one
that you're looking for.
In my case, I know exactly which one I'm looking for.
It's recon/domains-contacts/whois_pocs.
Hit enter and it's going to go ahead
and install that module.
You'll see right here, it installed the module
and then it reloaded it.
If I want to validate that actually happened, I can do that
by using the module search command like I did before
to show there was no modules found.
So let's go ahead and do that.
And now instead of no modules being found,
I see that I have one module found.
It's under the recon category
and it is the one I just installed.
Now, if you wanted to install all of the modules,
and there are a lot of them out there,
if you want to see them all, just type in marketplace
and then hit search and then hit enter.
And you'll see there is a bunch, there are so many
that is actually going off of my screen.
And so if you wanted to actually scroll up here,
you can go through and see that there are a ton
of different modules
and all of these have different functionality.
If you want to know what each one does,
you can actually look them up inside of recon-ng
using the info command
and you'll be able to learn more
about those particular tools.
Or again, go online, go ahead and Google or Bing
or DuckDuckGo or whatever your favorite search engine
or choice is and look up those
and you'll figure out which ones they are, what they do,
and which ones may be helpful in your reconnaissance.
Now, if you want to to install all of these,
you certainly can, and to do that
you just type in marketplace,
install all,
and if I hit enter right now, it's going to go
and install every single one of those.
Now, the reason I'm not going to do that
is because it's going to clutter up our screen.
It's going to throw a bunch of errors
because we haven't set up all the API keys
for all 40 or 50 different recon modules
and so I'm going to not do that right now.
But if you want to do that and you want to set up
your system fully, you can install all
and then go find API keys
for every single one of those modules.
Now the next thing we need to do is actually take
that module and load it.
Now again, if I type in modules search,
you're going to see that I have installed that module,
but it doesn't mean it's loaded and ready for me to go
because you can see where I am inside
of the recon structure.
I'm at recon-ng, Dion, I'm not inside a module,
I'm not inside any of the options.
I'm just inside the workspace.
So again, we're just going to type in modules and hit enter.
You're going to see the syntax.
We have load, reload and search.
I'm going to go ahead and type in modules,
load, and then I want to the whois_pocs,
which is the short name for the module I loaded.
There we go. Notice how my prompt changed.
Now I'm inside this module.
Now that we're in the module, we need to specify the options
that we want to use with the module,
and we're going to do that by entering information
into our database.
Now, inside a recon-ng,
there's actually a database installed with it
and you're able to put information into the database,
both things you want to search for
and information that's going to come back
when you do those searches.
Now, to do this and work with the database,
we need to use the db command.
And again, if you get lost at any time,
just type in help.
You'll notice there, the third line down is db,
which interfaces with the workspaces database.
Each workspace has its own database,
and that's another reason you want to have a workspace
dedicated to whatever engagement you're working on.
So we're going to go ahead and use db,
and then again, we can hit enter
and it'll tell us what things we can do with the database.
We can delete a database,
we can insert a line in a database,
we can add notes to a database,
we can query it using SQL commands,
or we can look at the database schema.
Now, if I wanted to see the schema of the database,
I can just type in db schema and hit enter.
And when I do that, again,
it scrolls off the screen a little bit,
but as we scroll up, you can see the different
tables that we're using.
Get up here to the top.
There we go.
So the first one we have is domains,
and this will hold domains, notes,
and the module that found the information.
Under companies, it has company description,
notes and module,
and you're going to see each of these are going to interact
with different parts of recon-ng and different modules.
Domains is the one we're actually going to be
working with here
because I'm using the Whois point of contacts module.
So let me go ahead and get back here to the bottom.
I'm just going to hit enter, it'll drop me right down.
Okay, and now what we want to do is go back
to our database command
and what we want to do is insert information
into that database, specifically the domain names
that we want to look at.
Now what I'm going to do is I'm just going to type in db,
insert and then domains, which is the field
that I want to insert information in.
And at that point I hit enter
and it's going to say, what do you want to enter in here?
Well, the thing I want to enter is the domain name,
so if I want to search diontraining.com,
I can enter that in.
If there's any notes I want to put in,
I could put them in here and true.
I'm just going to hit enter and make that blank
and it's going to insert that into one row.
Now if I want to enter another domain to search,
I can do that here as well.
Let's go ahead and do insert domains,
and in this case I'm going to use udemy.com.
Another one I want to enter, let's go ahead and do db insert
and we'll do domains, and I'm going to use tesla.com
and again, no notes.
All right, we have now inserted
three different domain names,
diontraining.com, udemy.com,
and tesla.com into our database.
Now if we want to see that
and verify it took, we can use the command show
and then the name of the table, in this case domains.
So show domains, and here is our table.
Boom, we have three rows, diontraining.com, udemy.com,
tesla.com, no notes on any of them,
and the module was user defined
because I may enter that information.
All right, now that we have loaded our module,
now that we have entered our information into the database,
we are ready to start searching the Whois database
for these points of contact.
Now, what we're going to do is we are going to use
this Whois POC module?
If you don't know what that module is,
well, we're inside of it now,
so we can just type in the command info
by typing info, it's going to tell me about
the module I'm currently in.
Notice when I typed info for Whois POCs,
it tells me this is the Whois POC Harvester,
who wrote it, the version
and a short description of it.
It's going to use the air and Whois database
to be able to harvest POC data
from the Whois queries for the given domain.
This is also going to update our contacts table
with the results inside of our workspace
so that data we get back is going to fill
part of our database in the workspace
so we can go back and look at that information later.
Now we have some options that we have to have.
We have a source, we have the value of default,
its required field,
and the description is the source of input.
Now, by default, that means it's going to go
and grab it from the table.
Those three domain names we just put in in rows one, two,
and three using the domains table.
If I wanted to do this from a file, I could do that as well
by changing this default value from default
to the file name, in our case we're going to do
it right from the database.
It keeps it nice, it keeps it clean,
and that's the way we're going to do it.
Now, if I wanted to change that, I could do that by changing
that by saying options set,
because I'm changing the options in this case,
I want to set the value of that from default
to something else.
Now in our case, I don't want to do that,
so I'm going to go ahead and delete that,
but you could do that if you wanted to go ahead
and read it from a file, read it from an SQL query,
or something else.
As you can see here, the default is
to select distinct domain from domains
where domain is not null,
which just basically means go into the database,
look for the domains table,
and any domain inside of the domains table.
I want to grab each one
that's not blank and we're going to test it.
So this is going to allow me test all three of them
with one command.
Now that we know that our options are good
to run this command,
we are just going to simply type run and hit enter.
When we do that, it's going to go off
and it is pulling that information
and it's grabbing all that information
and it dumps it to the screen.
Now, that's helpful,
but the screen makes it pretty hard to read
because I'd have to scroll up and look at that
because we just found 16 new records
and 12 of those were new contacts
that were either not duplicated
or were new things that we wanted to add.
Those all got put into our database.
Now, if you remember back,
I said we can show things from the database
using the show command.
Before we used show with domains
to show the three domains.
Now I want to go ahead and use show with contacts
to see the contacts table.
Here it is.
So we have those 12 entries going from row one
all the way down to row 12.
Now my screen is little bit zoomed in to make it easier
for you to read in the video.
If I was zoomed out,
it would all fit in one nice table.
Here you could see that three columns,
the phone notes and module row
went to the second line.
Now let's go ahead and read the first line.
The first line we have is based on Udemy.
You could see here that we found no first name,
no middle name, the last name was Operations Architect.
The email was NetOps plus aaron@udemy.com.
The title was Whois Contact
because that's the type of information we got.
The region, San Francisco, California,
the country, United States, and then phone, blank, notes,
blank, module, Whois POCS, which is the module we use
to find this information.
Now there's lots of modules in recon-ng.
You might be finding contacts by doing Twitter searches,
LinkedIn searches, Google searches, whatever it is,
all that will go into this database
to create our table of people.
But this tells us which module found that information
and which type of information it was.
In this case a Whois contact.
Next, we have lines two through 12,
and these are all based on Tesla.
Now you'll notice Dion Training didn't show up.
The reason for that is we actually have our Whois records
set up with privacy,
and so those are not going to be shared in the
whois database.
It just says this is a private record.
So there was no way to grab that information
and add it to the table.
That's why we have the one for Udemy,
which is a very common way of doing it
for a large organization,
they actually have a group email, not a person's email,
but if we look at Tesla, they chose not to do that.
Tesla actually has individual people's names.
Now as I look at them,
what is this information that's going to be useful?
Well, for one, I have names of people and emails I can use.
That's the obvious one.
But in addition to that, I might be able to figure out
what naming scheme that company uses.
For example, maybe you can't find everybody's email
when you're doing your open source research,
but you found their first and last name on LinkedIn.
Well, if you know that the company uses first name
dot last name like elon.musk@tesla.com,
then you could put that in for everybody you find
on LinkedIn and now you have their email.
Conversely though, we actually don't see that here.
With Tesla, I am seeing multiple different naming schemes.
The first one, Anna actually shows up as Anne,
which is three letters from her first name
and then her full last name, martinez@tesla.com.
The next one is just an abbreviation for the name Cameron,
and they called it cam@tesla.com.
The next one is Sherry,
but it's abbreviated down to CHE,
the first three letters again,
and then her last name lewis@tesla.com.
We get down to Elon Musk, it's first name dot last name.
We get down to Jian Gu.
We're seeing his full name slapped together with no period,
so that's another different convention.
We go down to line nine and we see Mahdesai,
and this is the first three letters
and the last name at tesla.com.
We get down to Paul Smicker and we just see paul@tesla.com.
We get down to Terry Chi and we see tchi@tesla.com.
So we're seeing a little bit of differentiation here,
but several of them I saw at least three
that had the first three letters
and then the last name,
so I'm thinking that might be the naming convention
at Tesla for their employees.
Now, if I wanted to validate that,
I could take some people's names who I find on LinkedIn
and I know their first name
and last name, put 'em into that format
of the first three letters in their last name at tesla.com,
and then see is it a valid email
by checking something like email dossier at Central Ops
or other things like that.
So hopefully you can start seeing
how we put all these different tools together
and get information from different places, consolidate it,
and then we can start doing things with it,
like spearfishing campaigns or wheeling campaigns
or social engineering in general.
Lots of different ways to start using this information.
Now, as I said at the beginning, there are a lot
of different modules to recon-ng,
and I just wanted to show you the basic usage
because all the modules work the same way.
As you go into a workspace
and then as you go into a module,
you're going to keep going through the directory structure
like you saw here,
recon-ng Dion, Whois POCs.
Now, if I wanted to go back,
I could just type in the word back
and it'll bring me up a level as well,
and then I can go ahead and load a different workspace
and then I can go ahead and do another assessment.
Now remember, you can always use the show command
to show anything you want, just like I did show domains
or show contacts,
you can also show companies, you can show credentials,
you can show hosts, leaks, locations, net blocks, ports,
profiles, push pins, repositories, and vulnerabilities,
because all of those have tables in the database,
as you saw when we looked at the database schema.
Now, all of those get things from various modules
that you may or may not install.
So that's the important thing, is to have the right module
for the right thing you're trying to grab.
In this demonstration, I only used one module
and I only searched for contacts,
so that's why we saw things in the show contacts
to display my findings as you see here on the screen.
But the great thing is if I ran other searches
from other modules and they found contacts,
they would also be in this table,
and I could easily see those too.
Now hopefully you're starting to see how all
of this starts coming together,
and you're going to play with it a little bit more
on your own.
As I said before, the thing that's going to make you
a great penetration tester is hands on
the keyboard, practicing with this stuff,
doing different reconnaissance evolutions,
trying to find information that's out there.
Now, the great thing about open source intelligence
and learning how to do this
is that all the information is out there.
You don't need permission from any of these companies
to go look up this information
because it's all public information that's sitting online,
and this is a great way for you to start building up
your skills early on in the reconnaissance process.
'Cause as long as you're doing passive reconnaissance,
you're not touching that company servers,
and you are not conducting any kind of hacking.
You're just in the preparation phases.
You're just learning information and using these tools
and getting better at them is going to make you
a better penetration tester in the long run.