Now we'll look at how to locate and analyze system log files in Linux.
Linux operating systems are largely server oriented
and on a server you always want to know what happened, who did, what, what worked,
what didn't work, what errors were encountered, who accessed the system and so on.
Everything important that happens on a Linux system is saved
as a text message somewhere which are called logs.
The Linux kernel and most programs that run on the Linux operating system,
generate status messages, error messages, warnings and so on.
There can be tens of different programs generating these messages all the time.
So we need a way to collect all of these and organize them nicely into files.
And this is the job of logging daemons.
These are simply applications that collect, organize and store logs.
The most popular one on Linux is rsyslog
and its name comes from rocket fast system for log processing
once logged in. Let's say we have no idea where SSH logs are stored
and we want to see who logged in through SSH.
Now,
we can search through all the files and look for anything that contains the word SSH D
to find stuff belonging to the SSH daemon.
And we'll notice a few lines like this
and this tells us that some SSH logs are stored in the slash bar slash log slash
A
dot log file.
We could then open up the file and start looking for SSH D log.
And in Vim, we would do a front slash type in SSH, D and press N to find the next match
and then Q to quit,
we could then open up the file with less and start looking for SSH D logs,
press the front slash type SSH D and press N to find the next match and
then queue to quit in the slash bar slash log slash off dot log file.
We'll find logs related to secure authentication or authorization.
We can see the date and time when this happened,
the hostname of the computer where this happened,
the application or source that generated this log message
and the log message itself.
Many log entries follow the same format,
especially system logs that keep track of what happened on the operating system.
And speaking of system logs,
you'll find most of these in the slash bar slash log slash cis
log file. Now, you might have noticed files with names like
a
dot log 0.1
or a
dot log 0.2 dot gz.
These are older logs and A
dot log is the latest A
dot log 0.1 is the one that was logged before the latest A
dot log
and A
dot log 0.2 dot GZ is the one that was created before
A
dot log 0.1.
And it's also archived with the gzip utility to make the file smaller.
Most of the time, you'll look at logs and see what happened in the past.
But sometimes you'll also want to take a look at what's currently happening.
Imagine you're debugging an application,
you want to try different things and see
what errors it generates while you push the buttons
and you can get a live view of a log file and
see log entries as soon as they appear with this command.
Now we use tail dash F and then the file name, the dash F makes tail enter follow mode.
You can test this out by using that command and then switching
to your virtual machine and logging in at the text console.
And now we can easily tell journal CTL.
Hey, only show me the logs generated by this command. Awesome, right.
This output opens up in the less pager so we can use the navigation keys,
we can search keys everything else we learn that we can do in the less utility
and then press Q to quit. In a previous lesson.
We learned how the operating system is managing important
system applications with the help of service units.
For example, the SSH daemon that's constantly running in the background
and such a service unit controls and monitors its activity.
Now, if we know the name of the service, we can view logs for it with this command
dash U tells journal CTL to display logs generated by the SSH dot service unit.
If we simply type journal CTL, we'll see all logs collected by the journal daemon
to go to the end of the output,
we can press the greater than sign.
If we wanna open the journal and instantly go to the end of the output.
We can use the dash E or end option
just like we opened up a log file in follow mode journal CTL also has a follow mode.
We can access that by passing the dash F option to our command.
And now we have a live view of all the new logs that
appear on the system and you press control C to exit this,
most logs are just informative.
They're telling us about normal stuff that's happening on the system,
but some are warnings about things that are not quite going well.
Others are errors about things that failed.
Others are huge alarm signals about things going critically wrong.
And these are tagged as info
warning
err
crit.
Now, if we only want to look for moderately serious errors,
we can use the dash P priority command-line option.
And one of those flag,
this shows us messages tagged with error or the error priority level.
The list of priorities we can use with the dash P
option are alert crit debug emerge error info notice and warning.
So if you forget this list, just type in journal CTL dash P,
add one space at the end and press tab two times,
you'll get a list of all the priority code names that you can use.
Now,
we can also use grep expressions to filter output
with the help of the dash G command-line option.
And of course, we can combine multiple command-line options.
For example,
we could filter out only logs with the info priority that
have a message beginning with the letter B with dash G.
And then our regex Carat B,
we pass the graph expression to only look for lines that begin with B.
We can also tell journal CTL to only display logs recorded after a certain time
with the dash S or sense option. This uses the 24 hour format.
So 0 200 shows us logs that were generated since
2 a.m. We also have an until dash U option.
So if we combine both options,
we could see logs generated since 1 a.m. and until 2 a.m. with a command like this,
the dash S and dash U options also support dates.
So we can use a command like this to see logs generated since 2024 March 23rd,
23 and 32nd note how we can fine tune this down to the second.
It's important to wrap this date between the single quotes.
Normally journal CTL will show us all logs collected
on the system starting with the first one available.
But the first entries might be super old, for example, since two months ago.
And in most cases, we don't need such old data
in a nutshell. The journal Daemon also groups logs based on when the system booted.
And often we'll get to see logs generated. Since the last time the system booted
to see logs generated for this current boot. We can use the dash bboot option.
In our case, dash B zero picks our current boot, boot zero
to see logs for the previous boot. We would type in this
to see logs generated two boots ago, we would type in journal CTL dash B dash two
on some operating systems.
Though the journaling daemon is configured to only keep
logs for the current boot and only in memory,
not on disk.
So as soon as we reboot, the journal log is lost.
It was a command like journal CTL dash B dash
one might show us that no persistent journal was found.
Although text logs are still preserved in the old fashioned logging format
in the
vlog directory,
but usually the journaling daemon is configured
to decide whether to save a persistent
journal depending on the existence of the
directory slash bar slash log slash journal.
If it doesn't exist, it won't preserve journal logs on disk, but if it exists,
it will start to preserve system D journal logs in that directory.
So by simply creating this directory,
we can instruct the journaling daemon to preserve these logs there.
The newest entries are at the top. So you might have to scroll up to see them.
The lines beginning with reboot system boot.
Also show you when the system was powered on.
An alternative command is last log.
This shows you when each user on the system logged in the last time for remote logins.
For example, through SSH,
you can also see the IP address from which they logged in.