In this lesson,
we'll take a look at how to verify the
integrity and availability of important resources and processes.
As time goes by a server will usually use more and more resources.
Storage space is a good example.
As the database grows, as users store more and more files,
more and more gigabytes of storage space are used
in the cloud. It's relatively easy to add more disks or
ssds
to a server.
But how do we know when we're running out of storage space?
Well, we can use the DF Diskfree utility
but this output is a bit hard to read.
For example, we can see how much space is used on a disk.
But we see this in one kilobyte blocks,
we can tell DF to make this output a bit more human readable with the dash H option.
Now, sizes are displayed in megabytes, gigabytes and terabytes. Instead
where we see the letters MG or T,
we can ignore the file systems that contain the word T MP FS.
Those are virtual file systems that only exist in the computer's memory,
not on storage devices.
In our case, we can see that we have two real file systems.
The one mounted on front slash is the root
file system where our Linux operating system is installed,
the one mounted on slash boot is a small file system where boot files are installed.
Now,
we'll learn what slash DEV slash mapper and slash DEV slash BD A two mean
in one of our later lessons where we discuss how to manage storage devices,
we can see that 3.9 gig are used on our root file system and we have 5.4 gig free.
Also, since 43% of our file system is used,
that means we still have plenty to spare around 57% of our free
space to see how much disk space a specific directory is using.
We can use the DU disk usage. Utility dash S is the summarize option.
This makes du display disk space used by the entire slash user directory alone.
Without the dash S option.
DU would show us disk space used by
that directory and also every subdirectory it contains
the H makes a display sizes in human readable format, megabytes,
gigabytes and so on.
Now let's move on to the server's ram or random access memory
to see how much memory is utilized we can use the free command.
And once again,
we use the dash H option to make the
command display sizes and megabytes gigabytes and so on.
Technically speaking, these are actually
mebibytes and gigabytes.
That's why we see the letters M I and G I here,
one mebibyte is actually equal to two times 20 or 1 million, 48,576 bytes
and one megabyte is equal to 1 million bytes. In
this output,
we see that we have around 3.6 gigabytes
usable memory and 1.0 gigabytes are currently used.
But how much memory is free and ready to be used by programs?
That's the amount we see in the column titled available.
Not the one named free,
which can be a bit confusing.
But the technical explanation is that some memory is used, hence not actually free.
And the operating system can make it available at any time.
For example, if we read a one gigabyte file,
the operating system will temporarily store it in memory
for faster access the next time we need it.
But if a program needs that one gigabyte of memory,
the operating system will free it up immediately.
So even though that one gig was used and not free,
it's still available any time it's needed.
Now, let's look at how the processor is used
with the uptime command.
We can see how the CPU cores were used by programs running on our server.
The important information here is the three numbers that we see after load average.
The first number tells us that the load average for the last one minute.
The second one tells us the load average for the last five minutes.
And finally, the third number refers to last 15 minutes.
So if we see a load average of 1.0 is our first number.
It means that one CPU core has been used at full capacity in the last one minute.
On average,
otherwise said, 100% of that CPU core was used intensely in the last minute.
Since it's an average, this could mean 100% utilization for an entire minute.
But it could also mean that for 30 seconds, 0% was used.
And for the other 30 seconds, 200% was used.
That is two cores
that would still lead to an average of 100% utilization
displayed as 1.0 in this field,
if we see 2.0 as our last number,
it means that on average two CPU cores were
fully utilized at 100% in the last 15 minutes.
Now, let's say we have eight CPU cores on the system and we see this load average,
that would mean that six CPU cores were used intensely in the last minute.
But in the last five and 15 minutes, the CPU cores were barely used.
So it's something that happened recently.
Some programs worked hard to do something for a very brief time.
But overall the system is not pushing the CPU too much.
So we shouldn't be too concerned.
But if we see averages like this,
then this would tell us that some programs use
our CPU CORS heavily almost all the time,
it's time to upgrade to a more powerful server or
optimize our setup so that it requires fewer CPU resources.
If we need to see some details about the CPU used on this system,
we can run this command LSCPU.
And if we need to see some details about other hardware on the system,
we can type LS PCI.
Now let's jump into the integrity of file systems
to check a file system for errors. We first must unmount it. In case it's mounted,
we'll learn more about file systems, partitions,
mounting and unmounting in the storage section of this course,
operating systems from the red hat family tend to use XFS file systems by default.
And operating systems from the
Ubuntu family use the EXT four file system.
But this can change in the future for both of them
to verify an
XFS file system. We'd use a command like this.
Now dash B turns on verbose,
output verbose means that more details will be
displayed about what the command is doing.
Dash B is not required,
but it's a useful option to throw in there to see what's going on
slash DEV slash B db. One points to the partition where this file system is stored on.
In
this case, it points to the first partition on the second virtual storage device.
In a real scenario. This could be something entirely different
such as slash DEV slash VD A two or slash DEV slash S
DC three depending on where the file system you want to scan is located
and we'll learn how to decode those device names later on
to check and repair an ext four file system. We can use a command like this.
Now, this is fsckext four and dash V turns on verbose output
dash F forces a check even if the file system reports that it's healthy
dash P turns on the so-called pre mode.
This lets the utility fix some simple
problems automatically without asking us any questions.
But if serious errors are encountered, yes, no questions will still be asked.
Dash V and dash F are optional.
If you can't remember all these options, at least remember to use the dash P option.
That one is more important. If you have a file system with many errors,
it can help you avoid typing.
Yes to dozens of questions and instead they'll be fixed automatically.
You could think about ext four file system problems to remember the dash P option.
Finally,
how do we make sure that key processes
important programs are working correctly on our system?
First, we can use this command
system CTL list dash dependencies.
This will show us a treelike structure of important
system D units that are active and inactive.
A transparent or white circle next to the unit
name means that this particular unit is currently inactive.
Usually that's no issue. Some units or services run once at boot and then exit.
Others are not meant to automatically start at boot time just when they're needed.
But there are some exceptions.
Some service units are very important and should actually run at all times.
An example is the
SSH
service unit that controls the life cycle of the
SSH Daemon.
If this service shows up as inactive, it would mean that SSH daemon is also inactive,
which would mean that administrators wouldn't be able to connect to
this operating system remotely if they need to manage it.
A green circle next to a unit name means it's currently active.
So we generally want to see this green circle next
to all important services meant to run all the time.
For example, besides the SSH dot service,
we also want the cron dot service and the A TD dot service to be active,
otherwise scheduled cron jobs and A T jobs wouldn't run automatically.
Now,
let's purposely terminate our at daemon to simulate an issue with
a service that should generally be active at all times.
Now, in this command systemctl list
dependencies,
we see a white light next to the A TD
service which signals a problem for such an important service
scheduled A T jobs won't be executed
to start investigating what went wrong. We can begin with this comment
system CTL status A TD service.
This will confirm that the service is currently inactive.
The progress that should run is dead or nonexistent
and the last log line will show us when the A TD service was deactivated.
So no serious errors that we need to fix.
And also we can see the service is enabled
to automatically start each time the system boots,
which means that our job is rather easy. All we need to do is start the service again.
In a real scenario, things might be slightly more complex.
But the log lines in the system CTL status command usually
point to what went wrong and why the service cannot start?
Which means that first,
the problem pointed in the logs should be fixed and only afterwards,
can we try to start the service again?
If the logs in the system CTL command don't show the real cause of the problem.
Remember from our previous lesson,
we can dive deeper and see all logs generated
by this service with a simple journal CTL command.
For example, to see logs for the A TD dot service unit, we just run this command.