There are two primary methods for
addressing vulnerabilities involving our systems.
One is applying patches or updates provided by the vendors and
adjusting the functionality or the security setings within our systems.
Typically vendors release patches on designated days depending
on the software that it's involved upon.
When we initially get that patch. We want to test it.
We want to make sure that it's operating in the way it should,
by testing the different patches within a secure system,
we can identify faults or flaws in how the
patch interacts with our other software within our system.
We want to make sure that if I'm getting
a Microsoft Windows patch that it's not interfering with Adobe
Acrobat or maybe one of our other
uh
pro
proprietary programs that we utilize on that system.
If it doesn't work, then the testing fails and we don't need to implement the patch.
We need to figure out what we need to fix the problem.
If it does work and it validates properly,
then we're going to continue the implementation,
moving to implementation means that we're going to proceed
with the patching process across one segment or more.
Usually we'll do one segment at a time we'll go through and we'll say, you know what,
let's do the HR department, they've got a low computer count.
Let's see how it pulls off outside the
testing environment into a real world working environment.
So we'll roll the patch out and again,
we'll validate it to make sure that it's working properly.
And if everything is
shiny and it functions as per demand, we move on with our lives,
if it doesn't and we see flaws, we see crashes within the program.
Maybe the hr systems that we just implemented the patches on it,
add some systems or had some tweaks in it that cause the system to crash.
Maybe different programs aren't working the way they were intended
or introduce vulnerabilities that we didn't anticipate if that occurs.
We're going to perform a rollback.
Usually a rollback implies that we're going to roll
back the system back to its primary operating function.
Before we implemented the patches,
we're putting the system back the way we found it.
That's all that rollback really is
usually patches go inside of a maintenance window.
I remember when I worked for several different companies.
The first one,
our maintenance window was from 10 p.m. to 6 a.m.
We had eight hours to work during those eight hours.
We had minimal customer impact,
meaning that customers didn't usually get on our systems.
Between that time
with another company that I worked for the maintenance
windows was from 12 a.m. to 4 a.m. Again,
not a very big maintenance window, but it didn't impact our customers.
Your maintenance window is gonna be dependent
upon how your customers or how your organization
deals with the different customers.
Uh There was another company that I was working with and I consulted with
and their maintenance window was literally from 6 p.m. to 6 a.m.
All their employees only worked from
8 a.m. to 5 p.m. And so we were able to
have free reign of the system in the enterprise environment,
literally for 12 hours, it was great. However, most systems are not like that.
You should anticipate having a maintenance window between about 11 pm at night
to about
3 to 4 a.m. in the morning. That's a typical maintenance windows for most companies.
When we provide a maintenance window,
it's scheduled even though we have this maintenance window,
even if that maintenance window is for 12 hours, we just don't say, oh,
it's a maintenance window.
I could do whatever I want.
We still want to schedule our activities.
If we're going through, we want to provide some insight into it.
We want to tell people, hey, our systems are gonna be down from this time to this time.
Imagine if you were,
if you're that employee working on a document and
you have a meeting first thing in the morning,
you have to work late at night
and somebody just took down all the servers without any notice,
you could be working on an important document that is badly needed for the,
for the security of your company
in a different department.
And your it team just took down the entire system without anybody knowing.
You're frantically calling the 24 hour help desk to figure out what's going on.
It's creating more work for everybody involved.
And you've just created not only hassles for that employee,
but for the network as a whole because you've involved
three or four other departments because you didn't schedule it properly
when we do maintenance window work,
we need to make sure that it's scheduled and reported properly.
However, if we have an emergency within our maintenance window,
that's when we want to do it.
Let's say we have a critical patch and it comes out at three o'clock in the afternoon,
you get with your boss and you're like, hey, this is an emergency patch.
We need to implement this as soon as possible or the
system is constantly crashing and it's almost got a 50% uptime.
So we can't continue to operate this way.
And even though I didn't schedule it within 24 to 48 hours notice,
which is normally our customer's policy,
we consider it an emergency patch you go through,
you're still going to notify the employees.
Hey, our systems are going to be down from this time to this town. Expect outages.
We still want to plan that out as much as possible.
That would be considered an emergency patch. We have both planned and unplanned.
When on my earlier example, when I talked about, hey,
we're going to roll out patches.
We have all this time. We want to provide that 24 to 48 hour notice.
We want to plan it ahead of schedule. We want to get approval.
We want to go through the process that our company has
outlined to do a planned outage during the maintenance window.
However, it's not uncommon to have an unscheduled or unplanned outage.
If we have an emergency patch or we have critical systems that are going down
unscheduled could be during the maintenance window or it could be during
the middle of the day more often than not whenever possible.
We want it to be during the maintenance window,
but that's not always possible given the criticality of the
problem or the severity of the problem that we're facing.
Finally, we have exceptions,
there are exceptions to everything that we do both in it and in life itself.
Sometimes we may have a critical problem but we
don't have the budget to properly fix it.
We can have an Exception to policy for budgeting.
When we have an Exception for policy for budgeting,
we can implement a patch or a problem but not go through all the nature of it.
Let's say that I've got a critical system problem and it requires
a specific framework or a specific hardware to fix the issue.
We can use a compensating
to facilitate an Exception to that hardware
in order to apply a software update that could
do the same thing while it's a compensating control,
we are providing an Exception via the budgeting problem.
There's also risk exceptions.
You may come to a point within your organization
where the risk is too high and your company just
doesn't want to deal with the risk attributed to
that fix or the risk attributed to that problem.
I know one company I was working with if it was raining or specifically during
Halloween, we weren't allowed on the roads with any company vehicles.
Halloween was off limits from 2 p.m. until
2 a.m. The company had over 30,000 vehicles in
its arsenal and it straight out said you are not to drive on the roads on Halloween,
October 31st from 2 p.m. to 2
a.m. under any circumstances without director approval.
This was a risk the company was not willing to take and the incident for
pr was one of the company vehicles possibly
hitting a child while they were triggered.
The risk was too great.
And so that provided a risk or excuse me, an Exception to a normal policy.
We can do lack of vendor support if there is a vendor support that isn't available.
Maybe there is a problem with one of our systems and we need that vendor support
to properly implement the fix because we just don't know we don't
have the inhouse specialties or the expertise to deal with that problem.
We may need vendor support if that vendor support isn't available on a Friday night.
We may say, you know what, let's push it
until Monday morning or Monday night when there's vendor support.
We can also have prioritization like we've stated before.
Some things are more critical than others.
If I have a higher priority to one item and we have this newer item that came up,
it may not matter that it's also a critical item.
Our vendors or our uh our company or organization could say,
you know what I realize this is a problem.
We already have this other critical function that's had a higher priority
and it's already scheduled. We're going to do that instead,
this is our exceptions to policy.