[Music]
good thank you uh good evening everyone
uh thank you for sticking around I know
it's it's it's late and a lot of us were
at work so uh it's great to see all of
you here uh my name is Peter VOR I will
talk to you about um the challenges that
I see on daily basis on in Security
operation centers and we will talk about
uh my Approach in in in helping them and
fixing them so who am I I'm a consultant
on 8 to5 basis um I work with various
types of organizations with assessing
their security operations assessing
their socks or assisting them with
designing a security Operation Center um
besides that I've been in the field for
quite a bit um I personally do a lot of
research and enjoy um thread modeling a
lot because um if you conduct thread
modeling properly you will get to any
system uh you want to get into right
it's not just clicking every single
button in in Metasploit or a cobal
strike you have to have a plan to get
into the system but anyway that's a
different talk um on the free time I
love to drive uh across our beautiful
country half a million miles um driving
under my belt I really enjoy to go to
places this is a picture from Badlands
that I visited uh last year so um agenda
for today so I will talk about the uh
sock challenges and I will base that on
on my dayto day work we will talk about
the reasons that I identified for sock
failure or issues and challenges then I
will talk about the security operations
framework or security operations
assessment framework that I use to um
assess U the organizations the teams and
to identify things that uh can help them
and then we will talk about um
recommendations so to do that I will use
the case studies from the organizations
that I have seen of course all the data
is um anonymized but those challenges
repeat in multiple organizations so case
study one um a global company they have
a Cel they have a sock manager and they
have one analyst and usually
the requirement from the uh board or
from the business will come in and say
hey we want
365 uh 24x7 monitoring right and you
have one
analyst two managers right one board
member how you will accomplish that and
it's a it's a global company they don't
want to hire especially in today's
economy and challenges and they don't
want to Outsource because they have
sensitive information that they don't
want to share right uh since they are
global company they would like to follow
the sun model right so here there is a
day in us there is a team monitors that
uh the environment then it's night we
move into Europe we move into Asia and
so on clearly here we can see that with
three members on the team you cannot
accomplish now there are some
opportunities since it's a global
company right they most likely have
locations in multiple countries across
the world and they most likely have some
kind of it team in each of the locations
so what they could potentially do is uh
start training and utilizing the
existing employees they have on the in
those sides but that would require um a
little bit reorganization and assigning
those individuals under the sock and of
course that would require training as
well but here you can see that there is
a disconnect between um the business the
requests and that and the sock and what
sock actually um can do so unrealistic
expectations case study two um we will
have a ciso and three analysts not bad
right with three analysts you can start
monitoring the organization pretty well
um you can
even start using the on call system to
have
24x7
monitoring however um in a lot of
organizations sock is being tasks tasked
with additional duties like um hey
conduct the vulnerability management for
the
organization um implement thread
intelligence Management program uh do
in-depth forensics or or pen testing and
so on you you can name various types of
tasks while it's a great idea to give
analysts additional tasks so they do not
burn out looking at the logs and alerts
on like 8 hour 10 hour or 12- hour
shifts right they have a little change
change around in the things that they
are doing implementing a full blown
program like vulnerability management um
for example uh takes that individual
completely out of the sock so if you
start
throwing um those programs on sock to to
do what you are doing is you are taking
away the analyst from the the so that's
functioning somehow and what is
happening you ending up with a um really
poor program right because the let's say
vulnerability management requires
someone full-time at least one engineer
fulltime to manage that so you have
taking this analyst and what's happening
sock sty failing and the program that
they were
uh task with implementing is failing as
well so in this scenario uh you would
still need additional individuals and
what's Happening Here is still lack of
understanding on higher level what it
takes to implement security program like
that on top of having um sock monitoring
and responding to security events within
the
organization excuse me um case study
number three let's say government
organization and as you can see here um
there is a pretty good
organization and and there are multiple
teams right there is enough Personnel to
conduct the
operations and the challenges that they
run into was they wanted to uh take care
of the alerts within an hour right
however when we conducted Workshop uh we
ended up with average of five plus days
of taking care of um the the security
events now and you can see here there is
a level old school level one level two
monitoring uh separation then you have
we have multiple teams for instant
response right for cloud for specific
system for backup systems and we even
have forensics so why does it take five
plus days to detect uh conduct investig
vation and respond one uh the team did
not have instant response plan so what
happens a lot an organization or team
will download a templated instant
response plan from Sans cisa merer or
any vendor you name it they download it
download it check the box for compliance
yes we have instant response plan
however they don't take the Second Step
which is customizing their instant
response plan to their business and to
their organization and infrastructure
that they have so uh whether you are
working with a financial institution
manufacturing organization Factory or
government organization the
requirements are different right the the
monitoring and response in high level
yeah it's the same thing however the
things that are important for this
organization critical assets processes
are different and and that's why you
need to have this understanding and
that's why you need to um customize your
instant response plan based on your
business and it it's needs based on the
teams that you have and critical assets
that you have I have seen multiple
organizations where they start building
in incident response plan during the
incident which is already too late or
after incident which is again too late
right download downloading templated
inant response plan great first step but
you actually have to customize it um to
your organization to your teams and then
you have to socialize it so every single
team within your organization knows
their role and at what point they need
to step in and uh what they have to do
so here no instant response plan and uh
no service level objectives and
agreements between the teams so let's
say U level one analy received a ticket
received an alert started
investigating it was up to the Analyst
at which point they wanted to escalate
to the next level it wasn't that okay
level one analyst receives an alert if
they cannot solve it within let's say 1
hour they escalate they didn't have that
sometimes analysts took a day before
they escalated right and that that
trickled down across all of the teams so
they didn't have that uh third item and
that's uh a really go good Workshop as
well and I um I it would be really
beneficial for you to do it take a look
at security
event uh the raw security event grab a
time stamp see when the security event
was generated then take a security take
the time stamp of the security event
after it was ingested to your system
whether using CIS loog Sim s whatever it
is grab that second Tim stamp see the
time difference then if you are using I
call it old school approach where in the
Sim you have a query that checks for
security events and then uh and then
triggers a ticket generates a ticket or
Pages the analyst right sometimes those
queries are set to run every 5 hours
every 12 hours and every 24 hours and
take a look at that time stamp or
frequency of that scheduled query and
now you have three time stamps right the
raow event when it was generated then
you have when it was ingested to the
central uh repository system right Sim
usually sore sis loock whatever it is
and then take a look at the time stamp
where analyst actually takes a look at
the alert I have seen um in multiple
organizations where uh the time between
the security event being generated an
analysts looking at the alert or looking
at the event is 24 hours right 24 hours
is a long time you are looking after the
fact and um again frequently
organizations are not even aware that
they are responding to um events or
alerts after the fact there was a there
was a best practice um rule
that industry is pushing right now it's
11060 right detect within one minute
investigate within 10 minutes and
respond within 60 minutes I have not
seen a one organization that actually um
successfully implemented that approach
and that practice and actually sustains
it I have seen organizations that detect
and investigate with within 30 minutes
which is a really good uh good outcome
good score but I have never seen
organization that detects within a
minute investigates with 10 and responds
within 60 I have seen organizations that
respond uh that responds takes five plus
days which is again uh to detect and
investigate five plus days that's after
the fact depending on the size of the
incident the response might take longer
but the detect and starting the
investigation should be the lowest time
possible because um you detect the
threat actor within the organization
faster and that gives you time to
respond and recover faster so they don't
they don't
completely um compromise your
environment So based on those um three
case studies and again those were three
case studies but they
frequently um repeat themselves across
the
organizations I identified the most
frequent reasons for the failure or
issues within the so and it's disconnect
between business and security operations
right so it's unclear and unrealistic
expectations that no one really uh talks
about lack of head count funding and
experience no agreements between the
teams that support Security operation
Center so uh service level agreement
service level objectives which is
basically time that you expect all
additional team to to give you a
response or support you so in this case
you have sock I'm sure um organizations
have Cloud teams network teams endpoint
teams messaging teams you name it there
has to be an agreement between the teams
especially when there is an incident
based on on severity that they respond
in specific time uh to support you in
instant response and of course no
instant response plan at the time of the
incident so now once we have those um uh
reasons I will talk a little bit about
how I approach and assess Security
operation Center to help them the first
thing is to understand the business to
align it to the security operations
right uh to align the security
operations to the business so again uh
reiterating depending on the business
focus the the security operations might
look a little bit different the critical
assets will be different um the
important processes might be
different and then um I move into
identifying gaps and opportunities in
the so whether it those are security
processes
or visibility Does the Sock has Does the
Sock have visibility to uh entire
environment does the so have tools to
conduct inant response and forensics uh
operations based on that I Define the
initiatives that the uh security team
would be most beneficial right what
would be the most beneficial for the for
that team and have a conversation either
with uh sock manager or seesaw or Chief
Architect and prioritize those
initiatives based on the goals that
their or organization has now
um you can have the best goals the best
plan right but if you don't have a
support from within the
organization um you will have hard time
implementing those so whenever you
identify those initiatives you also want
to identify the state stakeholders
whether it will be a network team
whether it will be the finance team um
Cloud team you want to have a discussion
with them and explain it to them why
those initiatives are important and have
their buying and support while you have
that it will be easier for you to
actually Implement those initiatives and
of course um document
that uh moving along um what we do next
I'm sure you have read white Papers
written by Myer Sans Gartner vendors
whoever and they all talk about process
people technology however they fail to
mention um what those security pillars
men mean uh and how they interconnect so
in our framework besides the process
people technology uh we also identified
Affiliates which are the teams that
support or assist Security operation
Center uh business because without
business and business needs uh there is
no sock right the the business defines
the need for the sock what needs to be
protected and that that's why this
pillar is here and then we have
visibility because sock needs visibility
and what provides the visibility
technology and we want to make sure that
so has the required visibility to
conduct its
um monitoring and other security
operations let's start diving deeper
into this so uh this is a high level
blueprint um I start with the mission
mission of the security operations so
what we are trying to do what we are
protecting uh the mission can be um well
is enforced by GRC by the governance and
what's governance using they are using
metrics and Reporting so they have they
need to have this information to
properly guide the organization uh we
are moving to planning phase preo a lot
of organizations still had a physical
facility um where they had a sock and
they had a war room now government
organizations still have have those um
private sector moved into having the the
sock uh remote with planning there is
collaboration because all the teams
identified uh to support so need to
collaborate need to have those tools and
agreements and processes defined and in
place um the teams need stuffing um
right and then individuals on those
teams need training and if you want to
retain them they need to have a career
path right there there is there are
often conversations that um analyst
leave within a year within two years
well guess what a lot of those jobs are
I would call it dead end jobs right that
you are sock analyst level one level two
and then there is nothing else for you
to do so what does that uh seasoned
analyst leave looks for another why did
they leave they look for another job
because um there is no career path
progression within the
organization and then you don't want to
burn out um your analy I have worked
with the socks where shifts were 12
hours right or night shifts I mean after
a while 12 hour shifts or especially
nights you you are done so you want to
monitor this U employee utilization if
you want to retain um your analyst and
again the analyst has been who has been
with your organization for a year or
longer he's he or she is finally trained
they know the organization they know the
business they know the infrastructure so
they can actually monitor and respond
properly so it is in your best interest
to keep those analysts and Engineers um
and at the end it all boils down to
budget right everything costs money you
have to have money for for your sock for
the operations for the solutions for the
personnel and and training so it does
boil down into the
budget um now let's talk
about um each one of those Main items so
the the first one was Mi Mission um ask
yourself do you know the mission of your
Security operation Center if you don't
ask your sock manager oriso hey uh
what's the mission of our sock um if
they can't answer that um the rule of to
is take a look at the mission statement
of your
organization and based on that build the
mission for your sock and I I listed
here a couple questions you can see that
that uh can assist you with that right
what are you trying to accomplish what
are you protecting what are your
critical assets and what are the threats
or threat actors that Target your
industry Target your organization and
that will help
you uh after the mission we have
governance and um it's interesting
because governance or GRC is looked at
as a um red Tech AP or a stepchild of
security in one corner you have
governance or GRC in the other one you
have so both of those teams are needed
and both of those teams need each other
and need to collaborate and and this is
why uh
GRC has capability of writing policies
and enforcing them but they have to have
data to do that who has data sock has
the data and they can provide the data
to the GRC team so both of those teams
can collaborate and can guide the um
organization right so again it's not the
um red tape um governance actually
guides the organization so the
organization is compliant uh and it's
lowering the risk and again um couple
questions that can assist you with that
so who uh right and approves the
security policies policies who enforces
them and how uh the measuring to report
the the progress as far as the risk
vulnerability management and so on and
then how are we going to communicate um
security efforts right so so this one
here would be bulletins would be brown
bag events webinars and so
on after that I move into planning so we
understand the business we understand
the business use cases we identified the
key stakeholders that will help us with
implementing those security initiatives
and here it's it's important that uh we
identify the individuals who have vested
interest in our security initiatives and
they understand why we are doing this
and they agree to support it and when we
have that we can start implementing that
with the in um our
organization and um I would like to add
here that you can have the best plan you
can have most skilled analysts and
Engineers but if you don't have buyin
you will not implement this and there is
a there is a quote uh used in in
Industry that culture ISS strategy for
breakfast right again best plan best
analyst if you don't have a buyin uh you
won't be successful with implementing
that um security operations initiative
within your
organization and
collaboration so
um back in the day there was approach
that if you are an it you can do
everything right networking programming
database Administration um scripting you
name it uh similar scenario happens
right now with security you are security
analyst you will do everything you will
monitor you will respond you will do
forensics you will do pen testing you
will do vulnerability management Trend
intelligence management pent tests
purple exercises I mean there is a lot
right well guess what a sock or security
analyst cannot do it on its own they
need support from additional teams they
need support from um the organization
and those roles and responsibilities
have to be outlined and and documented
and here couple things listed right
someone WR security policies someone
enforces them then there is a technology
deployment like firewalls EDR uh Sim you
name it configuration maintenance of
those tools uh monitoring itself Cloud
comes into play email then instant
response assisting with instant response
uh if needed forensics um development
and tuning of the security content now
very popular automation registration
well one analyst cannot do it alone even
team of analysts cannot do it alone you
will need to collaborate multiple teams
you will have to
um outline the roles of each of the
teams and how those teams will work
together uh
Staffing each team needs individuals
they will that will perform the work um
they need to be trained individuals
right so that you need to have a
training plan and then if you want to
ret retain them again career path
progression and if you don't want to
burn them out employee utilization you
want to measure that and you want to
rotate your analyst through various
security tasks that so they don't burn
out or they don't check out I again I
worked 12 hour shifts and we had uh
three types of analysts the ones that
would just come in and after a while
they wouldn't even look at the alerts
because 12 hour shift he just kills you
right so they didn't care anymore the
ones who cared were overwhelmed and
after the while after a while they were
um they would just quit or they would
move somewhere else right and and the
ones that try to keep up but again um
they couldn't because it was too much
for them so if you add those um
additional tasks uh for example do some
of the content development do some of
the tuning do some of the thread hunting
it keeps the analysts going keeps their
mind fresh and retains them for a longer
period of time keeps them uh protects
them from burning
out and um it all boils down to the
budet um and the statement that I like
the most here is do you have funding uh
for at least the minimum requirements of
security operations to accomplish the
mission so is it fun that to conduct the
minimum operations that it's supposed to
right if not then there is a problem now
um it's easy to ask for money it's easy
to um play with the feelings and use
fear but
T that will work maybe once or twice
that the med is doing enough of that for
for all of us um so I do not
recommending using fear or feelings to
to get money for your security
initiatives uh what I recommend is to
again identify those opportunities
initiatives um Define them prioritize
them get the buying of the stakeholders
documented and presented why uh specific
initiative or initiatives are important
and then um get the buy and Implement
them so uh this was a quick run through
through the the framework that we are
using um so what I have is um a QR code
is actually to the PDF to the book uh
that you can download and and read
through where it actually defines every
single security pillar and element that
we are using so you can read it in your
free time I have also posters for for
those um security P pillars and elements
so uh if you want grab grab yourself one
and we have some time right now for um
Q&A yes so what would you consider a
bare minimum
sack bare minimum sack Okay so
let me let me give you a little extra
that okay so minimum
personel and bare minimum functions okay
all right so say your your companies are
gr into where they want to have one
start fresh yes and it's like okay we
know we want to have security want to
have better visibility what's a minimum
place to start be a good pitch to say
hey look we can start here and then we
can gr
okay good question so first you want to
have visibility right um and you start
with network right Network visibility so
so of course firewalls uh however you
know we know that most of the traffic
now is encrypted probably 80% so without
NextGen firewall and breaking the
encryption your visibility will be
limited so now you you will want next
gen firewall and you want to see whether
you can break encryption in your
organization or not because some
organization allow that some do not
because you have a privacy challenges
right so the network visibility then
since um network is just part of it you
will want to have endpoint visibility
right so endpoint logs endpoint
operations um if you can afford it um
you or your organization can afford it
EDR right so you know exactly what's
happening on on the end points so you
have those two however if you want to
grow in the future there will be more
logs more alerts and more appliances so
you will need a central uh repository
for it back in the day it was a Sim
right you name it there were a lot of
them and they are good but there are
challenges with them because um when we
talked before there is a delay between
the time the alert or event gets into
the Sim the SIM escalates it to the
analyst so you have to take a look at
that so you have three items here uh if
you want to start looking at the emails
we know that most of the most of the
thread vectors that attackers are using
is through through fishing right and
sending emails so you will need to have
uh email visibility so now you have a
fourth
tool um so let's let's start with those
four tools
then do you want to have 24x7 365
monitoring
that that the question is important
because if you do then you need to have
either a Personnel to support that or
technology to support that right so like
having two analysts they will not handle
uh 24x7 365 they will be they will be
gone within 6 to 12 months right so you
have to take a look at that um the
industry is pushing Solutions like
automation orchestration which can
automate work of level one analyst
possibly level two however if we are
getting to Serious incidents you will
need um analysts uh on experienced
analysts doing the work right so and
that's one and two implementing
automation orchestration requires
skilled individuals and and takes time
so I gave you four Solutions and then
decision Point whether it's a 24x7 365
if if you can afford it and you don't
want to Outsource you will have to
calculate how your shifts are working to
support it now if you can Outsource it
there are plenty of companies MDR uh
msps that can do the
monitoring after hours on the weekends
and holidays for you however you have to
remember that they will not have a full
understanding of your business the
purpose they will not know which assets
are critical which which are not so
that's a decision Point again for you to
see uh what's worth more right having
that 24x7 365 monitoring with within
your organization inhouse or you are
okay with Outsourcing this work and
possibly not covering everything or 100%
with support so we are so we are running
into those decision points I know it
wasn't a clear cut answer but it it's
not that simple well help Che it
out we got we
got we
got um you know we
got we got a lot of that missing is
going be so that was good thank
you add that building block as far as
247 don't need okay not thatal now do
you have cloud cloud uh the only thing
that's s
cloud is a website okay so a server and
it's not it's not critical okay Noy
corporate dat or anything like that is
on there there is some Azure assets
okay they are only accessible
through but but now the this web server
that you have in the cloud does it have
direct access to your corporate
environment okay okay
complely got you okay so so if you would
have Cloud I or you have Cloud I would
add U Cloud monitoring
okay yes yes so okay so now um you have
the tools you have appliances uh are
they deployed across your environment
are are you sure you don't have any gaps
there's always gaps yes so you need to
start you need to start hunting them
right you need start looking for
that yeah it's just now to have getting
to the point we
needed start this and that that is a
challenge what is my pitch how many
people we l um what's theability which
is again we need that dashboard ability
look at stuff it's like looking at K
logs like you said it burns out it does
yeah and and now since you are running
into situation when you have multiple
appliances multiple vendors um does it
make sense for the analyst to learn 10
tools does it make sense to consolidate
because from the if it's a large
organization from the architectural
perspective The Simple Solution is the
best because there is a less number of
solution that analyst needs to learn and
use right so you don't want to go to 10
different tools and learn them and jump
from one to another you want to have it
Consolidated if if you have skilled
Engineers you might want to even move to
the source solution where some of the um
and and check which alerts are triggered
the most or or are simple to handle and
automate that that response you could
look into that as well but again it
depends what stage you are at well it's
do we just say simple goog and put
everything in there so that we can see
it filter do want stack so that we don't
with looking at stuff do we want to get
really complicated and do security dump
everything in there so then it's like
from end to end
[Music]
you yes and and here is here is another
note so um those solution just just for
the logs I again I know it's it's a best
practice from five plus years ago but I
I I call it old school why because if
you look at this uh those are those will
provide you uh information after the
fact so you have detect well actually if
you go through an entire one uh entire
cycle you have identify protect detect
respond recover right identify protect
it's called before the boom detect
respond recover it's after the boom so
so after alert after something already
happened and when you look just on the
RO logs and and hunting through them you
are on the right side so right right
side so after the boom after the fact uh
which where you want to move is to the
left which where is identify protect and
then detect and immediately recover um
the the tools like CIS log do not ort
that you want to move towards automation
where when something is detected your
tools immediately respond or prompt the
analyst to take a look and respond I
would look towards uh that direction if
your budget um allows that so then you
want to move
for then you got something where it
comes in you got more alerting
well partially right security monion
won't respond you would want to have
something like EDR and sore that if
something is detected the tool can
either immediately respond or provide
everything to the analyst and analyst
just clicks the button and response I
know that it won't be a 100% solution
but that will save your analyst time
from just scrolling through the logs
okay so a little more ACC than correct
yeah you're welcome so EDR and S Alo all
of that and what's your name Mathew
thank you thank you for the question
great question um anyone else you know I
know it's late anything goes go ahead
anything go can I ask about a cool
accent about what oh it's Polish Polish
yes any other
questions
okay I guess that's all thank you very
much for showing up and thank you very
much for your
time