[00:55.300 --> 00:59.600]  Welcome to my session.
[00:59.600 --> 01:08.000]  Today I'm going to talk about the potential risk of using telephone providers or telephone modules in a cloud environment.
[01:08.040 --> 01:11.680]  But before we jump with that, I would like to introduce myself.
[01:11.880 --> 01:13.000]  So my name is Uri.
[01:13.000 --> 01:15.480]  I'm the CEO and co-founder of Zest Security.
[01:15.620 --> 01:22.620]  I have spent over decades working in cybersecurity, specializing in both offensive and defensive security practices.
[01:22.640 --> 01:31.600]  As a former security architect, I have tackled countless of security challenges in my career, and some of them I would like to share with you today.
[01:33.100 --> 01:46.080]  So the agenda for today, we're going to talk about a telephone, specific for telephone modules and providers, and why we should evaluate any tools that we want to use in the organization.
[01:46.320 --> 01:53.800]  Then I will share with you some analysis we perform on those providers' modules from a vulnerability perspective.
[01:54.240 --> 02:01.420]  And after that, we're going to talk about why we should take care of that and what is the impact of those risks on your organization.
[02:02.520 --> 02:05.700]  And next, I'm going to show you two attack scenarios.
[02:05.700 --> 02:15.340]  One of them will be involving vulnerable providers, and the second one will be malicious modules, and how this could be a significant risk for your organization.
[02:15.880 --> 02:22.700]  And last, it's what we can do to prevent it or even to reduce the risk.
[02:24.560 --> 02:25.560]  All right.
[02:25.560 --> 02:31.420]  So as a security architect or security engineer, your main goal is to make sure your organization is secure.
[02:32.140 --> 02:41.560]  To do that, sometimes you need to read the documentation for the tool to understand how it operates in your organization, if there is any potential risk there.
[02:41.880 --> 02:44.360]  In this case, I'm talking about Terraform.
[02:44.360 --> 02:55.480]  Terraform is a powerful tool that helps an organization specific for DevOps and SRE to manage their cloud resources by code, or what we know as IAC.
[02:56.640 --> 03:00.280]  What cut my eyes here was this section from documentation.
[03:01.020 --> 03:04.940]  By the way, they did a great job about document everything.
[03:05.500 --> 03:11.580]  So in this case, Terraform configuration will have full access to the variable in Terraform state.
[03:11.820 --> 03:17.240]  And other than that, Terraform cannot prevent a malicious provider's module for exfiltrating the sensitive data.
[03:17.520 --> 03:24.460]  Other than that, they also recommend to use only a trust module and provider in your Terraform configuration.
[03:24.840 --> 03:26.820]  That raised me a few questions.
[03:26.820 --> 03:28.700]  First, what is Terraform providers?
[03:28.700 --> 03:31.300]  The second one, of course, is what is Terraform modules?
[03:31.300 --> 03:35.920]  And how can an attacker exploit this to their purpose?
[03:37.360 --> 03:41.740]  So let's understand what is Terraform providers and what is Terraform modules.
[03:41.740 --> 03:49.800]  Terraform is based on plug-in called the provider to interact with a cloud provider, a SaaS provider, and other API targets.
[03:50.120 --> 03:59.940]  Each provider defined a set of data resource type and data sources that help Terraform to manage the resources.
[04:00.560 --> 04:16.420]  In other end, modules are a combination of TFL that located in the same directory that help the user to define what kind of configuration resource they want to deploy to the cloud environment.
[04:17.220 --> 04:26.220]  So actually what happened, when user want to create a new resource, it will create a TFL file, what we call also modules.
[04:26.220 --> 04:40.700]  With all the configuration to apply to the resource, then the Terraform core will translate to the Terraform provider, and the Terraform provider actually will transmit the task to the cloud provider.
[04:41.620 --> 04:51.640]  From attack surface perspective, it's interesting because Terraform provider is written in the Go language program.
[04:51.960 --> 04:57.000]  The communication between Terraform core and the Terraform provider is based on protocol RPC.
[04:57.520 --> 05:03.740]  And the communication between Terraform to the target, Terraform provider to the target, is HTTPS.
[05:03.740 --> 05:07.960]  And I'm sure all of us know how many vulnerability each one of the components has.
[05:08.480 --> 05:13.260]  So again, Terraform is a powerful tool, but it's not bulletproof.
[05:13.260 --> 05:23.660]  And let's be honest, in this scenario, we need to take care of this attack surface, because it could be very dangerous for the organization.
[05:25.100 --> 05:35.220]  But before we jump to the risk analysis we perform, I would like to share with you how AshiCorp managed those providers modules.
[05:35.400 --> 05:38.020]  So AshiCorp have three different tiers.
[05:38.020 --> 05:45.100]  The official one is owned by AshiCorp itself, and they develop providers and modules.
[05:45.100 --> 05:54.580]  And the second one is the partner, other company that develop their own providers modules, but they have some partnership with AshiCorp itself.
[05:54.580 --> 06:01.500]  And the last one is community, is individual contributor that develop their own Terraform providers and modules.
[06:02.120 --> 06:22.260]  Another interesting part here Is that each one of the providers modules that are part Of those tiers are also public accessible in a repository in GitHub, which means everybody can read the code, everybody Can answer for a vulnerability, and review the code,
[06:22.260 --> 06:28.620]  and even If they do a great job, they can do a step further and inject Some malicious code.
[06:32.230 --> 06:32.750]  All right.
[06:32.750 --> 06:34.710]  Let's talk about the analysis we perform.
[06:34.710 --> 06:44.870]  So we tested closely the ten widely used Terraform providers And modules from each one of the tiers i mentioned before.
[06:45.190 --> 06:48.990]  And what we found was all of them have some critical Vulnerability.
[06:48.990 --> 06:57.190]  The highest number was in the Community, which makes sense because individual Contributor probably take care less of the vulnerability.
[06:57.670 --> 07:11.690]  But it doesn't mean the other official partner is not risky For us, because still we're talking about 11%, almost 12% For the official one that has critical vulnerability, and Almost 30% for the partner.
[07:12.450 --> 07:23.790]  Again, even if it's a critical To exploit this vulnerability, it doesn't mean we don't need to Take care or to understand what is the risk of that.
[07:26.750 --> 07:28.330]  Now i have a question for you.
[07:28.330 --> 07:29.590]  It's a tricky question.
[07:29.590 --> 07:33.070]  Does having more non-vulnerability make you more Secure?
[07:33.630 --> 07:36.790]  i think it's a tricky question.
[07:36.790 --> 07:46.510]  Because in my opinion, when you have more vulnerability, maybe It's because the owner or the organization trying to handle Those vulnerabilities in order to fix those.
[07:46.690 --> 07:51.850]  And if you have less, maybe the owner or the organization are Not checking for a vulnerability.
[07:53.630 --> 07:56.330]  This is why i think it's kind of tricky.
[07:56.330 --> 07:58.390]  But i have another question for you.
[07:58.390 --> 08:00.890]  And feel free to raise them.
[08:00.890 --> 08:04.170]  How many of you know using Telephone production?
[08:06.130 --> 08:07.450]  all right.
[08:07.450 --> 08:11.730]  How many of you know which telephone providers you are Using?
[08:14.420 --> 08:15.280]  all right.
[08:15.280 --> 08:18.320]  And the last question, are you scanning those for Vulnerabilities?
[08:21.420 --> 08:22.400]  all right.
[08:22.400 --> 08:27.620]  So let's understand why i asked those questions.
[08:28.800 --> 08:38.620]  So we took a closer look on the community, which has the highest Risk, and we tried to figure out how many downloads they have.
[08:38.620 --> 08:42.620]  Like how many users use those providers from the community.
[08:42.620 --> 08:47.300]  So what you can see here, we're talking about millions, and This is only for one month.
[08:47.460 --> 08:49.860]  Think about it for one second.
[08:50.100 --> 09:00.880]  One month, and we have a million downloads, which means every Download could be introduced for an attack for an attacker's Side.
[09:01.120 --> 09:03.160]  So this is like a huge number.
[09:03.160 --> 09:05.600]  And this is another reason why we need to take care.
[09:05.600 --> 09:10.400]  And of course it's also a reminder to understand what we Are pulling to our environment.
[09:13.410 --> 09:15.140]  So why it's important.
[09:15.670 --> 09:24.490]  As you have seen before, like the slide before, we're talking About millions, which means it's kind of a cold mine for Attackers.
[09:24.490 --> 09:33.270]  Because it's very attractive And a lot of users use it, it could be that the attacker will Try to exploit it or to use it to manipulate the users.
[09:34.250 --> 09:40.470]  Another spot is kind of a blind spot for application security Processes and sdlc.
[09:40.470 --> 09:51.410]  Somehow, as you see, almost none Of us raise their hand, take part of the scanning the Provider's modules for some vulnerabilities.
[09:52.650 --> 10:04.050]  And last, all of us, i'm sure all of us know how manual and Expensive it is to fix things and it's consuming a lot of time And it's not efficient.
[10:04.770 --> 10:11.970]  So i share with you about Terraform providers.
[10:11.970 --> 10:25.110]  I also share with you some Analysis we perform on those both terraform providers and Modules and what are the risk of that and why we need to take Care of that.
[10:25.710 --> 10:27.850]  Now it's time for the fun part.
[10:27.850 --> 10:29.430]  I'm going to share with you two scenarios.
[10:29.430 --> 10:32.550]  One of them, how can attacker exploit a vulnerable provider.
[10:32.550 --> 10:40.650]  And the second one is how can attacker manipulate the user to Install malicious terraform and install some backdoor there.
[10:43.140 --> 10:48.920]  So in this case, i'm going to share with you a cv from 2021.
[10:49.200 --> 10:55.860]  Even though it's a bit old, the reason i decided to share with You is because i faced it just two years ago.
[10:56.080 --> 11:01.100]  And it was very close to being in the production.
[11:01.100 --> 11:05.980]  And lucky for us, we managed to find it in time and prevent it.
[11:06.180 --> 11:14.860]  So the vulnerability itself is part of the vault provider, Which if you remember the tiers that i mentioned before, it's The official one.
[11:14.860 --> 11:18.260]  That should be more secure or Trusted.
[11:18.920 --> 11:35.220]  So the risk here is that the Vault provider didn't map the bound labels correctly, which Caused some unauthorized access to some service account from gcp.
[11:37.870 --> 11:40.690]  Let's see in reality how it will work.
[11:40.690 --> 11:45.550]  So the attacker read only access for some service account.
[11:45.870 --> 12:00.990]  So if we are talking about previous attack, we know that Many of the attacks that happened recently obtained tokens From some public repository or s3 bucket that was open to Everybody.
[12:01.410 --> 12:04.590]  So this step, it's kind of Easier for the attacker.
[12:04.590 --> 12:10.890]  Next, the attacker needs to Identify that the user defined vault by terraform itself.
[12:11.290 --> 12:16.090]  In this side, by the way, the user acts as it should be.
[12:16.090 --> 12:17.590]  Defined the bound service account.
[12:18.290 --> 12:21.350]  Defined bound label as you can see on the top.
[12:21.750 --> 12:36.250]  But what happened when he executed the terraform apply, The vault terraform provider took the bound label and mapped It to a non-variable, which make it the bound label empty.
[12:36.850 --> 12:49.670]  In this case, when the attacker identified that the user used The terraform vault to authenticate, he just needs to Use the token that he obtained from the repository.
[12:49.810 --> 12:57.350]  And even if he didn't have any access to that, when he will Authenticate with vault, surprisingly, we get access.
[12:58.030 --> 13:06.350]  From this point, what you need to do is just authenticate with Gcloud, and you will get access to the admin role.
[13:06.590 --> 13:09.190]  And from this point, you can do whatever you want.
[13:09.190 --> 13:13.090]  You can do lateral movement, data exfiltration, and service Disruption.
[13:13.490 --> 13:14.730]  You name it.
[13:18.880 --> 13:37.100]  Another interesting part about exploitation specific in the Terraform provider, because it's public accessible to the Internet, if you will go to the provider itself and search from Issues or some security issue, you will find a lot of new Security issues there,
[13:37.100 --> 13:37.800]  or old.
[13:37.800 --> 13:38.640]  Some of them open.
[13:38.640 --> 13:40.980]  Some of them closed.
[13:41.180 --> 13:45.980]  But the interesting part here, A lot of them are not associated with any cv.
[13:46.540 --> 13:53.860]  Which means if you will scan those providers for Vulnerability, you will miss it, because they don't have any cv.
[13:53.860 --> 14:03.140]  And this is another reason why we need to review the code, Review the issues to understand if they close the security issue Or not.
[14:06.400 --> 14:06.960]  All right.
[14:06.960 --> 14:10.000]  Let's jump for the second part.
[14:10.000 --> 14:11.760]  The malicious modules.
[14:12.140 --> 14:16.120]  So in this case, a target can upload a malicious module to Terraform registry or github.
[14:16.140 --> 14:20.200]  It has a lot of options to Convince the user to install it.
[14:20.200 --> 14:26.920]  Could be a local path, Terraform registry, github, and so on.
[14:27.400 --> 14:33.360]  In this example, i'm going to show you a terraform module That creates an ec2 instance.
[14:33.500 --> 14:37.580]  It will look very normal.
[14:37.580 --> 14:38.960]  Nothing suspicious there.
[14:38.960 --> 14:45.140]  But behind the scene, it will Use user data from aws to inject the backdoor.
[14:45.420 --> 14:59.120]  If you are not aware of the user data, user data is the kind of Feature that aws helped to install some software and Components before a launch.
[15:03.440 --> 15:11.560]  So in this case, the attacker Used a social engineer to convince the user to install the Malicious modules.
[15:11.560 --> 15:29.700]  Once the user installs it, you Will use it, as you can see in the bottom, import the malicious Modules, and there is nothing malicious, looking malicious, Just the word malicious, but in reality, you're not going to see That.
[15:30.340 --> 15:37.300]  So the user creates the ec2 Instance, as the module should do.
[15:37.740 --> 15:40.000]  And we execute the terraform apply.
[15:40.000 --> 15:48.180]  Once you do that, you basically not only create the ec2 Instance, but you also create the access to the attacker.
[15:50.960 --> 15:56.860]  How the malicious module looks from the attacker's side, it Looks exactly like that.
[15:56.860 --> 16:03.660]  Still, nothing malicious, Because he obfuscated all the malicious stuff in base 64.
[16:03.940 --> 16:21.520]  And another important thing to say here is in my career, i've Seen many devops team in sre use base 64 to encode the user Data, because it makes the code much more readable and easy for Them to manage it.
[16:26.850 --> 16:35.930]  So we talk about two attacks And one of them involving a vulnerable provider, the second One involved in a malicious module.
[16:35.930 --> 16:38.770]  There is another one, but Unfortunately, we don't have time.
[16:38.770 --> 16:40.190]  It's a malicious provider.
[16:41.110 --> 16:45.270]  But let's talk about what we can do to reduce the risk.
[16:45.770 --> 16:57.830]  So the best practice, read the documentation, read the code, Try to review all the issues that you have there, and try to Find a scandal repository for a vulnerability.
[16:58.030 --> 17:03.050]  Even if they have a lot of vulnerability, it's good to Understand how the owner takes care of that.
[17:03.130 --> 17:13.610]  If they have a vulnerability, for example, for a long time, Which means probably the attacker, the owner doesn't care About that, so it could raise a flag there.
[17:14.930 --> 17:15.930]  Version pinning.
[17:15.930 --> 17:21.050]  Pin the version of your Provider to reduce the possibility of introducing Vulnerability.
[17:22.130 --> 17:27.570]  It's where you can find all the Versions from providers.
[17:28.190 --> 17:43.010]  So you can keep tracking the Version that the provider has there, and you can also scan it With some open source tool to understand if you're using the Latest version or not.
[17:43.850 --> 17:46.650]  Another important thing is to Enable the state locking.
[17:46.650 --> 17:54.290]  State locking is kind of mirror What the user do with terraform, and what terraform Apply on the cloud.
[17:54.470 --> 18:03.350]  Which one, which attacker, When attacker have access to it, you can manipulate the state File and cause a lot of problem to your infrastructure.
[18:05.450 --> 18:13.870]  Monitoring your terraform plan and state file, and try to Identify some unexpected changes specifically in the state file.
[18:14.590 --> 18:19.670]  Scan your configuration with iac tool.
[18:19.670 --> 18:23.490]  That will help you to prevent some misconfiguration that Deploy to the cloud.
[18:27.410 --> 18:28.730]  All right.
[18:28.750 --> 18:30.370]  What about mitigation?
[18:30.770 --> 18:43.730]  you can also implement some Control to reduce the risk, to help you to kind of minimize The exposure or use some high privileges in your environment.
[18:43.730 --> 19:02.210]  So, for example, you can protect the cicd system Application log, especially the state file by using iam role And give terraform just these privileges and use only the Temporary cadential rather than the log lib secret.
[19:02.590 --> 19:10.210]  You can use a network restriction like vpc load Balancer to enable only noncommunication between services.
[19:10.210 --> 19:20.150]  You also can use cwpp to prevent nonmalicious communication From your infrastructure.
[19:20.930 --> 19:35.570]  And last, you can use cloud Watch, and this is specific for aws, to create an alert to Detect if someone tried to read your state file or to do some Manipulation there.
[19:39.560 --> 19:41.040]  Question?
[19:44.390 --> 19:44.970]  All right.
[19:46.170 --> 19:49.010]  One second.
[19:49.710 --> 19:54.890]  What scanning tools do you recommend?
[19:55.670 --> 19:57.990]  That's a good question.
[19:58.030 --> 20:02.510]  So there are tons of open Source that you can use to scan those.
[20:02.730 --> 20:11.570]  I really like for repository to scan, if you're familiar, part Of aqua repository.
[20:11.970 --> 20:14.270]  You can use checkup.
[20:16.550 --> 20:24.870]  There is another tool called clear for images, if you want to Scan those.
[20:25.690 --> 20:27.210]  There are a ton of tools.
[20:27.210 --> 20:30.050]  I can share with you later on if you want.
[20:30.750 --> 20:32.250]  That's it.
[20:37.740 --> 20:47.200]  No, so checkup, scanning a Telephone file, trivy can scan the repository where the Providers and modules are located.
[20:55.960 --> 20:56.820]  Okay.
[20:59.500 --> 21:01.360]  Any other questions?
[21:04.790 --> 21:06.170]  All right.
[21:07.150 --> 21:07.950]  All right.
[21:07.950 --> 21:08.550]  Thank you, guys.
[21:08.550 --> 21:09.830]  Thank you very much.
[21:24.350 --> 21:24.430]  Thank you.