In this lesson,
we'll explore three important tools and techniques used
for enumerating the web:
Web Application Firewalls, Web Crawling,
and Manual Enumeration.
We'll break down each concept,
explain why they are important,
and provide practical examples of how they can be used
to gather information during the Pentest.
First, let's talk about Web Application Firewalls,
commonly known as WAFs.
While Web Application Firewalls can typically be used
to protect web applications by filtering
and monitoring HTTP traffic,
they can also be leveraged by pentesters
to gather valuable information about the target environment.
Imagine you're trying to find the origin IP address
of a web application that is protected by a WAF.
The WAF masks the origin server's IP address
by acting as a proxy.
However, certain misconfigurations or vulnerabilities
in the WAF can reveal the origin IP address.
One method to discover the origin IP
is to analyze the DNS records.
If a WAF is not properly configured,
the DNS records might still point to the origin server.
Tools like dig or nslookup can help you
by enumerating the DNS records to find the origin IP.
Next, let's discuss web crawling.
Web crawling is the process
of systematically browsing the web
to collect information about websites.
This technique is useful in penetration testing
for discovering hidden content and vulnerabilities.
A common tool for web crawling is "DirBuster."
DirBuster automates the process
of finding hidden directories and files on a web server
by brute-forcing possible URLs.
For instance, running DirBuster against the target website
might reveal hidden admin panels
or configuration files that were not linked
to the main navigation.
Using DirBuster or similar tools,
you can create a map of the website's structure,
including directories and files
that may contain sensitive information or vulnerabilities.
Lastly, let's cover manual enumeration.
Manual enumeration is a process of manually probing
and investigating a target system to gather information.
It involves exploring the website's resources
such as robots.txt, sitemap files, and platform plugins
to uncover potential entry points and weaknesses.
The robots.txt file is used to instruct web crawlers
on which parts of a website should not be accessed.
However, attackers can use this file to identify directories
and file that a website owner wants to keep hidden.
These injuries reveal the existence
of potentially sensitive directories
that you can investigate further.
Now, let's discuss sitemaps.
A sitemap as an XML file that lists all URLs of a website
and is intended to help search engines index the site.
Sitemaps can also provide good information for pentesters
by revealing the structure of the website,
how often certain pages are updated,
and all its accessible pages.
For example, if we look at diontraining.com sitemap,
you can enumerate all the URLs listed
and explore each one for vulnerabilities
or sensitive information.
Finally, let's discuss how we use plugins in our tests.
Web application often use various plugins
or extensions to add functionality.
Enumerating these plugins
can reveal potential vulnerabilities
if the plugins are outdated or misconfigured.
We can manually inspect the source code of web pages
to identify platform plugins.
For instance, viewing the HTML source code
might reveal comments or scripts indicating the use
of specific plugins.
Knowing the exact version of a plugin allows you to check
for known vulnerabilities and plan your attacks accordingly.
For example, the WP GDPR Compliance plugin for WordPress,
ironically designed to help for privacy compliance,
had a critical vulnerability in versions before 1.4.3,
allowing unauthenticated attackers
to modify the WordPress options and inject malicious code
due to improper handling of its AJAX function.
Attackers could exploit this by sending crafted payloads
to the website's AJAX endpoint,
enabling new user registration
with administrative privileges.
This granted attackers full control over the site,
allowing them to install backdoors,
alter content, and other things.
The issue was resolved in version 1.4.3,
which validated and sanitized input properly,
and users were advised to update the plugin
as soon as possible to prevent exploitation.
So, remember, using Web Application Firewalls (WAF)
can help you uncover the origin address,
web crawling tools like DirBuster
help discover hidden content,
and manual enumeration techniques
such as inspecting robots.txt, sitemaps,
and platform plugins provide deeper insights
into the target's structure and potential vulnerabilities.