Classically speaking, information gathering techniques consist of the following two classes:

In this section we'll try to make use of different kinds of recon technique to do domain enumeration. Finding subdomains of a website can land us in surprising places. I remember a talk by Israeli security researcher, Nir Goldshlager, in which he performed a subdomain enumeration scan on a Google service, out of the bunch of subdomains he found there was one which ran a web application with a publicly disclosed local file inclusion vulnerability. Nir then used this to gain a shell on Google's server. Nir's intention wasn't evil, he reported this vulnerability responsibly to Google's security team.

Let us now learn some information gathering techniques. We'll use both active and passive methods.

The following recon tools will be discussed:

The following websites will be used for passive enumeration:

Fierce is an open source active recon tool to enumerate sub domains of a target website. This tool was written by Robert (RSnake) Hansen and comes pre-installed by default in Kali Linux.

The Fierce Perl script applies techniques such as zone transfer and wordlist brute-forcing to find subdomains of the target domain:

fierce -dns target.com

Let's run Fierce against iitk.ac.in and see how it performs. It is shown in the following screenshot:

Voila, Fierce presented us with a list of subdomains. One thing to note is that Fierce enumerated the name servers of iitk.ac.in, and then tried to do a zone transfer on each. Luckily one of the name servers was misconfigured and Fierce then grabbed a list of DNS entries including the subdomains from the misconfigured server.

We can also use a tool called dig which is available in *nix systems too, to perform a zone transfer without using Fierce. The command to perform a zone transfer using dig goes like this:

dig @<name-server-of-target> &lt...

theHarvester is an open source reconnaissance tool, it can dig out heaps of information, comprising of subdomains, email addresses, employee names, open ports, and so on. theHarvester mainly makes use of passive techniques and sometimes active techniques as well.

Let's run this amazing tool against my homepage:

theharvester –d prakharprasad.com –b google

Look at this! theHarvester found out a list of subdomains and an email address. We may use this email address to perform client side exploitation or phishing, but that's a different topic. The tool only utilized Google as a source of data to reveal this much information.

We can control the sources of data to be used with theHarvester by using the -b switch. The sources of data that theHarvester supports are:

google, googleCSE, bing, bingapi, pgp, linkedin, google-profiles, people123, jigsaw,twitter, googleplus, all

Let us try to run theHarvester on my domain and provide the data source as LinkedIn. Let's see what happens next...

SubBrute is an open source subdomain enumeration tool. It is community maintained and aims to be the fastest and most accurate domain finding tool. It makes use of open DNS resolvers to bypass rate-limiting restrictions.

This doesn't come preinstalled with Kali Linux and must be downloaded from https://github.com/TheRook/subbrute:

./subbrute.py target.com

Let us run SubBrute against PacktPub's website and see what results it yields:

You can see list of subdomains purging out. This tool utilizes open DNS resolvers to partially make this process somewhat passive. We have to use the –r switch to supply our own custom resolver list.

CeWL is a custom wordlist generator made by Robin Hood. It basically spiders the target site to a certain depth and then returns a list of words. This wordlist can later be used as a dictionary to bruteforce web application logins, for example an administrative portal.

CeWL is present in Kali Linux but can be downloaded from https://digi.ninja/projects/cewl.php#download.

./cewl target.com

Let me run this tool on my homepage with a link depth count of 1.

Look at that! It returned us a nice looking wordlist based on the scraped data from my website. CeWL also supports HTTP Basic Authentication and provide options to proxy the traffic. More options can be fiddled with by viewing its help switch --help. Instead of displaying the wordlist output on the console, we can save it to a file by using the -w switch.

You can clearly see the generated wordlist was written to the cewl.txt file. There's also the -v switch to increase the verbosity of the CeWL output, it comes in very handy when the site...

DirBuster is a file/directory brute-forcer. It's written in Java and programmed by the members of the OWASP community. It's a GUI application and comes with Kali Linux. DirBuster supports multithreading and is capable of brute-forcing targets at insane speeds.

DirBuster project: https://www.owasp.org/index.php/Category:OWASP_DirBuster_Project.

The GUI of this tool is straightforward as it provides a ton of options for brute-forcing. It can go up to 100 threads which is amazingly fast, provided that there is proper bandwidth supplied.

It comes with a set of wordlists for different requirements and conditions.

Let us run DirBuster against my website to look around for files/folders:

It found out some directories and files and although there a few false positives, not all results are incorrect. One thing that must be taken care of when using DirBuster is that it generates a lot of traffic which can easily slow down small websites, so the threads must be properly set to avoid taking down...

We can use WhatWeb, which is an active recon tool, to get basic information about a website.

WhatWeb listed cookies, country, and uncommon headers related to my website.

Maltego

Maltego is an Open-Source Intelligence (OSINT) tool developed by Paterva. It's a commercial tool, however the community edition comes by default alongside Kali Linux. We'll be using the community edition for this demonstration.

Maltego can be launched from the Information Gathering section of Kali Linux's Application menu. During the first launch, Maltego will ask you to register for the community edition license or login directly if already registered. This step must be done to access and run Maltego.

After the basic formalities are done, we can run Maltego again and we'll be presented with a dialog asking us the choice of machine to run. Machines are different categories or genres of information gathering we're interested in.

In the dialog, we're presented with different machines or information gathering categories...

Before I begin, I must say Shodan is a one-of-a-kind search engine. In their own words, it is the world's first computer search engine, often dubbed as the search engine for hackers. We can use Shodan to find different types of information about a target.

Let us to do a search on web servers running Microsoft IIS running version 8.0 through Shodan:

Shodan presented us with a page listing entries it has in its database. Shodan provides a very decent and useful way to filter our result by the following criterion:

Recently there was a publicly disclosed code execution flaw inside a Python-based debugger known as Werkzeug Debugger. We can give Shodan a shot and find out the computers running Werkzeug:

There we go! There is the list of computers running the vulnerable debugger.

Now let's find some ZTE OX253P routers. This particular brand of router is used widely by BSNL in India for providing WiMAX services...