What is Penetration Testing? Definition, Guide, Best Practices

Cyberattacks are terrifying because of their potential to wreak havoc on a massive scale. The interconnectedness that the internet provides can totally be exploited. Quality assurance teams around the world have to be prepared against such disastrous scenarios, so they sometimes launch authorized cyberattacks on their own systems to check for vulnerabilities. This process is known as penetration testing, or pen testing for short.
 

In this article, we will explore the concept of penetration testing, pen testing types, the steps to launch one, and popular pen testing tools you can use.

What is Penetration Testing?

Penetration testing, often referred to as "pen testing," is a cybersecurity testing practice where a trained professional, known as a penetration tester or ethical hacker, performs a simulated cyberattack on the organization’s system to assess its security.
 

Penetration testing is essentially an authorized cyberattack. These penetration testing sessions are usually supervised and carefully documented. The objective of pen testing is to help organizations understand their security posture and take proactive measures to mitigate potential risks.

Real-World Example of Penetration Testing

Let's use an e-commerce website as an example. Established e-commerce websites have to process and store thousands of sensitive payment information. If the database is attacked, the consequences can be disastrous. Penetration testing allows us to find those vulnerabilities and address them in a timely manner.
 

An external penetration testing specialist will perform some exploratory testing to discover security issues in the system, then launch an attack to see if they can achieve complete compromise of the specific system component they target.
 

For example, after some exploration, they find that there is an unattended staging environment for the payment process, and they can leverage that hole to access the payment API and initiate transactions on behalf of actual customers. They then launch an attack and achieve system compromise.
 

Of course, the penetration testing specialist must make sure that no harm was made to the organization as well as its customers. Their objective is simply to demonstrate that their system can be hacked into if that specific system area is not properly guarded.
 

A successful penetration reveals a lot of insights about the system, especially the areas to optimize for better security. 

Penetration Testing vs. Vulnerability Scanning

Vulnerability scanning is a fairly similar concept to penetration testing in the sense that they both try to find security issues in the system. However, as their names suggests, penetration testing takes the extra step to penetrate the system, while vulnerability scanning simply scans for issues in the system, yet does not exploit it.
 

Here is a comparison table to help you understand the differences between penetration testing versus vulnerability scanning:
 

Guide To Do Penetration Testing

1. Legal preparation

To launch a penetration test is to launch an authorized attack on a system, which is why it must be carefully planned and executed with great legal consideration.
 

Read more: What Is a Test Plan? Test Plan vs. Test Strategy
 

Obtain explicit, written consent from the organization or individual who is responsible for the target system. This consent is usually expressed in the form of a formal agreement or penetration testing contract. You can have a look at this template for Rules of Engagement for Penetration Testing from Microsoft.
 

You can see that the penetration test scope is carefully outlined in the template. For more detailed guidance on how to write test cases to be executed, you can check out Katalon’s test case template.
 

Other legal considerations include:

1. What are the legal regulations (including industry-specific and region-specific regulations) relevant to this penetration testing project?
2. Do we need to obtain any permits for the project to happen?
3. Should we involve a legal counsel to review contracts?
4. Have we established a “No Harm” policy where we explicitly state that the penetration testing should not cause any disruption or harm to the system under test?
5. Have we established a system to protect our data in case the penetration tests result in accidental data breach?
6. Is there enough documentation for all testing activities, tools used, and test results? 

Read more: How To Write a Test Strategy Document

2. Reconnaissance and information gathering

Reconnaissance is a military term that refers to the scouting activities to obtain intelligence. In the testing industry, we can think of it as part of an exploratory testing session, where the penetration testing specialist actively interacts and explores the system to find out the areas to be tested. They approach the system with an open mind, and with their experience and domain knowledge, they attempt to gain as much details about the target as possible.
 

There are two main ways to do this, including:
 

• Passive reconnaissance: gathering publicly available information about the target system, such as IP address, domain name, email addresses, employee names, and more. 
• Active reconnaissance: using more advanced methods to discover information about the target, such as DNS enumeration, Whois queries, or network scanning.

3. Vulnerability scanning

After the initial exploration, the penetration testing specialist now takes a deep dive into security issues. They leverage specialized tools like Nessus, OpenVAS, or Qualys to scan the target system for known vulnerabilities. After that, they can try to assess open ports to identify potential weaknesses.
 

Any vulnerabilities found will be documented, categorized, and prioritized based on severity.
 

Read more: What is Bug Life Cycle in Software Testing? Guide To Categorize Bugs

4. Exploitation

This is when the penetration test truly begins. The specialist attempts to exploit the vulnerabilities they found during the first and second step to gain unauthorized access and even control the system through a wide range of techniques.
 

Common techniques include social engineering, buffer overflow exploits, SQL injections, etc. There are many other techniques for specific vulnerabilities too, such as SSTI (server-side template injection), in which the attacker injects malicious code into server-side templates to gain control of the server. This technique is common on web application frameworks.
 

The end goal of this attack is not to retrieve sensitive data, but rather to demonstrate to the organization owning the system-under-attack that there exists vulnerabilities in their security, and the impact of those vulnerabilities are real if a real attack happens on a large enough scale.

5. Post-exploitation

Once the hacker has gained access to the system, they must first ensure continued access there if they want to retrieve anything of value. To do this, they need to establish backdoors, then escalate their privilege (i.e., access level) in the system, which should grant them the capability to expand the scope of the system compromise.
 

Once they reach the level of access they want, the hacker can start to harvest sensitive information from the system. To make sure no traces of the compromise are found, they can manipulate logs to erase all of the recordings about the attacks.
 

Of course, the scope of the attack must always stay within the penetration testing scope as outlined in the test plan for legal and ethical reasons.

6. Reporting

After the attack, it is time for the specialist to document all of their findings, with details on vulnerabilities found, the steps taken to compromise the system, and the impact of successful attacks. This report is then sent to the organization, which can then host a meeting with the specialist to discuss the attack, analyze the vulnerabilities, and align on the action items to be taken to improve security.

Popular Types of Penetration Testing

1. Social engineering

Social engineering is the act of psychologically manipulating people into disclosing confidential information. Why hack into a system when you can simply ask for access? At its core, social engineering exploits the cognitive biases that all humans have to get the victim to take actions for the hacker’s best interest. The system being hacked here is our mind. 
 

Below is a fairly good example of social engineering. This is an image taken from a Reddit post in the r/socialengineering subreddit. This image was from a Facebook group that garnered more than 120 comments.

Essentially, it asks you to comment under that post the combination of your grandparent’s name, first pet’s name, and street name, which together form your royal guest name. Many people fell for it, thinking it was only a harmless game, yet your grandparent’s name, first pet’s name, and street name provide answers to common password recovery questions. The hacker can now simply use those comments to create new passwords for your accounts and take all of your confidential information.
 

Currently, password security questions have gotten more and more complicated, and people also have greater security literacy.
 

There are several types of social engineering:
 

• Phishing: The most common technique (identified in 41% of incidents), phishing involves sending deceptive emails that look highly similar to the original email of a trusted source. The victim can be tricked into clicking on certain buttons/links that take them to a fake website where the hacker can easily gain access to their device and retrieve confidential information. Or worse, the victim can be tricked into sending confidential information themselves.
• Vishing: Vishing is essentially phishing that happens over the phone instead of through emails or the internet. With recent innovations in AI Deepfake, vishing has become increasingly more popular.
• Pretexting: Pretexting is when the attacker assumes the identity of a person in authority to extract information from the victim.
• Tailgating/Piggybacking: This is a physical security breach in which an unauthorized person follows an authorized person into a secure area. For example, restricted areas in some buildings have retina scanners to allow access for authorized personnel only. However, the attacker can follow that personnel and sneak into the restricted area without them knowing.
• Scareware: These are fake alerts that scare the victim (such as claiming that their computer has been infected by a virus, and that they should purchase their antivirus software). 

2. Buffer overflow exploits

You can think of a buffer as a temporary checkpoint when transferring data from one place to another. Note that data has to go from one source component and to a destination component, and these components do not always operate at the same speed or logic. These buffers allow the incoming data to be processed so that it can be stored in the destination component without causing any conflicts.
 

However, attackers can trigger a buffer overflow issue by feeding more data than the buffer can handle. If the input data exceeds the buffer size, it won’t be able to properly validate all of the input, and must overwrite other areas of the code to store such inputs. The attacker has control over what area they want to overwrite.
 

Usually the return address is the target. It keeps track of where the program should continue executing after the current function call is completed. The attacker can simply change the return address to a new value leading to a malicious location where they can take control of the victim’s application or extract sensitive data.

3. SQL injection

This is a fairly similar method to buffer overflow in the sense that it is also about injecting malicious code into the database. Here the attacker injects malicious SQL into a query to exploit poorly sanitized user inputs (i.e., inputs that do not go through proper validation to check if they are written in the correct format).
 

SQL usually happens on search queries, form data, or parameters in URL. For example, when filling in a form, instead of providing the typical username, they can enter:
 

' OR '1'='1
 

This results in the SQL query:
 

SELECT * FROM users WHERE username = ' ' OR '1'='1';
 

Since 1 = 1 is always true, the WHERE clause becomes always true, and all rows in the user’s table are returned from the database, granting the attacker a lot of sensitive data that they are not supposed to access.

4. Cross-site scripting (XSS)

Cross-site scripting is when the attacker injects malicious script (usually in the form of browser side script) into trusted webpages. When a user accesses that page, the script is executed because the browser thinks that the script is from a trusted site. The malicious script can access any cookies, session token, or sensitive information retained by the browser used for that site.

5. MITM attacks

Man-in-the-middle attacks are those where the attacker secretly places themselves into communications between two parties without their knowledge and gains information. It is similar to an eavesdropper or a spy listening in on a private conversation, but it happens on the internet.
 

For example, attackers intercept network traffic between a user and a server, capturing sensitive data like login credentials or payment information.
 

Top Pen Testing Tools

1. Nmap

Nmap (short for Network Mapper) is a network scanning tool used to find hosts and services on a computer network by sending packets and analyzing the responses. Nmap offers features like host discovery, service detection, and operating system detection.
 

You can extend its functionality with scripts that provide advanced service detection, vulnerability detection, and more. Nmap can adjust to network conditions, like latency and congestion, during a scan. Initially, it was a Linux utility but has been ported to other systems like Windows, macOS, and BSD. It's particularly popular on Linux, followed by Windows.

2. Metasploit

The Metasploit Project is a computer security initiative that focuses on security vulnerabilities, aids in penetration testing, and supports IDS signature development.
 

The most well-known part of the project is the open-source Metasploit Framework, a tool for creating and executing exploit code against a remote target machine. Other important aspects of the project include the Opcode Database, shellcode archive, and related research. The Metasploit Project includes anti-forensic and evasion tools, some of which are integrated into the Metasploit Framework. It comes pre-installed in the Kali Linux operating system.

3. Wireshark

Wireshark is a free and open-source packet analyzer used for network troubleshooting, analysis, software and communications protocol development, and education. It is cross-platform, with a user interface implemented using the Qt widget toolkit in current versions. It uses PCAP to capture packets and is compatible with Linux, macOS, BSD, Solaris, some other Unix-like operating systems, and Microsoft Windows. There's also a non-GUI version called TShark. Wireshark, along with its associated programs like TShark, is free software released under the terms of the GNU General Public License version 2 or any later version.

4. Burp Suite

Burp Suite is a software security application used for the penetration testing of web applications. It comes in both free and paid versions and is developed by PortSwigger. The suite includes tools like a proxy server (Burp Proxy), an indexing robot (Burp Spider), an intrusion tool (Burp Intruder), a vulnerability scanner (Burp Scanner), and an HTTP repeater (Burp Repeater).