Heuristics in Software Testing: Hunt Bugs With Style

Heuristics is the key to turn you into the Turkish Olympic guy.
 

It is something that comes with experience. At a certain point, experts all develop that “sense” in their field of work. Repeated exposure to hundreds of problems helps them recognize patterns that beginners don’t easily see. Heuristics is not always foolproof, but it's good for immediate problem-solving.

Source: r/memes

In this article, we’ll show you the best heuristics you can leverage to find better bugs when doing testing.

What is Heuristics?

Heuristics are mental shortcuts or rules of thumb that people use to make decisions or solve problems quickly and efficiently. They help simplify complex decision-making processes by focusing on a few key aspects rather than analyzing every detail. It is especially common and useful in pattern-heavy fields such as finance or chess.

Benefits of Heuristics in Software Testing

• Efficiency: Heuristics allow testers to focus on the most likely areas of failure, saving time when comprehensive testing isn’t possible.
• Adaptability: Since they are based on experience and intuition, heuristics can be adapted to different projects and situations.
• Focus on Risk: By prioritizing high-risk areas, heuristics help ensure that critical issues are found earlier in the process.
• Experience-Driven: Heuristics leverage the tester’s knowledge and insights, making them particularly useful in exploratory and ad-hoc testing.

Limitations of Using Heuristics

• Not Exhaustive: Heuristics can guide testers but don’t guarantee complete coverage of all potential issues.
• Subjectivity: They rely on experience, which can vary between testers, potentially leading to different outcomes based on the tester’s skill level.
• Over-reliance: Overuse of heuristics without structured test cases can lead to missed bugs, especially in complex systems.

5 Heuristics You Can Use When Testing

1. Error guessing

Error guessing is a testing technique where testers use their experience and intuition to anticipate the types of errors that might occur in the software. Instead of following a formal testing process, they "guess" potential problem areas based on their knowledge of common mistakes developers make.
 

A really interesting YouTube channel with that pure chaotic evil energy is Let’s Game It Out by Josh. Josh pushes games he plays to their absolute limits in ways that developers could never have intended. He really knows how to mess with the system, which is essentially what a tester should be doing on a daily basis.
 

If you’re a tester, you should watch his channel to see how he does error guessing and essentially breaks the game Raft:

In game testing, you usually use these heuristics:

1. Test if characters can pass through walls/objects or not
2. Test characters at the edge of the map
3. Test weapon damage
4. Test character animation glitches
5. Test loading screen
6. Test audio errors

2. Boundary value analysis

Now we’re getting technical. The idea behind boundary value analysis is that errors are most likely to occur at the edges of input ranges rather than in the middle, so testing the boundaries of these ranges is particularly important.  

For example,you are testing a function that validates user age for an online registration form, where the valid age range is between 18 and 60. Here are the equivalence classes:

• Valid Equivalence Class: [18-60]
• Invalid Equivalence Classes:
  • Ages below 18: [-∞ to 17]
  • Ages above 60: [61 to ∞]
  • Non-numeric inputs: ["abc", "#$%", etc.]

Once we have defined the equivalence classes, we can identify the boundary values for testing. Let’s begin with the numeric inputs:

1. Just below the lower boundary: 17
2. At the lower boundary: 18
3. Just above the lower boundary: 19
4. Just below the upper boundary: 59
5. At the upper boundary: 60
6. Just above the upper boundary: 61

3. HICCUPPS

The HICCUPPS is a mnemonic device used in exploratory testing. It is based on the idea that expectations about a product are based on several forms of desirable consistency, and deviations from these expectations may signal an issue. When testing a specific feature, think of HICCUPPS:

1. History: The system should behave similarly to past versions; deviations without explanation may indicate a problem.
2. Image: The system should align with the company’s brand or reputation; a complex interface may conflict with a simplicity-oriented brand.
3. Comparable Products: The system should be consistent with similar systems or competitors; underperformance compared to peers could raise concerns.
4. Claims: The system should behave according to advertised features or specifications; failing to meet them may be seen as a problem.
5. Users' Desires: The system should meet reasonable user expectations based on similar products; failure to do so could cause frustration.
6. Product: The internal elements should be consistent with each other; different design principles within the system may indicate issues.
7. Purpose: The system should support both explicit and implicit intended uses; failing to do so suggests a misalignment with its purpose.
8. Statutes: The system must comply with laws and regulations; failure to meet legal standards (e.g., accessibility) creates significant problems.

The HICCUPPS framework was originally developed by James Bach, the advocate of Rapid Software Testing (RST), one principle of which is to leverage heuristics to find practical (but not always foolproof) solutions.

4. SFDPOT (San Francisco Depot)

The SFDPOT is another useful mnemonic to help testers systematically think about different dimensions of a system that might require testing. When planning your tests or performing exploratory testing, use SFDPOT to systematically think about each aspect of the system:

1. Structure: What are the physical components, and how are they organized?
2. Function: What features or functions does the system provide, and how should they behave?
3. Data: How is data handled—entered, processed, stored, and retrieved?
4. Platform: On what platforms (OS, browser, device) should the system work, and how does it behave across them?
5. Operations: How does the system behave in real-world environments, including performance, monitoring, and error recovery?
6. Time: How does the system respond in terms of performance and over extended use or under specific time constraints?

5. Never - Always

The Never-Always heuristics in software testing help testers evaluate boundaries and potential defects by focusing on extreme cases where something should either never or always occur.

• Never: Test cases where a certain behavior, outcome, or condition should never happen. If it does, it indicates a defect.
• Always: Test cases where a certain behavior, outcome, or condition should always happen under specific circumstances. Failure to do so may indicate a problem.

Conclusions

Knowing how to leverage heuristics can really help you do better software testing. Once you have known which areas to test, you can start automating them.