Automated testing has become an indispensable part of the software testing process. By running pre-defined test cases automatically, automated testing provides fast and reliable regression testing and frees up testers to focus on more complex types of testing. However, automated testing has its limitations. This is where exploratory testing can act as a useful complement to automated testing.
What is Exploratory Testing?
Exploratory testing is an approach to testing where testers dynamically design and execute tests based on their knowledge, intuition and feedback from the system under test. The key characteristics of exploratory testing are:
Unlike automated testing where test cases are pre-defined, exploratory testing is flexible and reactive. Testers explore the system in an unscripted way, following leads and testing hypotheses. They might execute a test case once or several times, modifying it based on results.
- It is not scripted - tests are dynamically designed and executed
- It relies on the tester's skills, knowledge and intuition
- The tester explores the system as a user would, to find defects
- The tester learns about the system as they test and uses this information to design new tests
Unlike automated testing where test cases are pre-defined, exploratory testing is flexible and reactive. Testers explore the system in an unscripted way, following leads and testing hypotheses. They might execute a test case once or several times, modifying it based on results.
Key Benefits of Exploratory Testing
Some of the key benefits of exploratory testing are:
- Finding Defects Automated Testing Misses: Automated testing is only as good as the test cases authored. Due to its unscripted nature, exploratory testing can often uncover defects that automated tests miss. For example, an automated test would likely not be able to spot a usability issue or an anomaly in how the system functions. A human tester dynamically interacting with the system is better equipped to find such issues.
- Testing New or Changing Functionality: Automated test cases need to be re-written every time the application changes. Exploratory testing is well-suited for testing new or changing functionality as testers can dynamically design tests based on their understanding of the changes. No test case maintenance is needed.
- Providing Early Feedback: Exploratory testing can provide early feedback on the system, even before automated test cases can be written. Testers can start testing via exploration from the very first builds of the system.
- Testing Complex Systems: Systems that involve complex visual interfaces or workflows can be challenging to test thoroughly via automation alone. Exploratory testing can effectively complement automated testing in these cases.
- Assessing Quality: Well-designed exploratory testing sessions provide an indication of software quality and user satisfaction. Automated tests cannot assess subtler aspects like workflow and usability.
Key Challenges of Exploratory Testing
While powerful, exploratory testing also comes with some challenges:
How to Combine Automated and Exploratory Testing - Automated and exploratory testing are complementary techniques - combining them can lead to a comprehensive and efficient testing strategy.
Automation for Regression, Exploratory for New Features - Test new features, flows and usability via exploratory testing. Once stable, automate regression tests for these areas.
Documenting Exploratory Sessions - Have testers document their exploratory charters and test results. This improves repeatability.
Time-boxing Exploratory Testing - Schedule exploratory testing for new builds via timed sessions based on priority and risk.
Using Automation to Support Exploratory Testing - Tools like auto-highlighting can assist testers in exploratory sessions by drawing attention to changes.
Exploratory Testing to Validate Automation - Use exploratory methods to ensure automation provides sufficient coverage for critical areas.
Exploratory testing provides a valuable human perspective through its ability to probe, question and critically think. Automation provides unmatched test consistency at speed. Employing both automated and exploratory testing as complementary approaches can lead to elevated software quality and user satisfaction. With some thoughtfulness in blending these techniques, teams can unlock the benefits of both worlds.
- Results can vary based on the skill of individual testers
- Can be challenging for distributed teams across different geographies
- Less repeatable than automated scripted tests
- Test coverage can be inconsistent or unclear
How to Combine Automated and Exploratory Testing - Automated and exploratory testing are complementary techniques - combining them can lead to a comprehensive and efficient testing strategy.
Automation for Regression, Exploratory for New Features - Test new features, flows and usability via exploratory testing. Once stable, automate regression tests for these areas.
Documenting Exploratory Sessions - Have testers document their exploratory charters and test results. This improves repeatability.
Time-boxing Exploratory Testing - Schedule exploratory testing for new builds via timed sessions based on priority and risk.
Using Automation to Support Exploratory Testing - Tools like auto-highlighting can assist testers in exploratory sessions by drawing attention to changes.
Exploratory Testing to Validate Automation - Use exploratory methods to ensure automation provides sufficient coverage for critical areas.
Exploratory testing provides a valuable human perspective through its ability to probe, question and critically think. Automation provides unmatched test consistency at speed. Employing both automated and exploratory testing as complementary approaches can lead to elevated software quality and user satisfaction. With some thoughtfulness in blending these techniques, teams can unlock the benefits of both worlds.