So, lately I've been thinking a lot about how to carry out Exploratory Testing and some learning have sparked previous posts regarding why we should approach software in this way.
I mentioned in my last post I was attending a course in the busy city of London. So here goes, my attempt at simplifying my learning's
Exploratory Testing is an approach to testing. It is concisely described as a simultaneous learning, test design and test execution, let's not mix this up with a test type or technique:
There are a number of myths that surround Exploratory testing:
- Same as ad-hoc testing
- Can't be measured
- Can't reproduce defects
- Only for Agile teams
- Not Documented
- We need test-cases
As I said, these are just myths and make no sense at all!
Creating a test charter is a great way to document what it is you are testing, who is doing the testing, defining your tasks for your session, keeping test notes, recording bugs and logging issues. There are plenty of examples online so get googling!
There are some key elements that we should consider:
- Time - Limit your sessions to 60 minutes, this reduces the risk of losing concentration or being distracted
- Freedom gives more Responsibility - the more freedom you are given the more responsibility you have as you are responsible for interpreting back to the team
- Collaboration - This is key in the planning stages in order for you to understand the product under test
Let's all agree that structure is important. If you wonder around a big city you have never visited before aimlessly you are likely to miss all the big attractions unless you have done some planning.
Try a Scenario bash with your team, a great exercise when there is limited knowledge on the team. When you collaborate you will find that you can already come up with areas of test and/or areas of question to take back to the business owners... It really does work!
You have to LEARN what the project context is whether this is past experience, reading documentations or collaborating with the team. You need to DESIGN your tests, mapping out test ideas, considering potential tools you may use, defining your objective and agreeing what the metrics are to deliver. You will EXECUTE checking against acceptance criteria, exploring for important bugs and configuring the system under test. The last and probably the most important element is INTERPRET What have we learnt? How do we know we have found a bug?
We can use oracles: a heuristic principle or mechanism by which you recognise a potential problem.
How does this fit in Agile:
There is no reason this can't and shouldn't be a part of your agile process.
If you work in a scripted environment (Waterfall) we can use test scenarios as information objectives, using a scripted test in this way allows you to deviate from a step by step process and is more likely to uncover bugs! Why not give this ago?
More precisely, heuristics are strategies using readily accessible, though loosely applicable, information to control problem solving in human beings and machines.
There is a cheat sheet online: Heuristics Cheat Sheet some great hints on the sorts of things we as testers should be thinking about.
Here are a good number of examples from all the great tester figures out there to aid in your own Exploratory Testing; can you come up with your own?
Test Strategy Heuristics by James Bach
Structure, Function, Data, Integrations, Platform, Operations, Time
Test Techniques Heuristics by James Bach
Domain, User, Function, Flow, Stress, Scenario, Claims, Risk, Automatic
FCC CUTS VIDS
Touring Heuristics by Michael D Kelly
Feature Tour, Complexity Tour, Claims Tour, Configuration Tour, User Tour, Testability Tour, Scenario Tour, Variability Tour, Interoperability Tour, Data Tour, Structure Tour
API Testing Mnemonic by Ash Winter
Integration, Consumers, Endpoints, Operations, Volume, Error Handling, RESTful, Modularity, Authentication, Definitions
Microtest Mnemonic by Industrial Logic
Small, Precise, Isolated, Fast, Frequently Run
There are many things we can do to aid exploratory testing, many tools that we can use to record sessions, aid in execution and report bugs.
Visit Andy Glover's Blog Its brilliant - Cartoon tester
Start exploring but with structure. It's important to understand the software under tests to get the best out of YOU!