My thoughts on Test Plans…

I’m busy working through Google’s Software Testing book, How Google Tests Software, and got inspired when I read about the 10 minute test plan in the book. I had read this blog post a few years ago but only recently has it made an impression on me. I’m all for the main gist of the blog post to not create waste and to focus your planning efforts. If you do a google search for test plan templates and peruse some of them, some of these promote waste creation that frankly will never read by most of your team members.

Before I get to the main point of my blog post, lets unpack what a test plan is:

A test plan is a document detailing a systematic approach to testing a system such as a machine or software

Source: https://en.wikipedia.org/wiki/Test_plan

sum or intersection of strategy and logistics

Source: http://www.developsense.com/blog/2008/12/what-should-test-plan-contain/

A test plan should serve as a living document of your planning efforts on how you are going to test. This document should be the blueprint that dictates how testing will be performed for the application/project.

A test plan shouldn’t be about filling out a template because if that’s what your definition of a test plan then you are being inefficient with your use of time. A test plan should be specific, contextual and discuss the following:

• Why

Why are we testing on this project?  Do we understand why the customer wants this enhancement or new application?

• What

What problem are we trying to solve? What areas of the application are being affected?

• How

How are we going to test the application e.g. heuristics, oracles, scenarios?

• When

When are we planning to test e.g. testing estimates?

• Where

Where will these changes be tested e.g. environment?

At the end of the day, you should not need to document more information than the aforementioned points because then you are just taking precious time away from the most important responsibility of a tester i.e. to test!

If you would like to do some more reading on how to create more context specific test plans, I’d highly recommend reading through the following resources:

• Ministry of Testing – Test Planning Mindmap
• What Should a Test Plan Contain?
• Writing Good Test Plans and Writing Good Tests

Good luck with your future planning efforts!

Position Heuristic

This post I want to focus on the position heuristic, which I have found useful when it comes to testing. To clarify what a heuristic is:

A heuristic is a fallible method for solving a problem or making a decision

Source: http://www.developsense.com/blog/2012/04/heuristics-for-understanding-heuristics/

Heuristics are strategies using readily accessible, though loosely applicable, information to control problem solving in human beings and machines

Source: https://en.wikipedia.org/wiki/Heuristic

I found out about the position heuristic via Elisabeth Hendrickson Test Heuristics Cheat Sheet. The position heuristic focuses on testing data (or whatever you need to apply it to) by focusing on:

• Beginning
• Middle
• End

How to apply this heuristic to your testing? For example, you have an application that generates a report of employee information :

• First name
• Last name
• ID
• Cellphone

When you generate this report (hypothetically), a 1000  employee records are displayed. You can manually verify and validate each of the 1000 records and whether the information displayed are correct but that would take too long. If you apply the position heuristic (with a slight modification), you can verify and validate the report as follows (this is assuming you have written a SQL query that produces the report data for your own verification/validation purposes):

• Compare the top 20 records in the report to the top 20 records in the SQL query
• Compare the middle 20 records in the report to the middle 20 records in the SQL query
• Compare the bottom 20 records in the report to the bottom 20 records in the SQL query
• Randomly compare records between the top and middle data sets
• Randomly compare records between the middle and bottom data sets

Applying this heuristic, you will achieve good test coverage. Also, as a final test you can check whether the SQL query total (i.e. 1000 records) matches the total output from the report.

There are probably other ways to test the report with the use of tools (e.g. scraping the data from the report and the database and using excel or win merge to find any differences between the datasets) however I have found this heuristic very helpful when it comes to this testing reports.

Lessons learnt in mentoring testers…

I have been mentoring testers now for 5 years (still a novice in this domain) at my company. Predominately I focus on graduates however I do get the occasional experienced tester to train and mentor. I have 4 short lessons on detailing my experiences while mentoring testers.

• Have mentoring programme or structure in place

I have found that it’s important to have a training programme in place, but this programme should be flexible enough so that you can adapt it to the relevant tester. This programme should try to serve as a baseline of guidance for their role as a tester.

• Patience and support

As the saying goes, patience is a virtue and this couldn’t be truer when you are a mentor to someone. Mentoring requires patience! There are going to be times, when you are frustrated with them but try to not let this show. Rather work with them in building their confidence and trust and this will help them in the long run.

• Critical thinking

Very important point to drive home when working with the testers. “Think outside the box”, see things from different perspectives and always ask question. Introduce them to blogs, books and other resources that will help expand their knowledge on testing and quality assurance. Doing this from the beginning will greatly assist in building a smart tester!

• Work closely with them on their projects

I like to work closely with the tester on their project. This allows me to better understand how they test. I am then able to probe and ask questions about their thought process and approach to testing on the specific project. These answers that the tester provides, gives me insights into where there are gaps in the training and what to focus on next, which will assist them.

Mentoring is a constant challenge but a very rewarding challenge at the end of the day.

Tale of two UX experiences…

Over the past month, I have had two different user experiences with real world applications.

User Experience 1

When the demise of Google Reader was widely reported, I was severely disappointed because Google reader worked great. I started looking at alternatives and came across feedly. The UI looked very appealing and the migration process seemed simple. I migrated over to feedly and have been pleasantly surprised at how well feedly works and as an added benefit all my RSS feeds were migrated over.

User Experience 2

This week saw the end of MSN Messenger, with Skype replacing it. I knew this day would come that I would have to migrate to Skype (I had been ignoring the ads that kept popping up when I logged into MSN Messenger about migrating to Skype). Nonetheless, this Thursday the day had come where MSN Messenger was officially taken down. I proceeded to “upgrade” to Skype and followed instructions however when I finally managed to login to Skype, none of my MSN Messenger contacts had migrated over! I tried again and then again but with no success and no MSN Messenger contacts!

Frustrated I looked for another alternative and came across digsby. I signed up for an account, entered the relevant details and voila my MSN Messenger contacts were there.

Bugs can frustrate users especially when there are no viable workarounds or solutions. My poor user experience with the migration (or “upgrade”) process from MSN Messenger to Skype left me frustrated and in result moving over to a competing product.

User experiences are becoming more critical to building key customer relationships, but often ignored. As software testers, its important to take into account usability or user experience testing because if we don’t, we run the risk of losing customers to the competitors.

Basics…

“HOWZAT!!!!”

That sinking feeling that a batsmen feels when they know they have been cleaned bowl. The batsmen walks back to the pavilion and his inner voice is shouting at him, “Why did you play that shot. Why? Why?”. The batsmen in anger and frustration  knocks the bat against his helmet and everyone can see that his is angry with himself.

“Hard Luck”, says the captain to the batsmen, who has just entered the pavilion where his teammates are located. The coach, responds: “Hard Luck? Are you serious?” If he stuck to the basics and played a forward defensive stroke instead of that wild slog attempt, then he wouldn’t be here now.” The batsmen is taken aback by how the coach has responded but knows that the coach is right.

Basics Basics Basics!

What does this have to do with testing? A lot! Firstly, I’m not going to unpack the critical decisions that the batsmen took and why he went with the irrational choice of playing a slog instead of forward defensive stroke. The correlation I’m trying to make is when we are testing and you are under extreme pressures for variety of reasons, then do you abandon the basics of your testing process/strategy? I would say no and during times of heavy pressure you need to rely on the basics even more i.e. your training, your experience and your process (context specific).

I read an excellent article by Johanna Rothman on Management Myth 16: I Know How Long the Work Should Take and something that I took out of it, is that sometimes people cut corners to deliver a project quicker but while doing that they sacrifice their practice of professional software development. This couldn’t be more truer!

In conclusion – even when things seem bad, its important to rely on the basics because that is what is going to get you over the line.

Testing is…

Not about:

• Following some detailed test script in a test management solution and diligently, checking off the various steps for a test case, that was assigned to you by your test manager.
• Executing 20 test cases per day that has been assigned to you.
• Submitting a test report stating that you executed 100 test cases of which 95 passed, 5 failed.
• Developers throwing code over the wall and you not communicating and collaborating with them.
• etc.

The list is endless and it’s frustrating that people still nowadays think that the aforementioned points are what testing is about.

Testing is the process of evaluating a product by learning about it through experimentation, which includes to some degree: questioning, study, modelling, observation and inference.

Source: James Bach, Michael Bolton (http://www.satisfice.com/blog/archives/856)

This definition of what is testing is about (above) fits in perfectly with the way I view testing. Personally, testing is all about learning and I like that. I believe in that.

If you want to invest in your testing career, spend some time reading the following resources – your life will be better:

• Test and Checking Refined – James Bach and Michael Bolton.
• What testing can learn from social science (Part 1, Part 2, Part 3, Part 4, Part 5) – Huib Schoots.
• Creative and Critical Thinking and Testing (Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7) – John Stevenson.

Help your fellow testers and spread the knowledge of what testing is really about. Help your test managers see the light! And finally:

Learning lies at the heart of testing. Viva the learning tester.

Source: Iain McCowatt (http://exploringuncertainty.com/blog/archives/820)

How to Break Software – book review

I recently completed reading How To Break Software: A Practical Guide to Testing. It was a very interesting read even though it’s now 11-12 years old. How did I come across the book? James did a recent podcast and mentioned his book and because I’m trying to beef up software testing book collection, I decided to purchase it.

The gist of the book can be nicely summed up by the summary on the back cover:

How to Break Software takes a very applied and non-rigid approach to showing how to test software for common bugs

The book was easy to digest and is divided into different sections of ideas, approaches and techniques for testing (or testing attacks as James uses in the book). The book comprises of the following main ideas:

• Testing from the User Interface
• Inputs and Outputs
• Data and Computation
• Testing from the File System Interface
• Testing from the Software/OS Interface

Each section/chapter discusses the main idea of the software testing technique and provides different attacks that can be performed to reveal a common bug. These attacks can vary from forcing a function to call itself recursively to forcing the screen to refresh itself.

Even though the book is now 11-12 years old, I still managed to take something from the book. It provided me with another way of thinking when I approach software testing as well as a few software testing ideas to try out on my next project.

Some of the ideas in the book wasn’t really applicable to me, however its good to get different perspectives of how other testers approach software testing for different contexts.

Overall, a great reference of different ideas and there could be something for everyone in this book.