Agile Testing Days 2013: Tester Roundtable Workshop

At Agile Testing Days in Potsdam, I ran a workshop Tuesday afternoon demonstrating an approach to solving problems.  These may range across a variety of types of problems, but all of them are problems that are bothering people now.  I had run similar exercises before at conferences (most recently at CAST this past August), user groups (like at GR Testers earlier in August) and as an exercise for test organizations in companies.

Considering problems.

The real purpose at conferences is not so much to find solutions as it is to demonstrate the process so the participants can take the exercise back to their day-jobs and try it there.

The format is simple. Each participant describes problems or pain points they are dealing with; they are written down; participants vote on the problem they are most interested in discussing; when there is a single item selected, that is the one discussed.

And we're off...

We begin by describing the selected problem in greater depth, including what is the impact, what possible contributing factors may be involved and other aspects around it.

The list of problems. 

Voting!

The list of pain points and concerns were fairly extensive.  That there were people contributing more than one certainly helped with this.  After a brief summary of the problems, we began voting.  The consensus of the voting landed on the Potential 2^nd/3^rd Order Ignorance.  The explanation of this was that instead of "knowing there are things we don’t know," people don’t know there might be things that are not known.

Describe the problem. In explaining the situation, the participant whose problem it was walked  through a series of steps, including Deliberate Discovery of problems and the Accidental Discovery of the same problems.  (He had been in Dan North's tutorial the day before and this helped him frame the problem nicely.)  The difference seems to be the level of ignorance of stakeholders and the problems to be discovered. 

Discussing the Selected Problem

 The question at the root is, how can the people who will be impacted by problems not found in testing be made to recognize there may be a problem in the software that is, as yet, undetected?  The real issue is how can we help stakeholders understand the risks of not recognizing there may be problems lurking?

Describing the problem is extremely challenging and can be a problem for many people.  Without being able to do this, we will have an even greater challenge to sort out the problems in making software better.  

In describing the problem, several things became clear.  There are a number of references available for consideration that may give people food for thought for addressing this and similar issues. 

Consider, Nicholas Taleb’s Black Swan (this was mentioned in Matt Heusser’s keynote Wednesday afternoon.)  Looking at this from a slightly different angle, particularly on thinking and thought processes, 6 Thinking Hats gives help in considering how people think.  This can help form what we do and how we approach problems. 

Consider, in addition, Michael Bolton’s work around Frames and Test Framing.  Start here or simply engage the wonders of Google and try this broader search here.  Additionally, work on Focus/Defocusing approaches may have value in looking to answer the question “What else is going on?”

Define Possible Solutions.  Several ideas came up in the meeting.  

First Option:  Professional Experience/Authority 

The first possible solution drew a significant portion of the time available.   “In my professional opinion there are things we don’t know that we don’t know.”  

This may or may not have mixed results.  The context of the organization, the situation specific to the cause of concerns, may make this extremely difficult.  The real issue here appears to be “How do we get the attention of the stakeholders?”  This will be problematic.  In some companies, and aggressive approach may work.  In others, such an approach may not and could have significant consequences. 

Another option:  Change Expectations.

By demonstrating there are defects in production and defects/bugs found in each sprint, it may be possible to build a case.  Noting that the lack of finding problems does not mean the problems are there is only the beginning.    The question to consider becomes “Based on what we know, as established fact, what else might be going on?”

This brings us to the final stage of the exercise – This gives us the chance to identify, and more importantly put these ideas on paper.  Somehow, putting things on paper (or some other recorded form) in such instances helps us visualize the components we need to consider, if not address. 

Identify the Constraints (or Roadblocks) / Assumptions / Stakeholders / Tasks.  

Where the same item appears in more than one area, you have identified a key point to address.  HOW you address it will need to vary on the specifics.  For this exercise, we found this:

Constraints/Roadblocks                      Assumptions

===================                        =============

Product Owner                                   People want a better product

“Team is ‘OK with it.”

Tasks                                                  Stakeholders

===================                         =============

Retrospectives                                    Product Owner

-          Invite PO (and her boss)             Legal Team

-          Invite Legal                               Head of Channels

-          Invite anyone else who may

       contribute

-          Get the Tech staff to stays 

       after Demo

Final Suggestion for ideas on considering options with problems like this, Jerry Weinberg’s “Are Your Lights On?”

AND - we were out of time.