Improve defect reporting with these simple steps

January 12, 2019

Over the past few years, my brother John and I have worked with hundreds of clients to implement what is now being called “Insight Engines”. Through this we’ve created a repeatable process for the implementation phase of the technology and through implementing with so many organizations have come across an interesting phenomenon:

Testers articulate defects very poorly

This is often expressed in some manner frustration when they are then asked for qualification and clarification. Typically, the author of the defect ticket will respond with:

You want me to update the ticket?
But I put everything there!
Did you read the ticket?
We just need this fixed.

Which is the defense of some ticket created with a subject like:

Incremental crawling doesn’t work
Relevancy sucks
Documents are missing
Facet counts are wrong

Which has some summary reiterating the subject in a board manner like:

Incremental crawling not turned on, please fix
Search terms aren’t like previous system
Please crawl all of the documents
Please make the counts right

With Agile development and loose specifications, User Acceptance Testing (UAT) tends to be the point of where a functional system is actually tuned to meet the system. The users typically can not express what their expectations are having general focus on matters outside of the engine.

However, we promote the following simple form for the issue creation that if used drastically reduces the time of issue triage and allows the implementation team to begin address the issue.


# Issue Narrative
# Expected behavior
# Actual behavior
# Additional Information
# Steps to reproduce the behavior
 1.

Given this simple form or something derivative like:

I did this
I saw this
I expected this

with the urls or screenshots of the behavior can be passed almost immediately to an engineer to fix.

So many times, we don’t even know what the testers are doing or the test that is being performed.

Try it out and let me know what you think!