The 2016 CodeCrafConf took place in Glasgow today, and I ran a guided conversation called “Quality Time” – a session on quality and testing where the attendees contribute to the discussion, as opposed to one person speaking.
I wanted to do a very quick round-up of some of the things discussed, as the way the conversation and questions developed was really interesting, and not necessarily what I’d expected. I wanted to do this sooner rather than later, as my memory isn’t that great(!), so apologies if I’ve misremembered anything, or missed something out entirely.
For context, the conference was aimed at developers, but other attendees such as myself may be in other roles or have other responsibilities. The session lasted one hour in total, with the aim of spending five to eight minutes on each question.
What elements contribute to quality in software?
Most suggestions started off from a more technical / code perspective, with participants talking about code characteristics and code reviews. There were other suggestions around checking, automated tests and user testing. Towards the end, user experience was also touched upon.
What is the benchmark for quality?
Participants talked mostly about metrics relevant to the software / feature in question. For example, traffic, time spent on the site, profits, number of bugs, speed of fixing bugs. “Fluffier” indicators were also offered, such as frustration (from developers and users) and overall “happiness”.
How can quality code help create quality software?
Naturally, more technical suggestions in terms of simplicity and ease of maintenance. This led to discussion of faster devilvery and how that could be perceived as better quality to the end user.
What is testing?
One point was that testing isn’t just about the code or developed software. Participants seemed to agree that the requirements and potential solution are also subject to testing, questioning and scrutiny, before any actual coding begins. We also talked about types of testing (unit tests, automated tests, exploratory) and “throwing over the wall”.
What would be the impact to you if testing activities stopped?
I’m not sure that the group were expecting this question. After getting them to think about what it means to test, they were pushed to think about the value of testing. It was interesting to hear some admit that passing their work over to be tested was their “safety net”. It was also really nice for me, personally, to hear participants agree that testing is a valued skill and required as part of software development. The group also talked about who tests; developers, QA, test engineers.
When should we test?
To me, the answers seemed to be linked to feedback loops. Suggestions were that we should test early, coming back to the idea of testing the initial requirements and getting as much information as possible up front. Some pointed out that starting to test only after a chunk of development work has been done might be “too late”, or make it more difficult for developers to take what is perceived to be negative feedback about “their baby”. One participant asked, “Would it be too controversial to say never?” I understood their idea to be that testing shouldn’t be seen as a separate activity to development, and that developers / engineers should always be checking their work, and each others’ work, as they go along. This led to further discussions about who tests, with some very much seeing the value of an “outside eye” that has more of an overview of the full product and interactions between different features, while others said that, in their current teams, projects and way of doing things, they wouldn’t see any value to an outside or dedicated tester. I would have loved to talk about this more, but our time was coming to an end.
Testing and quality; what are people not talking about enough?
One last question in the final minutes. People aren’t talking enough about mutation testing. An Uncle Bob article on the subject was referenced; I believe it was this one.
Other questions
I’d prepared some other questions for the session that weren’t explicitly asked for one reason or another:
- Agile and DevOps have increased automation in testing. How has this impacted us? (Ran out of time)
- What are the disadvantages of developer / tester silos? (Ran out of time / touched upon during other discussions)
- How has the line between developers and testers blurred? (Discussed naturally with other questions)
- What are the best ways to work towards quality together? (Ran out of time)
I felt that the session was really worthwhile, and very interesting. Thanks again to everyone who took part and, of course, CodeCraft for organising and inviting me to guide the session.
Please do let me know what your answers would be to some of these questions, and feel free to ask them within your organisation. You might be surprised too!
Hi Cassandra, first of all congratulations on a well written and explained blog post. It is great that this kind of discussion about quality and testing is happening, and that you have shared your findings.
For me, any discussion about, or definition of, quality should be framed with people in mind; the people who will use our products (colleagues and customers primarily) but also the people working on it. If we aren’t adding value, and a good experience to the first group, any other metric or measure is not really going to help. The feelings and reactions of the second group tell us a great deal about whether we might be successful in our goals.
It’s not surprising that among technical people, the answers you heard had a technical focus, but it would be great to hear and read more about quality from a human perspective. It sounds as though the experience for the person using the software was something of an afterthought, only considered after factors relating to code. Not at all unusual but something that I would happily see reversed.
Rich
Hi Rich,
Thanks so much for your comments.
Yes, there were a few mentions of the user and value of the product, but I share your feelings that it could have been more of a consideration.
Thinking about it now, I suppose that might be another way developers rely on testing colleagues, to think about those things more and come up with ideas and scenarios when we might not be adding value or providing a good experience. Particularly in the cases of those who feel we don’t need to test outside of development, I wonder at which point those things might be looked at, and to what degree.
It was great to get a glimpse of the participants’ thoughts and mindset!
Thanks,
Cassandra