bookmark_borderRegression – Lets stop it!

I hate it.

You change something and you can’t tell if your change broke something in the system.

If you’re lucky, you did not break anything. Or nobody noticed it.
Next to that, on the same scale of luck, the potential bugs are found in the manual testing phase.

But often there is no time to do all the regression testing by hand. It will take days and days, and the change you made looked so insignificant. It should go live. What could possibly go wrong?.

Then it happens. You’re live, your changes work, but the inevitable happens. Regression!

Of course, this has to be fixed. We can’t let our customers have a product with new features while the features of the previous version(s) are broken.

And so the patching process begins.

I call it patching, because often you are not done with one patch. While you were working hard to get the first patch live, other regression bugs are found and need to be patched asap as well! And so you end up with a few patches. You could be done with with a few patch releases. But it could easily extend ten-fold.

This process is very stressful for the customer and the development team. As the team is working to get these patches out soon, the customer is unhappy with his ‘broken system’. Even worse, once a few bugs are found, more testing is done on the live system to make sure everything still works, and more regression bugs poor in, adding up to the stress. To the development team it begins to look like…

From the customer’s point of view, it looks like the team working on the product is not in control. It is as if the team does not seem to know what they are doing. To them their product, which seemed rock solid at start, is degrading to a house of cards.

You can debate about high and low impact issues, and the matter of urgency to fix these issues. The perception of the customer is likely to be the same, regardless.

This is how I see it:

It is us developers who are responsible for letting regression happen.

Not testers.
Not project managers.
Not stakeholders.
Not the customer.

It is us and us alone.

We write the code, we change the code, we are in control of the code (at least we should be!).

Even if you happened to be depending on a third-party system, it is your job to keep an eye out on that system. Verify that it behaves as you would expect it to. Why? Your system depends on the behaviour of another system, trusting that this behaviour does not change is not enough. You have to be *sure*.

Its all about attitude
Do you always deal with regression bugs after each release?

Stop accepting it, it is not normal.

Rather, start thinking about how you can prevent this. Don’t look how other people could prevent this. Think of what you could do right now. There are many ways to reduce the amount of regression bugs. For instance: add tests before changing any code. Fixate the behavior with black box tests. When you refactor, keep running your tests so you know you did not break existing behaviour. Add new tests for new features you introduce. Create a test suite that you can trust. Make integration tests. Is it hard to write tests? Make it easier. Don’t back away from the code, it is your code and you should be in control.

Besides the code, improve your own skills. Start reading about how to deal with legacy code. Attend a legacy code retreat to hone your skills. Practice, practice, practice! Get better.

Reap what you sow in your daily work.

But isn’t the whole team responsible?
Ah, of course! But does that mean that you, as a developer can now do less? Would it be okay in a team to not test, because you have testers? (“its their job right?”).

In a team we all have our strengths and weaknesses.

We understand code, and we can change code. No other role in your team is responsible for understanding the code then you. Being in a team does not make you less responsible.

Again, it is all about attitude. Stand for your craft, deliver high quality work and make sure the system is in check. It should be you who controls the system.

Attitude, again
There are developers out there who really think they know everything of the system. And to be honest, I once had a time where I always knew what changes had impact and what I could do. And even though I was right about the impact on changes…

…I was at least wrong as many times as I was right.

But sometimes it is not just being over-confident. Sometimes it is being ignorant, or even arrogant.

Please, don’t be like this guy…

Just because it is hard, doesn’t mean you shouldn’t do it
Regression is a pain. It can be dealt with.

It is not easy.

You will not completely eliminate regression bugs. But with the correct mindset, tools and safety-net(s), you will greatly reduce the amount of regression bugs.

It is necessary. Just do it. For the love of our (your!) craft, do it, for everyone who depends on us:

The customer.
The stakeholders.
The project managers.
Yes, even the testers.

bookmark_borderA Software Quality Model (Part I)

This is the first post, of more to come, about my thesis and the software quality model I have developed.

First I will explain in what context my thesis is placed. This post will focus on decisions made to actually form a ‘new’ model (actually, derived from…).

For my thesis I have investigated how to improve quality of a software product. The first step was to choose one of the five perspectives on quality, given by Garvin (see here for more information). The product based perspective is chosen for scoping reasons. With that perspective in mind, three software quality models where evaluated:

The first, and one of the most known models, is the model of Boehm (literally translated ‘the tree of Boehm’):

The second model was from Willmer and the third from Bemelmans. Basically Willmer takes Boehms model and took it a bit further in order to translate customer desires into metrics. Bemelmans takes a whole different approach: He takes three main characteristics: “requirements for usage”, “requirements for maintenance” and “requirements for product information”. Which each have their own sub-characteristcs. (though it does not look like a tree like Boehm, as it only has one dimension).

One model actually caught the attention because it tends to ‘translate’ customer language into a developer language. The basic idea is that a software quality characteristic is mapped with a metric. Unfortunately the book I had did not tell how to translate this. Also, there was a need to translate customer language into software quality characteristics. This was something that was not being explained either.

So how does the translation look like? Imagine a matrix, where at the left several software quality characteristics are given (for every row). And at the top, every column is a metric. With some means you could assign metrics to characteristics and get a basic idea what is important.

Taking the ‘translation matrix’ of Willmer as a starting point, I have developed a model that actually does three things:

1st. It translates customer language into software quality characteristics. Not only that, but it also gives a % of relevance. (more on that later).
2nd. The actual translation from characteristics to metrics. This looks a lot like how Willmer does this, but it also has a % per metric per characteristic.
3rd. The  interpretation of metrics. This is where things get really concrete. Here the actual scoring is defined. How will the metric LOC be interpreted?

With these three, it is possible to actually ‘calculate’ the total quality of a software product, with the ‘product based perspective’ of Garvin in mind! (So, it does not tell you how to improve your processes. Nor does it tell you how customers experience the website).

So how do you use this model? Using the “Plan/Do/Check/Act” (Circle of Deming) as basis, the suggested plan is:

– First ‘define’ quality.  (Plan)
– Meassure and analyse (Do)
– Evaluate (Check)
– Change definition / software (Act)

In my next posts I will explain briefly how the model looks like.

bookmark_borderChoosing a Software Quality Model

For my thesis (Software Quality) I am currently figuring out how to choose a proper Software Quality Model in order to lay down the foundation of a Software Quality Definition for the project I am working on.

For now I have choosen this approach, and I’d like to hear from others how they (would) have tackled this problem.

It is good to know that I have narrowed down what “Software Quality” is. I’ll be focussing on the product (Garvin, Gosby) quality and the targetted audience are Software Engineers.

At first I will determine what kind of product I am working on. In this case it is a website, selling things. Some characteristics come to mind, like “24/7” or “should display always the correct (price) information”.

With these characteristics, I’ll look up Software Quality Attributes that would fit in.

This gives me a table, with in the first column all the characteristics, and next to them a few Software Quality Attributes that would fall into that specific characteristic description.

Now that I have a list of Software Quality Attributes, I can look up what Quality Model has the most matching Software Quality Attributes in them as well. Models that focus on different attributes would not interest me anymore, because it would not apply to the kind of product I focus on.

I found a nice model comparison, with Software Quality Attributes, from NASA. For your convenience I show it here:

With this, it would be possible to choose a Software Quality Model and work from there.

I am half-way with this method, and so far I think this is the way to go. If you think this is a bit too easy, or I should really take other considerations. Please let me know.

Note: thanks to Wouter for his suggestion who got me into this direction.