Category: Quality

bookmark_borderA Software Quality Model (Part II) – Translating customer language into metrics, scoring quality

Posted on by stefanhendriks

In my previous post I have explained the context of my thesis, and the various software quality models that are evaluated.

For my thesis I have extended the Software Quality Model of Willmer. Although it is not an exact implementation of the model, it is inspired by it. Also, the influence of the customer is processed into this model. The goal of this model is to translate customer desires into metrics, in order to calculate the total quality.

When talking about software quality, it is either very concrete (I want a red car!) or very abstract (it has to be reliable!). Customers tend to tell their ‘experiences of quality’ in sentences. This is the first step of the model. Try to get a few (eight at most) of these sentences. They must be distinctive. (Don’t have four sentences preach about security…)

Translating these sentences into concrete, measurable ‘things’ for developers, is another story. But before doing that , ask your customer what the relative importance is of all these sentences you just have written down. Just imagine that there is this situation where you have to go live. But , there is an issue that needs to be tackled, all blocking, one for each aspect (sentence). If you could pick one, which would you pick? Would you tackle the first sentence, the second? etc?

Of course, your customer will would tell  in his own domain language what the most important thing is. Try to map that (and confirm) with these sentences. Score them and try to get an ‘order of importance’. After you have done that, you have your first important goal reached: You know the relative importance of each quality sentence (aspect from now on).

So, what now? The next step is to map these aspects/sentences to Software Quality Attributes (also known as non-functional requirements). You either need a Software Engineer to do that, or even better, try to do it with your team. Before mapping these, try to make a selection of Quality Attributes first that are most relevant to you. Ie, try to use tree of Boehm or ISO9160 as reference. Within my theses I have used 9 Quality Attributes, some Quality Attributes are ‘sub-quality attributes’ of others. Example of Quality Attributes: Understandability, Reliability, Security, Availability, Complexity, etc.

The result of this mapping is that you get for each aspect several quality attributes. Not all quality attributes are applicable to an aspect. Try to figure out how much quality attributes are applicable to an aspect. Do this by asking teammembers and for each member that selects a quality attribute, score it. This way you can calculate relatively how your team thinks the mapping should be. This is important, because the eventual result of your measurement (see below) should be the product of the customer and the team that work on that product.

So, you have a few aspects, and each aspect has a few quality attributes. All that is left is to map metrics to quality attributes. Mapping this is fairly easy, there are quite a bunch of metrics out there. Each metric tries to measure (a piece of) a quality attribute. Some are easy, like complexity (quality attribute) can be measured by (although it is not limited to) the Cyclomatic Complexity metric by McCabe.

So basically you end up with this:

Aspect(n..8) –> Quality Attribute(n) –> Metrics(n)

Where:

  • the total quality of the system, is the combination of all aspects (all aspects relativity make 100%)
  • you should keep eight aspects (believe me, more will only make it harder to distinguish and make decisions)
  • you should attach quality attributes to each aspect, and determine their relative applicability to this aspect
  • you should attach metrics to quality attributes

So in the end, how do you score quality? Is it possible with this model? Yes, certainly it is.

Once you have found metrics, and attached them to quality attributes. You should formulate ‘scoring rules’. This means, you need to write down how you will interpret results of a metric and translate that onto a scale of 1 to 10. A scoring rule could be:

“Lines of Code (LOC) may be 1000 for a file max. Every 100 lines more substracts one point. (Giving 10 points with 1000 or lower and with 2000 lines a 1)”

This means, a LOC of 1000 will score a 10. A LOC of 1500 scores a 5, a LOC of 2000 or higher scores a 1.

Do this for all metrics, and eventually you will be able to calculate the total quality of the system.

In order to make this more concrete, here an example of such a calculation:

Total Quality Score = sum of scores of each aspect

Aspect score = (Sum of all relative scores of all applicable attributes) * relative importance

Attribute score = (Sum of all relative scores of all applicable metrics)

Example (For the sake of this example, attributes are scored already)

Aspect #1

Is for 30% important (relative to Aspect #2)

Attributes:

A -> for 40% applicable

B -> for 60% applicable

Aspect #2

Is for 70% important (relative to Aspect #1)

Attributes:

C -> for 70% applicable

B -> for 30% applicable

Scoring:

Attribute A scores 7 (out of 10)

Attribute B scores 5 (out of 10)

Attribute C scores 8 (out of 10)

Total quality calculation:

Aspect #1

A = 7 * 40% = 2,8

B = 5 * 60% = 3,0

Absolute score is 2,8 + 3,0 = 5,8

Aspect #2

C = 8 * 70% = 5,6

B = 5 * 30% = 1,5

Absolute score is 5,6 + 1,5 = 7,1

Total quality is:

Aspect #1 –> 5,8 * 30% (importance) = 1,74

Aspect #2 –> 7,1 * 70% (importance) = 4,97

Total quality score is 1,74 + 6,71 ==> 6,71

6,71 on a score of 1 to 10 (1 being bad, 10 being best).

bookmark_borderA Software Quality Model (Part I)

Posted on by stefanhendriks

This is the first post, of more to come, about my thesis and the software quality model I have developed.

First I will explain in what context my thesis is placed. This post will focus on decisions made to actually form a ‘new’ model (actually, derived from…).

For my thesis I have investigated how to improve quality of a software product. The first step was to choose one of the five perspectives on quality, given by Garvin (see here for more information). The product based perspective is chosen for scoping reasons. With that perspective in mind, three software quality models where evaluated:

The first, and one of the most known models, is the model of Boehm (literally translated ‘the tree of Boehm’):

The second model was from Willmer and the third from Bemelmans. Basically Willmer takes Boehms model and took it a bit further in order to translate customer desires into metrics. Bemelmans takes a whole different approach: He takes three main characteristics: “requirements for usage”, “requirements for maintenance” and “requirements for product information”. Which each have their own sub-characteristcs. (though it does not look like a tree like Boehm, as it only has one dimension).

One model actually caught the attention because it tends to ‘translate’ customer language into a developer language. The basic idea is that a software quality characteristic is mapped with a metric. Unfortunately the book I had did not tell how to translate this. Also, there was a need to translate customer language into software quality characteristics. This was something that was not being explained either.

So how does the translation look like? Imagine a matrix, where at the left several software quality characteristics are given (for every row). And at the top, every column is a metric. With some means you could assign metrics to characteristics and get a basic idea what is important.

Taking the ‘translation matrix’ of Willmer as a starting point, I have developed a model that actually does three things:

1st. It translates customer language into software quality characteristics. Not only that, but it also gives a % of relevance. (more on that later).
2nd. The actual translation from characteristics to metrics. This looks a lot like how Willmer does this, but it also has a % per metric per characteristic.
3rd. The  interpretation of metrics. This is where things get really concrete. Here the actual scoring is defined. How will the metric LOC be interpreted?

With these three, it is possible to actually ‘calculate’ the total quality of a software product, with the ‘product based perspective’ of Garvin in mind! (So, it does not tell you how to improve your processes. Nor does it tell you how customers experience the website).

So how do you use this model? Using the “Plan/Do/Check/Act” (Circle of Deming) as basis, the suggested plan is:

– First ‘define’ quality.  (Plan)
– Meassure and analyse (Do)
– Evaluate (Check)
– Change definition / software (Act)

In my next posts I will explain briefly how the model looks like.

bookmark_borderThe need for quality of Web Applications

Posted on by stefanhendriks

I’ve been working on my thesis (about meassuring Software Quality) and its reaching its final state. I will blog some posts about my thoughts and findings about Software Quality from here. This post was a bit old, I dusted it off and actualised it a bit. More to come!

Here it comes:

Today, it is unthinkable to not use an internet related service in your daily life. Think about staying in touch with family and friends using email (gmail, yahoo, hotmail, etc), social networking (twitter, facebook, hyves, etc) or more.

If you want to book a trip, you arrange this using the internet. When you want to buy a new laptop, you find the specs and reviews on the internet. You use a comparison site to find the cheapest (web)store. And if you want to buy it, you use the internet.

It is fair to say that the internet has become more and more important to our lives. Even more than we imagined 5 years ago. How would we deal when our mail provider is down? How would we react when facebook is down? What about twitter? How do we do our bank business when we cannot reach the online-banking site?

I think you got the point.

Companies developing and maintaining (ecommerce) websites or services are constantly pressured to deliver high quality software in the shortest possible amount of time. Having these factors, time and quality; one cannot exclude the other. They do influence each other though. When you want to deliver fast, you ought to make sacrifices in terms of designing and testing the product. This will eventually lead to a lower quality. The opposite is true as well, if you spend more time in the design and testing, you will likely deliver higher quality. Also, unlike with traditional software, it is not always the first to deliver that is the best.

In the land of the internet, it is not only the case to get users to your website, but also to keep them revisiting. If you deliver a fancy new feature, you might get the attention. But if you did not put enough effort to make it high quality, you might lose them to your competitor who delivers a similiar feature a few days later, but twice as good. Remember, your visitors can easily switch if you cannot deliver what they expect.

So what does a visitor expect from a website? Looking around on the net you can find some definitions of quality of good websites. That is the biggest question of all. How can you deliver fast, and yet have some sort of quality to rely on?

You could use a Software Quality Model for that, like the ISO-9126. I would say, you’d use a model like the ISO-9126 and create/use a Web Application Quality Standard.

So what is this standard? I haven’t found a definite model, although I have found a good resource here.
In my opinion, the following Software Quality Attributes would belong in such a model:
– performance. Your visitors will leave if loading a page takes longer than 2 seconds. Directly influencing possible sales!
– security. Atleast be aware of the top ten of OWASP. Better yet, make sure you are not vulnerable to these exploits.
– maintainability. Developers should be able to easily extend code, or fix bugs without causing too much regression.
– reliability. Should your website simply shut down once the load is too high? How much ‘page faults’ may appear? One of thousand requests? One of hundred?.

Each of these attributes can be split into several measurable units. For example, performance wise you could measure the amount of time it takes to get a page loaded in the browser. Maintainability has some known measures (like McCabe’s Cyclomatic Complexity).

Note: Recently I figured that the ISO-9126 is actually not a model, but a ‘standard’. It really does not tell you *how* to do things, but it does give guidelines. The Software Quality model I’ve made does tell  you how to do things.

bookmark_borderChoosing a Software Quality Model

Posted on by stefanhendriks

For my thesis (Software Quality) I am currently figuring out how to choose a proper Software Quality Model in order to lay down the foundation of a Software Quality Definition for the project I am working on.

For now I have choosen this approach, and I’d like to hear from others how they (would) have tackled this problem.

It is good to know that I have narrowed down what “Software Quality” is. I’ll be focussing on the product (Garvin, Gosby) quality and the targetted audience are Software Engineers.

At first I will determine what kind of product I am working on. In this case it is a website, selling things. Some characteristics come to mind, like “24/7” or “should display always the correct (price) information”.

With these characteristics, I’ll look up Software Quality Attributes that would fit in.

This gives me a table, with in the first column all the characteristics, and next to them a few Software Quality Attributes that would fall into that specific characteristic description.

Now that I have a list of Software Quality Attributes, I can look up what Quality Model has the most matching Software Quality Attributes in them as well. Models that focus on different attributes would not interest me anymore, because it would not apply to the kind of product I focus on.

I found a nice model comparison, with Software Quality Attributes, from NASA. For your convenience I show it here:

With this, it would be possible to choose a Software Quality Model and work from there.

I am half-way with this method, and so far I think this is the way to go. If you think this is a bit too easy, or I should really take other considerations. Please let me know.

Note: thanks to Wouter for his suggestion who got me into this direction.