Inspection of the specification

“Would you please check the specification briefly and see if you notice anything?” – Do you know such tasks? You will certainly find various contradictions, gaps or errors in the specification if you only search long enough. How helpful such an approach is, however, depends on chance. 15% of all software bugs are delivered and a large part of these bugs are already in the specification. Customer requirements are misunderstood, use cases are described incompletely, different terms are used for the same thing, and expressions are interpreted differently. Of course, these problems not only affect specifications, but also architectural designs, test cases, all intermediate written software development results, and source code.

Best practices for successful inspections

The quality of the results of an inspection depends on the quality of the questions asked. A simple visual inspection in the sense of “Please briefly check the specification, perhaps you notice something” must be subjective. Scientific research has shown that the results of an inspection are never complete. Nobody finds all mistakes. Various inaccuracies, contradictions or errors are always identified. And even the same reader will not discover the same points in different passes through the same specification. Of course, this does not mean that the inspection of a specification is useless. On the contrary: it detects errors at an early stage that would otherwise be transported through the project instead of rectifying them at an early stage. To ensure that an inspection of the specification delivers the best possible results, I recommend six best practices:

1. A clear definition of objectives

Define a clear goal at the beginning. Does an initial review focus on the completeness of the project scope and the details are then worked out step by step? Or should the inspection be a final quality assurance before the implementation and reveal as many acceptance-relevant errors as possible?

2. The use of checklists

Checklists with yes-no questions have proven successful in troubleshooting inspections. Examples:

• Is a use case diagram included?
• Are the stakeholder names in Chapter 3 consistent with the actor names in Chapter 4?
• Are all error cases covered?

If the answer to a question is “Yes”, then everything is fine, if the answer is “No”, then there is room for improvement. The quality criteria can be found in the relevant standards. For requirements I use the ISO/IEC/IEEE 29148:2011 standard “Systems and software engineering – Life cycle processes – Requirements engineering”. This standard names the following quality criteria for requirements:

• necessary
• implementation-independent
• unambiguous
• consistent
• complete
• atomic
• realisable
• traceable

Strictly speaking, a checklist is a one-dimensional representation of a two-dimensional process: several chapters or contents of the document are checked against a list of quality criteria. I like to recommend that you proceed chapter by chapter and check the criteria for each chapter one after the other. In most cases, chapters or diagrams are so complex that you first have to think your way into them before a qualified assessment is possible.

In technical literature, it is often recommended that checklists should only contain one or two pages, but my experience is different. My ten chapter checklists for requirement specification templates are five to ten pages long. A list of abstract quality criteria such as “complete” or “consistent” is good in itself, but the result of the inspection is still subjective and arbitrary in practice. It is better to specify these criteria in a way that is appropriate for the template. The criteria usually multiply. Let’s take an exemplary look at “consistency”: the contents of one chapter should be consistent with each other and with several other chapters at the same time. For experts, breaking down is relatively easy, because it is known from standards and technical literature what belongs to a complete textual requirement or a complete use case diagram, or what consistency relationships between them look like. In my checklist, I have consolidated this knowledge and can easily reuse it in any inspection. If the criterion is formulated concretely enough, it can be processed quickly. I would hope that the results would also be more repeatable and less dependent on the inspector, but this has not yet been confirmed. The clear criteria are sometimes subjective. How, for example, do you want to answer objectively a question about comprehensibility such as: “Are the names of all use cases self-explanatory?

3. No hope for completeness of the error list

If the initial task is to find only the grossest contradictions, gaps or errors in a specification, a quick inspection by a single expert is sufficient. However, if you need a detailed inspection, even half a dozen independent reports will not reveal all contradictions, gaps or errors. A specification is such a complex document with countless interdependencies between its contents that it is impossible for people to find all the errors. Unfortunately, I cannot give you any hope for the completeness of an error list. Therefore, further quality assurance is necessary even after the inspection.

4. Working with several inspectors

There are semantic and syntactic inspection criteria: The syntax refers to the elements used such as words or model symbols, as well as permitted combinations, i.e. the grammar of text and graphical notations. This syntax can be evaluated very well by an external specification expert, often without fully understanding the content of the specification. The semantic criteria are used to check the quality of the content, such as the completeness of the requirements. These can best be assessed by stakeholders or industry experts. Since specification and industry expertise are often distributed among different people, several inspectors should assess the specification. In this way, you can competently cover criteria and achieve better completeness of results.

5. The repetition of the inspection

A specification should be quality assured several times. In practice, requirements change relatively frequently. As positive as these changes may be, they endanger quality and consistency in the sense of the specification. In addition, different questions arise at different points in time. By repeating the inspection you get a new chance to discover contradictions, gaps or errors. Remember: 15% of all software errors are delivered and the majority of these errors are already in the specification.

6. Automation is difficult

Wouldn’t it be great if you could get an error list for your specification at the push of a button and after a few seconds? Support by automated tools sounds really tempting. Unfortunately, the tools I have tested to date have not convinced me. Not all quality criteria can be checked automatically. Only syntactic criteria for which you can define simple rules can be identified in this way. And even then you still get too many “wrong errors”, i.e. error messages regarding places that meet the criteria of an error, but are nevertheless correct. This leads to high manual effort, because you have to decide whether the displayed errors really have to be corrected or whether they are just false messages? Only simple checks are relatively accurate, such as searching for sentences with too many words or for forbidden terms such as “anytime”. You can find lists with such forbidden terms in the ISO/IEC/IEEE 29148:2011 standard or in the CPRE Foundation Level¹. My recommendation: First define your checklist and then check whether you can check individual criteria with a tool. It costs more than it benefits you to go through all the testing possibilities of a tool and then try to find out whether the error list is really useful to you.

Conclusion

The inspection of a specification is an effort. An effort that pays off, especially if you compare it with the cost of eliminating delivered software errors. Defective software reduces customer satisfaction and loyalty, endangers the product or even the company image. You certainly want to avoid this. And last but not least a personal, not altruistic tip: If you want to check a specification, use an external professional. He or she can support you very individually with these best practices. He/she will develop a suitable checklist with numerous concrete suggestions for improvement to suit your situation. Of course, even the expert will not uncover all errors, but such an inspection is always cheaper than eliminating errors only after delivery.

Notes:

Dr Andrea Herrmann regularly blogs about requirements engineering and software engineering at http://www.herrmann-ehrlich.de/.

Here in our t2informatik Blog you can find more articles from her, including

What does t2informatik actually do? One click and you'll know.