Usability requirements: how to specify, test and report usability

What is the Common Industry Format?

The Common Industry Format (CIF) for usability test reports specifies the format for reporting the results of a summative usability evaluation. The most common type of usability evaluation is formative, i.e. designed to identify usability problems that can be fixed. A summative evaluation produces usability metrics that describe how usable a product is when used in a particular context of use [3,7]. The CIF report format and metrics are consistent with the ISO 9241-11 [5] definition of usability:

The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

The type of information and level of detail that is required in a CIF report is intended to ensure that:

  • Good practice in usability evaluation had been adhered to.
  • There is sufficient information for a usability specialist to judge the validity of the results (for example whether the evaluation context adequately reproduces the intended context of use).
  • If the test was replicated on the basis of the information given in the CIF, it should produce essentially the same results.
  • Specific effectiveness and efficiency metrics must be used, including the unassisted completion rate and the mean time on task.
  • Satisfaction must also be measured.

It was envisaged that a supplier would provide a CIF report to enable a corporate purchaser to take account of usability when making a purchase decision. A purchaser could compare CIF reports for alternative products (particularly if a common set of tasks had been used). The purchaser might specify in advance to a supplier the required values of the usability measures (for example based on the values for an existing product).

The original motivation for the CIF came from usability staff in purchasing companies who were frustrated at purchase decisions made exclusively on the basis of functionality. These companies experienced large uncontrolled overhead costs from supporting difficult to use software [4].

The CIF format was agreed by the IUSR working group of usability experts from purchasing and supplying companies, including companies such as IBM, Microsoft, Hewlett-Packard, Boeing, US West and Kodak.. It was based on collating good practice from the different companies, and aligning this with ISO 9241-11.

Copyright © 2002 Serco Ltd. Reproduction permitted provided the source is acknowledged.