# Card sorting

## Summary

This is a method for discovering the latent structure in an unsorted list of statements or ideas. The investigator writes each statement on a small index card and requests six or more informants to sort these cards into groups or clusters, working on their own. The results of the individual sorts are then combined and if necessary analysed statistically.

## Benefits

If the informants are representative of the user population for whom the application is being designed, then the result will reflect the structure in which the users expect the ideas or concepts should be presented.

## Method

### Planning

Collect the statements you wish to analyse. Write each statement on a separate card. If your statements already exist in computer format, print them out on labels and stick the labels on index cards. Number the cards uniquely on the back.

If you have pre-defined categories into which you want the statements sorted, prepare 'place mats' on which your informants may place the cards. Otherwise ensure there is a large empty table for the informant to place their piles of cards on.

Recruit informants who will be typical of the user population for whose benefit you are preparing the analysis. It is useful to have at least six such informants.

### Running

Shuffle the card deck so that all the informants don't get the same sequence of cards or worse, that each informant gets the previous informant's sequence. The informants receive the stack of cards, and then sort them into piles on the table in front of them, using place mats if desired. You may give a rough indication of the number of piles of cards you will expect to see, to give the informants a common understanding of the expected grain of analysis.

Explain that there may well be an 'unsortable' pile, but that they should attempt to place as many cards together into piles as they reasonably can.

Informants may attempt to show the relationship between piles of cards by spatial proximity on the table: ensure you note this information down: in the classic analytic technique, this information may get lost.

At the end, note which cards have been put together by the respondent by noting the numbers on the back of the cards. If you have not supplied place mats, invite the users to give a name to each pile of cards ("how would you describe the cards in this pile?")

### Analysis

If your clusters are relatively clear and straightforward, you may simply summarise the cards which are usually placed in each pile and give an overall name for the cluster. However, the result of card sorting is not always so straightforward.

The most common method of analysis for complex data from
card sorting is a statistical method called cluster analysis.
There are two main approaches to cluster analysis for this
kind of material: linkage and hierarchical. See [...] for
further details. In order to prepare for both types of analysis,
construct a similarity matrix. If there are *n* cards,
the matrix is *n* x *n *symmetric. Most methods
require the upper and lower quadrants to be filled redundantly,
and for the diagonals to be filled with 1.00. In each of the
cells, compute the probability *p* of the cards denoted
by the two common co-ordinates being together: that is, if
two cards *x* and *y* were sorted into the same
pile *q* times and there are *m* respondents altogether,
p = q / m

Some cluster analysis methods may require an index of dissimilarity,
so compute *p' =* *1 - p *and put 0.00 in the diagonals.

## More Information

A worksheet and an MS-DOS program for elementary linkage analysis may be found at

Computer supported methods for card sorting may be found at:

NIST tools: http://zing.ncsl.nist.gov/WebTools/

The popular IBM EZcalc and USort: http://web.archive.org/web/20040205000418/http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410

Websort: http://www.websort.net/

## Alternative Methods

The nearest alternative is the affinity diagram (concept wall) technique. This latter has the merit that a hierarchical arrangement can be deduced without recourse to mathematics.

## Next Steps

After card sorting you should proceed to design activities such as creating a (paper) prototype of the structure of the material you have been investigating. However, you may also wish to run a focus group or a brain storming session to further refine and to expand or enlarge on the results you have so far obtained.

## Case studies

http://www.acm.org/sigchi/web/chi97testing/mele.htm

## Background Reading

Lorr, Cluster Analysis for the Social Sciences

Infodesign: http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp

©UsabilityNet 2006. Reproduction permitted provided the source is acknowledged.__cookie policy__.