Logo for usabiliTEST card-sorting tool

Card Sorting online made easy (and affordable*)

What is Card Sorting?

Logo for usabiliTEST card-sorting tool Card sorting is a technique often used in usability testing and Information Architecture (IA). This user research involves participants using logic while sorting content or “cards” into categories or groups that make sense to them, given the information they are provided with. Very often this method is used in UX research for projects that help design teams to build or improve user interface (UI), information flow or user experience overall.

This is not a new research method. In fact the name itself gives it away, because originally participants were given a stack of index cards or peices of paper with content on them and asked to put them into optimal card sorting piles. If this isn't a part of your comprehensive UX research plan, perhaps it should.

Let's start with an example:

New web portal has a lot of links and the company decides to do some user testing in order to collect data that will help them arrange those links in such a way that it will be easier for consumers to find what they are looking for.

So we have a a problem, where too many links makes it difficult for users to use the site. A solution could be a simple categorization exercise in which participants organize options, in 10-15 minute sessions, into different groups based on their understanding of those concepts.

You may have noticed we emphasized the fact that it is participants will do the work that will help you to gain insights from your user experience about how to organize content in an intuitive way. And that's essentially what digital card-sorting is all about.

Screenshot of participant view of online test page
Screenshot of closed card sorting exercise page.

You’ve got options, when it comes to digital Card Sorting

Before you design your cardsort test you need to decide on the type, which often depends on the needs of your project. Typically, when you first hear about online card sorting software only two test types are mentioned: Open and Closed. At usabiliTEST we developed the third one — Hybrid, which is a combination of the first two. So, let’s take a closer look at what this means, how these types of card sorting are different from each other, and which are the best digital card sorting tools for your business needs.

Diagram of Closed Sort type

Participants are provided with a few category names and a set of cards. Then they place the value cards into these fixed categories. This test type will help either confirm your initial assumptions or allows to see where category labeling could be improved. The two demo links below provide examples of closed card sorting, one of which uses only text cards and the other images.

Diagram of Open Sort type

Participants sort index cards with some items written on them into logical groups. Once they create these groupings they must label them. Open sort is ideal for collecting user-generated ideas for logical content groupings. Please see demo button below for Open Card Sorting example.

Diagram of Hybrid Sort type

This is a combination of both Closed and Open Card Sorts. Test starts as a Closed Sort, where you provide predefined categories, however if participants do not find an appropriate category among those they were given, they can create and name their own and place their cards there.

Get it done in 3 simple steps

Please note that card sorting exercises can be conducted in person or remotely. There are benefits to both approaches however, since usabiliTEST's software is web-based, this article is written with remote card sorting session in mind.

  1. Create Test

    It only takes a few minutes to launch your own usability test. There's lots of flexibility in the setup process. So you can create either simple or complex tests — it all depends on your project's needs.

  2. Invite Participants

    Just because you built a test doesn't mean they will come. Furthermore, your test is worthless without real users and while the number of participants is often depends on your specific needs, it is said that a grup of some 30 participants is good enough to be able to overcome individual mental model bias. While you can tweet or email your test page's link, how do you know if they participated in your test? How do you know if they even opened your email containing the test?

    Out platform has been developed to assist you with all of this. Simply add your contacts into your address book and fire up the desired emails with a single click. Afterwards, sit back, relax, and let our system keeps track of your test responses for you. As for those slow responses, we'll follow-up with a gentle reminder.

    If your target audience is your office, you won't have to bother your colleagues about whether they have taken your test. You will already know if they have.
  3. Get Results

    Analyzing test results is where the fun begins. This could also be the longest part of any project. Imagine spending a lengthy amount of time tabulating all your numbers, when suddenly come late submissions…

    There's no need to worry about this scenario when you use usabiliTEST built-in analytics. All computations are done immediately and in real time. With just a click of a button, you can transform your raw numbers into percentages. These percentages can either be plotted on a graph or grouped into logical groupings. If you prefer to use other third-party statistical software, you can easily export all your data.

Screenshot of test data groupped by percentages
Screenshot of test data groupped by percentages.

Understanding test results

Test data is generated by your test participants, so it is only logical to begin your analysis by looking at the participants information. Depending on the motivation level of your panel some participants may abandon your test page before fully completing the task. If that's the case, you can exclude such results from the overall dataset by using include/exclude toggle switch.

Screenshot of usability test participants
Screenshot of participants table.

usabiliTEST provides built-in analysis tools that allow you to manipulate your test results on the fly.

Collected results are presented in several ways and we tried to make it easy to navigate by placing them under clearly marked tabs.

The first tab, raw numbers, is presenting sorts expressed by a numeric value and shows a correlation between cards and groupings, i.e. how many times each card was placed in each category by all participants. Now, in the Open card sorting test there could be quite a few categories, so to make it easier to understand the data, you can merge similar categories together. We pioneered this functionality and called it Virtual Merge.

The numbers also can be expressed in percentages, and the following tab uses special algorithm to group percentages together in color coordinated manner to let us visually see what the aggregated groups are.

The similarity matrix shows how close the relationships of the cards are and the MDS (multidimensional scaling) plots them on a plane.

The last one - an interactive dendrogram, which represents Hierarchical Cluster Analysis, arranges these relationships in a form of a tree. You can adjust the view by clicking on different nodes to see logical groupings. This could be especially helpful when you design a hierarchy structure, like a website navigation.

Screenshot of dendrogram view of test data
Screenshot of dendrogram view of test data.


Is Card Sorting a test?

Yes and no. While card sorting doesn't involve testing participants' cognitive abilities, it is a test designed to analyze Information Architecture. Instead of having right or wrong answers like some tests do, card sorting provides insights into available options based on the approach of each individual participant.

How can I use this tool?

With results obtained through a use of online card sorting sessions you can build your website’s structure. In addition, you’re more certain to organize information on your site in a logical manner for the sake of your users. For example, this technique is often used to redesign website navigation and menus.

How can usabiliTEST help me with card sorting?

All our tools have built-in data analytics that save you time by providing instant results. You can view your test data as raw numbers, percentages, or distance matrix. Charting features allow you to visualize it as either a multi-dimensional scaling (MDS) plot or a dendrogram. You also have the option of exporting your data in multiple formats so it can be used with your choice of tools.

Why is it called Card Sorting?

This method was developed before we used computers for usability testing. Originally, a person conducting Card Sorting exercise would use paper index cards and write terms or items on each of them. Then they would ask participants to sort these “cards” into logical groups. For example, “car” and “bus” could go into a group called “land” while “ship” and “boat” could be placed into a group called “water”.

Are there any free card sorting tools?

As far as we are aware there is only one free card sorting tool however it is not online but rather requires download and it installation on an Apple computer. This works well if you are conducting card sorting activity in a lab setting and can have your participants come to you, however if you rather not share your personal computer with multiple users or your potential participants are located elsewhere you should consider an online-based tool.
More at: FAQ #card-sorting

Blog articles

Open vs. Closed vs. Hybrid Card Sorting: When to Use Each Test Type

Helping you better understand user expectation, understand the various types of card sorting and when to use each test type.

Card Sorting in 5 simple steps [infographic]

This inforgaphic illustrates in 5 simple steps the simplicity of conducting Card Sorting and the benefits of such research.

* It doesn't matter if you’re a web designer, a student or a seasoned UX professional, usabiliTEST can help you achieve your goals at a fraction of the price you'd pay other providers.

And why is this? Because we're on a mission to change the world through improving usability and in the process of doing this, we don't want the cost to get in the way of those who can help us reach this goal.