Open vs. Closed vs. Hybrid Card Sorting: When to Use Each Test Type


Open vs. Closed vs. Hybrid Card Sorting: When to Use Each Test Type

Did you know that website menus with too many options can distract customers from finding what they want? Having a neat and organized website is essential to increase your customer base. However, how do you ensure your website has a structured layout?

The answer lies data that can be collected from the actual users and with the help of card sorting tests. Card sorting is a research technique in which people organize and categorize information for the benefit of providing, for example, an easy online navigation.

There are three test types: open, closed, and hybrid. With this basic guide, you can learn about each type and when you should use them.

Open Card Sorting

An open card sort is one of the easiest ways for you to gain insight into your customers. With an open card sort, you can learn what a customer thinks about your website content. However, here’s the easy part: they do all the work. During an open card sort, a user simply takes a bunch of cards and groups them into categories they deem to be logical.

For example, these cards could say words like Demi Lovato, Harry Potter, or even avocado. A person may categorize them into simple groups such as celebrities, books, and food.

If you’re unsure how to design or categorize your website, an open card sort can help. It provides feedback from your consumers as to what they find relevant and necessary to group. Thus, with these groupings, you can make your website more attuned to your user’s interests and needs.

Diagram of Open Card Sort

Closed Card Sorting

Similarly to an open sort, a closed card sort> provides users with a deck of cards that must be sorted into categories. However, the main difference between the two is in a closed card sort you assign category titles.

For instance, say there are a handful of cards with various words and phrases that need to be sorted into groups. Although, instead of the user creating a category, imagine there are three assigned categories called entertainment, shopping, and music. It’s then the user’s job to filter the cards into the selected categories how they see fit. Its purpose is to gain insight to see if your existing categories seem logical to your users or what they would change and how they would categorize your items.

Diagram of Closed Card Sort

Hybrid Card Sorting

A hybrid card sort is where you provide your users with predefined categories, similar to close card sort, but in addition, you allow them to create their own categories, if they don’t find a good fit from those provided and need to create their own. Normally, creators will use this technique when they already have some categories established but would like their customer’s input as to what the others should be.

For example, suppose there are already two categories established called fashion and relationships. However, you, as the customer, don’t believe the word “magic” would go under either of them. Instead, you can create another category called “fantasy” and drag the word magic as well as any other phrases you believe make sense into the category.

Diagram of Hybrid Card Sort

Test data analyses

Regardless what type of the test you conduct, the actions of your participants will generate data, such as which card was associated with which group. Unlike the closed sort, where the number of categories is known from the beginning, in the open and hybrid sorts there potentially could be as many new categories, created during the test, as provided the number of cards provided multiplied by the number of participants. This can get overwhelming and unless you do some adjustments to your data, the results may not be as clear as well. For example some of your users may create new category groups and name them:

  • Customer support
  • User support
  • Support
  • Technical support

While there names look different to a computer, you may be aware that all of them refer to the same feature of your project. To address this issue usabiliTEST pioneered and patented a Virtual Merge feature, where you, as project owner, can combine any groups together and even give a new name to that new combined group. This feature allows you to simplify the collected results and make the most of your data.

And once your data is ready for analyses our built-in tools allow you instantly convert combined responses into percentage, group these percentages into logical groups, create a distance matrix, plot multidimensional scaling diagram of the distance matrix and even create an interactive dendrogram, which represents Hierarchical Cluster Analysis and could be adjusted as broad or narrow as your research needs are with a simple click.

Screenshot of test data groupped by distance matrix
Screenshot of test data groupped by distance matrix.

Choosing the right sort type

Selecting the right card sort all depends on the needs of your project. There are always a number of questions one may have before launching a test, such as:

  • Are you creating a website menu from scratch or reordering existing navigation links?
  • Do you have access to existing customers who can be your test participants or do you need to recruit folks not familiar with your product?
  • Are your test participants motivated by incentives or some of your participants will drop cards without spending adequate time needed to make an informed decision?
  • How many cards are you going to ask them to sort?

These are all the questions that you need to have answered to ensure the success of your test and the quality of the data you will collect. If you are uncertain about some of these issues, we recommend to start with the Open Card Sorting test and once you have some data, that seems reasonable to you, run another test to validate the initial results, but this time use either Close or Hybrid card sorts.

If you’re ready to give card sorting a try, we offer all three types.

Also see

Card Sorting online made easy (and affordable)

Other articles

Card Sorting in 5 simple steps [infographic]

This inforgaphic illustrates in 5 simple steps the simplicity of conducting Card Sorting and the benefits of such research.