Use Cases for Taxonomy Development

Developing use cases in the initial design of a taxonomy is something I did not learn about until I went into consulting, but it is a useful approach to taxonomy and metadata design in any circumstance, regardless of the involvement of an external taxonomy consultant.
The use case technique comes from the field of systems analysis, and especially software and systems engineering, but use cases are increasingly applied in the development of systems and structures for knowledge management, content management, information management, etc. Typically use cases for a taxonomy are not limited to the taxonomy alone but are for the design of all metadata and the broader information or knowledge management system. “System” means the combination of software, content, metadata/taxonomies, and users.

What is a use case?

A use case describes a scenario of how a user uses a system to accomplish a particular goal. A use case should not be confused with a case study. It need not be long and detailed, although they may vary in their descriptive length. All use cases include:

1. A designated user type and role (sometimes called “actor”), which could be as simple as an internal organization job title. Examples of external users could be designated as: undergraduate college student, paralegal, pharmaceutical corporate librarian, experienced online shopper, etc.
2. A task that the user is engaged in which uses the system. This will likely be described in more detail than the description of the user. Taxonomy use cases would typically involve a specific aspect of one of the following tasks: indexing/tagging, using search to find information, using browse to find information, discovering/exploring for related information and finding/retrieving certain content items
3. A goal and perhaps ultimate purpose of the user’s task.

I had participated in a consulting project once whereby the stakeholders were advised to create use cases that went so far as identifying fictitious personas, a practice that is often done in marketing planning. I don’t think it’s necessary to go that far in taxonomy use case development, although it might be useful if there are users of the taxonomy who are external customers/clients.

Why create use cases for taxonomies and other metadata?

The task of developing taxonomies and other metadata can benefit from use cases in particular ways:

• It grounds the taxonomy in reality, ensuring that it is designed to be usable, rather than being an academic taxonomy on a subject domain.
• It engages the users and other stakeholders in the taxonomy development process, who then become more interested in supporting/promoting or using the taxonomy, especially when the taxonomy serves their user needs and solves their problems.
• It provides sample situations which can then be utilized for testing the draft taxonomy before the taxonomy and content are fully implemented in the system. As a taxonomist who has led taxonomy testing activities among sample users, I have personally found used cases to be valuable for this purpose. 

What are examples of use cases for taxonomies and other metadata?

The following brief fictitious use case examples are of the kind that could be used for taxonomy development.

Internal organization use cases:

• A subject-matter-expert author who is required to tag authored documents with subject categories so that users can find documents by subject.
• A digital asset manager in an advertising agency, who needs to ensure that image files are assigned the proper copyright information.
• A content manager at a publishing company who, as a major responsibility, needs to assign full metadata to XML file content for various downstream purposes to assembly digital content products.
• A marketing copywriter seeking an expert on a specific subject among a company’s employees to give feedback on the accuracy of a blog post the copywriter is writing and who is inclined to browse subjects if available. 
• A manager who wants to find historical information on product offered in order to prepare a presentation about the product.
• A digital marketer who needs to update the public website with seasonal images that were not used last year (but two years ago is OK).

External/customer use cases:

• An undergraduate student who uses the default search to look for information on the events leading up to the fall of the Berlin Wall for a history class paper.
• An experienced online shopper who is searching to purchase carry-on luggage and wants to filter results by price, color, and positive reviews.
• A corporate librarian conducting competitive intelligence research on market strategies of leading competitor companies in the same industry and who would like to use advance and/or Boolean searching, if possible.
• A lawyer specialized in commercial law who need to find out where and how to file a financing statement in the proper jurisdiction for a client of his who to secure a loan, but lacks experience in legal research.
• A cancer patient searching for an oncologist with a certain type of cancer specialty, acceptance of certain insurance, within a certain geographic region, and with a number of good patient reviews.
• A compliance officer who needs to find regulations and associated policies and procedures that pertain to various departments and products lines of his employer, who knows the names of statutes but not the titles of associated regulations.

How are taxonomy use cases utilized?

In addition to serving the purposes of engaging stakeholders and ensuring the taxonomy is content- and user-focused, use cases can have additional specific applications, such as:

• Identifying or validating who all the different types of users are, so that their issues and feedback can be taken into consideration in the future.
• Suggesting improvements in the user interface design.
• Developing walk-through scenarios, with specific search criteria or topics of browsing spelled out, for offline testing of the taxonomy usability (including adequate depth and breadth) for both indexing/tagging and retrieval. (Read more at the post "Testing Taxonomies.")
• Providing scenarios that can be used in other taxonomy/knowledge management project research, such as ROI (return on investment) research.

Card Sorting and Taxonomies

Card sorting is a common technique in information architecture for developing the organization of menu labels or categories on websites. It would thus seem to be a very suited methodology for developing all kinds of taxonomies, but in actual practice card sorting is not utilized for most taxonomy projects, at least not in my experience.

Card sorting gets its name from the paper-based approach of having numerous category or concept names written down each on a small index card, and then the cards can be sorted on a table into logical categories. Multiple stakeholders and/or test users are given the opportunity in turn to organize the cards as they deem appropriate, and the person administering the card sort, takes note of the choices and considers them for the actual organization structure. Today, card-sorting software, especially that which is web-based to allow remote access, has largely replaced the physical cards.

There are two variants to card-sorting exercises, the open card sort and the closed card sort. In an open card sort, participants sort the labeled cards in any groupings they see fit and then they assign their category groups with any group name they want. In a closed card sort, the participants are already presented with a set of named top category groups that they cannot change, and are asked to sort the labeled cards into the pre-assigned categories. Each type of card sort has distinct objectives and is suited for different stages of the project.

Open card sorting is a good way to get a new taxonomy from scratch off the ground when you have some concepts (extracted from the content) and don’t know how to organize them. However, this is increasingly no longer the scenario. It’s rare to start creating a taxonomy from scratch with no other reference for top categories. There are so many taxonomies in existence now for all subjects, that it’s easy to find a starting point as a model. Furthermore, the owner of a taxonomy may have already designated the top categories for business reasons.

The aim of closed card sorting is to determine in what broader category narrower categories belong, especially if there is uncertainty. But if a narrower category could rightfully belong under more than one category, rather than force a choice between one or the other based on a card sort, the subcategory could belong under both. This is what taxonomists call “polyhierarchy,” and it acceptable as long as the hierarchy is sound and valid in both locations. Thus, closed card sorting is only needed when you have decided you do not want polyhierarchy.  Polyhierarchy is generally a good thing, because it provides more than one navigation path to the same results, and different people choose different paths. Sometimes, however, polyhierarchy is avoided near the top levels of a taxonomy in order to maintain a sense of tree structure.

Card sorting is most practical for just two levels of hierarchy: concepts and their immediate parent categories. It’s possible but unwieldy to suggest to users that they may create three levels, and some card sorting software does not even allow it. Often it is more reliable to just run a second series of card sort testing for another hierarchical level in the taxonomy. However, running multiple card sort exercises for different hierarchical branches of a taxonomy can be quite impractical, if not also costly and time-consuming.

Finally, card sorting works only for traditionally hierarchical taxonomies. It does not work for faceted taxonomies, where terms from different facets/attributes are selected in combination to limit or filter search results. Faceted taxonomies are becoming increasingly common.

Card sorting continues to be useful for information architecture, though. When designing the structure of a website and its main and submenus, it can be difficult to decide what the categories should be, because the content of  a site can be unique or nonstandard. Additionally, polyhierarchy is not expected in submenus and could be confusing. Finally, website navigation is often not deeper than two or three levels, unlike many taxonomies that are often four or five levels deep and thus impractical to thoroughly design or validate with card sorting.

Testing Taxonomies

As mentioned in my previous blogpost, “Evaluating Taxonomies,” taxonomy evaluation and taxonomy testing differ. While the evaluation of a taxonomy by a taxonomist is needed when a taxonomy is created by non-taxonomists (such as by subject-matter experts instead), testing of a taxonomy, on the other hand, is recommended in all cases, no matter who created the taxonomy. Following is an overview of the different kinds of testing that can or should be performed on a taxonomy prior to its implementation.

Card-Sorting

Card-sorting is probably the best known kind of testing, especially now that the prevalence of online card-sorting tools facilitates set-up and enables remote participation. It is not necessarily the best kind of testing for all situations, though. Card-sorting serves to test categorization schemes, so while it is suited for hierarchical taxonomies, it is not so appropriate for faceted taxonomies, especially with regard to how the facets are to interact with each other. It is possible, though, to card-sort test an individual facet, if that facet comprises an internal hierarchy of terms.

There are two kinds of card-sort tests, open and closed. In open card-sorts, the testers group concepts/topics together and then assign a broader category of their own; whereas in closed card sorts, the broad categories are already designated, and the testers merely categorize the specific concepts/topics within those pre-determined categories. Open card-sorting, if chosen, is therefore done earlier in the taxonomy design process, when broad categories are uncertain. A single taxonomy project may have either or both kinds of card-sorting depending on where the greatest need is for this additional input of information. Testers could be test end-users or they could be stakeholders, depending on the needs of the test.

Card-sorting is actually not really a kind of taxonomy testing but rather a form of taxonomy idea testing. Card-sorting is not performed on a completed taxonomy to test it but rather to test ideas of categories/hierarchies which later will be combined to create the taxonomy. Therefore, card-sorting is not an alternative to the other kinds of testing described below, which may subsequently be done.

Use Testing

Use-testing or use-case-testing is a necessary step after a draft taxonomy is built or nearly completed but before it is finally implemented, allowing for revisions to be made based on the test results. It is at this point that the taxonomy is put to the test to see if it will perform as hoped in search/retrieval and (if applicable) for manual tagging. This type of testing might also be called taxonomy validation.

A cross-section of different kinds of test users should be recruited to prepare several typical use cases and perhaps one especially challenging use case of content search scenarios. The user is then presented with the taxonomy (which can be in any format at this stage, whether on paper, as an Excel file or as test web page) and asked to browse the taxonomy to look for terms under which the content for the use search scenario might be found. The user performs the test, either browsing in the tester’s physical presence or via screensharing with verbal narration of what the user is doing and why.  The test administrator takes notes regarding any problems in finding taxonomy terms for the use case. These findability problems should be considered as requirements for additional terms, additional nonpreferred (variant) terms to point to existing terms, or perhaps more polyhierarchy or associative relationships to help guide the user to find the desired concepts.

If the taxonomy is to be used for manual tagging or indexing, then a second, different set of use testing is needed, whereby users who perform this function should test the taxonomy for indexing of typical and challenging documents that they tend to deal with. Rather than coming up with use “cases”, the test-user-indexers merely need to come up with actual documents. The documents should represent a good cross-section of the various document types indexed. This exercise is even more straightforward than the user testing for finding content, so it could even be performed offline without the test administrator present, as long as the test-user-indexer takes good notes.

A-B Testing

In A-B Testing, the test-users are presented with two different possible scenarios and asked which they prefer. When comparing two different taxonomies or parts of taxonomies, only one or two variations should exist between the two that are compared to make the test clear-cut. You may set up a series of A-B test pairs to compare multiple variations. This kind of test is comparable to what an optometrist does for vision: “Which is better, A or B?”  Since only one or two differences should be compared and tested at a time, A-B testing is most suitable to compare proposed top-level categories, rather than getting into the depths of a taxonomy, where it is not practical to conduct a detailed term-by-term comparison. Thus, A-B testing focuses on high-level structural design, navigation and browsing, and not the effectiveness of finding and retrieving content.

A-B Testing can be done at any time in the taxonomy design and build process. It is also very useful when considering a taxonomy redesign for comparing the existing taxonomy (A) to a proposed change (B). A-B Testing is usually done by presenting the test users with graphical or interactive web page mock-ups. I’ve created the B image to an existing online A image, by taking a screenshot of A and then edit it in Microsoft’s Paint accessory. Although each individual A-B test is simple, deciding what to compare and how many comparison tests to make needs to be determined, since each test takes time and resources.

Conclusions

Taxonomies should be tested, but it’s not true that any test is good. Different tests are for different purposes and fit into different stages of the taxonomy process. An inappropriate test or inappropriately timed test can be a waste of time and money.