Use Cases for Taxonomy Development

Developing use cases in the initial design of a taxonomy is something I did not learn about until I went into consulting, but it is a useful approach to taxonomy and metadata design in any circumstance, regardless of the involvement of an external taxonomy consultant.
The use case technique comes from the field of systems analysis, and especially software and systems engineering, but use cases are increasingly applied in the development of systems and structures for knowledge management, content management, information management, etc. Typically use cases for a taxonomy are not limited to the taxonomy alone but are for the design of all metadata and the broader information or knowledge management system. “System” means the combination of software, content, metadata/taxonomies, and users.

What is a use case?

A use case describes a scenario of how a user uses a system to accomplish a particular goal. A use case should not be confused with a case study. It need not be long and detailed, although they may vary in their descriptive length. All use cases include:

1. A designated user type and role (sometimes called “actor”), which could be as simple as an internal organization job title. Examples of external users could be designated as: undergraduate college student, paralegal, pharmaceutical corporate librarian, experienced online shopper, etc.
2. A task that the user is engaged in which uses the system. This will likely be described in more detail than the description of the user. Taxonomy use cases would typically involve a specific aspect of one of the following tasks: indexing/tagging, using search to find information, using browse to find information, discovering/exploring for related information and finding/retrieving certain content items
3. A goal and perhaps ultimate purpose of the user’s task.

I had participated in a consulting project once whereby the stakeholders were advised to create use cases that went so far as identifying fictitious personas, a practice that is often done in marketing planning. I don’t think it’s necessary to go that far in taxonomy use case development, although it might be useful if there are users of the taxonomy who are external customers/clients.

Why create use cases for taxonomies and other metadata?

The task of developing taxonomies and other metadata can benefit from use cases in particular ways:

• It grounds the taxonomy in reality, ensuring that it is designed to be usable, rather than being an academic taxonomy on a subject domain.
• It engages the users and other stakeholders in the taxonomy development process, who then become more interested in supporting/promoting or using the taxonomy, especially when the taxonomy serves their user needs and solves their problems.
• It provides sample situations which can then be utilized for testing the draft taxonomy before the taxonomy and content are fully implemented in the system. As a taxonomist who has led taxonomy testing activities among sample users, I have personally found used cases to be valuable for this purpose. 

What are examples of use cases for taxonomies and other metadata?

The following brief fictitious use case examples are of the kind that could be used for taxonomy development.

Internal organization use cases:

• A subject-matter-expert author who is required to tag authored documents with subject categories so that users can find documents by subject.
• A digital asset manager in an advertising agency, who needs to ensure that image files are assigned the proper copyright information.
• A content manager at a publishing company who, as a major responsibility, needs to assign full metadata to XML file content for various downstream purposes to assembly digital content products.
• A marketing copywriter seeking an expert on a specific subject among a company’s employees to give feedback on the accuracy of a blog post the copywriter is writing and who is inclined to browse subjects if available. 
• A manager who wants to find historical information on product offered in order to prepare a presentation about the product.
• A digital marketer who needs to update the public website with seasonal images that were not used last year (but two years ago is OK).

External/customer use cases:

• An undergraduate student who uses the default search to look for information on the events leading up to the fall of the Berlin Wall for a history class paper.
• An experienced online shopper who is searching to purchase carry-on luggage and wants to filter results by price, color, and positive reviews.
• A corporate librarian conducting competitive intelligence research on market strategies of leading competitor companies in the same industry and who would like to use advance and/or Boolean searching, if possible.
• A lawyer specialized in commercial law who need to find out where and how to file a financing statement in the proper jurisdiction for a client of his who to secure a loan, but lacks experience in legal research.
• A cancer patient searching for an oncologist with a certain type of cancer specialty, acceptance of certain insurance, within a certain geographic region, and with a number of good patient reviews.
• A compliance officer who needs to find regulations and associated policies and procedures that pertain to various departments and products lines of his employer, who knows the names of statutes but not the titles of associated regulations.

How are taxonomy use cases utilized?

In addition to serving the purposes of engaging stakeholders and ensuring the taxonomy is content- and user-focused, use cases can have additional specific applications, such as:

• Identifying or validating who all the different types of users are, so that their issues and feedback can be taken into consideration in the future.
• Suggesting improvements in the user interface design.
• Developing walk-through scenarios, with specific search criteria or topics of browsing spelled out, for offline testing of the taxonomy usability (including adequate depth and breadth) for both indexing/tagging and retrieval. (Read more at the post "Testing Taxonomies.")
• Providing scenarios that can be used in other taxonomy/knowledge management project research, such as ROI (return on investment) research.

Testing Taxonomies

As mentioned in my previous blogpost, “Evaluating Taxonomies,” taxonomy evaluation and taxonomy testing differ. While the evaluation of a taxonomy by a taxonomist is needed when a taxonomy is created by non-taxonomists (such as by subject-matter experts instead), testing of a taxonomy, on the other hand, is recommended in all cases, no matter who created the taxonomy. Following is an overview of the different kinds of testing that can or should be performed on a taxonomy prior to its implementation.

Card-Sorting

Card-sorting is probably the best known kind of testing, especially now that the prevalence of online card-sorting tools facilitates set-up and enables remote participation. It is not necessarily the best kind of testing for all situations, though. Card-sorting serves to test categorization schemes, so while it is suited for hierarchical taxonomies, it is not so appropriate for faceted taxonomies, especially with regard to how the facets are to interact with each other. It is possible, though, to card-sort test an individual facet, if that facet comprises an internal hierarchy of terms.

There are two kinds of card-sort tests, open and closed. In open card-sorts, the testers group concepts/topics together and then assign a broader category of their own; whereas in closed card sorts, the broad categories are already designated, and the testers merely categorize the specific concepts/topics within those pre-determined categories. Open card-sorting, if chosen, is therefore done earlier in the taxonomy design process, when broad categories are uncertain. A single taxonomy project may have either or both kinds of card-sorting depending on where the greatest need is for this additional input of information. Testers could be test end-users or they could be stakeholders, depending on the needs of the test.

Card-sorting is actually not really a kind of taxonomy testing but rather a form of taxonomy idea testing. Card-sorting is not performed on a completed taxonomy to test it but rather to test ideas of categories/hierarchies which later will be combined to create the taxonomy. Therefore, card-sorting is not an alternative to the other kinds of testing described below, which may subsequently be done.

Use Testing

Use-testing or use-case-testing is a necessary step after a draft taxonomy is built or nearly completed but before it is finally implemented, allowing for revisions to be made based on the test results. It is at this point that the taxonomy is put to the test to see if it will perform as hoped in search/retrieval and (if applicable) for manual tagging. This type of testing might also be called taxonomy validation.

A cross-section of different kinds of test users should be recruited to prepare several typical use cases and perhaps one especially challenging use case of content search scenarios. The user is then presented with the taxonomy (which can be in any format at this stage, whether on paper, as an Excel file or as test web page) and asked to browse the taxonomy to look for terms under which the content for the use search scenario might be found. The user performs the test, either browsing in the tester’s physical presence or via screensharing with verbal narration of what the user is doing and why.  The test administrator takes notes regarding any problems in finding taxonomy terms for the use case. These findability problems should be considered as requirements for additional terms, additional nonpreferred (variant) terms to point to existing terms, or perhaps more polyhierarchy or associative relationships to help guide the user to find the desired concepts.

If the taxonomy is to be used for manual tagging or indexing, then a second, different set of use testing is needed, whereby users who perform this function should test the taxonomy for indexing of typical and challenging documents that they tend to deal with. Rather than coming up with use “cases”, the test-user-indexers merely need to come up with actual documents. The documents should represent a good cross-section of the various document types indexed. This exercise is even more straightforward than the user testing for finding content, so it could even be performed offline without the test administrator present, as long as the test-user-indexer takes good notes.

A-B Testing

In A-B Testing, the test-users are presented with two different possible scenarios and asked which they prefer. When comparing two different taxonomies or parts of taxonomies, only one or two variations should exist between the two that are compared to make the test clear-cut. You may set up a series of A-B test pairs to compare multiple variations. This kind of test is comparable to what an optometrist does for vision: “Which is better, A or B?”  Since only one or two differences should be compared and tested at a time, A-B testing is most suitable to compare proposed top-level categories, rather than getting into the depths of a taxonomy, where it is not practical to conduct a detailed term-by-term comparison. Thus, A-B testing focuses on high-level structural design, navigation and browsing, and not the effectiveness of finding and retrieving content.

A-B Testing can be done at any time in the taxonomy design and build process. It is also very useful when considering a taxonomy redesign for comparing the existing taxonomy (A) to a proposed change (B). A-B Testing is usually done by presenting the test users with graphical or interactive web page mock-ups. I’ve created the B image to an existing online A image, by taking a screenshot of A and then edit it in Microsoft’s Paint accessory. Although each individual A-B test is simple, deciding what to compare and how many comparison tests to make needs to be determined, since each test takes time and resources.

Conclusions

Taxonomies should be tested, but it’s not true that any test is good. Different tests are for different purposes and fit into different stages of the taxonomy process. An inappropriate test or inappropriately timed test can be a waste of time and money.