Topics and Document Types in Taxonomies

It’s quite common in a faceted taxonomy to have a Document/Content Type facet (I’ll call DocType here), whose terms define what a content item “is,” (a report, a blogpost, a form, a contract, a letter, a policy, etc.) and also a Topic or Subject facet, whose terms describe what a content item is “about” (legal compliance, training, new business, insurance, company information, etc.) While usually it’s pretty clear-cut what belongs in the DocType facet and what belongs in the Topic facet, occasionally there are some ambiguous concepts, so asking the questions “what is it?” versus “what is it about?” helps in making the distinction.

Often the taxonomist can resolve ambiguity by editing the term so that a one-word generic document type is appended to a descriptive word. For example “Marketing” by itself is a Topic, but “Marketing Material” is a DocType. This kind of decision is reached only after looking at the set of documents and determining whether there is a significant number of them that are really marketing materials versus a significant number of them that are really about marketing (and there could be both). You then have to decide how far to go with this. You could force otherwise topical concepts into DocTypes by adding the word “Document” to the end of many terms. For example, “Compliance” becomes “Compliance Document”, and Client Management” becomes “Client Management Document.”  Depending on your overall content set and taxonomy design, this may or may not be acceptable practice.

Another complicating issue that may come up in designing such a faceted taxonomy is what to do if certain Topics only occur in certain types of documents. This is not unusual. While DocTypes such as Report, Evaluation, Meeting Minutes, Memo, Article, Review, etc., are rather generic and could all be associated with any number of the same shared set of Topics, other DocTypes that a customized for a specific content set are more limited in their application. For example, Topics for different types of approval to be used only with a DocType of “Approval Letter,” or Topics for types of product information to be used only with a DocType of “Product Information Sheet.”

There are two ways to handle this issue: 

1. Create rules permitting certain Topics available as options only when certain DocTypes are assigned
This requires that DocType be assigned (tagged, indexed, matched, etc.) to a content item first, before the Topic is assigned. This can be seen as: the Topic is dependent on the DocType, or DocTypes terms drive the Topics, or the DocType takes precedence over the Topic. This is feasible with these facets, since a content item can be assigned only on DocType (in contrast with the possibility of getting assigned more than one Topic). What gets complicated, though, if there are additional rules between other facets, with the terms in one facet driving the availability of terms in other facets, such as File Type, Source, Department, etc. 

2. Merge the DocType and Topic facet into a single facet
This may seem extreme, but it could be practical, especially if it’s easier for the end-user. It works if the there are not so many Topic terms, such as not many more than the total number of DocType terms, the majority of them are applicable to a single  DocType term, and a user interface can be designed that supports an expandable/collapsible hierarchy, so a user clicks on a DocType and the applicable Topics underneath it display. Traditionally taxonomies are hierarchical after all. If a Topic term is valid for more than on DocType, then a valid polyhierarchy results. There could still be a distinct facet for File Type/Format (such as HTML, text, image, PDF, etc.), for which there would be no ambiguity, in contrast to the occasional ambiguity between DocTypes and Topics. 

In either case—whether rules for the terms of one facet driving the availability of terms in another or whether a merged expandable hierarchical facet is created—collaboration is needed between the taxonomist and the technical experts who configure the implementation of taxonomy in the content/document management system.

Capitalization in Taxonomies

The question often comes up: what is the preferred style for the capitalization of taxonomy terms? Other than all proper nouns being capitalized, there is no strict rule for generic terms. In making the determination, it’s important to address the following questions. What kind of taxonomy is it? How will it be used? Who are the users, and what might they be accustomed to or expect?

The ANSI/NISO standard Z39.19-2005 Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies states: “predominantly lowercase terms should be used for terms in controlled vocabularies” and continues: “capitals should be used only for the initial letter of proper names, trade names and those components of taxonomic names, such as genus, which are conventionally capitalized.” But remember that ANSI/NISO Z39.19 comprises guidelines and not strict requirements, so the stylistic matter of case does not have to follow ANSI/NISO Z39.19, if a house style dictates otherwise.

Note that there are three options, not just two for non-proper nouns/names, as these explanations themselves illustrate:
1.    all lower case (including the first letter of the first word)
2.    First letter of first word upper case
3.    Title Case (First Letter of the First and Main Words Capitalized)

While the distinctions between “controlled vocabularies,” “thesauri,”, and hierarchical or faceted “taxonomies” can be blurred, these different types do tend to have different practices for capitalization.

A “controlled vocabulary,” as the word “vocabulary” might suggest, is a list of terms (as single words or phrases), similar to what might be found in a glossary, with the possible added feature of synonyms/variants for each preferred term. Capitalization, therefore, could be expected to follow dictionary rules and thus not used except for proper names. A “synonym ring” type of controlled vocabulary, in which no terms are designated as “preferred” and none are even displayed to the user, has no need for any capitalization.

A “thesaurus” is a more complex type of controlled vocabulary with hierarchical and/or associative relationships relating various terms to each other. What are called thesauri tend to be more term-focused than hierarchically focused, and they tend to be large with many detailed terms. The terms can be quite specific, and proper nouns can be mixed in. Thesauri have traditionally been used by indexers to manually index multiple documents consistently over time. The resulting display of terms associated with content for the end-user to browse through is a type of index. Indexes (such as those at the backs of book) often follow the style of lower-case entries for non-proper names, too. If the terms are numerous and specific, they will appear to be and used as “index terms” rather than “categories.” Thus, if it’s called a thesaurus, it will more likely have terms in lower case. The choice of initial capitalization for a thesaurus, though, would not be incorrect, and is probably becoming more common, just as initial capitalization is becoming more common in main entries in back-of-the-book indexes.

A “taxonomy” implies a hierarchical classification or categorization of concepts. When we think of categories we think of labels or headings with subcategories. Headings in general tend to have initial capitalization or title capitalization. Thus, if it’s a strictly hierarchical taxonomy, where all terms are interconnected into a single hierarchy or a limited number of hierarchies, then it will more likely have initial capitalization or title capitalization. Such capitalization is particularly common on the relatively smaller/less detailed taxonomies that are proliferating on websites, intranets, and content management systems. It fits in with the web design style of capitalization on headings and categories.

In faceted taxonomies, which have become more popular in web/online taxonomies, proper names can be separated into their own facet(s), and confusion between proper names and generic terms is reduced. However, I would still recommend only the first letter of the first work capitalized, rather than title case, to minimize any confusion with proper names. The facet name itself, however, could be it title capitalization, since it represents a category heading and not a term for indexing. In fact, it might even be desirable to distinguish the facet labels from the values/terms within each facet by use of a different case style.

A mixed style of different capitalization at different levels is possible in hierarchical taxonomies, too. But I would recommend only the top terms, if any, have a different capitalization style. It would not be a good idea to have only the bottom level terms (“leaf nodes”) in a different case style, because they could change. If you decided that a leaf node should later have narrower terms added, you wouldn't want to have to worry about changing the case of the term. A good application of the mixed capitalization style is if the top level terms were not actually to be used in indexing/tagging but are really just categories/groupings of the actual index terms, which in-turn are arranged hierarchically underneath. (Other typographical methods of distinction could also be used for any non-indexible top-level categories.)

In sum, all-lower case is most appropriate for non-displayed controlled vocabularies, any controlled vocabularies or thesauri that integrate proper nouns into the same hierarchies as generic terms, and large thesauri used to support manual indexing. Initial capitalization is fine for end-user browsable hierarchical taxonomies on the web. Title capitalization is OK for facet labels or the top categories in a hierarchical taxonomy. Whichever style is chosen, however, should be applied consistently.

Taxonomies vs. Classification

A question had come up in one of my classes on how classification differs from taxonomies/thesauri. As part of an assignment to find thesauri on the web a student sought to find “how the Federal Government classifies its publications and was expecting to find a very elaborate Thesaurus … and instead found… the Superintendent of Documents classification system,” and so the student asked how that classification system fits into the scheme of definitions for taxonomies, controlled vocabularies, and thesauri. That I will attempt to explain here.

We are familiar with classification schemes used to catalog and locate books and other materials in libraries, such as the Dewy Decimal system or, for academic libraries, the Library of Congress Classification (letter-based call “numbers”). In addition to the U.S. federal government’s “Superintendent of Documents” classification system, many other national governments an international organizations also have their own document classification schemes, and states and provinces may have modified versions. There are also classification systems for industries, such as the NAICS (North American Industrial Classification System) codes. Corporations with large volumes of documents may have their own internal document classification systems.

I sum up the differences between classification schemes and taxonomies/thesauri as follows:

Classification:

• used for books, monographs, documents, reports, contracts, or other media
• developed for the classification of physical items for their location on shelves, drawers, or filing cabinets and physical file folders
• based on alpha-numeric codes
• involves assigning an item only one classification code
• manually assigned to each item
• classification codes may include additional information, such as date, title, author, or publishing department information within the same classification code
• rarely gets changed (due to the pre-established numeric code hierarchy)
• helps document managers and librarians organize documents and helps users locate pre-identified documents and materials

Taxonomy/Controlled Vocabulary/thesauri:

• used for articles, images, electronic files, paragraphs or sections of text if separated out as digital content units
• used primarily in online/digital space
• based on descriptive words and phrases (terms). Codes, if any, are secondary.
• involves assigning an item multiple taxonomy terms
• manually or automatically (auto-tagging, auto-classification, etc.) assigned to content items
• taxonomy terms restricted to subject information (not to include date, title, author, publishing department, etc.)
• can easily be revised and updated
• helps users identify which content items they want

Another way to think of the comparison:
Classification is for: where to put things/where does this document or item go.
Taxonomy is for: how to describe content/what is this text, image, or other media about.

So, while both classification and taxonomy are related and are within the realm of information science, they are really quite different. Since they serve different purposes, they can actually co-exist and both be applied to the same corpus of documents. Libraries utilize both at the same time: a classification system (the Dewy Decimal or Library of Congress Classification call numbers on books and media) and a form of a taxonomy in the catalog subject headings (usually Library of Congress Subject Headings, which are not to be confused with Library of Congress Classification).

Taxonomy and classification may each involve different people, too: catalogers for classification and taxonomists for taxonomies. While some information professionals may do both, you cannot assume that all catalogers know how to create taxonomies or that all taxonomists understand classification. There is, of course, a larger and growing need for taxonomies, in contrast to classification and cataloging systems, as more content migrates online. Furthermore, taxonomies are more adaptable to change and thus in need of continual maintenance, in comparison to the rather static classification systems. Many catalogers are taking an interest in learning about taxonomies these days.

Taxonomists who understand something about classification can also put that knowledge to use. There are many large corporations and agencies with documents organization by customized classification systems, which are now migrating over to dynamic online content/document management and taxonomies. The legacy classification systems then need to re-formed into (or replaced by) taxonomies, and then the legacy codes need to be mapped to the new taxonomy terms to ensure the continual retrieval of legacy documents. I did this kind of work as a consulting project for a large financial institution not long ago. There were thousands of legacy alpha-numeric codes, most of which combined both a document type attribute and a subject matter attribute into a single code, a typical feature of classification codes when a document can get only one code. A taxonomy, on the other hand may have one facet for document type and another facet for subject, and a document can be assigned multiple subject taxonomy terms in addition to the document type term.

As long as there are physical books, documents, and media, there is a need for classification, but if the entire content repository is digital, then taxonomies are the way to go.

Testing Taxonomies

As mentioned in my previous blogpost, “Evaluating Taxonomies,” taxonomy evaluation and taxonomy testing differ. While the evaluation of a taxonomy by a taxonomist is needed when a taxonomy is created by non-taxonomists (such as by subject-matter experts instead), testing of a taxonomy, on the other hand, is recommended in all cases, no matter who created the taxonomy. Following is an overview of the different kinds of testing that can or should be performed on a taxonomy prior to its implementation.

Card-Sorting

Card-sorting is probably the best known kind of testing, especially now that the prevalence of online card-sorting tools facilitates set-up and enables remote participation. It is not necessarily the best kind of testing for all situations, though. Card-sorting serves to test categorization schemes, so while it is suited for hierarchical taxonomies, it is not so appropriate for faceted taxonomies, especially with regard to how the facets are to interact with each other. It is possible, though, to card-sort test an individual facet, if that facet comprises an internal hierarchy of terms.

There are two kinds of card-sort tests, open and closed. In open card-sorts, the testers group concepts/topics together and then assign a broader category of their own; whereas in closed card sorts, the broad categories are already designated, and the testers merely categorize the specific concepts/topics within those pre-determined categories. Open card-sorting, if chosen, is therefore done earlier in the taxonomy design process, when broad categories are uncertain. A single taxonomy project may have either or both kinds of card-sorting depending on where the greatest need is for this additional input of information. Testers could be test end-users or they could be stakeholders, depending on the needs of the test.

Card-sorting is actually not really a kind of taxonomy testing but rather a form of taxonomy idea testing. Card-sorting is not performed on a completed taxonomy to test it but rather to test ideas of categories/hierarchies which later will be combined to create the taxonomy. Therefore, card-sorting is not an alternative to the other kinds of testing described below, which may subsequently be done.

Use Testing

Use-testing or use-case-testing is a necessary step after a draft taxonomy is built or nearly completed but before it is finally implemented, allowing for revisions to be made based on the test results. It is at this point that the taxonomy is put to the test to see if it will perform as hoped in search/retrieval and (if applicable) for manual tagging. This type of testing might also be called taxonomy validation.

A cross-section of different kinds of test users should be recruited to prepare several typical use cases and perhaps one especially challenging use case of content search scenarios. The user is then presented with the taxonomy (which can be in any format at this stage, whether on paper, as an Excel file or as test web page) and asked to browse the taxonomy to look for terms under which the content for the use search scenario might be found. The user performs the test, either browsing in the tester’s physical presence or via screensharing with verbal narration of what the user is doing and why.  The test administrator takes notes regarding any problems in finding taxonomy terms for the use case. These findability problems should be considered as requirements for additional terms, additional nonpreferred (variant) terms to point to existing terms, or perhaps more polyhierarchy or associative relationships to help guide the user to find the desired concepts.

If the taxonomy is to be used for manual tagging or indexing, then a second, different set of use testing is needed, whereby users who perform this function should test the taxonomy for indexing of typical and challenging documents that they tend to deal with. Rather than coming up with use “cases”, the test-user-indexers merely need to come up with actual documents. The documents should represent a good cross-section of the various document types indexed. This exercise is even more straightforward than the user testing for finding content, so it could even be performed offline without the test administrator present, as long as the test-user-indexer takes good notes.

A-B Testing

In A-B Testing, the test-users are presented with two different possible scenarios and asked which they prefer. When comparing two different taxonomies or parts of taxonomies, only one or two variations should exist between the two that are compared to make the test clear-cut. You may set up a series of A-B test pairs to compare multiple variations. This kind of test is comparable to what an optometrist does for vision: “Which is better, A or B?”  Since only one or two differences should be compared and tested at a time, A-B testing is most suitable to compare proposed top-level categories, rather than getting into the depths of a taxonomy, where it is not practical to conduct a detailed term-by-term comparison. Thus, A-B testing focuses on high-level structural design, navigation and browsing, and not the effectiveness of finding and retrieving content.

A-B Testing can be done at any time in the taxonomy design and build process. It is also very useful when considering a taxonomy redesign for comparing the existing taxonomy (A) to a proposed change (B). A-B Testing is usually done by presenting the test users with graphical or interactive web page mock-ups. I’ve created the B image to an existing online A image, by taking a screenshot of A and then edit it in Microsoft’s Paint accessory. Although each individual A-B test is simple, deciding what to compare and how many comparison tests to make needs to be determined, since each test takes time and resources.

Conclusions

Taxonomies should be tested, but it’s not true that any test is good. Different tests are for different purposes and fit into different stages of the taxonomy process. An inappropriate test or inappropriately timed test can be a waste of time and money.

Evaluating Taxonomies

In my last blog post, “Taxonomy Management Consulting,” I mentioned that more organizations now have taxonomies, so the need is shifting somewhat from designing and building new taxonomies to managing existing taxonomies. It might not be that simple, however, if the existing taxonomy was created and never used, created for a slightly different purpose or different content, or created by those not sufficiently knowledgeable in taxonomy design best practices.  I often find that an organization that has taxonomy consulting needs typically has some pre-existing taxonomies, but they are not adequate for one reason or another.

Any pre-existing taxonomies are important as part of a taxonomy development or redesign process and should be carefully considered. Whether pre-existing taxonomies will be only a source of terms for a new taxonomy or actually the basis of a new taxonomy with some editing depends on how structured, comprehensive, and sound these pre-existing taxonomies are.

• Structure: Pre-existing taxonomies may be of the type that is a simple flat list of terms with no hierarchy.  These are good sources of taxonomy terms but are rarely the basis for the taxonomy.

• Comprehensiveness: Often existing taxonomies cover only part of the scope of a desired full or enterprise-wide taxonomy,  in which case they will serve as part of the new taxonomy. 

• Soundness: This concerns to what extent the taxonomy is conforms to standards (such as ANSI/NISO Z39.19) and general best practices, so that it ought to work well with the content it is intending to reference. This is where taxonomy experts can come in and make such  determinations.

Evaluation Criteria 

Evaluating a taxonomy for soundness typically involves checking off or rating the taxonomy against a set of pre-defined criteria regarding terms, inter-term relationships, and overall structure and design. Some of the most important criteria include the following:

• Terms should be unambiguous and clear, yet not too wordy and long. If the taxonomy will be displayed for browsing, then terms should begin with key words and those that come under the same broader term should be in a somewhat consistent grammatical format. 
• Hierarchical relationships should conform to the ANSI/NISO Z39.19 standards of conforming to only one of the three types: generic-specific, instance, or whole-part, with perhaps limited exceptions in a corporate taxonomy that are intuitively logical and justified.  (See my blog post “Deviating from Taxonomy Standards”).
• Overall structure and design involves issues include the number of narrower terms for a broader term not being too few nor too many (such as 3-20), and the depth of the taxonomy being somewhat balanced and not too deep. For example, three levels deep in some places and four levels deep in others is OK, but two levels in some areas and five levels deep in others is not a well-balanced design.

Evaluation vs. Testing

Evaluating a taxonomy is not the same as testing a taxonomy. Testing a taxonomy involves using sample content and sample users in a controlled manner and can take considerable time and effort, so should not be done until after a taxonomy is determined to be generally sound. Evaluating a taxonomy, on the other hand, is to determine if it’s well constructed regardless of the content or users. Testing focuses on the specific application and use of the taxonomy and will be the topic of a future blogpost.

Taxonomy vs. Web Usability Heuristic Evaluation

Even if a numeric rating scale is used, the process is still more judgmental than scientific, and as such may be referred to as a “heuristic” analysis or evaluation. A “heuristic method” generally means evaluation, experimentation, or a trial-and-error method to find something out. The designation of heuristic evaluation has been used in website usability evaluation and from there has been carried over into taxonomy evaluation. User experience expert Jakob Nielsen first introduced the idea of heuristic evaluation to usability design back in 1990, described in his blogpost of 1995: “How to Conduct a Heuristic Evaluation.”

There are several differences, though, between taxonomy evaluation and web user interface evaluation. Although user testing of websites is not that much different from the testing of taxonomies, evaluation of taxonomies requires a more critical and analytical understanding and approach. Website usability evaluation does not require usability design experts, but taxonomy evaluation does require a level of expertise. Nielsen refers to “evaluators”, not experts, who are not much different from user testers. (Rather, the procedures in usability evaluating and testing differ.)

Another difference between website evaluation and taxonomy evaluation is that a website, even if a test dummy site, will have content, even if just mock-up pages with partial filler text, because navigation and content are integrally combined on websites. When a taxonomy, on the other hand, is at the evaluation stage, it is not implemented/linked to content, which makes it more difficult for the non-expert to evaluate. It might appear to look good on paper but not function well when implemented.

Nielson wrote: “Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles.” If the evaluators are not experts, then it’s easier and more affordable to have multiple evaluators. When a taxonomy requires evaluation, typically just one taxonomy expert is hired, but if you can afford two separate independent expert evaluations of your taxonomy, that’s all the better.