When starting a project to create a new taxonomy or expand an existing taxonomy, there is always the question of how large, and thus how detailed, the taxonomy should be. This is one of the first questions I ask a prospective client, but often they are not sure and ask for guidance on this question. A taxonomy should be designed to suit its intended tagged content and its users. So, the size of the taxonomy should also consider both of these aspects.
Taxonomy Covering Growing Content
On the content side, a larger volume of content, such as more articles or pages, requires a larger taxonomy. A more diverse range of subjects or products also requires a larger taxonomy. Earlier in my career, I worked on the controlled vocabularies of a library reference publisher, in which the number of articles tagged (“indexed”) and the subject area scope was huge, and thus the subject thesaurus had over 60,000 terms. Corporate taxonomy implementations, on the other hand, are much smaller, such as only the pages for the website, the intranet, or a content management system, and the scope is much smaller. For consulting projects, I have usually created taxonomies of only 100-400 terms.
However, the volume of digital content is increasing, and there is also a growing trend toward enterprise taxonomies to cover all the content in nearly all the systems of an organizations. So, corporate taxonomies need to be larger than in the past. An example is customer support articles accessed by customer support agents. Products and services and their features and the potential issues with them have grown and so has the collection of support articles, which in some cases has grown from hundreds to thousands. Another area of content growth is the size of a corporate intranet. My current consulting project involves an intranet with over 20,000 pages and linked documents. Text analytics or Generative AI technologies may assist with the analysis of this much content, but the actual creation of a suitable taxonomy with alternative labels (“synonyms”) within a limited time is a challenge.
Taxonomy Covering Evolving Uses and Implementations
On the user side, the type of front-end user interface implementation of a taxonomy plays an important role in determining the taxonomy size. If it’s fully displayed hierarchical browsable taxonomy, the taxonomy should not be too large. It’s recommended to have only three levels of hierarchy and no more terms per level than can be displayed in a single view. Hierarchical browsable taxonomies are not as popular now as in the past, though.
Faceted taxonomies have been very common over the past 20 years. The user selects terms from multiple facets (Topic, Location, Organization, Activity, Product/Service, etc.) in combination. This kind of taxonomy implementation does not require a large taxonomy, because multiple terms, each from different facets, are tagged to content items and are used in combination for retrieval. Furthermore, facets are often used to further refine a result set retrieved after executing a search. A typical faceted taxonomy may have 4-6 facets with the number of terms per facet in the tens, and its largest facets having not more than 100 terms. Although a faceted taxonomy is relatively small, it still takes time to create properly.
If a taxonomy is searched via a search box, with user-entered search strings matched against the terms, including their variants (“synonyms”), rather than browsed, it can be quite large, as large as needed to cover the content.
Now there are new methods of information retrieval that do not involve user browsing or searching. Generative AI and LLMs that are integrated into information retrieval user interfaces allow users to simply ask questions. Taxonomies are increasingly involved in such systems, whether supporting GraphRAG, AI agents, or simply to improve content metadata and make the content more “AI-ready.”
The size of taxonomies needed for such AI systems vary, but they tend to be larger than traditional taxonomies which are viewed by the users. The taxonomy doesn’t need to fit into a user interface. Instead, terms may need to be created based on parsing the words in hundreds of different anticipated user queries. The taxonomy terms would be more specific and more numerous than the grouping categories of traditional taxonomies. Furthermore, AI systems are sometime implemented instead of (or as an alternative to) search, rather than simply supplementing full-text search, as traditional taxonomies do. This means that if the term is not in the taxonomy, the content won’t be retrieved. So, a larger taxonomy is better.
Conclusions
Differences in the content and in the end-use implementations impact the size of the taxonomy. With multiple factors, though, there is some guess work in deciding how detailed a taxonomy should be. Thus, in all cases of taxonomy development, testing with search/query use case scenarios should be done to see if the taxonomy is sufficiently large and detailed.
No comments:
Post a Comment