With only a cursory look at the literature pertaining to the evaluation of ontologies, I already get the feeling that the current measures completely miss the point. The answer doesn't lie in the syntax (format) or structure of the ontology (the number of classes and properties, subsumed classes, axioms, etc), but rather the
effectiveness of an ontology ( a representation of knowledge ) is in whether the semantics can be used for some task. So what we really want, is to focus on the nature of the task, and whether ontology provides a competitive advantage over other technologies.
Off the top of my head, here are some tasks:
- search/browse (most sites using GO, etc)
- text annotation (gopubmed)
- data normalization and structured queries - bio2rdf
- answering questions that require background knowledge e.g. across a yeast database.
- data integration (heterogeneous types of data; data
from different sources, of differing granular detail) (see my translational medicine paper)
- classification e.g. domains or chemicals
- prediction e.g. predicting phenotypes
Perhaps others can suggest some?
