James Malone reflects on a panel discussion on evaluation and reuse of ontologies. He wants there to be a “a formal, objective and quantifiable process” for “making public judgements on ontologies”. Towards that, he suggests that we need:
- A formal set of engineering principles for systematic, disciplined, quantifiable approach to the design, development, operation, and maintenance of ontologies
- The use of test driven development, in particular using sets of (if appropriate, user collected) competency questions which an ontology guarantees to answer, with examples of those answers – think of this as similar to unit testing
- Cost benefit analysis for adopting frameworks such as upper ontologies, this includes aspects such as cost of training for use in development, cost to end users in understanding ontologies built using such frameworks, cost benefits measured as per metrics such as those above (e.g. answering competency questions) and risk of adoption (such as significant changes or longer term support).
– James Malone, in Why choosing ontologies should not be like choosing Pepsi or Coke, about his International Conference on Biomedical Ontology panel ‘How to deal with sectarianism in biomedical ontology.