TermFactory Manual

Introduction

This document describes the TermFactory approach and architecture. With suitable styling, it serves as a white paper, a status report, a history, and a manual. Of the styles listed above, white is for those who just want to read about TermFactory, black is for TF users, small is for administrators, and all is for developers. Style 'talk' shows selected parts of the document as a slideshow.

A model implementation of a TermFactory community platform is being built and will be tried out by an evaluation community. However, we try to keep the design as free of platform specific detail as possible. The main research focus is on making Semantic Web technology useful for global multilingual terminology work.

As for workflow, TF tries out these ideas in an experimental terminology network, and by so doing devises a guide of best practices as to how the collaborative distributed work pattern can be made to produce useful results with the least hassle. There are many practical questions to solve, not only technical ones but ones having to do with community building, access rights, authentication, division of labor, organisation and timing of different phases of work, allocation of authority, intellectual property rights, etc. Many of these issues it shares with other collaborative work processes, so TF does not have to invent everything, but rather survey and choose suitable solutions.

One is perhaps tempted to ask, what is new here, what is the gain from TF? There have been a lot of terminology data storage solutions before. There are more efficient uses of relational databases than current persistent ontology repositories. There is a lot of standards and tools for terminological data management, including XML based ones like TBX. There are old and well-tried query and reasoning tools, including SQL and Prolog. What is there to gain from using another burgeoning technology instead of the old tricks? Is it just the novelty value?

The expected benefits of as compared to the current state of the art terminology tools include

Ontology tools are not about runtime efficiency in (say) online web applications. They save human work and make possible the management of much larger collections of terminological data than before. Ontology work happens in the background.

The TF architecture

The TF architecture is not embodied in one software package, but an array of standards and software, both web and standalone tools, that allow different actors in different roles to collaborate to produce a shared resource. This shared resource is a distributed multi-domain special language ontology of multilingual terminology. It can be used to organise content and standardise communication in global multilingual organisations, enhance exchange of ideas and innovations across a multinational workforce, or facilitate understanding and support education across language barriers. The core of the system is formed by a hierarchy of Semantic Web ontologies served by a network of interconnected repositories. The human interfaces to the network are composed of online web collaborative work platforms and offline professional terminology and ontology tools.

TermFactory philosophy

From a philosophical perspective, all communication, including multilingual communication, involves translation, that is, conversion from one format to another. TF is specifically about interlingual translation of terms: internationalization of local content to make it globally shared, and localisation of globally shared content back home.

A familiar notion that helps explain TermFactory philosophy was explained in an old paper of mine (Questions of Identity in Discourse, 1989). Consider how we choose names or addresses we use to identify persons (places) depending on the size of the field of search. In the middle of a dialogue, we point or use pronouns. In talk among family and friends, we use first names and nicknames. Within a wider circle, full names are used. In official contexts, some ID is required. When there are none, a description is offered. In each context we use the name (address) that is most convenient among those that sufficiently identify the object. This also covers naming across cultures and languages: we switch to another code that works best in that context. Though TF primarily deals with multilingual terms, it is really about globalisation and localisation of names of any kind.

From this point of view, the simple answer to what TF TermFactory is about is this: it is a tool for facilitating choice of names and addresses to suit with changes of context. Starting with a local ordinary name for a thing in some language, it allows narrowing down the intended meaning to a special domain concept that has a global identifier, a URI. This global concept is in turn localized to terms and expressions in another language or culture. The facilities offered by TermFactory for interlingual intercultural communication conform to classical terminology theory, but are not confined to terminology only. Considered as an abstract machine, TermFactory creates a complex of string to string mappings, mediated by a semantic network expressed in description logic.

In a more mathematical vein: two category theoretic insights underlie much of TF.

1. Translations are mappings, morphisms in a category of languages. Globalisation, or interlingual translation of terms through the composition of internationalization followed by localization (i18n o L10n), uses the interlingua of language independent concepts as a category theoretic limit (universal element) of that category.
2. Universal elements are good for space efficiency and manipulation. Limits in a category use one thing to represent many, taking out the slack from redundant mappings.

Space (thereby maintenance) efficiency is the motivation for having global names in the first place. That is also the ultimate motivation for terminology theory for distinguishing concepts from terms. The TF distinction between terms and expressions is its dual, motivated by multilinguality.

All tasks in TF involve format conversion, and many tasks involve globalisation. URIs are global. Expressions in a (sub)language are local. TF tries to make them global, by giving them uris. At the same time, it goes to no end of trouble to make that global content locally usable. It is efficient and convenient in a given context to name anything with the shortest name that is unique in it. When the context is large, the names get longer. TF aims to make both optimisations possible at once: to globalize, and to localize at will.

Globality is the motivation of the Semantic Web. Its weakness is the downside of globalisation, namely loss of local (time) efficiency. Going through limit is space/resource efficient, like a tree is resource efficient compared to a graph, but it is time consuming for the same reason. Having direct routes is faster if the cost is no object. In the web, fast traffic is peer to peer. In the short run, nobody has the patience to look for long uris, and nobody cares to check if a given local thing already exists globally. Result: a lot of duplication, proliferation of different URIs for the same thing. The homonym problem globalisation sets out to solve is replaced by an equally untractable synonym problem. In a word, ontology hell .

The TermFactory architecture is meant to address this problem. (It cannot solve the problem at large, just think of the scale. It is enough if TF helps tame it in smaller circles.) The solution is something like this: before inventing another global concept associated to a local name, look up in TF what there is already under that local name, and borrow or subclass one of them. If not, share your innovation.

It is important to observe that there is relatively little in the mechanism of TF that is specifically hardcoded about multilingual terms. That is all in the data (the ontologies). Mostly, TF is a bunch of Semantic Web mechanisms to do mappings between strings using description logic based techniques.

Ontologies

An ontology, in the current Semantic Web sense of the word, is a collection of semantic descriptions of concepts in a formal language explicit enough to be processed by a machine.

Ontologies are a direct descendants of the semantic networks of 70's artificial intelligence. AI systems used semantic networks to describe the world to a machine so that the machine could behave intelligently. There was nothing wrong with the idea in itself, it just turned out to be unfeasible to describe enough of the world to make the machines behave intelligently enough, and the AI hype died out.

The idea got a new lease of life by the (already somewhat subdued) hype surrounding the Semantic Web, a new generation Internet where intelligent machines are able to understand and process information produced by and for people. It is supposed to upgrade the current human-interactive Web 2.0 to a human-machine interactive Web 3.0 .

At least for the time being, machines are worse than people at guessing meaning (IBM:s success in TV trivia notwithstanding). To make information accessible to them, people have to be more explicit about what they mean than they have to when communicating to other humans. Specifically, meanings must be annotated, marked up using some metalanguage in web documents for machines to read.

The first step in the 90's toward man-machine readability was XML, the eXtensible Markup Language to make document structure explicit to people and machines. XML gives documents a treelike structure but no particular semantics; the meaning, if any, is up to the user to provide. The ontology languages want to also fix meaning.

The base language for the Semantic Web is the Resource Description Framework language RDF. It is a language for constructing semantic networks as labeled graphs, not just trees like XML. RDF is actually independent of XML, although RDF graphs are by default written in XML. The native "language" of RDF is statements in the form of subject-predicate-object triples, for instance (1)

ont1:Fido rdf:type ont:Dog . ont:Dog rdfs:subClassOf ont:Pet . ont1:Sue ont:owns ont1:Fido .

saying that Fido is a dog, dogs are pets, and Sue owns Fido. RDF graphs are semantic networks built out of such triples.

The web being global, RDF allows identifying the concepts that appear in the triples with global identifiers, called, depending of variety, URLs (universal resource locators), URIs (universal resource identifiers), IRIs (international resource identifiers) or URNs (universal resource names). For instance, the dog's name ont1:Fido above is an abbreviation for the full TermFactory name of a particular globally unique dog, http://www.tfs.cc/ont1/Fido . There are other Fidos in the world, but only one by this name. Global identifiers is one of the main advantages of RDF that make TermFactory possible. Using global identifiers (URLs, URIs, IRIs or URNs), concepts can be identified uniquely and traced back to their owners. In the above example, the meaning of the predicates rdf:type and rdf:subClassOf is fixed (and explained) by the W3 consortium. The rest of the named entities are owned and documented by TermFactory. Another site might have their own Sue and Fido or a different concept of ownership, identified by different URIs.

The more semantics an ontology language has, the more meaning it can express and the fewer statements it needs to express it. But then all the consequences are no longer explicitly stated, and to unpack what is only implied, an inference engine, or reasoner, is needed. For instance, from (1) a reasoner can infer that Fido is a pet, or that Sue owns a pet, though these facts are not explicitly stated in the ontology. The richer the language, the more it can express, but the harder its inference problem, so the trick in defining an ontology language is to find a useful balance between expressive power and tractability.

RDF

RDF has a graph semantics. An RDF model can be visualized as a binary graph made up of labeled arcs.

Show/hide RDF graph

Such an RDF graph consists of triples each consisting of two nodes connected by a binary labeled arc. The nodes can be labeled or blank (anonymous). An RDF graph need not be connected. RDF graphs can be named. RDF nodes are either resources or literals. The arcs are called properties. Literals, for instance strings, numbers, or dates, have a label, but no URI and no properties. Resources may have a URI and properties. (They need not have either.) A resource with URI is a named resource, one without is an anonymous resource, alias blank (node). Blank nodes have fixed identity only inside the containing graph. Constant RDF triples (those without anonymous nodes) are completely self-sustained pieces of information that can be bartered between documents and ontologies. This is one of the strengths of RDF. A RDF model is an implementation level concept for a container holding one or more RDF graphs.

RDF has a standard way to assign properties to a triple by way of reification: a quad(ruple) of triples names a triple and associates to the name its predicate, subject, and object. Reification is not much used in practice.

There are a variety of formats to represent a RDF graph as text. Some are described in the section on TF formats further on.

RDFS

RDF Schema (RDFS) is a meta vocabulary for defining RDF vocabularies. It allows typing resources into classes and subclasses, and describing RDF properties.

An RDF property is a URI (named resource) that represents a binary relation viewed object-orientedly, as a property of members of its domain, taking values from a range. RDF properties are many-valued. RDF properties can be partially ordered with respect to generality, using the second order property rdfs:subPropertyOf . RDFS semantics says that properties rdfs:subClassOf and rdfs:subPropertyOf are transitive. A RDF query for a property should also return triples labeled with its subproperties.

OWL

The Web Ontology Language (OWL) is a family of knowledge representation languages for authoring ontologies, and is endorsed by the World Wide Web Consortium. OWL is considered one of the fundamental technologies underpinning the Semantic Web.

The Web Ontology Language OWL is a subset of classical predicate logic designed to express the kind of statements used by people in defining concepts since Aristotle. The most popular subset of OWL, the description logic fragment known as OWL DL, can express concepts and relationships like these: (translation to Manchester OWL syntax in grey)

The main forte of OWL is that it not only expresses those things (for the human reader), but machines can actually reason with them. Description logic engines like Fact++ or Pellet are able to relate such facts to one another automatically and give intelligent, that is, reasoned answers to questions about them, combining facts and drawing consequences that are only implicit in them. That is more than the average relational database does, maybe more than the average person is wont to do.

Instances, classes, properties and roles

As a species of description logic, OWL is a decidable subset of first order predicate logic with an object-oriented syntax. As any first order logic, it makes a sortal distinction between instances, or "things out there" and classes of such things. OWL goes past simple classification since it can talk about two-place relations. A binary relation is viewed object-orientedly (or should one say egocentrically) as a property of its subject, having the object as value. Nevertheless, such properties are by default multivalued, and they can have inverses, like parent and child. Most characteristically, OWL deals with roles. Roles are classes one belongs to in virtue of having a property (that is, a relation to something). Most of our apparent inherent properties are at depth roles in some bigger configuration. Something is big only in relation to something small, or a master only as the master of a slave. OWL cannot say very complicated things, but it is good at defining roles. And roles are a good approximation to the classical Aristotelian way of forming terminological definitions: "A species is that subclass of a genus which has the differentiae that...". To give the same point a linguistic turn: OWL has a good match with the natural language syntax of relational nouns and relative clauses traditionally used in terminological definitions.

The TermFactory ontology schema

This section describes the TermFactory ontology schema TFS.owl . The TermFactory ontology schema defines the skeleton of a multilingual terminological ontology in OWL DL, trying to conform to current terminology standards and other language technology standards , rich and precise enough to support semantic inference and language technology applications.

One of the liberating insights from describing terminology in a logic rather than by a fixed schema - whether hierarchical (tree structured) as XML or as tabular (record structured) like a relational database - is that there is nothing sacrosanct about the TFS schema vocabulary. Not only has TermFactory got a fixed entry format, there is no unique fixed signature either. As long as there are ontologies and reasoning, there does not have to be a fixed meta vocabulary for terms. A seed vocabulary like TFS.owl and its companions is there only to help thinking and provide a fixed point for automated reasoning. Beyond that, everyone is free to use their own vocabulary and conceptualization, provided an ontology is provided for it, plus bridge rules to map it to others, and eventually to TFS.owl in particular.

Like classical logic, OWL has an open world assumption built in. There is no way in first order logic to quantify over properties, hence to enumerate all and only the properties that a class allows to its members. As a consequence, an OWL schema can only name individual properties that must or must not be present. In this respect a RDF/OWL schema differs from a database or XML schema. which work with a closed world assumption (whatever is not specifically allowed is prohibited).

Unlike XML, there is no formal distinction in OWL between an ontology and an ontology schema. One just draws a line somewhere between statements that must hold in all TermFactory ontologies and the rest. The common core is called the TF ontology schema (TFS.owl). Together with its extensions (TFStrict.owl, TFTop.owl, TFSem.owl) and profiles (LegacyStrict.owl, DictionaryStrict.owl) it forms the top ontology in TF.

Like classical logic, OWL is weakly typed. It is possible, but not obligatory, to specify individual type, superclass of class, and domain and range of object properties. The default is the vacuous type owl:Thing . This weakness can be a strength. With strict typing, it can be necessary to create dummy objects just to satisfy typing requirements. In TF, it is rarely necessary. Therefore TF can accommodate a variety of approaches to terminological description. (See section on term properties below.)

The OWL standard in itself is about semantics, or concepts in abstraction from the (human) language in which they may be expressed. Each concept must have a global identifier to hang descriptions on, but as long as it is a valid URI, it matters little what that identifier is. Beyond that, the standard has little to say about language or multilinguality. Basically, the standard suggests using attribute rdfs:label with values in localised language-tagged Unicode strings to specify alternative human readable labels for classes. This is not nearly enough for proper multilingual language (technology) support. The main claim to fame for TF over and above OWL alone is that TF defines an explicit OWL ontology of human language expressions and terms that allows full control of the language associated to concepts.

Show/hide TF terms

TF top ontology

A thing to keep in mind is that an OWL ontology is not (just, or even primarily) a taxonomy in the classical Aristotelean or terminological sense of a tree of genera and species. It is a semantic network, a collection of nodes and arcs representing objects and binary relations between them. As just a special case of that structure, a set of distinguished binary relations including rdf:type , rdfs:subclassOf , owl:sameAs , owl:differentFrom and owl:disjointWith gives the graph a first order monadic class structure, like Boolean algebra or Venn diagrams. That structure allows many competing taxonomies as trees spanning the graph, none the more or less true than the others. A taxonomy can be thought of as a tree-formed index to the Boolean class structure, serving the purposes of efficient insertion and retrieval of instances.

A two-dimensional tree representation of an OWL ontology is misleading, if it suggests that the subclasses branching out from a class as its subclasses are disjoint. In general they aren't. Logically, the rdf:subClassOf relation is a partial order, not a tree. A class can have many supers in the same "tree" (directed acyclic graph). Thus an OWL ontology tree can have as sisters classes that at best belong to parallel cross-classifying taxononomies. This happens when there are orthogonal bases of classification, like classification by structure (parts) and by function (roles). In standard terminography, there is a special notation for such many-dimensional cross-classifying taxonomies (see e.g. ISO FDIS 704 example 8 ). Unfortunately OWL editors do not support it.

There are two instances of such cross-cuts at the top of the TF ontology. The tripartition sign:Sign-syn:Form-sem:Meaning is cross-cut by the class term:Description . TF descriptions can belong to any of the other three classes. sem:Meaning is subdivided to object-like ( ont:Concept ) and proposition-like ( ont:Content ) meanings on the one hand, and to count sem:Count and noncount sem:NonCount entities on the other hand. These two dichotomies cross-classify to form a fourfield.

What the very top of an ontology looks like is largely philosophical, very little hinges on that choice in practice. TF, as an ontology of terms, takes its inspiration from the category theoretic abstraction of morphism, a structure preserving mapping from one domain to another. This is taken as the germ of the idea of one thing meaning or representing another. A morphism is the reification of a meaning relation as an object. A sign is a special case of such a reification, as a pairing between some form and some meaning, or as the role played by a form at the receiving end (codomain) of such a mapping. A term is a linguistic sign, a special case of a sign. Other signs: images, traffic signs, formulae, are other special cases.

Unlike traditional approaches to terminology, TF has no fixed notion of a terminology entry as a record structure. (But see DESCRIBE queries .) TF builds on the semantic network metaphor, dual to the container metaphor of hierarchical databases (and more recently, XML). The container metaphor comes from physical media like paper or magnetic tape. Containment among convex objects naturally forms tree structures. In a rooted directed tree, it makes sense of talk of nodes as bigger elements containing smaller elements. In an undirected tree or graph, all nodes are equal, any node can be taken as root or focus. Nodes do not contain one another, rather, they are visualised as dots connected by links. RDF/OWL graphs are not rooted. The serialisation of a semantic network in RDF/XML need not respect connectedness. Information concerning a given node may be distributed freely among disconnected descriptions in an RDF/XML document.

As a descendant of classical logic, OWL is weakly typed. It is possible, but not obligatory, to specify individual type, superclass of class, and domain and range of object properties. The default is the vacuous type owl:Thing . TF tries to take advantage of this. Different typing regimes support different semantics (resoning) and different use cases of TF. With strict typing, it can be necessary to create dummy objects just to satisfy typing requirements. With weak typing, it is not necessary.

An even weaker ontological position is to consider concepts, terms, and expressions all intensional objects of a kind. An expression is a term without its semantic properties, and a concept is one without expression properties. Now all instances are signs. There is no sortal distinction made between things out there and their names. Individuals get identified with individual concepts. Jesus Christ belongs to classes Human and Noun. In the more extensional use case of traditional (Western) terminology theory, these classes would be asserted to be disjoint. By Aristotle's (and Quine's) definition of ambiguity, this use case makes individuals ambiguous. Christ is both human and not human (unless human is made equally vague). Ambiguity is not harmful as long as no reasoning is done on it that hinges on it. One can translate correctly on the basis of form when the translation retains the same ambiguity. This theme is pursued further in the section on TermFactory schema profiles .

Another application of the relativity of the TF top triad is that TF descriptions can be forms, meanings or signs. This allows the whole gamut of attaching uninterpreted labels, language specific messages, and language independent content as descriptions of other such entities.

TF algebra

An Aristotelian insight is that the distinction between meaning and form is not absolute: not a classification but a relation. What counts as matter and what form is relative in the way domain and codomain of a morphism are relative to the morphism. In normal TF usage, an expression is a general language vocabulary item whose meaning is narrowed or transferred when it is used as a special language term. The common language expression, and common language in general, is form relative to special-language terminology. The special language expression borrows linguistic properties from the conventional meaning of the general language expression. But that general language expression can be further analysed as a linguistic sign with form and meaning. This notion accords well with the spirit of the category theoretic abstraction: morphisms are closed under composition. Terms and expressions are also (instances of) concepts. TF does not say terms and expressions are disjoint classes. An instance which is at once a term and an expression can have properties of both classes.

The underlying motive of the form-meaning morphism is the core category theoretic notion of limit (universal element). A limit in a category is an object through which all the remaining objects can be factored. It takes out the slack from the category, represents what is common to it. In translation, a language-independent (interlingual) meaning, shared by a set of synonyms, is a limit (universal element) through which an equivalence class of synonyms is factored. In graph terms, the reified meaning constitutes the hub of a spanning subtree of form-meaning relations that removes the redundancy of a square matrix of bilateral synonymy relations. Recursively applied, the reification of synonymy into meaning allows logarithmic savings in the size of the representation (from m times n to m plus n). Dually, the meaning-form morphism allows analogous savings in the representation of homonymy. (In the original Aristotelian sense: two things are homonymous if they are called by the same name). Properties of the shared name need only be mentioned once. Monolingual sector terminologies have little need for this split because of the ideal of monosemy per special language: an expression should have just one sense per subject field. The situation is different across sectors and (sub)languages: one and the same expression (say operation ) is used in many fields.

An abstract way of looking at concepts, terms, and expressions emerges from Aristotle's insight. A term is the reification of a term:designates relationship, obtained by inserting a node and splitting the relation into the composition of two (term:designates equals term:designationOf o term:hasReferent).

More interestingly, the two extremes, concepts and expressions, appear dual. A concept is a "set" of expressions, and an expression is a "set" of concepts. This is not set theoretically sane as such, but makes category theoretic sense. Assuming both concepts and expressions form boolean algebras (closed under join, meet, and complement) and there is a bijection between atoms of one and coatoms with other ('basic' concepts have 'basic' designations, as they do in any ontology), it is possible to find for any concept an uniquely corresponding expression, and for an expression a uniquely corresponding concept. It is in the sense of this bijection that concepts "are" expressions.

Categorially taken, then, concepts "are" (match one-one with) expressions, just dually related: a concept "is" a synset, an expression "is" a homset (the set of all the concepts that it names). Except we should not talk about sets. Since we are metamodeling classes and set theoretic relationships already, we can use the metamodeling instances and properties to build a purely algebraic model. The dual map between concepts and expressions is term:designates. E designates C, where since both are Boolean algebras we can always choose E and C so that designates is one-one.

Vocabulary is not sacred in TF. Given reasoning, we can have it both ways, as long as we make the logic clear. If one wants to avoid reifying terms and signs, that too is possible, without leaving TF. Using OWL2 property chains, TFS can define a shortcut link between a concept and and expression term:designates as the composition of the relations term:designationOf and term:hasReferent , avoiding mention of terms entirely. Similarly, sign:signifies short-circuits between forms and referents of signs.

Show/hide limit diagram

With the composite property term:designates , a simple multilingual word list can be converted to equally simple TF triples:

<http://tfs.cc/exp0/Finnish> term:designatedBy [ exp:langCode "fi"; exp:baseForm "suomi" ] .

We may call this format TF Compact.

An advantage of using vocabulary consisting of term:designates and rdfs:label is that it avoids ontological questions. A rdfs:label with a language-coded string makes the least possible ontological commitment. We avoid awkward questions of expression identity and ontology mapping. Assume triples like the one above are added to a model. If one is just interested what labels a concept might have, a query for rdfs:label will include these. If the reasoner understood composition, it would also return TF baseforms that could be composed into labels. What if the query is for base forms of expressions? For the reasoner to answer a baseform query with a rdfs:label , it would need to do decomposition, that is apply property chain axioms in reverse to unpack rdfs:Label and infer the existence of a (blank) expression having as baseform that label. Existential instantiation is not what ontology reasoners typically do, so chances for real time queries are slim. But this reasoning could be implemented through some offline processing. The main thing is that the semantics is right. (In the long run, it is counterproductive to avoid ontological questions about expressions. Without ontological commitments, we cannot keep track on our vocabularies: manage sources, authorship, versioning etc.)

The compactification caused by composition is consistent with all TF profiles. TF profiles concern another dimension, namely which TF classes are disjoint and which TF properties are allowed on which TF class members.

TF Compact is an example of OWL turning what first looks like a syntatic format conversion into a matter of description logic reasoning. Instead of running a syntactic conversion script, we reason with a bridge ontology. One might say it is all just a matter of semantics in the end.

With property composition, we can make exact sense of the Aristotelian intuition that form-matter relationship is a relative one, forms a scale. To express that a special language concept is designated by an expression that in turn has a general language meaning, we go over duals like this:

C term:designatedBy E sign:designates M

where C meta:subClassOf M . Adding decomposition into terms and signs, this becomes

C term:referentOf T term:hasDesignation E sign:formOf S sign:hasMeaning M

where T is a special language term and S is a lexicographic sense of the same expression E.

TF ontology subsets

The core TF schema TFS.owl only defines the skeleton of the TermFactory special language term ontology. In its intended model, a term is a pair of an designation and referent. The designation is a natural language expression. The referent can be a general concept (class pun) or a named instance (a person, building, country etc.). TFS.owl does not entail all the properties of this intended model, not even all those expressible in OWL. More properties of the intended model can be imported from schema extensions as they are needed. In particular, they may be useful for integrity testing, to make sure that a term ontology reflects the intended model. The point is that not all of the properties are always needed, and the more of them are asserted, the more work there is for reasoning. In particular, axioms which entail separation of classes and distinctness, existence or uniqueness of instances are expensive to reason with, because increase the size of the model by they multiplying the number of entailments and blank nodes. Such complicating OWL axiom types include owl:disjointWith, owl:differentFrom, owl:someValuesFrom, owl:FunctionalProperty, and rdfs:domain/rdfs:range restrictions.

The WordNet to TF bridge TFwn.owl places the WordNet class Synset as a subclass of Meaning. Wordnet senses become instances of Sign (with property term:approximate "true" as default). WordNet synsets differ from TF Full style concepts by having an inherent part of speech, which in TF Full is a form property. Also, WordNet senses have label, which in TF Full is a form property. This makes WordNet an instance of Legacy profile with some traces of Lite: Synsets are instances of Meaning but they have part of speech; Senses are instances of Sign but they have base forms. There is an additional WordNet class Word that contains words (Forms). See section on morphology .

The following table and graph describe the subsets of the TF top ontology. The labeled clusters and connectors depict conceptual and functional groupings, not OWL import relationships. There are no built in import relationships between the subsets, to allow free mixing and matching of the subsets in different combinations.

Show/hide TF schema subset graph

The rest of this section is organised as a set of frequently asked questions about modeling classical terminology theory in TF.

Q: Has every term got a designation and a referent?

A: Designation is a named class in TFS. It could be defined as term:designationOf Some Term . Referent is not a named class in TFS. It could be defined as term:referentOf Some Term . Expression is a superclass of Designation that also contains Text. TFS.owl defines object properties term:hasDesignation and term:hasReferent . They are classified as basic term properties in TFProp.owl:

<owl:ObjectProperty rdf:about="&term;hasDesignation"> <rdfs:subPropertyOf rdf:resource="&meta;termObjectProperty"/> <rdfs:subPropertyOf rdf:resource="&meta;basicObjectProperty"/> </owl:ObjectProperty> <owl:ObjectProperty rdf:about="&term;hasReferent"> <rdfs:subPropertyOf rdf:resource="&meta;termObjectProperty"/> <rdfs:subPropertyOf rdf:resource="&meta;basicObjectProperty"/> </owl:ObjectProperty>

Q: Is the designation of a term always an expression?

A: Yes. The range restriction is asserted in TFS.owl:

<owl:ObjectProperty rdf:about="&term;hasDesignation"> <rdfs:range rdf:resource="&exp;Expression"/> </owl:ObjectProperty>

Q: Is the referent of a term always a concept?

A: No. It can be any TF object. The range restriction is asserted in TFS.owl:

<owl:ObjectProperty rdf:about="&term;hasReferent"> <rdfs:range rdf:resource="&meta;Object"/> </owl:ObjectProperty>

Q: Do only terms have designations?

A: Yes. The domain restriction is asserted in TFS.owl:

<owl:ObjectProperty rdf:about="&term;hasDesignation"> <rdfs:domain rdf:resource="&term;Term"/> </owl:ObjectProperty>

Q: Do only terms have referents?

A: Yes. The domain restriction is asserted in TFS.owl:

<owl:ObjectProperty rdf:about="&term;hasReferent"> <rdfs:domain rdf:resource="&term;Term"/> </owl:ObjectProperty>

Q: Does a term have a unique designation?

A: Yes. Functional property assertion in TFStrict.owl:

<owl:ObjectProperty rdf:about="&term;hasDesignation"> <rdf:type rdf:resource="&owl;FunctionalProperty"/> </owl:ObjectProperty>

Q: Does a term have a unique referent?

A: Yes. Functional property assertion in TFStrict.owl:

<owl:ObjectProperty rdf:about="&term;hasReferent"> <rdf:type rdf:resource="&owl;FunctionalProperty"/> </owl:ObjectProperty>

Q: Can two different terms have the same designation and referent?

A: No. A term is uniquely identified by its designation and referent. Uniqueness assertion in TFStrict.owl:

<owl:Class rdf:about="&term;Term"> <owl:hasKey rdf:parseType="Collection"> <rdf:Description rdf:about="&term;hasDesignation" /> <rdf:Description rdf:about="&term;hasReferent" /> </owl:hasKey>

Q: Are the term and its designation different things?

A: Asserted in DictionaryStrict.owl:

<rdf:Description rdf:about="&term;hasDesignation"> <rdf:type rdf:resource="&owl;IrreflexiveProperty"/> </rdf:Description>

Q: Are the term and its referent different things?

A: Asserted in LegacyStrict.owl:

<rdf:Description rdf:about="&term;hasReferent"> <rdf:type rdf:resource="&owl;IrreflexiveProperty"/> </rdf:Description>

Q: Are terms and expressions disjoint classes?

A: Disjointness asserted in DictionaryStrict.owl:

<owl:Class rdf:about="&term;Term"> <owl:disjointWith rdf:resource="&exp;Expression"/> </owl:Class>

Q: Are terms and concepts disjoint?

A: Disjointness asserted in LegacyStrict.owl:

<owl:Class rdf:about="&term;Term"> <owl:disjointWith rdf:resource="&ont;Concept"/> </owl:Class>

Q: Are designations and referents disjoint?

A: Not asserted either way. Counterexample: a quoted expression is an expression which designates an expression.

Q: Are all expressions designations of some term?

A: Not asserted either way.

Q: Are all concepts referents of some term or other?

A: Not asserted either way.

Q: Can a concept be referent of more than one term?

A: Not asserted either way.

TermFactory properties

The main classification tools in an OWL ontology are class and property hierarchies. A good TF sub-ontology bridges its top concepts (at least indirectly) to the TF schema and checks predefined classes and properties for fit before subclassing or inventing its own. It is good ontology writing practice to use classes ( rdf:type exp:English ) instead of distinguishing properties (like exp:langCode "en" ) for classification when workable, because description logic reasoners and editors support classes best. (The TF schema actually defines the class exp:English as the class of those things whose langCode is "en".) It is also good practice to build subproperty trees instead of a long flat list of properties. Subproperty relations allow alternative views on related properties and support querying properties by class instead of by name. This is particularly important with repository specific new types of properties. A query engine is able to return meaningful subsets on the basis of subproperty classifications without having to know the individual properties. This is valuable when the ontology collection is composed of contributions from independent sources.

TF divides properties first of all into properties of terms, expressions, and concepts. A group apart are semantic properties in the sem namespace, used in the TF natural language interface. To make the classification of properties as concept, term, or expression properties explicit, we build explicit hierarchies of TF object and data properties in TFS.owl. Here is a representative sample:

meta:objectProperty meta:conceptProperty meta:termProperty meta:expressionProperty

This property hierarchy is kept in a separate ontology document TFProp.owl . In it, each property also has a paronymous instance (pun) in namespace meta0. The puns are used in localising property names. (Paronyms for properties are needed in OWL 2, because even OWL 2 does not allow punning property names.) The puns are classified into a property instance classification that runs parallel to the property hierarchy:

meta:ObjectProperty meta:ConceptProperty meta:TermProperty meta:ExpressionProperty

The property classification simplifies the statement of property ranges, since the range of each type of property can be given just once for the type. Other properties inherit it from the type(s) they belong to.

Term properties

In the TermFactory terminology model, a term is a special case of a two-faced de Saussurean sign that links a meaning or referent (signifié) with a form or expression (signifiant). In practical terminology work, the distinction between term and expression is not always made explicitly. In TF, it is an option motivated by multitopicality and multilinguality. Among other things, it allows separation of those properties that expressions have by general-language grammar from those properties that are associated to them only in a specific domain-dependent or special-language meaning.

TF linguistics

One of the goals of the explicit machine-processable expression ontology is to make it possible to parse and generate natural language text from and to TF ontologies. Using a natural language generator and the TF schema vocabulary, it should be possible to generate definitions and explanations of concepts from an ontology automatically in the different languages covered by the expression ontology. Using a natural language parser and the TF schema vocabulary, it should be possible to parse natural language queries to the ontology and to carry out natural language commands from users. Putting this together, TF could support a multilingual human-machine dialogue system. In particular, TF should be able to explain and modify itself in natural language. The cparse NL multilingual parser/generator is going to be tested on TF vocabularies for multilingual parsing and generation of term definitions.

The EU MOLTO project shall apply Aarne Ranta's Grammatical Framework interlingual translation system to special domain translation. The special domains will be described as OWL ontologies. TF is to be used as a vocabulary acquisition tool in the project.

Here is a design sketch for the connections among MOLTO and TermFactory components.

Morphology in TF

In special language terminology, morphology plays a minor role. Terms as lexical innovations tend to have simple morphology. In terminology collections, expressions associated to concepts are given in some base form largely in abstraction of derivational or inflectional morphology. This is not always efficient or illuminating or even sufficient. For instance, an expression like exp:fi-epäkunnossa-P 'broken' has no base form. Its opposite exp:fi-kunnossa-P is an inflected case of base form exp:fi-kunto-N . Should exp:fi-epäkunnossa-P be related to exp:fi-kunto-N , and if so how?

Here is one proposal. exp:fi-kunnossa-P is simultaneously classified as an exp:Expression and as an inflected syn:Form whose syn:hasBaseForm is exp:fi-kunto-N

Inflection

. In general, when a form is a base form (lexeme), it is an exp:Expression as well as an inflected syn:Form, and related by syn:baseFormOf to its inflexions. The inflected form, which is perhaps not an exp:Expression at all, only a syn:Form, points to the base form with syn:hasBaseForm. For instance, form exp:fi-epäkunnossa-P can be connected to exp:fi-kunnossa-P by some appropriate expression property, say syn:hasDerivedForm.

Except for semantically or terminologically conditioned special cases, morphology is better not enumerated in TF, but by a morphology processor. It hardly makes sense to include paradigm listings into a static ontology document when such listings can be generated by rule. The natural approach is to provide a base form and enough tags and form to allow generating the whole paradigm from them using some morphological processor.

Number and count are best considered properties of concepts (class puns) not instances. An individual instance like ont1:Java can belong at the same time to ont:Program whose pun ont0:Program is count and to ont:Software whose pun ont0:Software is noncount.

Derivation

Say parse is a term in the category Verb. It has a productive agent noun parser. Are they the same term or two? We need to split this into two questions. Pro primo: Are there two expressions (lexemes) here with different resource identifiers? Pro secundo: do we need to enumerate and store them in the repository? The answer to the first question is probably positive. parser, though regularly derived from weld, is another lexeme, with its own set of forms. It belongs to a different part of speech than lead. The associated concepts are also different. One classifies processes, the other people. Assuming people are disjoint from processes, they cannot be the same class. One resource cannot have conflicting properties, so we have two. In another language, the relation of the corresponding terms need not be as predictable, and we want to be able to tell which translates which. On the other hand, if the derivation is fully predictable, we need not store both in the repository. One can be automatically derived from the other using an ontology/rule schema for derivation or an external morphological processor. Only axioms need to be enumerated, theorems follow.

How to implement this in practice needs thought. Assume there are resources en-parse-V and ont:Parse in the ontology, but no resource en-parser-N or ont:Parser. How does one query for 'the agentive noun for the verb parse'? Will a query for string pattern "parse" include the agentive noun? Will a query for "parser" find it? Can one find an entry for en-parser-n_-_ont:Parser if it is not already stored?

For the first question, we might write a special query handler for a triple like en-parse-n exp:hasAgentiveNoun ?x. The query handler for this predicate would dispatche to a special purpose reasoner, perhaps implemented by a morphological processor. For the string match query the query engine would need to expand derivatives to find matches for unlisted strings. As for the last question, parsing unknown resource names on the fly requires rewrite rules that unpack the unknown resource identifier into a query. (Location mappings might just swing it in this particular case, but that is not what they are good for.)

The [<c>=parse] natural language parser/generator was adapted to serve this purpose in the predecessors of TermFactory, the 4M and Cogks dialogue systems (see graph below). The cparse generator has a Java/Jena converter that converts between linguistic feature structures and RDF graphs. There is a tool that converts TF ontologies into cparse dictionaries. cparse version 71 was completed in May 2009. Test grammars have been developed for multilingual parsing and generation of concept definitions between Finnish, English, and Chinese.

Show/hide CoGKS architecture

TF natural language semantics (TFSem.owl)

The purpose of the TF semantics namespace is to provide enough semantic analysis of general language to support a simple interlingua suitable for typical terminological definitions which can be parsed from and generated to multiple natural languages. The longer term aim is to reduce or obviate the need to manually verbalise definitions that are built following standard rules of terminology from a concept system formalised in TF.

For instance, the concept ont:Parser could be defined in a TF ontology with superclass ont:Program and with property sem:hasFunction value ont:Parse (which in turn could be dethe fined with superclass ont:Analyze having property sem:Object value ont:Text . This already constitutes an interlingua from which it is not difficult to machine generate simple phrases like a program to analyze text to verbalize the OWL definition. (The language independent parser/generator [<c>=parse] has been tested on this example.)

How to add localizations

There are many ways to find and edit concepts which have not got localizations in a given language. The following scripts are provided in file:io/sparql as a starter.

locals.sparql is the query used by command line script localize. It produces all the localizations found in the dataset. It can also be run through the query form for instance. nolocals-fi.sparql is its complement for Finnish. It produces a table of concepts missing localizations in Finnish. With JSON output setting, it produces a JSON table that can be read into the TF/MOLTO equivalents editor. (It needs the ARQ projection syntax extension of SPARQL, see here). describe-unlocalized-fi.sparql runs a TF DESCRIBE query on concepts which have no Finnish localization. It shows among other things their localizations in other languages. It does not create templates for the missing equivalents. Templates can be created individually in the editor using the editor TF Inseret menu's input templates. construct-unlocalized-fi.sparql creates templates for missing Finnish localizations. Running first the describe script and then the construct script on the result with the accumulation (-M) option set produces both.

pellet4tf query -e MIXED -q sparql/describe-unlocalized-fi.sparql ../owl/TFL10n.owl > step1.ttl time pellet4tf query -o XHTML -R ont:Concept -e MIXED -q sparql/construct-unlocalized-fi.sparql -M step1.ttl ../owl/TFL10n.owl > step2.xhtml

Show/hide step2.xhtml

To avoid duplication of work, one should first check if there are candidates for localization already elsewhere in TF before submitting new proposals.

Ontologies in the Web

Much of the complexity of TF reflects the complexity of distributed management of large ontologies in the Web. The tactics is one of divide and conquer: divide large ontologies into manageable size pieces, reduce complexity using reasoning, inheritance, redirection, and whatnot.

Reasoning in TF

Reasoning can reduce the traveling size of an ontology by letting it assert much less than it implies. It is possible to define for a given TF term ontology, a notion of a minimal set of axioms from which all statements of the ontology can be derived, and from which no axiom can be removed without losing statements in the theory. An axiom set is not generally unique for a given theory; there can be many axiom sets of the same smallest size. At another end, there is the notion of materialization , or inferential closure of the ontology, which contains all the (non-tautological) statements (in the ontology's vocabulary) derivable from the axiom set. This is unique for an ontology, but not necessarily finite. The closure of a TF term ontology is a finite (though possibly large) set. An axiom set for a term ontology is useful to have for editing and versioning because the set is minimal and edits only need to be done in one place. A (partial) materialization can be useful for viewing and querying because it allows faster access to implicit content than online reasoning - provided the closure is not prohibitively large. (cf Ontotext on reasoning strategies.)

The difference between statements, axioms, and theorems must be observed when editing an ontology. In TF, editing is a syntactic operation on a set of triples. Statements, not facts, are edited. This means that a fact may stay in an ontology although a triple gets deleted, if it is a theorem from some axioms that remain. Deleting facts would not be just more complex, but would bring in tricky problems of nonmonotonic reasoning or belief revision.

Property inheritance in OWL

One of the attractions of ontologies is property inheritance. Instead of having to assign properties to each instance (say a term or expression) individually, one assigns shared properties to classes, whence they get inherited to class instances. In description logic, inheritance becomes classical logical entailment. OWL allows expressing some property inheritance by means of defining classes by description.

Take as an example the ontology of languages and language codes in TF. Class ont:Language contains concepts (puns) for individual languages like exp0:English, which reifies the language adjective exp:English. exp:English denotes the set of all pieces of the English language, including English terms and expressions, that have language code "en". The language code "en" in turn is the base form of a term (an abbreviation) that refers to English. Terms and expressions that belong to English should have the language code "en". Having "en" as language code means being English. Intuitively, then, the first triple below should be equivalent to the second:

[] exp:langCode "en" . [] rdf:type exp:English .

Indeed, this equivalence can be expressed in OWL as follows.

<owl:Class rdf:about="&exp;English"> <owl:equivalentClass> <owl:Restriction> <owl:onProperty rdf:resource="&exp;langCode"/> <owl:hasValue rdf:datatype="&xsd;string">en</owl:hasValue> </owl:Restriction> </owl:equivalentClass> <rdfs:subClassOf rdf:resource="&exp;Language"/> <rdfs:comment rdf:datatype="&xsd;string">anything in the English language</rdfs:comment> </owl:Class>

Here the element owl:Restriction defines a class by description, namely, the class of those things which have language code 'en'. Any instance that belongs to this class has that language code. Or that is what the axiom says. It is up to a reasoner to enforce this classification. (OWL restrictions tend to look ugly, so it may be convenient to invent a class whose main job is to entail restrictions. Then the restrictions are easily imposed by just subclassing from that class. exp:English above is an example - its instances inherit language code 'en'.)

Rules

Different varieties of rules have been proposed as extensions of description logic. Besides extending classical reasoning power past the confines of decidable description logic, some types of rules can do nonmonotone reasoning, signature transformation and refactoring (rewriting the vocabulary of resource names and literals). Because rules can express more, they are less well behaved than descriptions. For the same reason, there is less agreement about what rules should be like. There is no W3C recommendation for a rule language, just a W3C recommendation for a rule interchange format RIF . A well known Semantic Web rule language is SWRL. SWRL is partially supported by Pellet .

TF supports limited ontology transformation using SPARQL CONSTRUCT queries with pellet4tf query -M and -m options. The TF rewrite utility can do refactoring of TF URIs. Seee also SPARUL in the next section.

Querying as reasoning

A query language expresses questions about a dataset (set of models). The answer may be yes/no, a list of bindings of answer values to variables (question words) in the query, or another set of triples. The answer to a query is computed by a query engine. Querying is one if not the main way of accessing knowledge stored in a TF repository. The Web of Data concept builds on a network of linked data consisting of machine readable query endpoints parallel to the current network of human readable websites.

In Hintikka's logic of questions, an answer to a question is set of formulas in epistemic logic that entails the question, expressed as another epistemic formula. Answers we won and we did not win to question did we win (or not) . The logic is that the answer formula (I know that) we won entails the question formula I know that we won or I know that we did not win . Since the answer matches the question, it can be abbreviated to yes or no.

Question words are quantifiers into an epistemic context. I know what something is when there is something I know it to be. When the form of the answer reflects that of the question, an answer to a question can be abbreviated to a list of (bindings of) values to question words. So one can answer Who won WW2? with USA, Britain, ... , instead of USA won WW2 and Britain won WW2 and ... . This type of answer to a query constitutes a set of bindings of values to question variables to produce instances of the query in the data. But note that Hintikka's definition of answerhood also covers indirect answers, where inference intervenes between the question and the answer, or between the answer and the dataset.

In query languages, question variables are distinguished from the rest in some way depending on the query language. Query languages may have imperative variants as well, i.e. they don't just answer questions but change things on the basis of the answer to a query.

Many query languages for RDF and OWL have been proposed, including RDQL, SERQL and SPARQL for RDF or OWL-QL, SPARQL-DL and SAIQL for OWL.

SPARQL

SPARQL is the best known RDF query language and a w3c recommendation. Its syntax resembles that of the well established relational database query language SQL, which may be one of its selling points. Compared to SQL, a major missing feature compared to SQL is UPDATE queries. SPARQL 1.1 Update is a W3C Working Draft 22 since October 2009. Once standard, SPARQL Update can do some of the jobs now done by pellet4TF and edit4tf tools.

Other SQL features missing from SPARQL are aggregator functions like COUNT, nested queries (query whose dataset is another query).

SPARQL query answering is based on the semantics of RDF. A query is answered by a set of subgraphs of the dataset that match (by graph isomorphism) the query graph. With this semantics, SPARQL only finds triples that are explicitly listed in the dataset. There is a provision in the SPARQL standard to extend SPARQL query answering to other entailment regimes beyond subgraph matching . Given such extensions, SPARQL can be used to extract entailments from a dataset.

Realization in DL reasoning finds the most specific classes that an individual belongs to; i.e., realization computes the direct types for each individual. Realization can only be performed after classification since direct types are defined with respect to a class hierarchy. Using the classification hierarchy, it is also possible to get all the types for each individual.

Realization is typically a slow undertaking, since it in effect multiplies the number of instances with the number of classes. It usually does not make sense to ask TF DESCRIBE queries using the Mixed engine on larger ontologies, since the query can easily take more time than one cares to wait. One solution is to do realization on a large ontology once in a while and cache the results for direct consultation. Another way to go is to apply the TF Stacked engine.

Answer formatting in SPARQL is restricted. The answer of a SPARQL query can be yes/no (ASK queries), a list of bindings as text, xml, or json (SELECT), or a set of RDF triples constructed from the answer bindings and constant strings (CONSTRUCT). The result set can be filtered with functions that express syntactic conditions on resource names and literals. But there is no way to construct new resource names and literals from the answer bindings. This restricts the use of SPARQL as a RDF graph transformation tool (but see SPARQL-U or SPARUL). Such facilities are included in Semantic Web rule languages.

Ontology imports

Industrial strength ontologies, for instance in the biomedical domain or in the Linked Data initiative, can become quite big. We are talking of hundreds of thousands, millions, or billions of entities. At that scale, ontologies become unmanageable unless some kind of divide and conquer tactics is available. (Desktop ontology editors can have difficulties with tens of thousands of items, not to speak of us human users who get lost with hundreds). There is a growing literature on ontology sharing and importing, ontology decomposition, modularity etc. On the level of the OWL standard, there is as yet little support for all this. The only primitive in the standard is the owl:imports element that creates import relationships between ontology resources. In terms of RDF graphs, an ontology is just another named resource, a graph node. An URL is identified as an ontology in a statement like

<http://puls.cs.helsinki.fi/med> rdf:type owl:Ontology .

What IS an ontology, concretely? An ontology is identified by an ontology resource URI ("ontology header"), and it should probably consist of nodes (resources and literals) and statements (triples) describing them. But which triples belong to an ontology? The OWL standard does not explicitly say. The connection between an ontology URI and the associated resources and triples is not expressed in OWL. There are conventions based on physical document containment relations and URI sharing, but no explicit statement in OWL to the effect that a given triple belongs to a given ontology.

The association of an ontology "header" URI to an ontology document URL is physical and not standardized. The OWL recommendations talk loosely about "the ontology header", but there is nothing to enforce existence or uniqueness of such a header in an ontology document. There is no semantic or syntactic connection between an ontology document, the ontology header, and the triples in the document, any more than there is one between the name of a file and its contents. Membership of a statement in an ontology might be represented in RDF using reification, but it is not done in practice. There may be good reasons for not making membership of a triple in an ontology explicit. At least doing so with reification would be quite inefficient. Cf. Wikipedia .

The archetypal OWL RDF/XML ontology document format for simple ontologies was one where the ontology document URL matches the (only) ontology resource URI described in the ontology header, and the ontology URI is the URI prefix of all the resources "defined" in the ontology. But this is too simple to be true in real life.

In the default setup, resources and statements are "defined" in an ontology document and "belong to" that ontology. What that might mean semantically is nowhere defined. Triples in an ontology document are normally not marked in any way as being "from" one ontology rather than another. There is an RDFS utility property rdfs:isDefinedBy for pointing a resource as belonging to a vocabulary, but it is vaguely defined and rarely used. The same statement may occur in many ontology documents. Import statements create an imports graph of the ontology resources, and as a side effect pool together triples from the ontology document associated to (accessible by) the ontology URI. Do (all of) these triples thereby belong to the importing ontology? Hard to say. What triples "belong" in an ontology, and how the namespaces of the triples are related to that of the ontology URI, is not regulated.

Having the same URI prefix as an ontology URI proves nothing. An ontology's triples may (and typically do) contain vocabulary sharing URI prefix with the ontology URL, but that is again conventional. An ontology document may quite well contain items that don't have the same base URI as the ontology (they typically do). There is no constraint between an ontology URI and the URIs of other resources in "it" (i.e. its defining document). In particular, nothing can be inferred from such conventions in RDF or OWL.

In OWL 1, the external URL of an imported ontology and the ontology header URI were related only indirectly. The standard says "An OWL document consists of optional ontology headers (generally at most one)". Protege 3 used to require that an imported document's xml:base attribute matched the imported URI. This convention was predicated on the default setup explained above. See http://www.w3.org/TR/owl-ref/#imports-def . For OWL 2.0, see http://www.w3.org/TR/owl2-syntax/ .)This made it hard to locate imports. In OWL 2, imports are identified by URL or the document location, not by "internal" ontology URI. See http://protegewiki.stanford.edu/wiki/How_Owl_2.0_Imports_Work .

Since there is no necessary association between an ontology document URL and an ontology (resource) URI ("header"), many documents could contain and describe the same ontology resource (like any other resource), and one document could describe more than one ontology resource. It is just a convention related to Semantic Web addressing orthodoxy that an association between an ontology URI and an ontology document (accessible at that) URI exists and is one-to-one.

What happens if an ontology imports some URL, and that causes loading a document which does not contain an ontology header for the imported URI? Nothing specific happens in fact. If an imported ontology document has no (different) associated ontology resource description, importing it works like an include. If an ontology is too big to have in one file, it can be split into many in this way.

Ontology URLs are important for ontology imports. A statement of form

<http://puls.cs.helsinki.fi/med> owl:imports <http://puls.cs.helsinki.fi/> .

tells that the subject ontology imports the object ontology. Even if the purpose is just to split the ontology to manageable pieces, technically, the pieces become ontologies with their own URI. It may be safer to avoid using relative URI's in writing ontologies because they may cause trouble later if the ontology is split.

Resolving ontology URIs vs. URLs in ontology import statements can cause problems. An ontology loader goes and tries to locate a document describing the object ontology at the ontology URL. Failing that, it may ask for an association of the ontology URL with some file. Given a file, the loader looks for a RDF description about the ontology URL in it. If one is found, the file is loaded as a document describing the object ontology.

Jena resolves the object URI of an import statement to a physical URL against an external catalogue ont-policy.rdf . OWLAPI does not implement a catalogue mechanism, it is up to the user of the API to implement one. Protege 4 implements an XML catalogue mechanism. The current Pellet reasoner just fails if an imported URL does not point to a readable ontology. Further discussion:

http://protege.stanford.edu/doc/owl/owl-imports.html
http://www.nabble.com/Creating-local-repository-for-a-project-td24969842.html
http://protegewiki.stanford.edu/index.php/How_Owl_2.0_Imports_Work http://www.w3.org/2007/OWL/wiki/LC_Responses/TR1 http://www.w3.org/2007/OWL/wiki/Imports

In at least some versions of Protege the convention is that the xml:base attribute identifies "the" ontology defined in an ontology file. The idea behind this convention seems to be that the ontology resource uri should be the same as the xml:base prefix of the document containing it and that the shared prefix should also identify resources belonging to the ontology. A document imports the ontology that is mentioned in its xml:base element. The base attribute value must match the external URL of the ontology document, otherwise the ontology won't load. TF does not follow this convention. It prevents splitting an ontology into several files without changing the original namespaces. Things are supposed to change with Protege 4.1. In Protege 4.0 (build 113) import locations of Turtle (.ttl) files apparently are not cached, unlike imports of RDF/XML (.owl) files. Apparently in Protege 4, owl RDF/XML file locations are cached in per-directory XML catalogue files (catalogv00.xml), Turtle files are not.

The TF ontology has contended with the question of ontology imports from its beginning as the Tekes 4M project ontology. Already in that stage, term ontologies formed an inheritance hierarchy using the owl:imports primitive. In terms of the following figure, loading the company instance database on the top right corner would import the ontologies needed by it directly or indirectly. For example, a ship engine numbered #12345 could belong to a class xyz of engines described in a company ontology, which could define it as a given type of combustion engine ABC , describe in a more general ontology of diesels, and so on up.

Bridge ontologies

For importing third party OWL ontologies to TF, one method is using hand-made bridge ontologies. Bridge ontologies are (preferably relatively small) ontologies which interface between other ontologies. They import all or part of the component ontologies plus define the contact points where the imported entities "plug in" to one another, by adding properties or concepts to relate them.

The following query uri uses TF uri retry facility to fetch the ontology associated to an uri.

http://localhost:8080/TermFactory/query?uri=%3Chttp%3a%2f%2ftfs.cc%2f%ont%2fFinnish%3E

The following query uri also specifies repositories to serve as the dataset of the query. Items in a list of repositories to search are delimited by an URL encoded newline character %0A . More than one repository parameter can be given per query string.

http://localhost:8080/TermFactory/query?uri=http%3a%2f%2ftfs.cc%2f%ont%2fFinnish&repos=repo1.uri%0arepo2.uri%0a

Here is an example of using the result of one query in the dataset of another.

http://localhost:8080/TermFactory/query?pattern=Finn&repos=http%3a%2f%2ftfs.cc:8080%query?pattern=Country&repos=TFS.owl%0aPlace.owl%0a%0a%2fDomain%2fGeography%2fCountry%2f%0a

What this does is use the local query engine to query for items related to Finns in a dataset composed of (i) the result of a query to TF schema about countries and (ii) the local ontology index under countries. Note the double newline (percent encoded as %0a%0a ) marking the end of the embedded query's repo parameter.

If the http server at grapson.com is further configured to redirect its incoming uris (perhaps just those of them that satisfy some further condition) to the TermFactory webapp, the import element can be made maximally simple:

<owl:Ontology rdf:about=""> <owl:imports rdf:resource="http://tfs.cc/ont0/English"/> </owl:Ontology>

This request is first rewritten by the grapson.com web server to a TermFactory query uri which is looked up or queried (from repositories specified in the query uri or in the service's settings). The resulting model is what gets imported.

The query engine used in the selection stage and the engine used in the description stage of a DESCRIBE query should sometimes be different. To find all that is known about some topic, the best bet is to use a reasoner all the way. For editing purposes, it may make good sense to use a reasoner to select relevant items for editing to be sure nothing is missed, but then describe those resources for editing with a SPARQL reasoner. It only makes sense to edit asserted triples, because implied triples will not go away unless one edits the axioms that imply them. Editing is a syntactic operation which should be applied to a minimal nonredundant representation.

To tell the TF query facility to switch engine midway of a DESCRIBE query, the query engine parameter can be of form ENGINE1+ENGINE2. For the editing scenario above, the parameter value should be MIXED+SPARQL. Parameter value MIXED (or MIXED+MIXED) uses the MIXED engine in both phases. The second engine in the composition is just ignored if the query is not of DESCRIBE query. The Stacked engine has its engine choices built in, so it cannot be composed with the other engines in this way.

Naming resources in TF

A central source of complexity concerns naming of resources in TF. As explained in the section on TF philosophy, the local/global naming problem is very much at the core of TermFactory. The global web resource identifiers (URLs, URIs, IRIs, URNs) are unambiguous all right, but they are also long, ugly, and hard to remember.

Semantic Web orthodoxy only requires that global resource identifiers are monosemous, viz. identify a unique resource. There is no converse requirement that entities associated to resource IDs should be mononymous, viz. have just one URI associated to them. There would be no way to enforce it. For TF, this means that though a given TF URI should identify just one terminology entity, there is nothing to prevent one terminology entity to have many URIs aliased to it. This is a useful insight, because it allows a variety of different URIs associated to a given term or other entity depending on need. For some purposes (decentralised creation, search, or debugging), a URI descriptive of the thing named is useful. For other purposes (say, obfuscation, channel independence) an encrypted or character-encoded version of a descriptive URI, or just a simple anonymous numeric URI (version maintenance), may be preferable. The TF rewrite utility supports some forms of such aliasing.

Descriptive identifiers

The main reason to worry about the choice of URIs for expressions and terms is what is sometimes termed the ontology hell . That is the situation (already present) where effectively the same concept is reinvented many times with only slightly varying URI and essentially the same meaning by many authors, who then face the problem of ontology matching to find if they are really talking about the same thing or not. The problem of polysemy besetting natural language is just replaced by an equally confusing problem of synonymy, known in terminology theory as the harmonisation problem between different terminology standards. This problem can be alleviated (not removed) by a naming convention that exposes the similarity of competing conceptualisations.

The TF descriptive naming convention for expressions and terms is a compromise between a human readable name and a perfect hash. The idea is to choose key properties of an expression/term and form the descriptive name from them. The properties are supposed to be real keys in the relational database or OWL 2 hasKey sense, so that they uniquely identify the expression/term. Two different items should not end up with the same descriptive name, and one item shoud get only one such name. The main brunt of identification is borne by the site URL prefix. The local name of an expression is formed from the expressions language code, base form, and category (usually, part of speech code is sufficient, but it can be another sufficiently distinguishing tag if not). Different authors should be able to arrive at the descriptive name of an item independently and be reasonably confident that another author adopting the same or similar descriptive name is after the same or a closely related notion. What we want to avoid as far as possible is the need of a separate global catalogue of resources.

The three parts of the local name are separated by hyphens. For example, the English language greeting 'hi' in TFS (if it belonged to the TFS vocabulary) would get the descriptive URI

http://tfs.cc/exp1/en-hi-S

The descriptive label for a term is formed by concatenating the descriptive label of its designation with a namespace prefix for its referent and the referent's local name, separated by hyphen. Since namespace prefixes are not globally registered, this naming convention is only suggestive of the referent of the term, and its main purpose is to serve as a mnemonic distinguisher. The designation and referent parts of the term label are separated by the string "_-_". For example, if TFS had a concept for greetings whose prefix name were sem:Greeting , the sense of English "hi" as a greeting could be labeled descriptively as

http://tfs.cc/exp1/en-hi-S_-_sem-Greeting

Hyphen and underscore are used as separators in the labels because they are the least reserved separator like characters in the many Semantic Web character sets. These separator strings should not occur inside the parts they separatein a way that could cause ambiguity.

The validity of the assumption that the key properties of a descriptive identifier are really keys can be stated and tested in OWL 2 using the OWL 2 construct HasKey. It allows defining keys for a given class. An HasKey axiom states that each named instance of a class is uniquely identified by a (data or object) property or a set of properties - that is, if two named instances of the class coincide on values for each of key properties, then these two individuals are the same.

There is no equally catholic naming convention for concepts in TF. However, when the rewrite relabeler finds a concept without a name associated to a term that has an expression designating it, the relabeler tries to use that expression's base form to generate a camel cased name for the concept. For instance, a blank concept designated by an expression with base form 'concept without name' will get a URI with local name ConceptWithoutName . This convention can be useful when converting third party word lists to ontologies.

TFS has not got a descriptive labeling convention for URIs of multilingual messages or their texts. The justification is that texts have a weaker identity. They are less generally reused, have fewer unpredictable properties, and are less likely to get accidentally reinvented by different authors, so there is less motivation for them to carry their identities on their sleeves.

An important advantage of using descriptive URIs for TF expressions and terms, i.e. ones that code the key identifying properties of the resource in the resource name, is that such URIs are least likely to cause ambiguity in situations where the creation of new URIs is not centrally controlled. Namespaces avoid name collisions across sites, but inside a given namespace, contention between URIs is at least more easily detected and remedied when the name is descriptive of the resource. A corresponding weakness of this convention is that terms which differ only insignificantly (say, by spelling variant) may get created alongside one another and need to be mapped as equal or related after the fact. This danger could be minimised by additional conventions on the choice of representative name. On the other hand, even with a slightly leaky convention, the similarity of close variants is less likely to go undetected.

Besides genuine aliasing where different URIs are used to point to the same TF entity, resource identifirs may need to get character encoded to conform to different standards. Non-Latin script support in today's Semantic Web tools is surprisingly weak and the character conventions in different SW standards and tools far from uniform ( Auer et al. 2010 ). In OWL 2.0, ontologies and their elements are identified using Internationalized Resource Identifiers (IRIs) [RFC3987]; while OWL 1.x uses Uniform Resource Identifiers (URIs). For some purposes, it is safer to use URI-encoded (percent encoded) versions of non-ascii URIs.

Absolute TermFactory URIs like http://tfs.cc/ont/Concept or http://localhost:8080/TermFactory/query?uri=http%3a%2f%2ftfs.cc%2f%ont%2fEnglish can contain a scheme (protocol) http:// , authority (host:port) localhost:8080 , path /concept , and query ?query=... . A fragment identifier after hash sign like #Concept at the end of a URI is not officially part of the URI, but just a URI reference. Such references are often used for ontology concept identifiers (though not in TF). A relative URI is a suffix of an absolute URI with scheme left out.

TermFactory URIs

This section proposes naming conventions for TermFactory resource URIs.

Hash vs.slash vocabularies

A common convention for naming resource URIS is to append the local name of a resource as a fragment identifier to the name of the ontology, separated by the cross-hatch or hash character #. This convention suggests that an ontology URI like http://tfs.cc/ont0/ctryCode should point to a location in an ontology document at URI http://tfs.cc/ont0 . In actual fact, this is not how it works most of the time.

As things go, an ontology resource URI like http://host/path#Resource may not resolve to any document in the websphere. If there is an ontology document at http://host/path at all, unless the document is HTML, the fragment identifer after the hash (#) will not single out any particular part of it.

Besides, it in general makes little sense to think of an ontology resource as a fragment of any one particular ontology document, since TF resources can be described at many locations in one document and in many different documents. A technical difficulty of the hash vocabulary convention is also that fragment identifiers are not part of the http url that gets sent between servers. A client can only ask for and receive a complete document from a server. Fragment identifiers are only meaningful for clients. (For discussion, see RDF best practices , WordNet URIs , Jeni Tennison's blog , and Hebeler et al. 2009:58 ).

A better design decision for TF ontologies is to use resolvable URLs (known as slash vocabulary) for ontology resources to begin with. TermFactory's own vocabulary is a slash vocabulary. However fragment identifiers (hash vocabularies) are in general use, so TF had better have ways to handle them too.

Redirection in TermFactory

The Semantic Web addressing orthodoxy is to serve a document from its URL. But documents at given URLs may not be always easily accessible to all, and certainly not editable by all. Often one needs to hold copies of a document obtained from a given URL. So we need a way to keep a mapping from the URL to the copy we want to access at a given time.

A whole host of solutions have been invented in the websphere for web address resolution, forwarding or URL redirection . Jena invented location mappings for remapping RDF documents and ontology policy files for remapping OWL ontologies. ( Protege 3.1 used ont-policy files. Protege 3.2 had a home-grown repository mechanism.) XML provides XML catalogs. ( Protege 4 uses them.) In addition, client or server side scripts, web servers and application containers have URL mapping or rewriting facilities. Finally, domain names can get forwarded.

This chapter covers different ways of redirecting resource identifiers (URIs) and web addresses (URLs) in TF. There are many, for different needs. One is specific for TF, others are borrowed.

• TF location mappings and the retry utility
• Jena RDF location mappings and OWL ont-policy documents
• webpage forwarding with apache php
• webpage forwarding with Tomcat servlets
• Tomcat url rewriting
• Apache url rewriting

TF location mappings

The TF retry location mapper is a general finite state string rewriting mechanism. It can rewrite any string at all into another string, not just a web identifier or web address of a resource. It serves to localize and globalize (web) addresses much like the rest of TF serves to localize and globalize (resource) names.

The TermFactory back end uses the retry facility and jena style location mappings to redirect ontology urls. A given uri, e.g. http:tfs.cc/ont/Finnish can be retrieved with a query string of form

http://localhost:8080/TermFactory/query?url=http%3a%2f%2ftfs.cc%2fowl%2fTFS.owl

The url to be sought is given here as value of parameter named url in URL encoded form. What the TF query service does with a request of this form is try locating the url using TF location mappings repeatedly until a hit is found. The mappings are given in a jena rdf format location mapping file whose name is specified in the TF option settings file tf.properties , by default, it is called file:etc/location-mapping.n3 . (The file, like the rest of TF settings, are kept in the TF web service directory.)

Formats

This section documents data formats handled by TF.

Semantic Web file formats

There are a variety of formats to represent a RDF graph as text. The normative (standard) syntax for RDF is RDF/XML , an XML document format for RDF triples. Unfortunately, XML makes RDF look more complicated than it is. Moreover, there is no canonical or normal form for RDF, so different tools may generate different (though equivalent) serialisations for the same RDF. (For a remedy, see below .)

A relatively human-friendly syntax for RDF is Turtle. N3 has several features that go beyond a serialization for RDF models, such as support for RDF-based rules. Turtle is a simplified, RDF-only subset of Tim Berner-Lee's Notation 3 . Here is the result a TF query around the concept of ISO country codes in Turtle format.

Show/hide Turtle

The document starts with a long list of namespace abbreviations (prefixes). Turtle RDF triple terms are separated by whitespace, except quoted strings can contain whitespace. Minor punctuation can be used to fold together similar triples as follows. Triples that only differ by object can be written by writing the common subject and predicate followed by a comma-separated list of the objects. Triples that share subject can be written by writing the common subject followed by a list of predicate-object pairs between semicolons. Each group ends in a full stop.

In Turtle, (absolute or relative) URIs are surrounded by angle brackets. Directive @base can be used to resolve relative URIs against given base URI. By default, the base is the URI of the document. In addition namespace prefixes can be defined with directive @prefix . Turtle base and prefix work the same way as corresponding XML attributes explained below.

The Turtle format handles unabbreviated non-Latin IRIS fine. In Turtle, character encoding problems are avoided by foregoing prefixes and keeping resource names as unabbreviated IRIs (e.g. <http://biocaster.nii.ac.jp/biocaster1#鳥インフルエンザ-N>). Turtle is more restrictive about qualified (prefixed) resource names. More precisely, Turtle qualified names allow XML qualified name ( QName ) characters except dot \u002e. TF3 encoding can be used for encoding non-ascii URIs in Turtle as prefixed resource names.

Blank nodes can be written with prefix _: followed by an arbitrary ID. Alternatively, a node can be written with a matching pair of left and right brackets in front or around its properties: [] :property :value or [ :property :value ] .

Literal strings are surrounded by single or triple double-quote characters (triple quotes include newlines inside them). The datatype of a literal is written after the literal linked to it with two carets (^^).

Below is an example of the TF entry for the concept of ISO country code in RDF/XML format.

Show/hide RDF/XML

The root element (tag) of the document is rdf:RDF, and the content is a list of rdf:Description elements, describing resources (rdf nodes). Ontology namespace prefixes are declared by the xmlns attributes. Namespace prefixes only help abbreviate uris. They make it easier to write and change uris textually should the namespace change. They are local (so they can change between rewrites) and have no fixed meaning (except for prefixes reserved by w3c, such as xml, rdf, rdfs, owl ...) The empty XML namespace prefix xmlns: defines bare XML element and attribute names.

The xml:base URI attribute sets the base URI for resolving relative RDF URI references. By default, the base URI is the URI of the document. The xml:base URI applies to all RDF/XML attributes that deal with RDF URI references: rdf:about, rdf:resource, rdf:ID and rdf:datatype.

The node described by a rdf:Description is identified by an rdf:about, rdf:ID or rdf:nodeID attribute (if at all). A rdf:ID attribute on a node element can be used instead of rdf:about. It has the hash character built in, so that rdf:ID="name" is equivalent to rdf:about="#name". While there can be any number of descriptions rdf:about a given resource, rdf:ID is an XML ID, so it can only appear once in the scope of a given xml:base.

Blank nodes (anonymous nodes which have no resource URI) can be given an xml document local ID with attribute rdf:nodeID. This is needed when same blank node is referred to more than once. The node ID is local, meaning it may change between rewrites of the same document.

To recap: XML entity names can be used to abbreviate URIs in XML attribute values. XML namespace prefixes are used to abbreviate URIs in element names. xml:base attribute allows truncating absolute URIs into relative ones in its scope. These are all XML document related abbreviatory conventions to avoid writing URIs in full, nothing to do with the RDF graph or OWL ontology being described.

Here is a representative beginning of an archetypal OWL RDF/XML ontology document :

TF ontologies typically contain resources from many different namespaces. Resources in them are identified by explicit prefixes or entity references instead of relative URIs. TF ontology URIs identify documents. The document/ontology URI is not related to the URIs of the resources defined inside the ontology. For instance, TF Schema ontology uri is http://tfs.cc/owl/TFS.owl, but no other resources in it have the same prefix.

RDF/XML can be abbreviated in various ways (see the standard ). A variant format called RDF/XML-ABBREV groups and nests triples sharing subject, predicate, or object nodes inside one another much the same way as Turtle does.

Besides Turtle and RDF/XML, OWL has a number of formats of its own. OWL 2 has a normative functional-style syntax . These OWL specific formats are not used in TF for now. Nothing prevents using them with third party tools that support them. There is an online converter between different OWL syntaxes at http://owl.cs.manchester.ac.uk/converter/restful.jsp .

A naive term list enumerates terms one by one, listing all properties associated directly or indirectly to each in one go, without distinction: properties related to its referent(s), lexical properties, and properties related to the term as such. This shallow end of the term pool is what we call TF Lite. At the deep end, the commonalities between terms that share meaning is reified to a separate entity, the shared meaning, and the shared semantic properties associated to that, and the meaning associated to the term. Symmetrically, the commonalities between terms that share lexical and grammatical properties are collected to a new entity, the shared expression. Intermediate cases are obtained by making or not making the term/concept or term/expression splits individually. The aim is non-redundancy: every property of a resource must provide a fact about the resource, the whole resource, and nothing but the resource, "so help me Codd".

Nothing is absolute here: terminology is an applied science. What seems one and the same concept in one analysis can get refined into many later. By parity, the category of expressions allow refinement: what counts as an expression for one purpose can become a term with a meaning under further analysis.

Orientation, literally, has to do with the orientation of the rdf graph around ("about") a node in the graph. A given layout is determined by the choice of root and the order of traversing the edges of the graph to obtain a spanning subtree that constitutesa tree formed hierarchical representation of it, and, eventually, the choice of serialisation of the tree into a one dimensional stream of characters.

It is a hallmark of the TF approach to separate these concerns from the real objects terminology is about, namely terminological resources: instances, classes, properties, and statements connecting them. At the same time, the TF abstraction of terminological content from any fixed grouping or ordering of data leaves room for different mind sets or approaches to terminology work. Such approaches can be seen as different choices of granularity and orientation. Different choices may fit different tasks.

The Western standard normative approach to terminology founded by Eugen Wüster, codified in ISO terminology standards, aims at standardisation and harmonisation of international terminology. This starts with language independent concept analysis, where the expert community agrees on the referents of special language terminology, whatever they are called. In standardization, choices are made of standard expressions to designate the referents in each language. In harmonization, national differences are ironed out. The Western view of a well-behaved term comes quite close to the TF Full profile of the TF schema, depicted above .

Traditional descriptive lexicography distinguishes between expressions (lemmas) and terms (word senses). A traditional lexicographical entry is rooted in a lemma and enumerates the senses of the expression in some order usually based on partial similarity of meaning or usage of the senses. A bilingual dictionary entry is rooted in a target expression and lists target language expressions arranged in a similar way by shared or at least similar senses of the source and the target expressions.

Further approaches are possible. A term oriented approach (Kudashev) can wind out the graph starting from a term as root and enumerate other terms which the root bears selected (e.g. semantic) relations with, in the same language or in other languages. Such a contrastive layout can be particularly useful for translators.

This freedom from fixed structure offers both advantages and disadvantages considering format conversion. It can make conversion toward TF easier, because the target format is relatively free; no particular topology need be followed in converting to TF, so conversion need not change the source topology. On the other hand, it means that conversion from TF to more rigid formats can involve search (like queries from relational databases). A query language supported by a reasoner can be of much help here. For one thing, just because RDF/OWL serialisation to XML is not constrained, the resulting topology is rather unpredictable. This makes applying XML tools like XSLT to RDF/XML fragile. A standard serialisation of TF to some (more) fixed XML format, say TBX, is one solution to this. Another one is to divide and conquer the problem using TF profiles .

The TF XHTML format allows choosing between concept-oriented, term-oriented and lemma-oriented layouts.

TF terms can be distributed through the collaborative interfaces, they can be queried directly from the repository system, or they can be converted to other existing formats and distributed through existing distribution channels (electronically, in print, by phone etc.) Formats handled by TF will include at least the LISA TBX terminology exchange format, the commercial SDL MultiTerm xml export format. Further formats can be added on demand.

LMF

ISO 24613:2008, Language resource management - Lexical markup framework (LMF), is the ISO International Organization for Standardization ISO/TC37 standard for natural language processing (NLP) and machine-readable dictionary (MRD) lexicons. The scope is standardization of principles and methods relating to language resources in the contexts of multilingual communication and cultural diversity.

LMF appears to follow the same Saussurean model as TF, except that a LMF lexical entry allows many senses. This is the lexicographical notion of a lemma-keyed lexical entry.

• Show LMF example

Conversion

Third party ontologies can be imported to TF in different ways depending on the available formats and the character of the relationship between TF and the third party contents.

If the third party ontology (call it O3) is already in a TF native format, there is nothing to do but link it to other TF ontologies. In practice, this usually means building a bridge ontology that imports O3, TF schema, and possibly other related ontologies and statements that help bridge O3 to the TF schema. For example, an ontology of ship diesel engines can be related to TF via a bridge ontology that loads a generic engine ontology, the TF schema, and contains bridge statements that place its classes and properties inside the engine ontology and/or relate its classes and properties to TF schema.

Another example is the BioCaster epidemic ontology, which is an OWL ontology that describes diseases and their names in many languages. Though its syntax is OWL, the class and property structure differs from TF Full (it is best construed as an instance of TF Lite). Here, there are two extremes (with various intermediate mixtures), that are analogous to compilation and interpretation in computing.

The syntactic extreme is to write a conversion script that converts BioCaster classes and properties to corresponding TF classes and properties, perhaps factoring in new TF style instances, classes and namespaces. The conversion script can work at different levels of the Semantic Web language stack (OWL/RDF/XML). On the RDF/OWL level, conversion can happen by a program using some RDF/OWL API like Jena, or it can be written as a sequence of SPARQL CONSTRUCT queries. The advantage of doing an RDF/OWL level conversion is that the conversion is robust against variations in file formats. Or one can write an XSL script to transform an RDF/XML ontology file to another, TF compliant file. XSLT scripting is fast, but the disadvantage is that the conversion becomes dependent on the many variations of the RDF/XML format. The advantage of doing syntax conversion is that once converted, the third party ontology is a native TF ontology that can be queried maintained using TF tools without any intermediate steps. On the minus side, if the third party ontology changes outside of TF, a new conversion is needed.

The semantic extreme is to leave the third party ontology unchanged and relate it to TF in OWL using bridge statements and/or rules. For instance, a bridge statement could relate BioCaster property englishTerm to TF term:referentOf as a subproperty with appropriate restriction on it, saying the target term must be an English term. The advantage of this option is that less (perhaps no) conversion is needed, as the work relating the ontologies is done by the reasoner. The third party ontology can evolve freely and stay available for TF queries at any time. On the minus side, the third party ontology cannot be used in TF without a reasoner that does the bridging. If reasoning is done at runtime, this will slow down response times.

Semantic conversion using sparql queries can take the form of an OWL ontology that (recursively) imports sparql CONSTRUCT queries on one or more other ontologies. This type of conversion document gets automatically updated whenever the imported ontologies change. Although a conversion may be too complicated to do with one sparql query, by query imports it is possible to join results of several queries in parallel or in series so that the final composite outcome is the desired one. This idea of virtual conversion is discussed in the section on TF profile conversion .

A variety of ontologies and third party terminology collections in legacy formats have been imported or converted to TF form to test the TF concept. The 4M project ontology is the historical starting point of TF. The mobilite space ontology was imported as is, as an example of importing a third party OWL ontology. A fragment of the Finnish YSO library ontology (in OWL Full) was extracted as an example of a subject field ontology. A paper industry vocabulary was converted from MultiTerm format. A third party glossary of building management terms was chosen as an example of importing from a legacy format first to TBX and from TBX to TF. An example of ad hoc conversion a multilingual glossary of chemistry terms converted directly from excel sheets to TF OWL/XML using perl.

TBX to TF conversion

This section is a guide to the conversion of a terminology in the Term Base Exchange (TBX) format into TF.

(version 0.0) The converter tbx2owl.xsl is an XSL(T) 1.0 script which transforms a TBX xml document into an OWL RDF/XML document.

TBX Basic is a simpler subset of TBX. An XSL 2.0 script tbx2tfs.xsl converts TBX Basic documents to TF. The converter comes with a separate rdf/xml document tbx-mapping.rdf (by default, located in $TF_HOME/etc ) which spells out the element by element correspondences between TBX data categories and TFS. (The mapping file format is rdf/xml in order to be applicable as is both ways.) The mapping file consists of correspondence rules like

<rdf:Description> <tbx rdf:parseType="Resource"> <name>termNote</name> <type>partOfSpeech</type> <text>noun</text> </tbx> <tfs> <exp:Designation> <exp:catCode>N</exp:catCode> </exp:Designation> </tfs> </rdf:Description> <rdf:Description> <tbx rdf:parseType="Resource"> <name>termNote</name> <type>partOfSpeech</type> <text>&DUMMY1;</text> </tbx> <tfs> <exp:Designation> <exp:catCode>&DUMMY1;</exp:catCode> </exp:Designation> </tfs> </rdf:Description> <rdf:Description> <tbx rdf:parseType="Resource"> <name>descrip</name> <name>other</name> <text>&DUMMY1;</text> </tbx> <tfs> <owl:Individual> <term:hasDescription> <term:Description> <exp:Text> <exp:text>&DUMMY1;</exp:text> </exp:Text> </term:Description> </term:hasDescription> </owl:Individual> </tfs> </rdf:Description>

The tbx node of a rule describes the TBX element in terms of the element name, type and content. The tfs node describes the TF side. The top node defines the subject of the target graph. The graph under it gives the properties that in TF correspond to the TBX element. &DUMMY1; is a variable indicating content shared by both sides. other is a dummy for "other" tbx types. A match is exact if context, name, type and content all match, and a partial match if context and name match. A partial match is more exact if type matches. The logic followed by the converters TF2TBXWriter.java and tbx2tf.xsl in matching such correspondences is as follows.

1. A given input matches a correspondence if it is an exact match to the source element.
2. A partial match matches input except for &DUMMY1; and other .
3. An exact match is preferred over partial matches, a more exact match is preferred over a less exact match.
4. The first match wins if there are many.
5. A target element is written only if context matches.
6. If there is no target element nothing is written.
7. Text matching &DUMMY1; in the source is substituted for &DUMMY1; in the target.

This logic supports various strategies:

• In order to change some values and keep the rest, write exact match rules and a dummy to dummy general rule.
• In order to keep some values and suppress the rest, write exact rules and a dummy empty target rule.
• In order to pass values except some, write a dummy general rule and exact match empty target rules.
• In order to suppress values except some, write a dummy empty target rule and exact match rules.
• In order to convert values (except some) to a default, write a dummy to default rule (plus some exact rules).

The current version of the converter only handles one shared variable.

TF to TBX conversion

The TFS to TBX converter is a Jena writer TF2TBXWriter.java embedded in the TF rewrite utility . It can be run with a command line like

rewrite ctryCode.owl TBX > ctryCode.xml

The converter uses the same rdf document (by default, etc/tbx-mapping.rdf ) as the TBX to TF converter for finding conversion correspondences.

With rewrite command line switch verbose , the converter prints information about mapping rule matches:

rewrite file:Place.owl TBX verbose ... </termEntry> <termEntry id="http://tfs.cc/term/hasDefinition"> <descrip type="instanceOf">owl:ObjectProperty</descrip> <!-- *** No rule: *** http://www.w3.org/2002/07/owl# rdfs:subPropertyOf term:hasDescription --> ...

The conversion mapping between TF and TBX can be extended using bridge ontologies and a reasoner. Bridge ontologies define correspondences between TF properties and TBX data categories. Using a reasoner, properties in the TF ontology which are not covered by rules in the mapping file can be rephrased into properties which are covered there. For instance, it may be enough to define a general mapping rule for the class meta:Description and let the reasoner and bridge ontology entail the mapping from a variety of different types of description to that common class.

The general insight from our conversion efforts is that syntactic rewriting in terminology format conversion can be significantly simplified by using TF semantic conversion (RDF/OWL entailment with bridge axioms) as an intermediary. Also the conversion pipeline becomes more transparent because the TF internal conversion steps have a clear semantics.

Show/hide TF conversion

TF to cparse conversion

A TF terminology can be converted into multilingual lexicons for the constrained language parser/generator cparse . (version 0.0) The converter (Terminator2) is implemented in Java using the Jena RDF/OWL library.

MultiTerm2xhtml and MultiTerm2FO

Conversion scripts MultiTerm2xhtml.xsl and MultiTerm2FO.xsl convert MultiTerm vocabularies into xhtml and FO formats. Using the Apache FOP processor, MultiTerm vocabularies get further converted into multilingual PDF (including simplified Chinese).

Conversion scripts MultiTerm2xhtml.xsl and MultiTerm2FO.xsl convert MultiTerm vocabularies into xhtml and FO formats. Using the Apache FOP processor, MultiTerm vocabularies get further converted into multilingual PDF (including simplified Chinese).

The conversion used the following commands. The sed script does tf3 encoding and fixes entity references.

xsltproc fiwn-sumo.xsl fiwn-all.xml > fiwn-all-sumo-raw.owl saxon -xsl:fiwn-vunl.xsl fiwn-all.xml > fiwn-all-vunl-raw.owl sed -f tf3encode.sed fiwn-all-sumo-raw.owl > fiwn-all-sumo.owl sed -f tf3encode.sed fiwn-all-vunl-raw.owl > fiwn-all-vunl.owl

The vu.nl conversion needs synset-align.ttl for vu.nl synset URIs. After the conversion, doctype entity declarations in fiwn-doctype.txt are inserted in the prolog just after the xml declaration. The namespace for the Finnish translations is provisionally http://tfs.cc/wn (set in the xml:base attribute in the xsl scripts).

The following scripts extract a sample collection of words, senses, and meanings from each the two WordNet conversions and their corresponding Finnish translations. The result can be checked in Protege by loading either pair of files and classifying the ontologies with a reasoner. (For the vu.nl conversion, also load schema-align.ttl that defines vu.nl property inverses.)

# vunl sparql --query d-nwordsynsets-vunl-1000.sparql --data ./download/raw/rdf_wordnet-synset.ttl --data ./download/full/rdf_full_wordnet-wordsense-synset-relations.ttl --data ./download/full/rdf_full_wordnet-wordsensesandwords.ttl > d-nwordsynsets-vunl-1000.ttl sparql --query d-fiwordsynsets-vunl-1000.sparql --data fiwn-all-vunl.owl --data rdf_wordnet-synset.ttl > d-fiwordsynsets-vunl-1000.ttl #sumo sparql --query d-wordsynsets-sumo-1000.sparql --data wordnet-sumo.owl > d-wordsynsets-sumo-1000.ttl time sparql --query d-fiwordsynsets-sumo-1000.sparql --data fiwn-all-sumo.owl > d-fiwordsynsets-sumo-1000.ttl

The above scripts take of the order of ten minutes to run per entry. In order to mass produce synset entries, we indexed the rdf descriptions in TURTLE format wordnet files by synset id and split the description files textually at paragraph boundaries into synset entry size files. The Finnish translations file fiwn-all-vunl.ttl and the relevant vu.nl English WordNet files are shown below with sample contents.

./fiwn/fiwn-all-vunl-ttl fiwn:fi-Akheron-joki-N rdf:type wn20schema:Word ; rdfs:label "Akheron-joki"@fi ; wn20schema:sense fiwn:fi-Akheron-joki-N_-_WN30-109186709 . ./vunl/download/raw/rdf_wordnet-synset.ttl wn30:synset-1530s-noun-1 wn20schema:synsetId 115148787 . ./vunl/download/full/rdf_full_wordnet-wordsensesandwords.ttl: wn30:wordsense-zymurgy-noun-1 wn20schema:word wn30:word-zymurgy . ./vunl/download/full/rdf_full_wordnet-wordsense-synset-relations.ttl wn30:synset-zymotic-adjective-2 wn20schema:containsWordSense wn30:wordsense-zymotic-adjective-2 .

The translations are already indexed by synset id. We used sparql to index the vunl words and senses files with the synsetId datatype property present in the synset file. These synset-indexed files were then split per synset id into over 100K entry files. This took just over an hour wallclock time.

time perl split.perl finnish.ttl synset-relations.ttl senses.ttl real 61m33.559s user 0m27.950s sys 0m31.198s

s(Synset_ID,W_num,Word,Ss_type,Sense_number,Tag_count).

Ad hoc conversion to TF

As an example of conversion from a typical tabular file format (entry per row, concept and language sets per column) through TBX to TF, we converted a multilingual welfare vocabulary via TBX Basic to TF. The table-to-TBX conversion was made with a simple Perl script txt2tbx.perl .

A conversion from TF back to puls ontologies is in the plans. It will allow PULS to use ontologies developed in TF without changing the internals of the PULS system.

TF schema profiles

TF profiles harp on a theme familiar from database design and relational database normalisation, now transposed to the ontology instrument. The question is what players on the terminology field become resources, i.e. first class (= first order) entities, and which can remain virtual (classes, properties and roles). Thre are two reasons to reify some feature into a resource: if it can have properties, or if positing it makes the model smaller. An answer to this question is also implicit in the various terminology theoretical definitions of what a term really "is".

An old standard, DIN 2342 (Begriffe der Terminologielehre) defines a term as „zusammengehörige Paar aus einem Begriff und seiner Benennung als Element einer Terminologie.“ TF Full conforms to this definition.

In contrast, the ISO 1087-1:2000 definition of term reads:

verbal designation (3.4.1) of a general concept (3.2.3) in a specific subject field (3.1.2)
Note:
A term may contain symbols and can have variants, e.g. different forms of spelling.

One (perhap not the intended) reading of this definition is that a term is a verbal expression, considered in the role of a designation of a general concept. (Another ISO formulation is 'Designation of a defined concept in a special language by a linguistic expression' [ISO 1087:1990] is vague between two construals.) A literal ontological reading of the latter definition would be

term:Term rdfs:subClassOf exp:Expression , [ rdf:type owl:Restriction ; owl:onProperty term:designationOf ; owl:allValuesForm ont:Concept ] .

In this construal, term:Term is a role of exp:Expression in the technical sense of OWL. It is that subclass of exp:Expression whose members designate special language general concepts. This reading is a variety of the TF Legacy profile. On this reading, terms/expressions shared between domains are not just similar, they are the same. This is because here term:Term is not reified into an individual. An expression may have many names corresponding to its roles as designation for different concepts, but the names denote the same entity. The terms cannot go with have conflicting term properties, or conflicting term and expression properties (say, owner), since they denote the same thing.

More specifically, we can distinguish two limiting case profiles for TF schema, provisionally called TF Lite and TF Full. TF Lite is the base case, other profiles are implemented as additional axiom sets on the TF schema. TF Lite adds no axioms. Intermediate profiles differ in whether they separate concepts from terms (Legacy) or expressions from senses (Dictionary). In Legacy terminology work, expressions need not be separated from terms, since in normative monolingual terminology expressions are monosemic. In descriptive Dictionary work, where word senses are assumed to be unique (no full synonyms), there is no need for language-independent concepts. TF Full separates all three. The three-way separation is motivated in multilingual cross-domain term collections.

There is an asymmetry between the two separations. The separation of terms from expressions is fine tuning. In Legacy, a term is a role: a term IS an expression that denotes a given concept. Also TF Full allows term:Term to be a subclass of exp:Expression. Terms can stay a special case of expressions. Mixing concepts with terms causes an object-metalanguage metonymy.

A term and a concept are on different sides of the Tarskian semantic divide: the former is a piece of language, the latter a piece of the world. For a concept (class) to include another is thus different from a term (expression) to include another. Men are included in animals, but the word (sense) man is not included in (the word sense) animal . (If anything, man , as a string, is included in woman ). On pain of category error, the TF Dictionary profile must distinguish meta level entailment properties between terms to match object level denotational relations between concepts. E.g. sign:hyponymOf, sign:synonymWith, sign:meronymOf are term instance properties matching rdfs:subClassOf, owl:equivalentClass, ont:partOf in TFS concept (class/instance) vocabulary. Which vocabulary should be used between WordNet synsets depends on whether synsets are construed syntactically as (representatives of collections of) signs or semantically as instances of class Meaning. (We choose the latter.) The relation of two terms that share the referent is sign.synonymWith . It is also the relation holding between a term and and a terminological definition of the same concept. This property lets us associate a definition text to a term sharing language with it, while maintaining with terminology theory that a definition defines the concept, not the term.

Comparing Lite semantics to Full semantics, the main difference is whether meaning is treated as a first order (syntactic) relation between terms or a second order relation between classes. Technically, the distinction is between using application specific instance level properties like sign:hyponymOf or OWL properties like owl:subClassOf . The former support dealing with taxonomies of named entities. OWL class semantics provides the power of description logic reasoning: Boolean and some relational reasoning about classes defined by description not just by name.

To support the different semantics, tools will be provided to rewrite an ontology from one level to another. Among other things, this simplifies conversion to TF. Conversion libraries only need to support direct conversion to the closest subset of TF. TF specific SPARQL conversion scripts automate upgrading/downgrading between levels by automating the necessary splits/merges. TF Lite may be an easier target to convert to from contrastive terminologies of the lexicographical type. On the other hand, TF Full is more transparent to OWL reasoning. See section on TF profile conversion .

TF Lite

A TF Lite ontology does not require that the expressions, terms and concepts are disjoint. TF Lite need not split terms, concepts, and expressions into different resources. The same resource can play many roles. Since TF does not apply the unique name convention, a given resource can be given different aliases in different roles. These properties of TF Lite leave many descriptive options to play with. One and the same resource, perhaps aliased using different URIs, can play all three roles of term, concept, and expression.

Descriptive translation-oriented contrastive terminography does not start from, or aim at, interlingual concept harmonisation, but operates with pairwise term comparisons. In a TF Lite implementation of this approach, terms start out as undivided entities. At the start of the game, we collect a cluster of similar terms like fi-metsä-N_-_FI-Forest , en-wood-N_-_GB-Wood , en-forest_-_GB_Forest , ru-les-N_-_RU-Les . Instead of creating separate instances for term en-forest_N_-_GB-Forest , its expression en-forest-N and meaning GB-Forest , we just use en-forest-N and GB-Forest as aliases for the term itself. Comparisons between terms are reified by contrastive term instances which have the source term of the comparison as the referent and the target term of the comparison as the expression.

For instance, comparing English forest to wood , the latter appears to be a narrower term than the former (a wood is a small forest). This observation is reified into a contrastive term en-wood-N_-_GB-Forest . This contrastive term is marked as an approximate match, specifically, it has the property term:approximate "sub" . A sample of this approach is shown below.

The description of the same contrastive facts in TF Full, where terms, concepts and expressions are separated, is not very different. We eliminate each contrastive/approximate term by replacing it with a new exact term whose concept is related to that of the contrastive/approximate term with rdfs:subClassOf as appropriate. To wit: if approximate term E -_R has property term:approximate "sub" , and term E_-_C is exact, then C rdfs:subClassOf R . If E_-_R has property term:approximate "super" , then R rdfs:subClassOf C . If E_-_R has property term:approximate "true" , then C and R have a nonempty common subclass in common. The best we can do about this is add a class S for the common subclass and assert S rdfs:subClassOf C. B rdfs:subClassOf R . (There is no explicit way in OWL to say that S is nonempty, short of putting named instances into it.) The previous contrastive sample converted to TF Full might look as follows.

Separation axioms

To be precise, separation of terms and concepts can mean just conceptual separation where a resource is typed as either a term:Term or ont:Concept, and it has concept or term properties according as it belongs to one or the other of the classes. Relations like term:hasReferent relate members of the former class to members of the latter class. This conceptual separation does not imply that terms and concepts are disjoint. It only concerns domains and ranges of properties and class membership. It is still consistent for a term and its referent to denote the same resource, even to have the same URI. If they are not, then we may say that the separation is strict.

A TF ontology can be tested for strict conformance to one of the TF profiles using separation axioms in LegacyStrict.owl and DictionaryStrict.owl . LegacyStrict checks that signs are separate from their meanings. DictionaryStrict asserts that signs are separate from their forms. The following Pellet queries check whether signs, forms, and meanings can be separated in ContrastiveLite.owl:

pellet consistency LegacyStrict.owl ContrastiveLite.owl pellet entail -e LegacyStrict.owl ContrastiveLite.owl

However, as was pointed out earlier, the points of the TF triad are relative, not absolute. While it makes sense to make semiotic relations like term:hasReferent irreflexive, so as to separate object and metalanguage levels, categorical separation axioms between the members of the semiotic triad are not strictly true. A special language term can be designated by a general one, like the English mathematical term for plus or minus is designated by the common English noun (sense of the English word) sign . Or a sign can be the referent of another sign; for example, a quote refers to the text without the quotes. Given no unique name assumption, a TF ontology can choose whether URIs for terms, concept instances, and expressions point to different entities. They can be identified using owl:sameAs and separated using owl:differentFrom . The following figure illustrates different ways of cutting the TF pie.

Show/hide TF top ontology

TF profile conversion

Consider what the freedom from ontological commitment in TF means from the point of view of data complexity and terminology evolution. Say we get a simple list of names. We can first enter them as instances of class Term. Sometime later the names get associated with some content. As long as there is no ambiguity, we may associate the content directly to the term instances. There is no need to split terms from expressions yet because there is nothing to share. When a need arises to disambiguate the expression, we may split the shared expression off the different terms. Dually, when a need arises to share content, we can split the shared meaning into a concept and relate synonyms to that. All this can be done just in time, on demand, not ex ante just because the data structure requires it.

Say we find at some point that we need to split a vague concept into two. For instance, the English word parse can be a noun or a verb, denoting the process or parsing or its result. For other languages like Finnish, the two concepts need to be separated, because they go with different words. We can keep the vague concept that holds the shared information and create new concepts/expressions that inherit shared features from the vague concept, and add the distinguishing features to new subconcepts. If the process/result ambiguity is pervasive, it can be coded as an axiom, rule and/or template.

Show/hide concept split

The TF language profiles TF Lite, TF Full, and their intermediates serve among other things to help conversion of content to TF. The idea is that a given third party terminology collection can be converted to the TF profile that is closest to the source structure. Afterwards, when appropriate, the content can be further converted inside TF to conform to another profile.

Experience with various conversions (echa and kyamk glossaries from excel tables and puls from lisp source) showed that conversion from ad hoc formats directly to TF is often easier than conversion through TBX, thanks to the non-hierarchical nature of RDF/OWL, that makes it able to flexibly merge piecemeal partial information about resources obtained from different places in the source.

An experimental MS Excel workbook macro has been written which converts a simple excel tbable of term information into RDF triples conforming to TF Lite. The triples are further stepwise converted into more complex TF profiles using TF internal ontology tools.

TF Lite does not separate concepts and expressions from terms. It is the easiest target to convert to from contrastive terminologies of the lexicographical type.

Virtual ontologies

TF profile conversion on demand can be supported by the TF services as follows. The sequence of sparql queries and rewrite steps that stepwise bring a TF Lite ontology into the TF Full normal form can also be carried out through the TF services and coded in the form of TF query URLs. There are at least two ways of doing such virtual conversion:

• use query import where the query generates the TF ontology from the skos ontology at load time
• use bridge ontology and reasoner to generate TF ontology triples from skos triples at query time

A complete conversion procedure can be coded in the form of an ontology which imports the results of a number of queries on the source ontologies and intermediate ontologies based on the same principle. At the leaves of the imports tree are instances of the ontologies to convert. The conversion ontology represents the conversion process and its result at the same time. The conversion gets in effect rerun every time the ontology is loaded. Since the process can take a while, TF caching is useful to avoid actually doing the conversion at every load. But by proper version control on the cache, the conversion can be set up to run automatically every time the input ontologies change. In this approach, conversion ceases to be a separate offline affair, and becomes an active part of the TF distributed repository access system. The conversion is not just a once-off process but a type of ontology, a virtual TF ontology.

An even looser coupling to TF can be maintained with the help of bridge ontologies. In this case, the third party ontology is not converted at all. Instead, TF queries are carried out against it by bridging its concepts to TF with a separate bridge ontology. An example is the bridging of the BioCaster epidemic ontology to TF with the bridge ontology biobridge.ttl .

The BioCaster ontology is an example of a Legacy profile term ontology. It distinguishes concepts and terms, but not expressions. Here is an example of how terms look in it:

@prefix : <http://biocaster.nii.ac.jp/biocaster#> . @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . @prefix p1: <http://biocaster.nii.ac.jp/> . @prefix owl: <http://www.w3.org/2002/07/owl#> . @prefix xsd: <http://www.w3.org/2001/XMLSchema#> . @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . :Country_235 rdf:type :Country ; :ISOCode "VN"^^xsd:string ; :hasLink :Wikipedia_04977 , :Wikipedia_04976 ; :label "Vietnam"^^xsd:string ; :synonymTerm :vietnameseTerm_1188 . :vietnameseTerm_1188 rdf:type :vietnameseTerm ; :hasRootTerm :Country_235 ; :label "Việt nam"^^xsd:string .

The English term for a country is of type Country and has an English language label. It is linked to its synonyms in other languages which are linked back to it with the hasRootTerm property. Concepts, terms, and expressions are not separated as distinct entities. Biocaster can be construed as a special case of a TF Full ontology where terms are self-referential and self-designative. These identifications can be spelled out as the following bridge ontology:

term:selfReferent rdfs:subPropertyOf term:hasReferent ; rdf:type owl:ReflexiveProperty . :hasRootTerm rdfs:subPropertyOf term:selfReferent . :Country rdfs:subClassOf :englishTerm . :englishTerm rdf:type term:Term ; rdfs:subClassOf [ rdf:type owl:Restriction ; owl:hasValue "en"^^xsd:string ; owl:onProperty exp:langCode ] . term:hasDesignation rdf:type owl:ReflexiveProperty . :label rdfs:subPropertyOf exp:baseForm .

The first two axioms say that root terms are self referential. The next pair express that country names are in English. The last two axioms say terms are designations. Using this bridge ontology, a reasoner is able to parse BioCaster on the fly as a TF Full ontology.

There is a conversion script io/script/convert4tf similar to the localization script io/script/localize for carrying out on-demand conversions like

convert4tf -t bio:Country -e Stacked -X biobridge.ttl biocaster.ttl

One possibility worth considering is to link ISOcat to TF as another term repository through its web service interface, as a collection of virtual TF ontologies. This idea could also work with the ONKI ontology library. However, as of fall 2011, ONKI is not planning to provide a SPARQL entry point (Osma Suominen, p.c.)

Repositories

We now define what we mean by a TermFactory site and a TF repository. A TF site is a web site identified by a URL, maintained by some organization that comprises one or more TF back end repositories, one or more TF front end platforms/tools, and some maintenance staff. A TF repository consists typically of one or more web servers with file space, one or more persistent-ontology databases (for now, usually relational databases), and TermFactory web services. TF repositories can communicate with one another through web services without the mediation of TF front ends.

A TermFactory system of repositories consists of RDF/OWL databases at different sites, connected to one another through a web service protocol. In the following figure, the green cans represent each a site with a TF repository. The arrows represent TF web service connections . The repositories are laid out in a tree to suggest inheritance relationships between the terminology domains that each repository "owns". The situation need not be that simple in practice, sites may also reciprocate (the blue arrow), and all repositories can connect through the protocol. As the image suggests, different (language) regions can take responsibility within each sector for their vernacular terms and reciprocate with their peers for multilinguality.

Show/hide TF repository network

Each database can store several term collections as OWL ontologies. Each terminology collection is named by a common URI (universal resource identifier) and an associated namespace prefix, for instance, prefix tfs abbreviates URI http://tfs.cc and goes with the TermFactory ontology schema. Each site stores those ontologies which it owns and manages, plus it can cache or mirror ontologies which are owned by some other site.

Persistent repositories

A persistent RDF/OWL database can be a triple store or a relational database (MySQL, for instance) which stores OWL data (as ontology triples or some more storage-efficient form). An OWL database manager built on top of the relational database knows how to store OWL ontologies into the database and retrieve them or answer queries about them. The TF web service layer knows how to send queries between the repositories and relay the answers back to the client.

TF includes a Jena TDB native triple store database that maintains RDF graphs in indexed data files on the file system. It can be used by the TermFactory utilities and services as temporary storage. By default, TermFactory queries and edits database models in place instead of reading them in memory. Query results get cached in database if the wait flag is set. A read only ontology gets cached in a database first time it is edited if the database cache flag is set. The default is to have cacheFS flag on and cacheDB flag off. The default assumption is that retry location mappings are set up so that a database cached version has precedence, so that editing continues where it stopped.

Revision control

It may not be worth the trouble to manage TF ontologies as fully dynamic ontology databases with real time shared access, record level locking, etc. Terminology information needs a slower process of collaborative editing on something like a wiki platform (which of course can be backed up or under revision control). Once approved, terminology is moved to a read only section of the database, where it gets updated through batch updates.

OWL revision control literature contains many proposals to log ontology change history explicitly and in full detail as OWL annotations or meta ontologies ( Redmond et al. ). Versioning systems like parsia.com Redmond et al. paper do not use a syntactic or semantic approach, but a pragmatic one, as relations between versions are defined by a history of user actions. An item is related to its predecessors through a history of editing change by a user. We avoid going down this route. In our view, ontology versions should be compared on their current merits, whatever their editing history. Instead, we are thinking in terms of two extremes:

1. syntactic: standard serialisation of ontologies plus existing third party source revision control systems like svn.
2. semantic: a facility to run semantic diffs using a reasoner. Horrock et al. (ref.) show how to test entailment in OWL by doing tabular proof on a source ontology and a complement of a target ontology.

If the tableau construction succeeds, the complement of the closure contains facts not entailed by the source. The semantic diff of two ontologies is their symmetric difference (exclusive or). The first extreme documents editing changes, the second extreme semantic changes.

For TF revision control, there are pre-existing tools for versioning document, memory, and database representations of ontologies. Ontologies as web documents have and surely need both version information and version control. Version information can be provided by OWL annotation properties. Ontology tools like editors allow the user to manage working versions of the ontology under offline development. Persistent ontology revision control, which allows retrieving and comparing to older versions of the same ontology, is best handled with dedicated version control systems outside of the ontology proper. For databases (like mySQL there are ways to back up and recover (restore) snapshots of the database content either as a whole or incrementally (see HowtoForge ). Web content management platforms and Wikis have their own version control solutions. (Cf. a discussion of version control for web content. .

During the TF project, the software and ontologies are being kept under the Subversion revision control system. Subversion has extensions for revision management through websites or local filesystems (e.g. Tortoise SVN for Windows, eSVN for Linux).

OWL documentation discusses version annotation per ontology, but notes that the owl:versionInfo property can be used to annotate classes and properties as well as ontologies. With TF versioning a question of grain of such annotations arises (cf. source indications ). Versioning per ontology may be too coarse, versioning on every triple is too fine. TF conventions for annotation property assignment and property inheritance are called for, with rules for conflict resolution for when a triple inherits more than one source or version indication. In XML based entries like TBX, administrative information is scoped by the term entry tree. Semantic Web reasoners enable stating property inheritance using some rule language. Two main approaches present themselves for TF.

• scoping by query. An entry query retrieves annotation triples relevant for the entry. The scope of the triples is implicitly defined by the structure of the entry.
• inheritance by rule. Annotations for a triple are inferred by a reasoner using explicit rules .

The approaches can be combined. An entry query collects the annotations, and rules distribute them to the component triples.

Normal forms

This section considers normal forms for TF ontology documents. A normal form is a unique choice among equivalent representations. Reduction to normal form by term rewriting is what many reasoners in effect do. Tabular reasoners virtually reduce a model to disjunctive normal form (DNF). Resolution reasoners reduce to conjunctive normal form (CNF). Normal forms for many different logics exist, which can be made unique by sorting terms. Reduction to normal form reduces equivalence comparison to textual identity. More generally, normalization converts semantic reasoning to syntactic processing. Runtime is saved at the cost of offline compilation and storage.

Carroll/Stickler 2004 is an early proposal for an xml normal form for rdf triples. Dau 2006 discusses normal forms for rdf graphs. He defines two extreme normal forms for rdf graphs as concerns the proportion of nodes to triples. One extreme is a normal form where each fact is represented as a separate triple. The other extreme is a representation where every node appears just once. The duality is similar to that between nested tree (graph or matrix) and nonnested (path or mrs) representations of feature structures or Turtle files.

What about OWL? OWL ontologies are theories in DL, which has a a sizable theory of equivalence. There are also a number of proposed CNF normal forms for some description logics ( Hitzler/Eberhart 2007 , Bienvenu 2008 ).

The most promising semantic normal form for TF may be the prime implicant normal form or its dual, the prime implicate normal form. The prime implicant normal form is basically a pruned syntactic version of a tableau (Hintikka model set) construction. (Dually, the prime implicate normal form is a pruned clausal normal form.) Bienvenu 2008 defines PINF for the modal logic K, but it should be extensible to other modal logics with finite model property (Bull and Segerberg ref), including KB, KT and K4. The construction is not efficient (possibly exponential in time and space), but once in (suitably sorted) PINF, two ontologies can be compared syntactically for semantic differences.

Date and user

The XHTML writer records the date and time of the write of a TF document in a meta element in the header of the document in the XML date format . This information is preserved in conversion of the document to other TF formats as a date-valued datatype property meta:date .

When a user submits an edit, the MediaWiki plugin sends the currently logged in user name to the edit service, which records it in the XHTML header as a meta element. This information is preserved in conversion of the document to other TF formats as datatype property meta:user . Other services or operations do not change the last recorded user.

Efficiency

This section surveys ways and means to improve runtime efficiency of TF installations.

DB cache

A TF location mapping retry query takes optional parameters queryDB and cacheDB with boolean values ( true or false ). The value queryDB=true lets the retry utility try default database location mappings (those prefixed with rdb+ ) in etc/location-mapping.n3 . With queryDB=false (the default), the retry facility skips any rdb+ prefixed location mappings. Note that queryDB only restricts existing location mappings, it does not create any mappings.

The value cacheDB=true makes the retry facility save the result set of the retry query on the default DB database using the query uri as modelID. With the default value cacheDB=false the result of a query is not stored in the database. During editing, it makes sense to have both flags on, so that editing changes are immediately available to subsequent queries.

These options implement a database cache for TF query result sets. Command

retry lsRDB

retry notry rdb+file:Place.owl

retry notry file:Place.owl cacheDB=true

retry clearDB rdb+file:Place.owl notry

retry cleanDB

Directory index

The TF repository URI convention allow storing ontology subsets (for instance, individual or collections of entries) indexed under filename paths that reflect the position of the entry or collection in some alternative TF class taxonomy or other. Such a directory index can share structure through symbolic links, so that the same item(s) can be indexed under many alternative paths. One example is an index based on the concept URL as follows.

Cache updates

According to the design, TF repositories may cache copies of ontology models in a database. This raises the question how to tell upstream servers to invalidate cache items when an ontology has changed.

A partial solution for this is provided by location mappings. Location map a given persistent url prefix, say, http://tfs.cc/owl/TFS.owl , first to a versioned url like http://tfs.cc/owl/2.0/TFS . When there is a version change, the persistent url is mapped to a new version number, say, http://tfs.cc/owl/2.1/TFS.owl . Queries for the persistent url after the remapping will be looking up items whose urls are not in the cache (DB items are cached against the last url in the mapping), so they will be queried anew and the new version cached in the DB against the new version number. The older version stays in the cache, but it is no longer fetched, and nobody is the wiser.

Tools

TF terminologies are maintained with the help of various user interfaces and tools. These include TermFactory specific tools, collaborative terminology platforms with TF plugins, and third party professional ontology tools.

Retry utility

com.grapson.tf.rev.jena.Retry.java is a java class for applying TF location mappings. The TF ontology model class OntModel4TF reader is set up to use Retry. There is a command line wrapper script retry in io/script . This utility helps testing a site's location mapping and caching policy. Options as shown below.

The retry utility database tasks manipulate the relational database. The corresponding tasks listFS, delRFS, clearFS, cleanFS, putFS, getFS manipulate the jena TDB native database held in the local filesystem. The location mapping task url tests location mappings to retrieve TF addresses. retry url foo just tries to locate and print the resolved location of foo. retry getDB foo only looks for foo in the database. It does the same as retry notry rdb+foo . Unless explicit format is given, the format of a cached model is guessed from the modelID. To get expected xhtml, xhtml entry write parameters may need to be supplied.

The url task tries to fetch the url using location mappings. Option file=true tells a command line utility to resolve subsequent schemeless file path parameters against the current working directory before passing them along. For instance, retry file=true foo says the same as retry file:$PWD/foo. . Here are some examples of usage:

With url option, retry just locates the document and returns its location. With readAll=false retry downloads the document it finds. It downloads any kind of file, not just rdf models. If a RDF format is given and does not match the extension of the document at the location, retry tries to read the downloaded document as RDF and convert it into the requested format. With readAll=true, retry always tries to read the document into a RDF model and include its imports if any. In contrast, pellet4tf query -u always reads the requested document as RDF and includes imports unless option --ignore-imports is given.

Universal resource identifiers (URIs) may globally identify a resource (they point to just one thing in the world), but not all of them are related to universal resource locators (URLs) identifying a web source holding a description of that resource. By the Semantic Web addressing orthodoxy, the usual RDF resource URI naming convention (a URL plus a fragment identifier) suggests that a resource URI should be described in a document obtained from the given URL at a location pointed to by the fragment identifier. This in general is not practical. Web servers only serve complete URL documents, fragments are located at the client end. The whole ontology would have to be downloaded to access the description of just one URI in it.

This is why web addressing orthodoxy is not completely true of TF either. A TF URI is related to an URL describing it by the TF retry facility. Retry option uri2url shows the file URL of a given TF resource for given links and format. Bona fide URLs without fragment hash should stay the way they are, the rest get prefixed with links and suffixed with a file extension matching the given format. The url2url option tests where Retry location mapping finds the first available copy of a given TF file. Only availability of files is tested, the file is not downloaded.

Below is a schematic picture of URI and URL redirection in TF.

Show/hide TF URL redirection

TF does URL redirection using Retry on jena location mappings. The web server (apache2 for instance) or the web application container (Tomcat for instance) can do their own URL rewriting.

Rewrite utility

com.grapson.tf.rev.jena.Rewrite.java is a java class for rewriting TF ontologies in different formats, and maintaining and changing TF term, expression, concept and individual URI's. It reads an ontology file and rewrites it adding labels to these TF objects and/or rewriting URIs of these objects. There is a shell script wrapper to Rewrite.java in io/script by name rewrite .

RDF/OWL node identifiers can be chosen arbitrarily. It may make sense to choose a fixed arbitrary identifier to a resource, so that all of its properties can change without losing its identity. An editor may show the user a node's label instead of (or alongside) its ID, or let the user tell the editor how to construct a mnemonic name for a node from its properties. But for manual inspection of an ontology in text form, or when transferring data across a variety of converters and editors, it may help to use less arbitrary, descriptive identifiers for entities. Numeric IDs are very easy to get wrong and the errors are hard to catch. TF tries to satisfy both conflicting needs with the Rewrite utility. Rewrite can be run with shell script io/script/rewrite . Query and edit utilities and services can apply one or more of certain labeled rewrite operations. Currently, the labeled operations are deblank , deblank , and relabel , which remove and restore blank nodes and create descriptive labels for anonymous TF resources, respectively.

Usage: rewrite raw | label | blanks | id=<int> remove replace create base=<URI> label=<URI> in=<URI> out=<URI> uuid urlencode | urldecode | tf3encode | tf3decode | normalize <source> ... <format> (readAll|writeAll)=(true|false) | editAll trim imports | queryFS=true|false queryDB=true|false cacheDB=true|false notry help format file=true|false verbose

Minimally, rewrite reads one or more TF files and prints a TF model in different formats. When there are more than one input file, it merges the contents of the files into one model. Help on formats is given with

rewrite help format Rewrite output formats: RDF/XML RDF/XML-ABBREV N3-PP TURTLE TF3 XHTML template=<URI> root=<URI> schema=<URI> active=<URI> original=<URI> edits=<URI> locals=<URI> lang=<ISO langcode> links=<URI> JSON TBX

For the XHTML format parameters, see the section on the XHTML format.

If a source consists of a TF filetype only (e.g. ".owl"), rewrite reads the source from standard input in the indicated syntax. It is then possible to use rewrite as a filter (e.g. cat test | rewrite .tf3 converts test from tf3 into rdf/xml).

Beyond that, rewrite allows rewriting resource URIs in various ways governed by the following option switches. Switch in= gives the prefix of URIs that come in (get factored out) in the rewrite. Switch label= gives the URI of a label (string or annotation) property used by rewrite. It defaults to meta:label . Switch base= gives the URI of a string property used for the lemma of the expression. It defaults to exp:baseForm . Option remove tells rewrite to remove any old label properties from the affected instances. If switch out= is given, new URIs that get created are factored into this namespace. If not, the default is the instance's original namespace. Only named resources are refactored. Task option blanks turns blanks into named resources.

Option create makes rewrite create new URIs for TF resources. What kind of URIs get created depend on the task, which can be one of id and label . With option create id , rewrite forms sequential numerical ids. With option create label , rewrite tries to form descriptive URIs for expressions and terms on the basis of their language, base form, and referent properties.

Option uuid makes rewrite turn the new URIs into uuids. The other encoding and decoding options try applying the corresponding character conversions on the URIs.

Option replace makes rewrite actually replace URIs of the affected resources. The URIs are taken from the label properties of the instances (if any) unless create is chosen. In that case, newly created URIs are used in the renaming. In either case, unless remove is specified, the old URI is saved as a label property of the renamed instance.

For expressions and terms, a descriptive URI is generated only when the entity has the requisite property values (langCode, catCode, and baseForm or romanisation in case of expressions; hasDesignation, hasReferent in the case of terms) and the namespace prefixes for the designation and referent namespaces are known.

The following command line provides descriptive URIs for blank node expressions and terms:

rewrite create label replace sourcefile > targetfile

Option readAll forces a full read of a RDF file

Option writeAll outputs all statements in the model and its imports.

Option editAll includes readonly triples from an xhtml file

Option trim trims off namespace prefixes for namespaces that are not used in the model.

Option imports prints out the names of the ontologies imported by the sources.

Command rewrite raw tf3encode source.tf3 tf3-encodes a triple file. This can be useful as a step in conversion.

Command rewrite raw normalize source.owl normalizes a file into unicode normal form C. It may be necessary to normalize incoming unicode for jena.

Protege 4 TURTLE reader (build 4.0.114) breaks on TURTLE string escape \" . Use triple quoted multiline strings?

Here is the round trip from an RDF/XML or TURTLE format ontology file to TF3 form and back.

1. Read an RDF/XML or TURTLE ontology file with rewrite and write it out in TF3

   rewrite TFS.owl TF3 > TFS.ttl

2. Read the TF3 ontology with rewrite and write it out in rdf/xml This form is equivalent to the original ontology.

   rewrite TFS.ttl > TFS.owl

Usage examples:

rewrite TFS.owl replace label urlencode TF3

The example command prints to standard output a relabeled version of TFS.owl in TermFactory triple format with entity URI's URL-encoded.

The following command line rewrites a turtle TF3 encoded ontology into RDF/XML decoding the TF3 encoded items.

rewrite file:puls-locations.ttl replace label tf3decode > puls-locations.owl

(version 1.9) The rewrite utility can also be run through the TF query services. An example query string is http://localhost:8080/TermFactory/query?uri=blank.owl&how=relabel . So far, only task relabel is covered for use in TF internal profile conversions See TF query string parameters .

The TF reasoner Pellet4TF

(version 1.0) The pellet reasoner command line tool Pellet has been adapted to carry out TF query and classify tasks. The new TF query engine is packaged in io/lib/tf-io.jar . It can be run using command line script io/script/pellet4tf.sh:

Pellet4TF classify adds the RDF/XML output format, and the ability to query the class tree round a given seed up and down to a given depth.

script/pellet4tf.sh help classify PelletClassify for TF: Classify a TF ontology [around a seed class] and display the hierarchy Usage: <pellet> classify [options] <file URI>... Argument description: --loader, -l (Jena | OWLAPI | KRSS) Use Jena, OWLAPI, or KRSS to load the ontology (Default: OWLAPI) --input-format (RDF/XML | N3 | N-TRIPLE) Format of the input file (valid only for the Jena loader). Default behaviour is to guess the input format based on the file extension. --output-format, -o (RDF/XML | text) Format of the output. (Default: text) --seed, -s (C) One class URI or local name to build a taxonomy for. Example: "Animal" --up, -u (I) Number of levels up from seed in classification tree. Default -1 (all) --down, -d (I) Number of levels down from seed in classification tree. Default -1 (all) --help, -h Print this message

Here is an example of using pellet4tf classify. TF prefixes are not supported (yet), so that full resource URI must be used as seed.

lcarlson@lhc:~/Data/CF/TF/io/ pellet4tf classify -u -1 -d -1 -s http://tfs.cc/ont/Language_industry ../owl/TFS.owl tree print http://tfs.cc/ont/Language_industry up -1 down -1 http://tfs.cc/ont/Language_industry http://tfs.cc/ont/Language_technology http://tfs.cc/ont/Multilingual_language_technology http://tfs.cc/ont/Linguistics http://tfs.cc/ont/Terminology http://tfs.cc/ont/Language_industry http://tfs.cc/meta/Domain http://tfs.cc/meta/Meta _TOP_ http://tfs.cc/sem/Place http://tfs.cc/sem/Role http://tfs.cc/sem/Meaning http://tfs.cc/meta/Object _TOP_ lcarlson@lhc:~/Data/CF/TF/io/

Pellet4TF query adds to pellet a facility for querying persistent ontologies. More concretely, this means that it can save ontology files into Jena relational databases and fetch ontologies from them. An ontology url is allowed to have the form of a Java database connection url, for example

jdbc:mysql://localhost/rdb&user=tfuser&url=http%3a%2f%2ftfs.cc%2fowl%2fTFS.owl

The name of the ontology is given in the query parameter named uri in the url of the database connection. This kind of a persistent-ontology uri can also be given as an redirection entry ( AltURL resource) in a Jena policy file ont-policy.rdf . PelletQuery4TF also understands the following kind of abbreviation:

rdb+http://tfs.cc/owl/TFS.owl

rdb+file://home/user/TF/owl/TFS.owl

Given a pseudo uri of this form, Pellet4TF tries to load the ontology uri after the rdb+ prefix from a pre-set database connection URL specified as option TF_DBCONNNECTION in pellet properties file etc/tf.properties . The default connection is jdbc:mysql://localhost/rdb .

Usage: <pellet> query [options] <file URI>... Argument description: --help, -h Print this message --verbose, -v Print full stack trace for errors. --config, -C (configuration file) Use the selected configuration file --query-file, -q (<file URL>) Read the SPARQL (or RDQL) query from the given file --url, -u (<URL>) URL to look up using retry. --uri, -U (<URI or QName>) URI to look up or describe. --describes, -D (<resource URIs or TF QNames>) List of resource URIs to describe. Pellet4TF query implements DESCRIBE query for TermFactory. --output-format, -o (Tabular | XML | JSON | TF3 | RDF/XML | RDF/XML-ABBREV | XHTML | TURTLE) Format of result set: Tabular through JSON for SELECT or ASK queries, rest for CONSTRUCT or DESCRIBE queries --matrix, -M (<file URL>) model to receive query results --minus, -m subtract query result from matrix --template, -T XHTML entry template --root, -R XHTML entry root --depth, -r DESCRIBE query recursion depth (Default: -1) --schema, -S XHTML schema ontology --axioms, -X Stacked engine axioms ontology --active, -A XHTML active ontology --original, -O XHTML original ontology --locals, -L XHTML localization ontology --lang, -l XHTML localization language --links, -H XHTML hyperlinks prefix --output-file, -f Output file. --output-encoding, -c Output encoding (UTF-8 or UTF-16). (Default: UTF-8) --query-format (SPARQL | ARQ | RDQL) The query format (Default: SPARQL) --input-format (RDF/XML | Turtle | N-Triples) Format of the input file (valid only for the Jena loader). Default behaviour is to guess the input format based on the file extension. --ignore-imports Ignore imported ontologies --display-query, -d Display the input query --sites-flag, -s Broadcast query to other sites --query-engine, -e (Pellet | ARQ | Mixed | SPARQL | Stacked) The query engine that will be used. Default behavior is to auto select the engine that can handle the given query with best performance. Pellet query engine is the typically fastest but cannot handle FILTER, OPTIONAL, UNION, DESCRIBE or named graphs. Mixed engine uses ARQ to handle SPARQL algebra and uses Pellet to answer Basic Graph Patterns (BGP) which can be expressed in SPARQL-DL. ARQ engine uses Pellet to answer single triple patterns and can handle queries that do not fit into SPARQL-DL. As a consequence SPARQL-DL extensions and complex class expressions encoded inside the SPARQL query are not supported. SPARQL is just plain Jena ARQ without pellet. Stacked engine first runs sparql to get a smaller model and then runs Pellet on that model plus a separate schema (--axioms). --bnode Treat bnodes in the query as undistinguished variables. Undistinguished variables can match individuals whose existence is inferred by the reasoner, e.g. due to a someValuesFrom restriction. This option has no effect if ARQ engine is selected. --timing, -t Print detailed timing information --queryFS query from FS cache --cacheFS save to FS cache --queryDB query from DB cache --cacheDB save to DB cache --notry, -n no location mapping --nowait, -w wait for results pellet4tf help query PelletQuery4TF: SPARQL-DL Query Engine for TermFactory Usage: <pellet> query [options] <file URI>... Argument description: --query-file, -q (<file URL>) Read the SPARQL (or RDQL) query from the given file --url, -u (<URL>) URL to look up using retry. --uri, -U (<URI or QName>) URI to look up or describe. --describe-resource, -D (<resource URIs or TF QNames>) List of resource URIs to describe. Pellet4TF query implements DESCRIBE query for TermFactory. --output-format, -o (Tabular | XML | JSON | TF3 | RDF/XML | RDF/XML-ABBREV | XHTML | TURTLE) Format of result set: Tabular through JSON for SELECT or ASK queries, rest for CONSTRUCT or DESCRIBE queries --matrix, -M (<file URL>) model to receive query results --minus, -m subtract query result from matrix --entry, -E XHTML entry type --depth, -r DESCRIBE query recursion depth --schema, -S XHTML schema ontology --axioms, -X Stacked engine axioms ontology --active, -A XHTML active ontology --original, -P XHTML original ontology --locals, -L XHTML localization ontology --lang, -l XHTML localization language --links, -p XHTML links prefix --output-file, -f Output file. --output-encoding, -c Output encoding (UTF-8 or UTF-16). (Default: UTF-8) --query-format (SPARQL | ARQ | RDQL) The query format (Default: SPARQL) --input-format (RDF/XML | N3 | N-TRIPLE) Format of the input file (valid only for the Jena loader). Default behaviour is to guess the input format based on the file extension. --display-query, -d Display the input query --sites-flag, -s Broadcast query to other sites --query-engine, -e (Pellet | ARQ | Mixed | SPARQL | Stacked) The query engine that will be used. Default behavior is to auto select the engine that can handle the given query with best performance. Pellet query engine is the typically fastest but cannot handle FILTER, OPTIONAL, UNION, DESCRIBE or named graphs. Mixed engine uses ARQ to handle SPARQL algebra and uses Pellet to answer Basic Graph Patterns (BGP) which can be expressed in SPARQL-DL. ARQ engine uses Pellet to answer single triple patterns and can handle queries that do not fit into SPARQL-DL. As a consequence SPARQL-DL extensions and complex class expressions encoded inside the SPARQL query are not supported. SPARQL is just plain Jena ARQ without pellet. Stacked engine first runs sparql to get a smaller model and then runs Pellet on that model plus an axioms file. --bnode Treat bnodes in the query as undistinguished variables. Undistinguished variables can match individuals whose existence is inferred by the reasoner, e.g. due to a someValuesFrom restriction. This option has no effect if ARQ engine is selected. --timing, -t Print detailed timing information --verbose, -v Implies -t -d --queryDB, -Q query from DB cache --cacheDB, -C save to DB cache --notry, -n no location mapping --help, -h Print this message

pellet4TF SPARQL engine uses TF ontology loader, unlike original Jena sparql. As a RDF tool, sparql does not understand ontology imports. For instance, the multipart ontology epi.owl cannot be directly queried with Jena sparql, but it can be so queried with pellet4TF SPARQL engine.

sparql --data=file:../owl/epi/epi.owl --query=s-inst.sparql ------------------------------------------- | inst | =========================================== | <http://tfs.cc/owl/epi/epi.owl> | ------------------------------------------- vs. time pellet4tf query -e SPARQL -q ../../io/s-all.sparql epi.owl ... Query Results (220977 answers): ...

The default value of the site-flag parameter from the command line is false. A query gets broadcast only if -s flag is given on the command line.

The -D parameter allows shorthands like the following:

pellet4tf query -D exp:Language pellet4tf query -D "exp:Language exp0:Language" pellet4tf query -D http://tfs.cc/ont/Country pellet4tf query -D "<http://tfs.cc/ont/Country>"

Prefixed names like exp:Language work for prefixes defined in etc/prefix.sparql , otherwise use full URI. Angle brackets around a URI are optional. If they are used, better quote the string to avoid ambiguity.

When a matrix model URL is given, CONSTRUCT or DESCRIBE query results are merged to the matrix model and the merger returned. This option is useful when a model needs to be enriched with reasoner results; for instance, when doing format conversion with queries. Example: pellet4tf query -q explit.sparql -M test.tf3 test.tf3 adds the result of query explit.sparql to test.tf3 .

When a matrix URL and flag -m are given, CONSTRUCT or DESCRIBE query results are removed from the matrix model and the matrix minus the results is returned. This option is useful when a model needs to be modified with reasoner results; for instance, when doing format conversion with queries. Example: pellet4tf query -q explit.sparql -M -m test.tf3 test.tf3 removes the result of query explit.sparql from test.tf3 .

A Pellet Mixed engine DESCRIBE query on anything but small datasets is often too slow, typically due to the inefficiency of realization (rdf:type inference). The TermFactory specific STACKED engine is an attempt to remedy this. The Stacked engine runs a first round DESCRIBE query with Jena SPARQL engine for the arguments of the input query, then runs the main querey with Pellet MIXED engine using as dataset the results of the first round. A schema ontology to reason with on the second round can be specified with property TF_AXIOMS or an axioms=... query string or --axioms=... command line parameter.

The first stage uses the more efficient SPARQL rdf query engine to select a set of asserted triples from a large ontology. The second stage uses as input the selected triples plus the supplied schema ontology to run the slower Pellet MIXED engine query on this (hopefully smaller but still sufficient) dataset. For instance, if the input query is a SELECT query, the pre-query replaces SELECT with DESCRIBE for the first round and applies the SELECT querey to the result of the DESCRIBE query.

The Stacked engine is sound but not safe: it may miss long distance entailments entailed by the original dataset but not by the extract made by the pre-query. For example, an instance base with statements :a :lt :b . :b :lt :c . and schema :lt a owl:TransitiveProperty . entails triple :a :lt :c . This statement will be included in DESCRIBE :a under the Mixed engine, but not under the Stacked engine, if :b :lt :c does not happen to get included in the pre-query DESCRIBE result. With unlimited DESCRIBE depth, Stacked engine is safe, but then the extracted model easily gets to be too big to allow any useful savings. (Compare pellet modularity.)

edit4tf

The TF editing back-end is implemented with class grapson.tf.rev.jena.Edit4TF. It can be run from command line using the script edit4tf in io/script :

Usage: edit4tf querystring operand URI, string, or - for stdin del | add | ed | pop | rw=relabel|deblank|reblank|... postfix operator verbose notry no location mapping queryFS=true|false query local filesystem database. Default true. queryDB=true|false query remote database. Default false. cacheDB=true|false save to database. Default false. readAll reads rdf file including imports writeAll outputs all statements in the model and its imports. editdAll includes xhtml content that is marked readonly into xhtml read. rewrite=relabel|deblank|reblank|... <format> give output format as in rewrite utility. template=<URI> output template for xhtml entry write root=<URI> root filter (one or more instances/classes) for xhtml entry write schema=<URI> schema ontology for xhtml entry write ` active=<URI> active ontology for xhtml entry write original=<URI> original ontology for xhtml entry write edits=<URI> edited ontology for xhtml entry write locals=<URI> localisation ontology for xhtml entry write lang=<ISO langcode> localisation language code for xhtml entry write links=<URI> hyperlink prefix for xhtml entry write

The querystring argument is identical to the edit service edit operation querystring. This allows testing query service operations offline. A rewrite operation given as argument of form rewrite= is carried out last. Arguments in the form rw= are carried out where they occur in the edit operation stack.

Edit operations are read in postfix (reverse Polish) notation: operations add and del pop the last two operands from the stack and carry out the indicated operation (subtract or add the second model from the first). Operation ed pops three operands and carry out the more complex edit operation described below. The remaining parameters can be freely interspersed. Here is an example of command line scripting the edit API. entity.ttl is a bilingual WordNet entry and entity-fi.ttl is its editable Finnish language content (the active model). entity-edits.ttl is the result of changing occurrences of Finnish word kokonaisuus 'whole' in entity.ttl to the more appropriate word olio 'being, entity'.

edit4tf entity-fi.ttl entity.ttl entity-edits.ttl del del entity-edits.ttl entity.ttl del add > entity-fi-edited.ttl deletes from entity.fi.ttl what entity-edits.ttl deleted from entity.ttl and then adds what it added. edit4tf entity.ttl entity.ttl entity-edits.ttl del del entity-edits.ttl entity.ttl del add > entity-edited.ttl does the same to entity.ttl rewrite XHTML entity-edited.ttl active=entity-fi-edited.ttl schema=../owl/wn/TFwn.owl > entity-edited.xhtml lays out entity-edited.ttl in xhtml for further editing against the updated Finnish file entity-fi.edited as active model.

Operation ed telescopes into one operation the edits on the three lines above in one go.

The operands to edit4tf can be model URIs or model strings from standard input. A bare filetype like .ttl is read from standard input in the format indicated by the filetype. An operand of that starts with the string ?p= is interpreted as an EditForm edit URL . Empty operands can be indicated by supplying names of nonexistent files:

edit4tf modelID=olio.ttl - - .ttl ed < olio.ttl

All this edit operation does is save the edits in olio.ttl in the database cache under the given model ID. The dash placeholders for missing active model and original on the line are not reserved, there just happens to be no file named dash.

index4tf

The index service can also be run from command line with the index4tf script:

index4tf Usage: index4tf <fromURL> <toURL> notry verbose site=<DAV URL> user=<DAV username> pass=<DAV password>

For instance, command line

index4tf file:entity.xhtml http://purl.org/vocabularies/princeton/wn30/synset-entity-noun-1

uploads Wordnet entry entity.xhtml to the default webdav directory under its official purl.org URI. The entry can be downloaded from the webdav cache as http://localhost/webdav/purl.org/vocabularies/princeton/wn30/synset-entity-noun-1/synset-entity-noun-1 . The default site is set with TF setting TF_INDEX_URL.

If toURL is not specified, the document located at fromURL is uploaded to webdav as fromURL. The user definable pseudo scheme dav+ is defined by a prefix mapping in etc/location-mapping.n3 as pointing to current value of TF_INDEX_URL. Note that dav+ is not a built in pseudo scheme, so it only works when location mappings are turned on.

To upload to the TDB triple store or to relational database, add pseudo scheme tdb+ or rdb+ respectively in the target upload location url. A pseudo scheme alone as toURL indicates to use the source URL as model ID, so that the following two commands mean the same.

index4tf file:ctryCode.xhtml tdb+file:ctryCode.xhtml index4tf file:ctryCode.xhtml tdb+

Indexing of documents to a webdav directory has the following safety feature against accidental overwriting.

index4tf http://localhost/active.ttl copy ok http://localhost/dav/localhost/active.ttl length 1024 index4tf http://localhost/active.ttl copy at http://localhost/dav/localhost/active.ttl exists, not overwritten index4tf http://localhost/dav/localhost/active.ttl copy ok http://localhost/dav/localhost/active.ttl length 211

If the location of the original does not point to the webdav directory, a copy is created for it in webdav. The copy can only be overwritten when it is accessed by its webdav directory url. This prevents accidentally overwriting the webdav copy another time with the (possibly already outdated) original.

Say we try to commit an edit of an original for which there already is a version on dav. The new edited version of the original fails to overwrite the existing version on dav. We can now choose between the following actions.

1. Enter the address of the dav version as active ontology instead of the original and commit the edit. This edits the version on dav, creating a new version based on the existing version on dav.
2. Enter a different upload address as toURL and commit the edit. This creates a new a branch from the official version at the alternative address.
3. Enter the location of the dav version as toURL and redo the edit. This overwrites the version on dav, amounting to a revert to original followed by edit.

It is of course wiser to check at the outset that the ontology to edit is the latest version. Location mappings can be set up to map the original to the editable version, making the check automatic.

The remainder of this chapter documents third party tools that may be used in working with TF.

Editors

This section documents editors for manual refinement of ontology and terminology data in different formats.

XML editors

Ontologies in RDF/XML and TF entries in XHTML, TBX or any other XML format can be syntactically edited using any XML editor. The advantage may be that XML can be told to be pedantic about syntax, and there are a lot more tools for XML than for RDF or OWL. The disadvantage is that XML cannot check the semantics.

XMLmind XML editor

The XMLmind XML Editor from Pixware is a customisable XML editor written in Java which allows to edit large, complex, modular, XML documents in a structured WYSIWYG mode, i.e. the page looks uncluttered but you control what you do as precisely as if you were using an XML programmer's editor. XML can be rendered to a large variety of formats using the open source Formatting Objects toolchain. It natively supports MathML 2 Presentation Markup. XXE is highly customizable, without programming, by local gurus and consultants.

The TermFactory toolkit includes a plugin (configuration addon) for the free XML document editor XMLmind which allows structured editing of the LISA Oscar TBX terminology format in WYSIWYG mode. The TermFactory toolkit includes a plugin (configuration addon) for the free XML document editor XMLmind which allows structured editing of MultiTerm xml export format in WYSIWYG mode. The XMLmind XML editor user interface has been localized into Finnish.

It would not be difficult to construct a XMLmind skin for editing TF XHTML entries offline with XMLmind.

RDF/OWL editors

There is quite a selection of ontology editors to choose from. Listings are maintained at W3C , Wikipedia , and other places. A small sample is surveyed below.

Swoop

MINDSWAP Swoop OWL editor (version 2.3beta4 in 2007, no longer under development) was something like a testbed for the pellet reasoner. It still finds some use as a more or less direct graphical interface to pellet services. It has a funny venn diagram visualiser, an ontology partitioner which seems to only do proper partitions, so it may not be able to split a well-connected ontology. It has a query evaluator, but only for RQL queries. Keep in mind as a model or source for editing functionality in TF.

Protege

Protégé is a free, open source ontology editor and a knowledge acquisition system. Like Eclipse, Protégé is a framework for which various other projects suggest plugins. This application is written in Java and heavily uses Swing to create the rather complex user interface. Protege recently has over 100,000 registered users. Protégé is being developed at Stanford University in collaboration with the University of Manchester. Version 3 was developed at Stanford and has the most contributed extensions. Version 4 is being develped by Manchester University and is build on the Java Eclipse IDE. It supports OWL 2.0. A side-by-side comparison of Protege 3 and 4 is found in the Protege Wiki .

(version 0.4) The current version of Protege used in TF development is version 4. It is downloadable from http://protege.stanford.edu/ .

Protege 4 (as of version 4.1, Aug 2011) has limited support for working with anonymous individuals. There are no editing facilities for anonymous individuals. There are several places where anonymous individuals don't show up in the Protege 4.1 graphical interface. This makes working with anonymous individuals difficult.

Protege plugins

There are many contributed plugins to Protege 3 and 4 , of which the following deserve mention here.

• Collaborative Protege (3)
• ProSE (4)

Collaborative Protege

Collaborative Protege is an extension of Protege 3 that supports collaborative ontology editing. (It is not known if or when Collaborative Protege comes to Protege 4.) In addition to the common ontology editing operations, it enables annotation of both ontology components and ontology changes. It supports the searching and filtering of user annotations, also known as notes, based on different criteria. Collaborative Protege implements two types of voting mechanisms that can be used for voting of change proposals. Multiple users may edit the same ontology at the same time. In multi-user mode, all changes made by one user are seen immediately by other users. There are two working modes available for Collaborative Protege. Both modes support multiple users working on an ontology:

1. The multi-user mode - allows multiple clients to edit simultaneously the same ontology hosted on a Protege server. All changes made by one client are immediately visible by other clients. This mode is also referred to as client-server mode, or concurrent mode and requires a client-server setup. This mode is based on the implementation of the multi-user Protege and is the preferred mode in which Collaborative Protege should be run.
2. The standalone mode - allows multiple users to access the same ontology in succession. The ontology can be stored on a shared network drive and all clients will access the same project files. However, simultaneous access is not possible. This mode is also referred to as the consecutive mode.

The next figure shows the collaborative protege graphical user interface.

Show/hide Collaborative Protege

ProSE

The ProSE plugin guides a human user in choosing what to import from an ontology into another. In particular, it helps the user choose big enough a subset of the external ontology so that the risk of unintentionally indirectly asserting new relations between the imported concepts is minimised. It also helps make sure that the imported subset is no bigger than required for that purpose. In the TF scenaario, one may or may not want to enrich relations between the imported concepts. depending on the case at hand. When needed, the ProSE plugin can be used to determine a suitable set to import.

TopBraid Suite

As part of TopBraid Suite, Composer incorporates a flexible and extensible framework with a published API for developing semantic client/server or browser-based solutions, that can integrate disparate applications and data sources.

Implemented as an Eclipse plug-in, Composer serves as a development environment for TopBraid Ensemble™ and for all the applications delivered using TopBraid Live™. Composer is used to develop ontology models, configure data source integration as well as to customize dynamic forms and reports.

Two versions are available - Standard Edition and Maestro Edition.

NeON Toolkit

NeOn is a 14.7 million Euros project involving 14 European partners and co-funded by the European Commission’s Sixth Framework Programme under grant number IST-2005-027595. NeOn started in March 2006 and has a duration of 4 years. Our aim is to advance the state of the art in using ontologies for large-scale semantic applications in the distributed organizations. Particularly, we aim at improving the capability to handle multiple networked ontologies that exist in a particular context, are created collaboratively, and might be highly dynamic and constantly evolving.

The first release of the NeOn Toolkit, one of the core outcomes of the NeOn project, is available for download and testing from the NeOn Toolkit and Community site.

(From NeOn Wiki:) The NeOn toolkit is a state-of-the-art, open source multi-platform ontology engineering environment, which aims to provide comprehensive support for all activities in the ontology engineering life-cycle. The toolkit is based on the Eclipse platform, a leading development environment. The toolkit provides an extensive set of plug-ins (currently 45 plug-ins are available) covering all aspects of ontology engineering, including:

• relational database integration
• modularisation
• visualisation
• alignment
• project management

The NeOn Toolkit is part of the reference implementation of the NeOn architecture. The major goal of the NeOn project is to provide methodology, infrastructure and tools for designing and managing a new generation of knowledge-intensive semantic applications. The toolkit is implemented as an Eclipse application.

NeOn DIG plugin

A DIG reasoner interface plugin is advertised for NeOn as follows:

The purpose of the DIG Plugin is the implementation of the DIG Interface version 1.1. The DIG Description Logics Interface Version 1.1 is a specification for defining a new interface for DL Systems. It is effectively an XML Schema for a DL concept language along with ask/tell functionality. In two words, the DIG interface provides a standardized way to access and query a reasoner. The user initially chooses the desired action she is interested in. Then the ontology is translated into the DIG interface and sent to the reasoner along with the queries that have been posed. After the query processing inside the reasoner has taken place, the reasoner sends back to the user the response encoded in the DIG Interface and the user can extract the answer to her query. The interested reader is referred to the protocol specification for further information.

The entire concept language and tell/ask functionality is enough to capture every functionality that is usually provided by a reasoner. In the context of the use cases in the NeOn-Project the reasoning tasks that are of utmost importance to us are ontology coherency and classification. More concretely:

• Ontology Coherency: The reasoner takes as input the (translated into the DIG protocol) ontology and for each concept it returns true or false, depending on whether the corresponding concept is satisfiable or unsatisfiable, respectively.
• Classification: The reasoner takes as input the (translated into the DIG protocol) ontology and returns the inferred classification of the various concepts, as opposed to the explicit one that the user initially sees. Moreover, for every concept in the inferred hierarchy we get whether it is satisfiable or not.

It must be highlighted at this point that the DIG Description Logics Interface that has been partially implemented in this plugin does not come bound to any reasoner. Contrary to that, it only provides an interface to query any reasoner that supports the DIG Protocol. The motivation behind this is that the current plugin is a general-purpose plugin that is intended to be used with different reasoners, so we found it meaningful not to restrict it to a particular reasoner, but to rather provide the user with the flexibility to do that themselves, in full accordance with their needs.

OwlSight

OwlSight is a web based ontology browser from ClarkParsia that uses the Pellet reasoner. The browser is written with the Google Web Toolkit and it uses the OWL API to access ontologies.

Ontology Browser

Ontology Browser is a tool to dynamically gneate documentation for ontologies, based on the OWLDoc software. It uses OWL API to access ontologies and has an inteface to Fact++ reasoner.

OntoTrack

OntoTrack is a browsing and editing tool for OWL ontologies developed at Ulm University using the OWL API.

OntoWiki

OntoWiki is a tool providing support for agile, distributed knowledge engineering scenarios.

OntoWiki facilitates the visual presentation of a knowledge base as an information map, with different views on instance data. It enables intuitive authoring of semantic content, with an inline editing mode for editing RDF content, similar to WYSIWIG for text documents.

Comparing OWL editors for TF

We have tried three java Eclipse based OWL editors with TF ontologies. Here are some of our observations:

The combined PULS/BioCaster epidemic ontology contains about 3000 classes (owl:Class 2709) and 20K instances (owl:Thing 22286). There are about 300K asserted triples. The OWLIM reasoner adds another 600K so the inference closure contains about 1M triples.

TopBraid loads the ontology in 00 seconds. It takes a couple of minutes to form the closure with OWLIMSwift. TopBraid feels triple oriented. It seems reasonably robust and provides some useful services (e.g. instance statistics exported in spreadsheet form).

NeOn toolkit takes several minutes to load epi.owl (downloading from the web). Most of the time is reported as spent reading RDF triples and building indices. The class editor is comparable to Protege. The individual editor does not create hyperlinks to object property objects. Reasoners are not mentioned on the NeOn Toolkit GUI (Version: 1.2.3 Build id: B1023 (2009-07-30)). NeOn has little built-in support for browsing and editing TF ontologies.

Protege 4 (version 4.0.114) is has fewer features than Protege 3, but best suited of the three for browsing TF ontologies, thanks to bundled reasoner support and hyperlinks.

For Protege 4 help, see http://protegewiki.stanford.edu/wiki/Protege4UserDocs .

Validators

TF validation can be done using XML validators, RDF validators OWL syntax validators, OWL semantic reasoners, and TermFactory special built tools (see TF3 ). It is advisable to check third party ontologies with a validator before conversion to spot coding errors in incoming data.

Reasoners

A semantic reasoner, reasoning engine, rules engine, or simply a reasoner, is a piece of software able to infer logical consequences from a set of asserted facts or axioms. The notion of a semantic reasoner generalizes that of an inference engine, by providing a richer set of mechanisms to work with. The inference rules are commonly specified by means of an ontology language, and often a description language. Many reasoners use first-order predicate logic to perform reasoning; inference commonly proceeds by forward chaining and backward chaining.

This section documents ontology query and rule languages and reasoners that may be used in TF. For a survey see Sattler .

Pellet does not support DESCRIBE queries. The PELLET engine only queries graph patterns. The Mixed engine uses the Pellet engine to do the graph pattern part of a query and Jena ARQ for other SPARQL constructs. The Pellet ARQ engine option loads an ontology into a pellet kb and runs the ARQ engine on triples retrieved by the Pellet reasoner.

Visualizers

This section documents tools for graphical presentation of ontologies

The TermFactory Visualizer TFVisu

A first version of the TermFactory graph visualizer exists. The core graphics and web service code was written by Seppo Nyrkkö. TFVisu has now been integrated in the TF web service backend architecture. The visualizer service allows reading ontology URL's in TURTLE format and choosing seed resources whose neighborhood(s) the tool visualizes as a RDF style labeled line-and-circle diagram (cirles represent nodes, lines arcs). Images requested from the visualizer are currently cached into a sql database held at the web service client side.

Given two URL's representing two versions of the same ontology, TFVisu shows their join and differences in three diagrams. Example:

Bramble

Bramble is a framework which visualises RDF graphs in a native browser environment, using the SVG standard and JavaScript technology.. Graphs can be directly expanded, modified and explored. Users select nodes and edges from a central data repository containing millions of statements. The resulting graph can be shared with other users retaining full interactivity for collaborative work or presentation purposes.

Software libraries

There are a number of open-source program libraries for handling RDF graphs and OWL ontologies, notably, OpenRDF, Jena, and the OWL API. The TF project currently uses the HP Jena Java RDF/OWL code library as well (in a small way) the OpenRDF Sesame RDF stack. We also use OWL API in connection with the Pellet TF reasoner. .

WS overview

Show/hide TF services overview

TermFactory Services overview

Termitehtaan palvelutavat

Behind the scenes, TF nodes communicate using web services. As shown in the figure above, there are more than one way how TF involves web services. Starting from the bottom, the network of TF repositories appears to higher layers as an ontology service. Above that, the TF query backend includes the TF federated reasoner which allows submitting distributed queries to a repository network. The TF edit service allows saving and retrieving entries for editing from repositories. On top of that, there is the TermFactory API which supports embeddable services to TF front ends such as text area editors and commenting / voting systems. TF front ends can be implemented natively on different collaborative platforms, included into them as more standalone server side plugins, or merged on existing pages as client side href="http://en.wikipedia.org/wiki/Mashup_%28web_application_hybrid%29">mashups .

A similar approach was taken in the Finnish KITES Multilingual Workbench/Desktop project. MLWB tools are typically used through organization’s own productivity tools such as Microsoft Word, business portals, and content management systems, or through their own user interfaces.

Show/hide MLWB Architecture

We first considered using "big web services", including the XML based SOAP, WSDL, BPEL, and UDDI standards. The advantage seemed to be that there was a lot of existing code. On the downside, the resulting overall system could be expected to become like a tectonic plate: huge and slow. There were many inertial components involved: Java, XML, OWL, Web...

The following was an early cast for a TF web service system design: A QueryForm client forms a sparql-dl query from user input. A TF service contains the distributed reasoning engine, an extension of the Pellet reasoner along the lines of federated reasoners Drago and PDL. The reasoning engine uses UDDI to map graph and term URLs on TF instance URL's. As a reasoner proceeds, it sends subqueries for data through the BPEL service or to one or more ``local'' Query Services or to other TF repositories. The BPEL service schedules and optimises such subqueries and relays the results back. The TF service aggregates the results and returns them to the client. Labels like QueryForm, BPEL and UDDI are only suggestive for the type of functionality meant.

A central ontology registry and workflow manager was first planned to be implemented using XML based web service techniques ("Big Web Services", including UDDI and BPEL). But a centralised registry architecture was subsequently abandoned as too complex and fragile. Lucky for TF, for by now UDDI seems dead and BPEL is not much better off. The Axis2 WSDL service layer is not doing much work either, to tell the truth.

Another proposal for an ontology registry is Oyster provided in the Neon Toolkit . It uses peer-to-peer networking to contact repositories.

The TF needs that first seemed to call for centralized cataloguing and orchestrating services include these:

1. How to tell which TF concepts belong to which TF ontology
2. How to tell which TF ontologies are managed with which TF repository server
3. How to delegate term queries down the line when ontologies import terms from ontologies managed by other repositories
4. How to check that a given cached description of a term is up to date relative to the ontologies it imports from

On further thought, it started to appear that many if not all of these problems can be solved (or circumvented) by judicious application of existing web addressing technology. The key is to conform to standard web addressing orthodoxy. The orthodoxy is to make web content associated to a given URI simply available at that location (see e.g. http://www.w3.org/Addressing/ ).

1. An example of the first question is how to locate the TF description of a concept uri like http://tfs.cc/ont/English in the TF repository system. The answer is now relatively simple. It is enough for a web client to send a GET request to the URL http://tfs.cc/ont/English . The request goes to the webserver at http://tfs.cc . Since this server is a TF repository server, it uses the server redirect capability to translate the incoming uri concept/English to http://localhost:8080/TermFactory/query?uri=http://tfs.cc/ont/English . This is a TF query service url which is handled by the local Tomcat application server at the tfs host. Further redirects cen be defined in TF location mappings . Such mappings can also register which ontologies should be searched for a given term URI or prefix name (QName).

2. The second question is similarly a non-issue: by web addressing orthodoxy, the ontology URL tells where the ontology is.

3. As for TF imports , TermFactory imports are identified by TF URLS. That is, a TF query import URI points to a TF service which returns the result of a query as an OWL model (possibly one cached in the repository database or one saved as text files on the server).

Loading an ontology at a node can indirectly cause relayed queries at any number of other nodes, possibly rounding back to the first node. In the OWL standard, cyclic import (causing mutual or self-import) is allowed and entails identity (equivalence). Cycling is checked in jena by keeping a list of imported ontologies during model load and checking when the url of a model to loacd is already on the list. TF imports are covered, a special case of jena OWL imports.

4. The question about version updates is solved internally on per server basis, as explained in the section on cache updates . The main point is that each term query result brings along with it version information about imported ontologies. When a site updates an ontology, it updates the version information saved on the server's ont-policy file, where it gets looked up by the location mapping retry facility.

So it is turning out that the TF repository network can be built without big web services. The functions of boxes labeled UDDI and BPEL in the blueprint are getting solved with existing uri redirection services running locally in each TF instance. Some background theory and a general outline of the design is given in a separate paper .

In version 1.2, a TF repository network exists as it were implicitly, through TF uri naming conventions. Each TF server operates independently; there is no privileged communication between such servers beyond mutual query service requests. It is not the servers that depend on one another, but the ontologies stored in their repositories. An ontology housed in one server becomes dependent on another TF server through importing some (small) ontology whose uri is maintained by that other server (simply because the imported ontology uri, directly or through location mappings, queries the other server).

Version 1.2 TF repositories work completely independently of one another, communicating only through repository URLs. Adding a new repository does not require any updating in the rest of the network. Removal of a repository just means that some queries will no longer return answers. The only way how a TF repository hierarchy comes about (at the current state of the implementation, when no attention has yet been paid to business aspects), is through the internal logic of the participating ontologies. A top ontology is one which is imported (from) by many ontologies; a leaf ontology is not imported (from) by other ontologies.

(version 0.1) An initial implementation of the TF backend exists. It sends Jena SPARQL queries to a persistent FT Jena OWL database in MySQL over the web using Axis2 POJO (plain old Java object) web service calls. This implements a baby version of a standalone TF repository, i.e. the aqua, green, blue, and red boxes in the following figure.

(version 1.0) An improved implementation of the TF backend exists. It is implemented as a set of Axis2 web services serving a webapp / RESTful web api done with java servlets. The following is the current Axis2 listing of TF services. The orange boxes labeled BPEL and UDDI, i.e. the repository network choreography, remain to implement.

TF API

This section documents the TermFactory Web API (web application programming interface). It lists the functionalities offered by the TF repository server network which different user interfaces or platforms can exploit according to their needs and capabilities.

• Query
• Edit
• Index

The TF web services

A simple-minded but reasonably full implementation of the TF ontology repository query backend exists. It is implemented as a Tomcat webapp backed up by a set of Tomcat hosted Axis2 web services. The servlets in the web application access the Axis2 services as usual in java through a Axis2 client. On top of this, the webapp servlets constitute a more lightweight REST style web API layer, accessed using HTTP GET (urls) or POST. The following is the current Axis2 listing of the backed TF services. The box labeled UDDI is in effect implemented with Jena location mappings, and the box labeled BPEL is implemented with the location mapping retry facility in QueryService.

TFServices

QueryService

QueryService serves queries against file or database TF repositories using the new Pellet-based TF query engine Pellet4TF. Compared to v. 0.1, the improvements include the following.

• version 1.0 uses the Pellet4TF reasoner.
  • choice between ARQ, PELLET, Mixed, SPARQL and Stacked query engine.
  • supports querying both and file and database repositories.
  • TF DESCRIBE query implemented.
  • TF query imports implemented.
  • choice of output format
  • choice of output encoding

QueryService implements the setQuery and getQuery operations. These operations set query bean parameters sent to the service and get them after the query has been carried out, respectively. The fields of the query bean are listed here.

Relayed queries

Instead of letting one query engine to do all the work, it should be possible to relay a query through other sites in the TF repository network and collect the results. A relayed query is a query run on the query service of a different repository. Instead of a list of ontologies as dataset, a relayed query is against a list of query services. Instead of fetching all the ontologies in the repositories and running the query locally, this type of query distributes the query for the several repositories to run and merges the results from them.

Since datasets are identified by URLs in TF whatever their size or location, nothing extra is needed in TF to implement relayed queries. A relayed query is a query whose repository is the result set of another query to a TF query service.

Setting TF_REPOS in a site's TF options etc/tf.properties defines the default repositories queried from that site (that list is consulted if there is no repo parameter in the query). Here is an example of a relayed query:

http://localhost:8080/TermFactory/query?pattern=Finn&repos=http%3a%2f%2ftfs.cc:8080%query?pattern=Country&repos=TFS.owl%0aPlace.owl%0a%0a%2fDomain%2fGeography%2fCountry%2f%0a

What this does is use the local query engine to query for items related to Finns in a dataset composed of (i) the result of a query to TF schema about countries and (ii) the local ontology index under countries. Note the double newline (percent encoded as %0a%0a ) marking the end of the embedded query's repo parameter.

Another observation: Since query engines are web services in TF also identified by URL, a TF site does not even need to have a local query service, it can use a remote one to query local ontologies as well as remote repositories (provided it has permission to do so). A TF site can also be a virtual construction implemented by a cluster of nodes.

Broadcast queries

A broadcast query in TF is a query that is recursively relayed through a network of sites. Broadcasting in TF is not completely general, but each site only relays a query to such further sites as are specified by the site. The set of sites to which a given site will relay a broadcast query to is looked up from the site's TF option TF_SITES. Assume the site list contains just the one site http://tfs.cc:8080/TermFactory/query . The local query

http://localhost:8080/TermFactory/query?pattern=China

gets in effect rewritten to

http://localhost:8080/TermFactory/query?pattern=China&repos=http%3a%2f%2ftfs.cc:8080%2fTermFactory%2fquery?pattern=China

This causes the local query engine to first relay the broadcast query to http://tfs.cc and then use the results as a dataset for its own query.

A provision is needed against cycles caused by a query getting sent back to where it has already been. This is prevented by giving a broadcast query an UUID. Each site keeps a list of served query IDs. Queries on the list are ignored. When a query is done, the originator broadcasts an OK to remove the query ID from the network. Broadcasting is the default. It can be explicitly prevented with query parameter sites=false .

TF resources have URIs that identify the resource's owner, so the default is to get a named resource from the owner. A network broadcast is involved when we don't exactly know what we want Then the best bet is to search our own and neighboring sites, which in turn may query their neighboring sites. Cycles in the network are stopped with the query ID. For a more general solution, compare federated SPARQL queries .

EditService

EditService setEdit/getEdit operations implements four main actions: edit, save, delete, and add. edit does delete and add in one transaction. An edit operation works on RDF triples. A matrix triple set is edited by subtracting a delendum set and/or adding an addendum set to it. Triple sets can be specified by an URL, a jena model, a W3C DOM tree, or an XHTML string. In Edit service, the edit parameters are conveyed in an Edit bean with the following fields.

The operation carried out is given in the operation field, matching the operations described for edit4tf . The default opcode (assumed when no explicit opcode is given) is ed . For accessing the edit service with a URL see EditForm .

IndexService

IndexService provides TermFactory webserver directory management using webDAV. It offers various ways of indexing a given ontology file (ontology or entry file) under different descriptions. For discussion, see section on URI conventions.

The webDAV protocol does not support filesystem links. As a substitute, TF IndexService creates a directory for the (slash vocabulary) resource URI and puts in it a php directory index file index.php which forwards user agents to the actual resource document using a http redirect header.

TF IndexService can be used to upload an xhtml entry produced by the query service, say ctryCode.xhtml for resource http://tfs.cc/ont0/ctryCode , to a mirroring server grapson.com 's TF webdav directory using the webdav url http://grapson.com/webdav/tfs.cc/ont/ctryCode.xhtml . The file is then accessible for reading and writing at grapson as http://grapson.com/tfs.cc/ont/ctryCode.xhtml . In addition, a php redirection file (link file) http://grapson.com/tfs.cc/ont/ctryCode/index.php is created which makes the directory url http://grapson.com/tfs.cc/ont/ctryCode fetch the same xhtml file. Further such php links to the file can be created elsewhere for multiple indexing. Alternatively the grapson.com webserver is told to rewrite http://tfs.cc/ont/.* to http://grapson.com/tfs.cc/ont/*.xhtml .

In the home server tfs.cc , the directory /ont can be redirected or linked to a webdav directory http://tfs.cc/webdav/ont . This makes the TF resource uri http://tfs.cc/ont/ctryCode directly web addressable and web editable as a TF entry, in conformance to web addressing orthodoxy. For hash vocabularies. the only blemish is the necessity to substitute a slash for the fragment hash, in order to have a resolvable URL visible to web servers in the first place.

IndexService implements the setIndex and getIndex operations. These operations set query parameters in the bean sent to the service and get them after the query has been carried out:

String modelID; String fromURL; String toURL; String contents; String format; String encoding; String answer; String message;

The operation depends on the parameters given. If modelID and toURL are given, the model cached at modelID is uploaded to toURL. Else if contents and toURL are given, the contents are uploaded to toURL. Else if contents and modelID are given, the contents are cached in DB.

GateService

TF GateService user management is based on the user management of the GlobalSight open source TMS (translation management system). TermFactory users are maintained in and managed through GlobalSight server instance with TF specific extensions. GlobalSight user management is based on an "ontology" of users, companies, locale pairs, activity types, and roles. Users belong to a company, and can belong to one or more roles in the company. A GlobalSight role is identified by company, activity type (for instance, translation, revision, terminology), and locale pair (for instance, from English to Chinese). TF adds one more attribute to the list, called domain. A TF domain is a regular expression on ontology URIs, for instance, "http://tfs/cc/.*" matches all TermFactory ontology uris.

The TF extension of GlobalSight is packaged as a GlobalSight patch that can be installed on top of a current GlobalSight release (the current version is 8.2). The patch adds a new optional attribute type "tfdomain" to the GlobalSight LDAP schema, and a corresponding column TFDOMAIN in the GlobalSight relational database table USER_DEFAULT_ROLES. Requisite changes are made to the GlobalSight front end and middleware to support viewing and editing domain fields on roles.

Below is a sample user role entry from GlobalSight LDAP directory. The activity type determines the type of permission and the value of the tfdomain attribute its domain. Currently, the locale pair is not used to constrain access to ontologies by locale, but it might at some point.

# 57 EditTFS_3 fi_FI fi_FI http://foo/bar KOE, Groups, globalsight.com dn: cn=57 EditTFS_3 fi_FI fi_FI http://foo/bar KOE,ou=Groups,o=globalsight.com roletype: U sourcelocale: fi_FI uniqueMember: uid=KOE,ou=People,o=globalsight.com status: ACTIVE targetlocale: fi_FI objectClass: top objectClass: groupOfUniqueNames objectClass: localizationrole cn: 57 EditTFS_3 fi_FI fi_FI http://foo/bar KOE tfdomain: http://foo/bar activitytype: EditTFS_3

Since TF user management is delegated to GlobalSight, The TF GateService can remain quite simple. All it needs is a stripped-down version of the GlobalSight user management API to check access, read user information and read/write permits from the GlobalSight back end.

GateService implements the setGate and getGate operations. These operations respectively send login parameters and receive an access token, or send an access token and receive user information and read/write permissions.

String site; String user; String pass; String token; String uinfo; String readPermit; String writePermit; String message;

The GateService is currently used by QueryForm and EditForm to login a user, by QueryService to filter query repositories, and by EditService to check write permission on the current active ontology before trying to commit edits. At present, all domains qualify for read permissions, while only domains in roles whose activity type contains the word "edit" (case insensitive) qualify for write permissions.

SparqlService

TF SparqlService is a sparql protocol compliant query endpoint for TF. The w3c sparql protocol recommendation defines the sparql webservice protocol by way of a WSDL 2.0 schema document sparql-protocol.wsdl . The protocol also specifies valid query requests, results and faults (errors) with xml schema protocol-types.xsd . (This schema imports further schemas for its subelements).

The wsdl definition is meant to make possible automatic generation of service and client implementations in different programming languages from the specification. The xml schemas for data formats similarly serve automatic translation of associated data into xml and back for transmission over the web and are used for data validation.

TF SparqlService was generated from the WSDL specification using the Axis2 WSDL2Java code generator as follows.

1. Downloaded the Axis2 standard distribution (version 1.5.4) that contains the code generator.
2. Added the following service endpoint inside the protocol-query.wsdl description element. (The code generator throws a null pointer if there are no endpoints in the wsdl.)

   <service name="SparqlService" interface="tns:SparqlQuery"> <endpoint name="SparqlServiceHttpGet" binding="tns:queryHttpGet" address="http://localhost:8080/axis2/services/SparqlService.SparqlServiceHttpEndpoint/"> </endpoint> </service>

3. Generated service skeleton files using the WSDL2Java utility. (The classpath contains . and the axis2 library jars.)

   . classpath && java org.apache.axis2.wsdl.WSDL2Java -uri protocol-query.wsdl -p com.grapson.tf.ws.service -wv 2 -d jaxbri -ss -sd -ssi

4. Deleted occurrences of word Skeleton from the generated source and stuck the TF query engine into SparqlService.java. (Needed to write a class Jena2Sparql to convert Jena result set to sparql protocol format.)
5. Added TF libraries to the generated build.xml and built the service .aar archive with command ant
6. Deployed the service by copying the build/lib/*.aar file to CATALINA_HOME/webapps/axis2/WEB-INF/services
7. Checked http://localhost:8080/axis2/services/listServices to make sure the service has been properly deployed.
8. Tested the service with a query URI, e.g.

   http://localhost:8080/axis2/services/SparqlService/query?query=SELECT+?inst+?class+WHERE+{?inst+rdf:type+?class}&default-graph-uri=http%3A%2F%2Ftfs.cc%2Fowl%2FTFS.owl

EditForm servlet

The TF webservices editing API supports saving a model in the database under a model ID and deleting a model from it by model ID. There are also methods for deleting a model (specified as a URI, a jena model, or the DOM tree of an TF2XML format document) from a given DB model and adding such a model to another DB model.

The least platform dependent way to implement collaborative structured editing (e.g. on a Wiki platform) is to let the platform provide the editor and just read the result as an xml element off the edited page. TF provides a transform back from TF2XML to jena model with a jena model reader XML2TFReader. The TF Edit web service allows editing a persistent model (set of triples) by adding or deleting another model to it.

A terminology query can produce content from several different repositories through relays. Edits should only be allowed on the currently active ontology, which should be one to which the Wiki has edit permissions. Some solution like the Protege facility of choosing the active ontology among the ontologies shown (greying the rest) is indicated. OWL 2 has only just begun to support per-statement source annotations, using a variant of the RDF reification (quad) technique. It takes four more triples to annotate a triple in the ontology. Pending a more efficient technique for source indication (Ontotext has one), it seems easier to do a second query for editable content from the active ontology and use that to single out the editable elements at the edit interface level.

Editing triples about named resources is straightforward enough. Blank triples are more problematic. A brute force way to edit blanks is to rewrite them with universal resource names for the time of the editing. Then they are treated as named resources and cause no problems. After editing, they are written back as blank nodes. This works as long as the blanks all come from the same active model. In general, however, edits may be the result of querying some other dataset. Then there is no way but match blanks across models, that is, do rdf reasoning.

The TF jena model XHTML writer can be given as parameter the active model. Given this parameter, the writer marks those triples that come from the active model with attribute class="editable" and those that do not with class="readonly" . An editing tool can then restrict editing to the active ontology, so that a subsequent save of the edits can be included in the right ontology. (Compare the forthcoming HTML 5 contenteditable attribute .)

The Axis2 web api passes all data (request parameters, TF settings, edits, and messages) in an XML bean. With the servlet API, it is less straightforward to return side effects like messages. There is no problem if the request does not return edits, for instance, a save or copy request. In this case, the message is returned as the contents of the response page. If the return page is XHTML, the XHTML writer includes service messages and other metadata in the XHTML header. If the return pages is RDF, messages and other metadata is included in the RDF triples as properties of a blank node of type meta:Entry. Conversions between formats keep the metadata. In editing an active model, the rule is the same as with other edits: metadata specific to the original gets removed, and metadata specific to the edits gets added. Metadata shared by the original and edited entry is left as is. The XHTML writer omits subjects with rdf:type meta:Entry unless meta:Entry is explicitly included in the root filter.

Besides these basic editor capabilities, the TF editing support shall provide a set of widgets that can be used in enriching native X(HT)ML editing in third party platforms. Such widgets can include picklist contents and/or menus for choosing existing resources such as properties and values cached from repository content. To manage large selections of values (typically object property values), an autocomplete facility can be more useful than than menu picklists. In any case, what the back-end edit api shall provide is a method to generate picklist and autocomplete contents from ontologies using appropriate TF queries.

TF web application

The TermFactory web application is a sample client for the TF Axis2 web service engine TFServices. In the demo implementation, it is an Apache Tomcat servlet accessible at address http://localhost:8080/TermFactory/ . The url http://localhost:8080/TermFactory/query starts an interactive form. The same address with an uri or pattern parameter serves ontology requests. In Tomcat, the TermFactory.war archive file gets extracted to the Tomcat WEBAPPS directory at deployment. To hot update the application, one can edit or replace files in this directory.

TF query URIs can range from simple ontology-fetch or term-describe URIs to URIs containing literal SPARQL query strings. Ontology fetch is implemented with the location mapping retry facility. The retry facility and the associated location mappings allow defining a sequence by which content is queried from alternative data sources: web documents or databases, ontologies or cached term entries. QueryForm HTTP GET returns a query form if it has no query string or the query parameters include form=true, and a result page for the given query string parameters otherwise.

A TF QueryForm url without parameters like http://localhost:8080/TermFactory/query opens QueryForm with default settings. The form can be started with user defined initial settings using a url like http://localhost:8080/TermFactory/query?form=true&... where the subsequent parameters are some of the TF query parameters below.

QueryForm parameters

Following is a listing of the parameters understood by QueryForm. Recognized abbreviations in parentheses. For those parameters that allow multiple occurrences in a query string (currently repos and rewrite), all supplied values will be applied. For the rest, the last occurrence wins. Note that the QueryForm parameter names 'url' and 'uri' are misleading. The ontology document and the repos can be anything resolvable to URL/s by the TF retry facility. The ontology resource/s can be anything resolvable to URI/s by the TF localization facility.

• Query parameters
  • url an ontology document
  • uri one or more ontology resources
  • queryURL (q) address of a SPARQL query
  • query text of a SPARQL query
  • pattern (p) a regular expression
  • repos (r) list of repositories separated with newlines (percent coded as %0a). Multiple.
  • queryType type of query
    • URL a URL query
    • URI a URI query
    • SELECT a SELECT query
    • CONSTRUCT a CONSTRUCT query
    • DESCRIBE a TF DESCRIBE query
    • ANY (or null, default) any TF query
• engine (e) TF query engine to use
  • ARQ
  • PELLET
  • Mixed
  • SPARQL
  • Stacked
• format (f) CONSTRUCT or DESCRIBE format
  • RDF/XML
  • XHTML
  • TF3
• format2 (f2) SELECT format
  • XML
  • JSON
  • Tabular
• encoding character encoding
  • UTF-16
  • UTF-8
• DB settings
  • queryDB query database. Default false.
  • cacheDB save to database. Default false.
  • notry no location mapping. Default false.
• XHTML settings
  • template (t) output template
  • root entry root (one or more instances or classes)
  • schema (s) ontology schema. Bridges third party vocabulary to TF. Default TFS.owl
  • active editable ontology. No default
  • original deleted ontology. No default
  • locals localization ontology. Default TF10n.owl
  • links link mapping file. Default etc/links.n3
  • lang localization language code. Default en.
• rewrite (rw) rewrite operation name. Multiple.

The TF query engine and the Axis2 service web API for it has three built in query types, URL, URI, and ANY.

A TF URL query looks for a file of the given name using location mappings to try alternative locations (URLs).

A TF URI query looks for an up-to-date version of an entry for the given resource/s and returns it in the specified format. Each resource is first tried with retry to look for a ready-made entry for the resource. Resources that have not got an entry are described with a DESCRIBE query from the given repositories.

The QueryForm servlet provides a few more queryType options. The SELECT, CONSTRUCT and DESCRIBE queryType options in the query form respectively send a hardcoded sample SELECT or CONSTRUCT query to the query service. The hardcoded queries reside in class QueryPatterns and can be modified by overriding or rewriting that class. It will not be difficult to add additional predefined query types as need arises.

The edit service can be called through the EditForm with query parameters as follows:

More complicated edit requests can be made by specifying a stack of desired edit operations with query parameters p and op (currently one of del,add,ed ). The operations are carried out in reverse Polish fashion:

http://localhost:8080/TermFactory/edit?&p=.&p=.&op=del&p=.&op=add

This deletes the model indicated by the second p parameter value from the first and then adds the third one to the result. The edit operations can be interspersed with rewrite operations in the form of rw=<operation> . Each rewrite pops the model on top of the stack and pushes the rewritten model back in its place.

EditForm parameters

Following is a listing of the parameters understood by EditForm.

• submit type of action
  • Query do a query
  • Save do a save
  • Edit do an edit
  • Upload do an upload
• form type of response
  • true a form
  • false a page
• Query parameters
  • queryString ontology document or resource/s according to queryType checkbox
  • queryType type of query
    • URL a URL query
    • URI a URI query
• format output format
  • SELECT formats
    • XML
    • JSON
    • Tabular
  • CONSTRUCT or DESCRIBE formats
    • RDF/XML
    • XHTML
    • TF3
• encoding character encoding
  • UTF-16
  • UTF-8
• DB settings
  • queryDB query database. Default false.
  • cacheDB save to database. Default false.
  • notry no location mapping. Default false.
• XHTML settings
  • schema ontology schema. Bridges third party vocabulary to TF. Default TFS.owl
  • template output template
  • root entry root filter
  • active name or content of editable ontology. No default
  • original name or content of deleted ontology. No default
  • locals localization ontology. Default TF10n.owl
  • lang localization language code. Default en.
  • links link mapping file. Default etc/links.n3
• rewrite rewrite option
  • deblank replace blanks with URNs
  • reblank restore URNs as blanks
  • relabel create descriptive labels
• Edit parameters
  • source source URL or string
  • original original URL or string
  • edits edits URL or string
  • op edit operation
    • del delete
    • add add
    • ed edit
• fromURL URL to download from
• toURL URL to upload to
• message service message

The EditForm doQuery action implements two of the query types defined in QueryForm, viz. query for a document by address, and query for one or more named resources (preferably by address, failing that, by description from given repositories). The doSave action uploads edit area contents (as an xhtml string) to a default location (by default, a webdav directory). With appropriate location mapping settings, a subsequent query for the source of the edits produces the last saved version. (To revert to the original version, query the original location with location mappings turned off.) The doEdit action validates the contents of the edit area (roundtrips the html through rdf). If an active ontology is specified, it updates the active ontology with the edits. If the active ontology comes from a web document, the edited version of the active ontology is saved in the default output format to a default location (by default, a webdav directory.)

User interfaces

This section describes the TF front end user interfaces.

The mockup

At the outset of the TermFactory project, the plan was to develop specific dedicated platforms for TermFactory for querying, browsing, editing and discussing terms. There were a lot of promising platforms to borrow from, and many of them were surveyed for adaptation to TermFactory. With time, it became apparent that such platforms are a rapidly moving target. The very existence of many popular platforms competing for clientele made it less enticing to have to get trapped in any one of them. TermFactory is too generic a concept to be imprisoned in some particular user interface. Now, the focus is to provide plugins and mashups that can be embedded in a variety of present or future platforms.

At this time (Jan 2012), it seems what little there was by way of a will for common standards (thanks to the common enemy Windows) is wilting away. It is a free-for-all, a new dogfight where the big players are scrambling for position in the brave new world of handheld devices . It is to be hoped that no one wins. Meanwhile, it seems best not to stake much on the user end, since that is going to change very fast.

The initial concept of at least four main use scenarios: Query, Browse, Edit, and Discuss, is still there. Therefore it still makes sense to start discussion with the original mockup.

In addition to the four views shown here, there may be simplified layouts for simple term search and entry, as well as text oriented TFPedia content, following the spirit of Wikipedia. Entry to TermFactory web can be controlled by registration and login, as usual. Different user categories should be distinguished with different credentials for seeing or modifying content.

In retrospect, the four functions: browse, view detail, edit, discuss, stay with us. The main difference to the plan is that the association of tasks and views is much less cut and dried now, thanks to connecting and mashing up third party tools and platforms all served by the TermFactory back end. Many tools and platforms provide more functions than one. Some functions are served by many platforms, with often useful variations. This plurality is part of the web experience.

TermFactory Search

TermFactory Search shows a term browser on the left half of the screen. The fields shown on the screen can be modified by the user and the sort order of the results are user modifiable much in the same way as in email programs. The right half of the screen has a fill-out search form and a SPARQL query editor, both for searching TF repositories. The results of a query ars shown in the browser. Naming and saving search filters and SPARQL queries by user profile should be added (not yet shown here).

TermFactory View

By clicking entries shown in the browser, the user can view them in more detail in TermFactory View. The left hand side of the screen arranges term information in a way standard in terminographical entry layout. The Term section shows the different (cross-language) synonyms for a given concept. The Concept section shows a fragment of a concept's neighborhood in the ontology. The Description section shows textual fields related to the concept. Naming and saving views by user profile should be added (not yet shown here).

At the margins of the entry section, there are the vote and comment forms. A user can give immediate feedback about items shown by voting them up or down at this stage. She can also make comments or suggestions through the Suggest/Comment form on bottom of page. These comments and suggestions will be stored in her local TF forum and spread to the other sites through its RSS feed.

On the right half of the screen, headings of discussion items fetched from TermFactory sites related to the chosen item(s) are shown. These items are fetched along with the terminology content proper in connection with a repository query (filtering through TF site RSS 1.0 streams or in other ways). The query language allows the user to control this part of the search as well.

TermFactory Forum

If the user wants to check the discussion around an item, she can move to TF Forum. The layout of TF Forum follows usual news forum layout. The main novelty is that threads are associated to TF resources (URLs or URNs). Users can also vote about other discussion contributions. As usual in user moderated forums, such votes can then accrue to the account of the authors of the contributions. The "karma" thus accruing to authors of successful contributions can be exploited e.g. to sort messages or to evaluate new suggestions), to help site moderators, even to automate some aspects of the certification process. Naming and saving forum settings by user profile should be added (not yet shown here).

Again we do not want to get stuck with any specific forum tool. Some recent commenting tools like Discqus allow inserting comment elements mashup style on various platforms and support relaying comments between platforms through a common store using RSS. This fits well the TF distributed design.

TermFactory Wiki

In a professional terminology use scenario, TF repository contents are updated by a selected set of experts on the basis of community input as a separate stage of the workflow. That is, suggestions, discussions or votes are not stored on TF repos nor do they affect other TF content directly. Some TF updates might be made automatic on the basis of community polls, but that requires coding a separate workflow mechanism; it is not built in.

More precisely, free form discussion of terms and verbal descriptions for human use not leaving the Wiki platform can be free for all. But structured editing of TF entries and acceptance for inclusion to a repository is to happen by qualified people only. Unlike the discussion and general information sections, and forum discussions, such sections of such a TF Wiki are not open for all to write. Once a steady state or agreement has been reached by the Wiki participants, the result of the edits should be saved on a repository. Besides free form text, the results of the editing should thus be eventually brought to a form which conforms to TF ontology format. A term entry is fetched from a database repository. (For now, at least, more global editing of ontologies is supposed to happen offline.) The Wiki page can use TF Retry to fetch a term for editing. The entry is rendered on the wiki page in some editable form. Changes to the entry are saved back to the repository.

TermFactory Wiki is a collaborative editing platform. Like Wikipedia, it has Edit and Discuss tabs for changing object content and meta discussion about the content, respectively. Like Collaborative Protege, it may have a Chat section for real time chat between editors. Editing TermFactory entries on a Wiki is not open for all, but only for an accredited team of editors. (It can be a big and distributed team, nevertheless.) TermFactory Wiki does not manage edit locks but trusts simultaneous users to manage commits between them. The chat keeps a list of currently logged-in users and their activity. Contributions of non-editing users start as suggestions in the forum and follow the longer community accreditation (discussion and voting) process. Some aspects of this process can be automated, for instance, promoting translation suggestions to a repository either after they have a sufficient high vote/status (an editor or other user with sufficient authority may be needed cast a decisive vote). The discussion tab can be less restricted. A channel exchanging information between TF Forum and TF Wiki discussion page must be provided. Naming and saving settings by user profile should be added (not yet shown here).

TF Wikis may be connected to the repository back end through a structured entry editor, shown as an editable area on the Wiki page.

TF Wikis may be connected to TF forums through a comment collection mashup component like Discqus .

The actual front end

This section covers the actual TermFactory front end toolkit. A noteworthy feature of the TF approach that we try to get by with as few TermFactory specific adaptations as possible, so that the tools remain usable for their original purposes as well. Here are a few cases in point:

The TF editor is implemented as plugin on the open-source javascript wysiwyg editor CKEditor .

• TF XHTML entries are a special case of XHTML. CKEditor is a generic HTML editor. Thus i can be used to edit any HTML content besides TF entries.
• TF entries are a special case of RDF/OWL models. The back end is able to convert any RDF/OWL model into XHTML. Thus the TF Editor can be used as a general purpose RDF/OWL ontology editor.
• The TF default term entry template is a special case of entry template. The editor can be adapted to other formats, like the WordNet lexicon format, by changing editor and back end parameters.

Login

TermFactory user management is described in the section on the TF gate service. A front end to the gate service is the GateForm servlet. TermFactory QueryForm and EditForm forward to the GateForm servlet to check user credentials. GateForm then makes a new request to the original form with the user's credentials. GateForm can be run alone with the webapp URL TermFactory/login. Currently, the GateForm form just checks the login and shows some user information.

Search

A number of likely query types have been built into the query form from among the more or less boundless variety expressible in the SPARQL query language and supported by the TF query engine.

The fetch document/s query type fetches one or more documents identified by URL or by a location mapped TF address using the TF retry facility. With the no routing option, TF location mappings are not used. TF built in naming conventions, including TF built-in pseudo schemes (tdb+ and rdb+) and relative file url resolution with respect to TF_HOME are still applied. The no routing option applies to all document retrievals during a query execution, including repository locations and XHTML parameters.

The default assumption is that the documents fetched by a URL query are RDF documents in one or another of the formats understood by TF. If the fetched document contains an ontology that imports other ontologies, then they are included in the query result. The output format of combined graph can be selected from the graph format radio buttons. If the requested format is different from the source, the source is read into a model and rewritten in the requested format.

With the no imports option, ontology imports are not read and included in the result. If, moreover, the requested format matches the original format (or format any is checked), the document is downloaded as is. An individual document (rdf or not) can fetched and downloaded as is with URL query settings no imports and format any. Alternatively, the download address and submit button can be used for the same purpose.

The accumulate checkbox joins the contents of the document at the download address to the result of the query. Together with the to file option, it allows collecting a complex query result incrementally with a sequence of simpler queries.

If to file is not checked, QueryForm tries to return the query result in a response page while you wait. If the query takes too long, check the to file checkbox. Then the form returns immediately with the user selected or a machine generated model ID. The query results will be saved to the displayed download address in due time. They be can be fetched from the download address with the download button when the query is complete.

The fetch or describe resource/s query option is a mix of the fetch document and a TF DESCRIBE query. The input line can contain for one or more whitespace separated TF resource names (URIs, prefixed names defined whose prefix is defined in the repositories, or nicknames defined in location mappings). Each such name is first looked up as an address for a document as in the fetch document query type. This allows precomputing the results of a common or complex describe query for X into a canned entry X. A DESCRIBE query is carried out for those resources that had no associated document. (A fresh DESCRIBE query for a list of resources X can be requested by entering DESCRIBE X in the query textarea.)

The next three radio buttons on the left side of the form search and describe terms by baseform string pattern using the SPARQL regular expression filter.

The right side of the form runs user defined SPARQL queries. A query file can be specified by name or the query text can be just typed or copypasted in the text area.

The Query button executes the chosen query with the settings shown at the bottom of the form. The Clear button removes messages from the page.

Save settings let the user specify a descriptive name for the query result. The place where the results are saved appear in the download address. If no modelID is specified, TF creates a long random name (a UUID).

The Download button downloads the document specified in the download address.

The Edit button copies the download address to EditForm and opens EditForm. The EditForm does not fetch the document automatically, in case some options should be set before downloading the document in the editor.

The Copy button copies the contents of the download address to the upload address. (The upload address must be a writable TF address.)

Edit

The edit form lets the user fetch a prefabricated TF document to a textarea editor (the URL button), or make a TF describe query for a resource URI (the URI button). The results of the query are shown in the editor in TF xhtml format. If the url of an active model url was given, resources and statements found in the active model are boldfaced. If localisation language (and ontology) if given, resource labels are localised according to them.

The user can then edit the XHTML document using the editor. On Save, the editing changes are interpreted against the active model and an edited version of the active model saved in the TF database cache. The database name (modelID) of the saved version is shown on the page. On Upload, the saved version is uploaded to the given url on a TF webserver.

Make note that it is an edited copy of the active model, not the editor contents, that gets saved. The point is that the contents in the editor may come from many ontologies, as the result of a multi-repository or multi-site query. The extra information in the query result may be needed to decide how to edit the active content, but only the currently active ontology is subject to change. Also, separating the edits from the active model makes it possible to edit selected parts of a large ontology without dragging all of it to the editor.

As a special case of the above, consider the case when the edits and the active model coincide. This happens, in particular, in editing an existing entry (an XHTML format document). In this particular case, if the active model is not set, the servlet sets it is equal to the source. On Save, what is saved is a new version of the edited entry. In this particular case, too, the servlet tries to update the edit area with the saved active model. (Normally, when the active model is different from the edited contents, a Save does not update the edit area.)

Edit form

The edit form contains, besides the term edits textarea housing the CKEditor, a menu of settings. This part of the form pops up from the Set button on the CKEditor toolbar.

Show/hide edit settings popup menu

Although the settings menu appears to pop up "from inside" the textarea editor, it actually is a part of the the same form as the termarea editor. Its looks are determined by the edit form's stylesheet editform.css . The controls of the menu are described below.

The database provides a persistent cache to hold edits between edit sessions. When 'save in DB' are checked, the results of edit actions are saved in the default TF database. When 'search in DB' is checked, and location mappings are set up to look first for a database version of a resource, the next query for the resource gets the version last saved in the database. This lets the editor stop editing on a version of an entry, go away to do other things, and return to the version he worked on last. Or it can be a community of editors sharing a version. Then there are the usual difficulties about document sharing here that have to be solved by some locking or conflict resolution mechanism. If the editor sits on some collaborative platform like Mediawiki, shared versions can be saved on the platform and let the platform take care of turn taking.

The subsequent options from template to hyperlinks instruct the XHTML writer . They are usually defaulted so that one need not set them. The defaults are shown in the menu. Unless set by user, they are fetched from the entry's meta triples or html header.

The radio button provides the first two options of the TF query form. The document button does a URL query, i.e. uses Retry to fetch a document by name. (If TF routing is used, the name can be anything defined in the location mappings. If not, it should be an accessible URL.) The resource button calls for a DESCRIBE query for the resources whose names are listed in the source. The editor Query button currently only provides these two query types. When a more complex query is called for, use the query form and save the results somewhere to fetch into the editor.

The Edit button sends the contents of the editor to the edit servlet that relays it to the EditService back end for validation. If the edits are well-formed, the edit service rewrites the XHTML (to show what the edits actually amount to) and returns them to the editor. If the database save checkbox is checked, the edits are also saved in the database. Further, if an active ontology was specified, the edit service also updates the active ontology with the edits and saves an updated version of the active ontology in the database. The safe strategy is first uncheck the save checkbox and use Edit just to validate the edits. If they look right, then check the save box and click Edit another time. When returning to editing after a while, if there is a risk that the edits in the editor have gone stale, get the newest version from the database by checking 'search in DB'. (This assumes location mappings are turned on and configured to check the default database first.)

The blanks select menu gives choices for rewriting blank nodes in an entry. Option deblank replaces blank nodes with blank resource names of form urn:blank:... . This gives the blanks a persistent if precarious identity that remains between rewrites. By the RDF standard , RDF reads/writes do not preserve blank IDs . Option reblank rewrites blank URNs back to normal blanks. Option relabel tries to use the TF rewrite utility to invent descriptive labels to terms and expressions according to the TF descriptive label naming convention .

The Upload button uses IndexService to upload the edits to the given upload location (a TermFactory web directory URI). If there are no edits but a modelID, the model at the given modelID is uploaded.

CKEditor

TermFactory extends CKEditor with a TF specific plugin for CKEditor.

The toolbar shortcut buttons apply the settings in force in the edit form. They just save the trouble of opening the popup when there is no need to change the settings. The TF Insert menu is also on the CKEditor context menu (right button of mouse).

Since the TF editor is a fully featured HTML editor, it is always possible for a user to enter content anywhere in an entry by just typing it there, or by cutting and pasting content anywhere and then editing it. Once one is at home with some terminology and entry format, that may well be the most convenient way of working. However, should one forget what can go where, or just want to avoid typos, there is the TF Insert menu.

Show/hide TF Insert menu

The purpose of the TF Insert menu is to help users choose properties and values suited for each type of resource. The menu has (currently) four tabs. The menu is opened at a tab that matches the current selection in the editor text area: term for terms (signs) and definitions (messages), exp for expressions and texts (forms), ont for concepts (meanings), and any for unclassified properties. Each tab has (currently) just two text inputs, one for property and the other for its value. The four tabs differ only in how the input fields are autocompleted. A click on a property input tells TF Insert menu to look for a property or value among those available for the tab. Another click opens a selection of subproperties of the property. Clicking on the selection repeats the process, until the desired property appears.

A click on a value input makes TF Insert menu to look for classes in the range of the currently selected property. (If the selected property is a datatype property, TF Insert menu autocompletes with literal values for the property. If there is no input property, search starts with the top class for the tab.) Another click opens a menu of subclasses of the value to choose from. Clicking on the selected value starts the subclassing process again with the selection. If the selected class has no subclasses, TF Insert menu looks for its instances.

The TF Insert menu has one submit button called Insert and two exit buttons, Cancel and OK. The Insert button uses the input value to locate a template from a predefined collection of input templates. An input template is a predefined bit of XHTML associated to the currently selected value. The Insert button tells TF Insert menu to insert the template as the value of the input property in the property list containing the current selection in the text area. If no template exists for the input value, the input value as such gets inserted. Insertion is undoable (select Undo from CKEditor toolbar or hit Ctrl-Z).

The OK button closes the menu but remembers the current inputs for the next time the menu is opened. It also creates a shortcut to the current inputs in the input autocomplete list. Such shortcuts are of form property=value . Selecting a shortcut restores its property and value as the current inputs. Once the inputs are restored, one can use them to do another autocomplete or insert.

The Cancel button closes the menu and clears the tab and the shortcuts. The next time, the canceled tab opens with its startup values.

Wiki

A TermFactory Wiki has been implemented on MediaWiki as a Special page run by a MediaWiki extension that embeds the CKEditor javascript textarea editor. MediaWiki hosts the QueryForm servlet in an inline frame on the page. The query result shown in the frame is transferred on the editor text area, where it can be worked on using a TF extension of the CKEditor javascript editor. The edits get saved on the MediaWiki page (database) between editing sessions. The wikipage version of an entry becomes XHTML insert on the page. It can be edited as is on the MediaWiki page, or copy-pasted to another XHTML editor. The edits can be cached to the repository database with the term editor page Save button. The Save button also saves the changes to the active ontology in the database. An ontology can be published on a site's ontology collection in a WebDAV file server using the Upload button.

Comment

In forums, too, we want to keep a distance from specific implementations. One message standard in this area is RSS , which supports internet messaging between forums, blogs and other similar discussion platforms. More recently, json (jsonp) has gained popularity. It seems that the RDF based RSS 1.0 specification never caught on, and it has been superseded in practice with the XML based RSS 2.0 specification.

Disqus

So far, there is no forum implementation for TermFactory. The TermFactory Mediawiki platform is set up to generate Disqus comments threaded by the TF resource uri given in the page page title. In disqus terms, the TF resource uri is used as the disqus identifier instead of hosting page url. In this way, comments on the same resource coming from different platforms get collected together and can be reviewed as a group.

Workflows

This section documents workflows and best practices in TF based terminology work. There is a general section on division of labor and the terminology worklow, and a section on how the work happens on different platforms. The latter is further divided to sections on professional terminology tools, The TF wiki platform, and TF panel inserts on other platforms

The increase in user created content and interactivity gives rise to issues of control over the community and ownership of the jointly-created content. This gives rise to fundamental legal issues, such as Intellectual Property and property ownership rights.

Collaborative terminology work vs. traditional terminology

Since there is no regimented methodology for collaborative terminology work yet, there are both possibilities and risks. The following table compares traditional terminology work per work phase to the TF workflow. TF methods do not oust traditional ones, but complement them

A virtual expert community is a natural accumulation point for links to relevant documentary sources. The TermFactory user base keeps a steady flow of term proposals just by failing to find terms in the repository. Community portals can also be actively harvested for terms not covered by the repository using ContentFactory information retrieval, fact extraction and term / keyword extraction tools. The user base's preferences for term usage can be monitored in the community using original or active voting schemes. Such statistics can be used as (descriptive terminology) is or to support authoritative decisions in grading or (de)selecting term candidates (normative terminology, harmonisation). Term choices can also be evaluated against the expression ontology of the complete TF repository system so as to avoid inter-sector term clashes, accidental homonymy, and to enhance terminological consistency. The primary source for TF terminology is the repository system (or a subset of authoritative servers). When separate compilations are needed for special purposes, for publication on some restricted channel or the like, the desired subset can be retrieved from the system using suitable database queries. The results of the queries, being in a structured standard form, can be rendered in desired formats according to publication channel using fully automatic transformation pipelines.

Another table of comparisons can be made according to the roles and responsibilities of the actors in the traditional versus TF terminology workflow. Here, there are many opportunities but also potential risks and losses, not the least for those whose roles are susceptible to change in the process.

In the traditional workflow, terminologies are made to order by professional terminologists working either in-house (relatively few organizations can afford to have in-house terminologists) or as a paid service by terminology organizations or language services. As a separate process, terminology is not a very profitable business, owing to the high manpower cost of quality terminology and the relatively low priority given to terminoloy by clients. Worldwide, there are not many companies or organisations solely specialising in terminology work; many of the existing ones are small and/or get public funding. The profits from high quality terminology for a company or organisation whose main business is elsewhere are long term and indirect, and usually relegated under some low profile budget item. It makes more sense to integrate terminology with some more pressing mainstream business item or service. Thus larger language services can do terminology as a part of a wider language service offering, like language training, translation, multilingual content management, or the like.

Part of the high cost of terms comes from expensive expert time. Expert committees are ill attended and tend to converge slowly because terminology disagreements often hide territorial conflicts. While no technology can remove these problems, collaborative terminology platforms can help reduce these bottlenecks by allowing a wider range of opinions and more freedom from calendar conflicts. The fear might be raised that web communities are too amorphous and unruly to contribute reliable input. This criticism uses the free-for-all web communities like Wikipedia as the thought model. Even Wikipedia is surprisingly good at correcting itself, but there is no need to think that TF expert communities are anonymous or free for all. Membership of a given community may well be restricted and the contributors identified. Technology is not the limiting factor here.

An interesting possibility is to use experience-based attribution of authority by the community, a method that works well in many web-based expert communities. An expert whose advice has been useful for a large number of users gains authority points (counting thanks from users, or some other measurable proof of quality). The community can use this information to sort or weigh alternative opinions on a given topic. Although one can imagine many ways how this idea might not work, perhaps surprisingly, it practice it does.

Another possible criticism is that the expert community can only have public discussions. Nothing prevents one-to-one or other private discussion en petit comité among members of on an expert forum, either on the forum or separately, whether real time chat, forum or email style.

It is important to separate technology from its application. A case in point is the question of motivation. Professional terminology work is done for a fee, while one of the seductions of the idea of collaborative terminology work is that it might happen for a shared interest, on a tit-for-tat basis - or for indirect profit or gratification, like the visibility and authority gained from contributing to communities like Wikipedia or LinkedIn. But all this is again independent of the technology. It is quite as possible to build a TermFactory installation working on a fee-for-service basis. For information providers (terminologists, experts) the payment is based on content rendered and approved by the buyer(s). For information consumers (terminology users), the payment could be on a subscription basis, or measured by term downloads. Different user roles, rights and obligations can be defined as is done in many existing collaborative work platforms.

Professional terminologists are trained to follow standard work methods and quality norms in their work. If the work is spread on a larger number of subject experts and users, quality may deteriorate. True; this is one of the areas where TF needs to come up with innovative solutions. Roughly, we need a regimented upward flow in a TF repository system, rather reminiscent of the Wikipedia echelons of quality checking. When term suggestions come in, they are made available to the communmity with a low reliability status. When they have been revised by the commumity and perhaps passed inspection by a select group of experts, they get a higher status. At the end, they may be adopted by the repository authority as part of the "normative" core of the collection. (This process is one where the users too may get promoted to a higher status.)

One crucial quality requirement for professional terminology is source indication. This plays an important role in the subsequent authorisation of terminology. Here, information technology can help a lot. First, the platform may make it easy for users to to add source indications in a way that minimally is traceable to the source and at best follows a given norm (say, using web addresses and/or one of the many bibliography formats in use). Second, terminology suggestions can be evaluated by the reliability and authority of the source and/or the proposer(s).

TFS terminology repositories as a rule are an ongoing concern. This is a major improvement to the status quo, where typically a terminology project is started when earlier terminology collections are hopelessly obsolete, and by the time the new collection is out it is already obsolescent itself. A TF repository is open 24/7, so it can catch the newest fads. On the other hand, the repository system is under revision control, so that earlier editions can be kept on the accessible, or an authorised edition alongside the nightly release. It is also possible to open and close special purpose scheduled terminology projects using the repository system as a base of data. There is no need to throw away anything that works, just because there are more alternatives available.

TermFactory roles around MediaWiki

Here we consider the division of labor in a Wiki based TermFactory community and the associated user roles.

Approved TermFactory specific descriptive information about some resource, say 'cat' meant for human consumption can be entered on a wiki page associated to the resource. The content of the page should agree with the TermFactory entry about the this sense of the word 'cat'. If it does not, one or the other needs updating.

Also on the page is the snapshot of a TermFactory entry for this term resource that goes with this version of the page. It is easy to browse to an earlier version of the enty and the accompanying text using MediaWiki page history.

One plausible division of labor is the following.

• Unregisterd users can browse the open collection.
• Unregisterd users can discuss terms using Disqus commenting system,
• Unregistered users can discuss MediaWiki pages trough the MediaWiki Discussion tab.
• A wiki page in an open collection can be edited by self-registered (i.e. identifiable) users.
• A wiki page in a closed collection can be edited by other-registered (i.e. approved) users.
• The TF entry associated to a resource can be edited by approved users.
• An ontology can be edited with a TF entry by authoritative users (owners of a collection).

Each echelon can be further divided by category to specialists of each category.

Ontology work

This section documents one aspect of ontology based professional terminology work. viz. mixing and matching of term collections using OWL editors, reasoners, and converters. Using ontology tools can save manual labor in matching and merging data from differerent sources. Existing owl reasoners and query engines are generic tools whose application to terminological ontologies requires a high level of ontology expertise and remains semiautomatic at best. The aim of the TF specific ontology tools is to telescope the workflow so as to minimise expert intervention.

Extracting

One of the basic tasks in using TF is extracting some desired subset of data from one or more repositories as dataset. This section surveys different ways of going about this task.

Merging

One of the basic tasks in using TF is extracting some desired subset of data from one or more repositories as dataset. This section surveys different ways of going about this task.

Extracting, combining and matching terminological ontologies using generic and TF ontology tools has been studied in connection with merging the epidemic ontologies PULS and Biocaster. The epi related sparql scripts are in TF svn in directory cnv/script .

There are two main ways of merging ontologies in TF, by rewriting, or by importing and bridging. In rewriting, the contents of the donor ontology are reshaped into TF entities, removing the original structure. In importing/merging, a donor OWL ontology imports TFS and its concepts are related to TFS top ontology using bridge axioms. Rewriting avoids redundancy, saves space and creates better integration, but concurrent development of ontologies is difficult. Importing and bridging creates redundancy, bloat and clutter, but allows ontologies to keep their identity under the merger. The two methods can also apply in sequence, which may give something of both. Query imports offer a dynamic method of partial merging of ontologies.

Survey

This section surveys collaborative software and platforms that might suit TermFactory.

Applications

Equivalents editor

The MOLTO project is to use TF as middleware between domain ontologies and GF grammar. The MOLTO translation editor includes a tab for editing multilingual term equivalents. Content to the translation editor can be sent in the form of a TF localization file. This raises the question how to communicate the json content to TermFactory. The section on json format details one way.

The changes made in an equivalents editor client can be carried back to a matrix TF ontology maintaining those terms using the TF web API. A RESTful solution is this. First convert the json back to TF using a query like

http://localhost:8080/TermFactory/query?url=changes.json&schema=TFS.owl&rw=relabel&f=TURTLE &cacheDB=true

The result is saved in the default TF database. Then use the following query to edit the matrix ontology with the cached equivalents ontology.

http://localhost:8080/TermFactory/edit?m=matrix&d=original&a=changes.json&cacheDB=true

Here, matrix stands for the URL of the matrix ontology, original to the previous version of the equivalents and changes.json is the database location of the new converted equivalents ontology. As a result, an updated copy of the matrix ontology appears in the database, from which it can be fetched to TermFactory Wiki.

TF resources

TermFactory Manual

In many browsers you can choose page style from a built-in menu. As of end of 2011, Firefox, IE, and Opera have page style menus. Chrome does not, but the manual's on-page style menu seems to work. The print settings are defined in stylesheet print.css that gets imported by the other styles when css media is print (when the document is printed or viewed in the browser's print preview mode).

The slides of the Manual are generated with xslt script slide.xsl . An example of a Ubuntu command line is

xsltproc slides.xsl TFManual25en.xhtml > slides.xhtml

The table of contents of the Manual is generated with xslt2 script identity.xsl . An example of a Ubuntu command line is

saxonb-xslt -xsl:identity.xsl -ext:on -s:TFManual26en.xhtml

The result file is generated in the same directory with name TFManual_en.xhtml .

The TF white paper can be generated from the manual with xslt script white.xsl using a xslt processor. The white paper is linked under name TFmanual.html . The latest version of the complete manual is linked under name TFManual_en.xhtml . Slides only work from a file with extension .xhtml in Firefox.

TF config files

Currently, TermFactory config files are held in directory $TF_HOME/etc , where $TF_HOME stands for TermFactory installation directory, in the cases of web services, the webservice root. Set environment variable TF_HOME (in /etc/environment or equivalent) to point to the TF installation directory. The files in this directory so far are

The Jena policy file etc/ont-policy.rdf is a facility for mapping fully specified ontology uris to alternative urls. A TF site uses this file to list the ontologies it maintains. It also contains version information. The uri mappings in this ontology policy file are applied before prefix mappings. (Jena rdf location mappings are looked up for other non-ontology rdf files too, including rdf-format settings files.) If models you get are not those you wanted, there are a lot of places to check: webserver mappings, web container (tomcat) mappings, Jena ont-policy (both mappings and caching policy), Jena location mappings, TF_NOTRY option.

TF command line script option verbose turns on logging in a fixed set of classes. For more control over log messages, the files etc/logging.properties and etc/log4f.properties hold logging settings for TF command line scripts in io/script directory. The command line scripts get the location of the configuration file from java properties, see the scripts in io/script. The locations of the log4j.properties , logging.properties and pellet.properties files can be set with Java properties log4j.configuration , java.util.logging.config.file and pellet.configuration , respectively. These properties can be set as -D java command line options to the java runtime start command. Example:

TF_HOME=/home/lcarlson/Data/CF/TF export JAVA_OPTS=" -Dpellet.configuration=file:$TF_HOME/etc/pellet.properties"

Another pair of logging property files in etc/tomcat/ get included in the service archives when the services are built. Logging settings on installed services and web applications can be changed at runtime by editing copies of these property files at root level of the service aar (for services) and TermFactory.war/WEB-INF/classes (for TermFactory webapp).

Pellet and Jena mostly use java logging. TF code mostly uses the log4j logger. The log4j.properties file must be on the java classpath.

TF properties

TF options are by default read from TF property file $TF_HOME/etc/tf.properties . The location of the property file can be overridden with java property tf.configuration . For instance, command line scripts can use etc/tf.default.properties by adding to the java command line a parameter like -Dtf.configuration=${TF_HOME}/etc/tf.default.properties .

Locations are given in the configuration file as URLs or paths. As a special case, urls or paths of form file://foo or just //foo are resolved against the environment variable TF_HOME if that is set; else against configuration file property TF_HOME. If TF_HOME is not set or "", //foo resolves to current directory, i.e. ./foo . The location of the pellet configuration file can also be overridden by setting java property pellet.configuration to point to another location.

The TF_REPOS option gives the default data set to search by the site query service. This option is used when the query does not list the dataset explicitly. The TF_REPOS parameter is either a url to a list of repositories, or a hard-coded list of uris delimited by "\n".

The TF_SITES option gives the default sites to broadcast a query to. The TF_SITES parameter is either a url to a list of repositories, or a hard-coded list of uris delimited by "\n".

The TF DESCRIBE query classification depth is controlled with options TF_DESCRIBE_UP and TF_DESCRIBE_DOWN . The PelletClassify4TF taxonomy depth is controlled by TF_CLASSIFY_UP and TF_CLASSIFY_DOWN . The TF_INVERT flag controls the direction of classification triples in RDF and JSON output. The default (false) is to write triples bottom up (from particular to general), using properties rdfs:subClassOf and rdf:type . The invert option writes triples directed top down (from general to particular), using TF specific meta properties meta:hasSubClass and meta:instance . There is an analogous punned inverse of rdfs:subPropertyOf called meta:hasSubProperty for listing subproperty relationships top down.

The default output format is used by the TF query engine. The default output character encoding UTF-8 can be changed to UTF-16. Pellet4TF tries to guess the input character encoding of sparql files from the beginning of the file.

TF keeps its ontology policy file ont-policy.rdf at TF_HOME/etc . This file can be used to redirect ontology URL's to local files on a long term basis. The location mappings defined in ont-policy.rdf are included into the TF retry location mappings.

Boolean setting TF_VERSIONINFO turns on or off version checking of models in the database cache against version info given in the ontology policy file ont-policy.rdf . When true (the default), the TF location mapping retry facility warns about a model whose versionInfo differs from versionInfo associated to the same URI in the ont-policy file.

File URL TF_PREFIX_SPARQL specifies the file name of a sparql query for pellet4tf to copy default namespace prefixes from in order to simplify writing TF sparql queries. This option allows using conventional qualified names of TF URIs in sparql queries. For instance, to describe the country China, one can just submit the query DESCRIBE ont1:China . (One can also declare prefixes in the query prolog, or use full URLs like <http://tfs.cc/ont1/China> .) The same abbreviatory convenience applies to the pellet4tf describe-resource command line option -D .

Boolean option TF_QUERYDB tells the retry facility whether to query the default database cache for ontologies. (The default is true). Boolean option TF_CACHEDB tells whether the retry facility should cache ontologies in the default database cache. (The default is false). The configured settings can be overridden by command line scripts.

TF_TBX_MAPPING points to the RDF document containing the mapping rules used in conversions between TF and the TBX format.

TF_TEMPLATE allows setting the default template to write XHTML entries .

TF_ROOT allows setting the default root filter (one or more instances/classes) to include in entry XHTML entries .

TermFactory schema (TFS.owl)

The url of the current version of the TermFactory OWL schema is http://tfs.cc/owl/TFS.owl . It can be downloaded from the url (for instance, with a browser or an OWL editor like Protege ).

Listing of TF schema vocabulary

The following table is a listing of the TF schema vocabulary with comments. It was generated with pellet query c-comment.sparql using command line pellet4tf query -o RDF/XML -q c-comment.sparql ../owl/TFS.owl > comments.owl and converted to html with script comment.xsl using command line xsltproc comment.xsl comments.owl > comments.html . These commands can also be run with shell script doc/comment.sh .

Show/hide TF schema vocabulary

List of converted vocabularies

4m.owl

the 4M project ontology (ca 2000 entities) converted to TF (a one-time conversion) using a purpose-built Java Jena OWL converter FM2TF.java .

space.owl

the mobilite space ontology (ca 400 classes) downloaded from web and imported using a bridge ontology

yso-raken.owl

a library taxonomy of Finnish construction terms (ca 70 concepts) extracted from FinnOnto thesaurus ontology using sesame rdf

paperi.owl

A fi-de-en paper industry terminology converted from MultiTerm format to TF over TBX (ca 200 concepts)

rakli8.owl

A fi-en building maintenance glossary (ca 100 concepts). converted from PDF via TBX

ECHA-glossary.owl

A multilingual glossary of EU chemical terms.(ca 200 concepts). converted from csv exports of excel sheets using a dedicated perl script.

kyamk.owl

A multilingual glossary of customs terms.collected by KYAMK (ca 300 concepts). converted from csv exports of excel sheets using a dedicated perl script.

puls.owl

The PULS epidemics ontology converted from lisp sources using lisp and perl into Turtle format (ca 3000 terms).

puls-locations.owl

The PULS country ontology converted from lisp source using lisp into Turtle format (ca 1700 terms).

bio.owl

The BioCaster multilingual epidemics ontology converted from owl rdf/xml source using xslt (ca 8000 terms).

cml.owl

An en-fi-ru EU project ontology converted from html source into TF using xslt (ca 5000 terms).

sost.owl

An en-fi-ru welfare vocabulary converted from Windows Word content xml into TF using xslt (ca 800 concepts, 7000 terms).

hs.owl

Another multilingual welfare vocabulary converted from fixed-width tabular format through TBX into TF using Perl and xslt (ca 1000 concepts, 9000 terms).

TFCtry-??.owl

Iso 3199 country codes and names converted from Wikipedia using xslt and TF. en-fi-ru-zh, ca 250 codes per language, 2000 terms, 20K triples

TFLang-??.owl

Iso 639 language codes plus English and vernacular language names converted from Wikipedia using xslt and TF. 185 codes, 368 terms, ca 11K triples.

TFLang-??.owl

TermFactory subject field classification converted from MSWord over OpenOffice wiki text export and Perl. fi-en-zh, 97 headings, 591 subheadings, 4570 terms, ca 11K triples.

TermFactory sample ontologies

These links point to sample TermFactory ontologies. They import the TF schema plus possibly other ontologies. They can be downloaded and/or inspected with OWL tools like Protege .

The sample ontologies are copyrighted and confidential.

TermFactory ontology

The TermFactory ontology is an example of a layered ontology that contains information about expressions relevant for automatic natural language processing.

KYAMK ontology

The KYAMK glossary is a raw import of a recent multilingual glossary of customs terms.

ContentFactory wikipages

http://sysdb.cs.helsinki.fi/~tkt_plus/twiki/bin/view/ContentFactory/TermFactory

TermFactory source code repository

External libraries (Jena, Pellet) are kept separately and included into the download through svn:externals definitions.

Currently, the root structure of the code base is

cnv doc etc fe io owl txt viz ws

More specifically:

cnv

conversions between TF and other formats

doc

TF manual and other external documentation of TF

etc

TF setting files

fe

TF front-end code

io

TF back-end i/o and the query engine

owl

TF ontologies

txt

TF internal developer notes

viz

TF graphical visualizer

ws

TF web services

Version history

• version 2.6 31.05.12 gate service
• version 2.5 01.01.12, 12.02.12 pellet 2.3.0, tdb database
• version 2.4 24.12.11 TF API docs 2.4
• version 2.3 12.12.11 xhtml input templates
• version 2.2 15.11.11 xhtml output templates
• version 2.1 18.08.11 convert4tf, tfs.cc server version
• version 2.0 14.12.10 rewrite as service, virtual ontologies
• version 1.9 29.11.10 index services, EditServlet and QueryForm v. 1.9., TFStrict
• version 1.8 30.10.10 Separated TFProp, TFDom, TFCtry, TFLang, TF profiles
• version 1.7 15.09.10 TF for WordNet, separated TFL10n
• version 1.6 10.06.10 June 2010 demo
• version 1.5 19.05.10 TF self documentation by reflection
• version 1.4 13.04.10 TF profiles, front end design
• version 1.3 16.03.10 TF edit services, repository network
• version 1.2 15.01.10 TF webapp: query service with retry of location mappings; 20.03.10 TFS v 1.2
• version 1.1 15.12.09 query imports using query URLs
• version 1.0 08.11.09 tf io merged, 21.11.09 TF backend v 1.0 completed, 07.12.09 TF2XML format
• version 0.9 28.10.09 describe queries
• version 0.8 21.09.09 ontology matching
• version 0.7 05.04.09 TF3 format, more conversions
• version 0.6 03.04.09 June 2009 demo
• version 0.5 03.04.09 White paper
• version 0.4 01.04.09 TF query import specification
• version 0.3 25.02.09 TF schema in OWL DL, Protege 4 + Fact++, Pellet check ok
• version 0.2 02.02.09 TF schema, TermFactory, Protege 3.3.1
• version 0.1 01.11.08 TF schema, ECHA demos
• version 0.0 07.02.07 rakli8.owl converts and loads

Some bibliography

• Eberhart, Andreas. 2005. Ontology-based infrastructure for intelligent applications. Ph.D.diss., U des Saarlandes, Saarbrücken 2005
• Elena Montiel-Ponsoda, Guadalupe Aguado de Cea, Mari Carmen Suárez-Figueroa, Raúl Palma, Asunción Gómez-Pérez and Wim Peters. LexOMV: an OMV extension to capture multilinguality. http://neon-project.org/web-content/images/Publications/ontolex2007-lexomv-montiel-crv.pdf
• Timothy Redmond, Michael Smith, Nick Drummond, and Tania Tudorache. Managing Change: An Ontology Version Control System. clarkparsia.com/files/pdf/change-owled2008-eu.pdf
• Walsh, N. 2003. RDF Twig: accessing RDF graphs in XSLT. Extreme Markup Languages 2003. Montréal, Québec August 4-8, 2003.
• Baowen Xu, Peng Wang, Jianjiang Lu, Yanhui Li, Dazhou Kang. 2004 . Theory and Semantic Refinement of Bridge Ontology Based on Multi-ontologies. Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2004) 1082-3409/04 http://ieeexplore.ieee.org/iel5/9460/30023/01374221.pdf?arnumber=1374221

Table of contents