Filtering annotations

It is often useful to view, use, or process some subset of the annotations that are in a Knowtator annotation project. We have created a simple mechanism for creating annotation filters for defining some subset of the annotations in a Knowtator project. For example, it may the case that you have part-of-speech tags created by one annotator and named entities created separately by three annotators. Filters can be used to view only named entities from one of the annotators or to run IAA metrics on two of the annotators.

A filter is defined as a set of conditions that an annotation must satisfy to not be "filtered out". There are three conditions an annotation must meet:

It must be an annotation of one of the types specified by the filter.
It must have been created by one of the annotators specified by the filter.
It must be in one of the sets specified by the filter.

Thus, an annotation filter has three slots: types, annotators, sets. If any of these slots are empty, then it will not be used to filter out annotations. The Empty Filter has all three slots empty and therefore does not filter out any annotations.

To create an annotation filter do the following:

Click on the "Instances" tab. Below the Protégé menus and the tool bar are tabbed panes. Typical tabs that you will see open are labelled Classes, Slots, Forms, Instances, etc.
The Instances tab is split into three vertical sections. The left-most section displays the class hierarchy. Expand the subclass of :THING called knowtator support class by clicking the tree expansion icon.
Select the class called knowtator filter.
The instances of knowtator filter appear in the middle section. To create a new filter instance click the create instance icon near the top of the section.
If the new instance is selected from the list of instances in the middle section, then the slot values for the new instance will appear in the rightmost section. Verify that the new filter instance is selected.
Enter the name of the filter in the slot labeled name.
Select the types that the filter will allow by clicking the Add Class icon above the types slot.
Select the annotators that the filter will allow by clicking the Add Instance icon above the annotators slot.
Select the sets that the filter will allow by clicking the Add Instance icon above the sets slot.

The following are some ways that filters are used in Knowtator:

Annotation Display - The most basic usage of filters is to determine which annotations are displayed. The currently selected filter can be set by using the filter selection navigation buttons which look like this: . Only "active filters" can be selected. Active filters are specified in the configure dialog. By carefully selecting and ordering the active filters, one can set up a workflow for the annotators if the annotation task is very complex. That is, the first active filter may contain annotations that correspond to some subtask while the next active filter may contain annotatins that correspond to another subtask. When the annotator finishes one subtask she can advance to the next filter and work on the next subtask.
Annotation Removal - It is common in the course of an annotation project to end up with a lot of "clutter" in an annotation project. A filter can be used in conjunction with the Menu->Knowtator->Remove Annotations option to define a set of annotations to keep (discarding all others).
Exporting to XML - when exporting annotatins to XML you must specify a filter that determines which annotations to export.
Merge annotations
Inter-annotator agreement - when you calculate IAA you will be prompted for a filter that defines the annotations you want compared.
Consensus mode - Consensus mode makes heavy use of filters. Please see the consensus page for details.