Searching for items (and collections) [VC 21.3.1 GEN]

Searching in Vidispine

Searching in Vidispine is implemented using either Solr or Elasticsearch as the backend. This allows functionality such as boolean operators, faceted searching, term highlighting, search term suggestions, etc. It is possible to search for just items, just collections, or both at the same time, depending on which RESTful resource the search request is made to (/item, /collection or /search). The search criteria are expressed using an XML or JSON document of type ItemSearchDocument, described in more detail below.

For best performance

Don’t retrieve the hit count if you don’t use it.
Use filters if possible as these can be cached separately and do not affect the score nor highlighting.
Disable full text indexing for fields that contain JSON or other content that should not be included in the full text index.
Only fetch the specific metadata fields and groups that you need instead of fetching all metadata. See Get information.
If you only want to search in the generic metadata, or if your application does not use timed metadata, then make sure to specify <intervals>generic</intervals>.

Search history

Vidispine stores the search document, as well as the timestamp and user for all searches that are made. If the same user makes an identical search twice, only one entry will be shown in the search history, with the timestamp of the last search.

Queries

Boolean operators

Boolean operators AND, OR and NOT can be used in search queries. A boolean operator can contain zero or more field-value pairs and zero or more boolean operators.

Implicit operators

If no operators are specified operators are implicitly added using the following rules:

Multiple values within a field

If a field contains multiple values, an implicit OR operator is added.

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>originalFormat</name>
    <value>dv</value>
    <value>mp4</value>
  </field>
</ItemSearchDocument>

is logically equivalent to

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="OR">
    <field>
      <name>originalFormat</name>
      <value>dv</value>
    </field>
    <field>
      <name>originalFormat</name>
      <value>mp4</value>
    </field>
  </operator>
</ItemSearchDocument>

Multiple field elements at top level

If a document has multiple field elements at top level, an implicit AND operator is added.

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>originalFormat</name>
    <value>dv</value>
  </field>
  <field>
    <name>originalFormat</name>
    <value>mp4</value>
  </field>
</ItemSearchDocument>

is logically equivalent to

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="AND">
    <field>
      <name>originalFormat</name>
      <value>dv</value>
    </field>
    <field>
      <name>originalFormat</name>
      <value>mp4</value>
    </field>
  </operator>
</ItemSearchDocument>

Text elements

In version 1 of the query syntax text elements are added with an implicit AND operator, and in version 2 it’s an implicit OR.

Example

Searching for items that were not created within the last week and have either the formats mp4 or dv.

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="AND">
    <operator operation="NOT">
      <field>
        <name>created</name>
        <range>
          <value>NOW-7DAYS</value>
          <value>NOW</value>
        </range>
      </field>
    </operator>
    <field>
      <name>originalFormat</name>
      <value>mp4</value>
      <value>dv</value>
    </field>
  </operator>
</ItemSearchDocument>

Phrase search

Vidispine supports wildcard search and phrase search for field type string and string-exact. A phrase is a group of words surrounded by double quotes, such as “foo bar”.

Wildcard search

The wildcard special character in Vidispine is *, meaning matching zero or more sequential characters.

he*	words start with”he”, like he, hey, hello
h*e	will match he, hope, house, etc.
*he	words end with “he”, like he, the.

Note: wildcard in a phrase search is not supported (e.g. "foo b*" won’t be able to find foo bar).

Range search

Use a range query to find fields with values within a certain range. The minimum and maximum values for the data type of the field can be specified using the attributes minimum and maximum.

To search for any values:

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>app_score</name>
    <range>
      <value minimum="true"/>
      <value maximum="true"/>
    </range>
  </field>
</ItemSearchDocument>

To search for values in range [10..20], that is, inclusive of 10 and 20:

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>app_score</name>
    <range>
      <value>10</value>
      <value>20</value>
    </range>
  </field>
</ItemSearchDocument>

The exclusiveMinimum and/or the exclusiveMaximum attributes can be used to perform exclusive range queries.

For example, to search for values between 10 and 20 (exclusive):

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>app_score</name>
    <range exclusiveMinimum="true" exclusiveMaximum="true">
      <value>10</value>
      <value>20</value>
    </range>
  </field>
</ItemSearchDocument>

Search intervals

By setting <intervals> in the ItemSearchDocument, search criteria can be applied to metadata within different ranges accordingly:

generic	only search generic metadata, a.k.a metadata inside (-INF, +INF)
timed	search metadata within ranges other than (-INF, +INF)
all	search metadata both timed and generic metadata (default option)

For example, search items with only timed metadata containing originalFormat=dv:

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>originalFormat</name>
    <value>dv</value>
  </field>
  <intervals>timed</intervals>
</ItemSearchDocument>

Group search

Searching items by its metadata groups are supported.

Example

To find items with any groups:

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <group>
  </group>
</ItemSearchDocument>

To find items without any groups:

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="NOT">
    <group>
    </group>
  </operator>
</ItemSearchDocument>

To find items without a “movie_info” group:

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="NOT">
    <group>
      <name>movie_info</name>
    </group>
  </operator>
</ItemSearchDocument>

To find items with a “movie_info” group containing two fields with specific values. Note that the AND is implicit.

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <group>
    <name>movie_info</name>
    <field>
      <name>movie_name</name>
      <value>StarWars</value>
    </field>
    <field>
      <name>episode_no</name>
      <value>1</value>
    </field>
  </group>
</ItemSearchDocument>

To find items with a “movie_info” group with an episode number of either 1 or 2.

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <group>
    <name>movie_info</name>
    <operator operation="OR">
      <field>
        <name>episode_no</name>
        <value>1</value>
      </field>
      <field>
        <name>episode_no</name>
        <value>2</value>
      </field>
    </operator>
  </group>
</ItemSearchDocument>

To find items with either a “movie_info” or a “video_info” group.

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <operator operation="OR">
      <group>
        <name>movie_info</name>
      </group>
      <group>
        <name>video_info</name>
      </group>
  </operator>
</ItemSearchDocument>

Query syntax versions

In 4.2.0 a new query syntax was introduced. In order to use the new syntax, set the version attribute in the search document to 2. If no version is set, the old query syntax will be used (version 1).

Version 1

The search value of a string-exact field is always interpreted literally.
The search value of a string field is interpreted literally only if it’s surrounded by quotation marks. In other cases, implicit OR s are used in between the words.
Multiple values means OR. Multiple text elements means AND.

The noescape attribute is needed, if user want to search quotation marks or wildcard characters literally in a string field;

<?xml version="1.0"?>
<ItemSearchDocument>
  <field>
    <name></name>
    <value noescape="true">\"foo bar\"</value>
    <value noescape="true">foo\*</value>
  </field>
</ItemSearchDocument>

Version 2

One or more SPACE characters means logical AND. So <value>foo bar</value> and <value>foo bar</value> means searching a field value containing both foo and bar.

Multiple values means OR. To search for foo or bar, in the title or text:

<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>title</name>
    <value>foo</value>
    <value>bar</value>
  </field>
</ItemSearchDocument>

<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <text>foo</text>
  <text>bar</text>
</ItemSearchDocument>

Special characters in Vidispine search are ", SPACE, \, and *. Any character followed by \ is considered as literal. so \* means literal *, and \f is the same as the single character f.
The characters inside quotes are consider as literal, except ". A \" is still needed to represent a literal quote inside quotes.
The noescape attribute of a metadata field value has been removed since Vidispine 4.2.

Operators and special characters

To highlight the differences between the two versions:

Query	Version 1	Version 2
<text>foo bar</text>	foo OR bar	foo AND bar
<field>foo bar</field>	foo OR bar	foo AND bar
<text>foo</text> <text>bar</text>	foo AND bar	foo OR bar
<field>foo</field> <field>bar</field>	foo OR bar ^[1]	foo OR bar
“foo bar”	“foo bar”	“foo bar”
\”foo\”	\\”foo\\” ^[2]	\”foo\”
foo*	foo*	foo*
foo\*	foo\\* ^[2]	foo\*
foo\_bar ^[3]	foo\\ OR bar ^[2]	foo\_bar

String types

An example of the differences when searching string fields, assuming a field value of foo bar.

	`foo bar`
	Version 1		Version 2
	string	string-exact	string	string-exact
`foo`	Y	N	Y	N
`FOO`	Y	N	Y	N
`foo bar`	Y	Y	Y	N ^[5]
`"foo bar"`	Y	N ^[4]	Y	Y ^[6]
`foo\ bar`	Y	N	Y	Y ^[5]
`"foo xy"`	N	N	N	N
`foo xy`	Y	N	N ^[7]	N

1	Use `<operator operation="AND">` to search for `foo AND bar` for example.
2 (1, 2, 3)	Use `noescape=true` to search for literal `"`, `*` and `SPACE`.
3	Here `_` means SPACE.
4	The character `"` is interpreted literally.
5 (1, 2)	`SPACE` is a special character and needs to be escaped in order to get literally meaning.
6	It’s a phrase search, and “string-exact” only have one token in the index, which the same as the query in this case.
7	It’s `foo OR xy` in version 1, and `foo AND xy` in version 2.

Filters

A search filter is a query does not affect scoring nor highlighting, similar to a filter query in Solr. A filter can:

Contain both fields and operators.
Be named and excluded from facets.

Example

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <filter operation="OR" name="productType">
     <field>
       <name>type</name>
       <value>pc</value>
     </field>
     <field>
       <name>type</name>
       <value>phone</value>
     </field>
   </filter>
</ItemSearchDocument>

Joins

Joint searches on metadata of item, share and file are supported. The old search schema is extended with three new search criterion types: <item>, <shape>, and <file>. Please refer to xmlSchema.xsd for the full schema.

Depending on the search result needed(items, shapes, or files), ItemSearchDocument, ShapeSearchDocument or FileSearchDocument should be sent to Vidispine respectively. Those three search documents use the same syntax, only the document names are different.

Note:

A version = 2 document is needed in order to perform the joint search.
The <intervals> constrain only works for item metadata in a ItemSearchDocument. It has not effect in ShapeSearchDocument and FileSearchDocument.

Examples

Joins on item search

Find items containing shapes with metadata shapeCodec=mp4:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <shape>
    <field>
      <name>shapeCodec</name>
      <value>mp4</value>
    </field>
  </shape>
</ItemSearchDocument>

Find items that have generic metadata title = vidispine, and contain a shape with shapeCodec=mp4, and contain a file with metadata filetitle = demo:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <item>
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
  </item>
  <shape>
    <field>
      <name>shapeCodec</name>
      <value>mp4</value>
    </field>
  </shape>
  <file>
    <field>
      <name>filetitle</name>
      <value>demo</value>
    </field>
  </file>
  <intervals>generic</intervals>
</ItemSearchDocument>

Find items that have generic metadata title = vidispine, and contain a shape with shapeCodec=mp4, and contain a file with metadata filetitle = demo:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <item>
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
  </item>
  <shape>
    <field>
      <name>shapeCodec</name>
      <value>mp4</value>
    </field>
  </shape>
  <file>
    <field>
      <name>filetitle</name>
      <value>demo</value>
    </field>
  </file>
  <intervals>generic</intervals>
</ItemSearchDocument>

Find items that have metadata title = vidispine, and contain a shape with shapeCodec=mp4; the shape must contain a file with metadata filetitle = demo:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <item>
    <field>
      <name>title</name>
      <value>item</value>
    </field>
  </item>
  <shape>
    <field>
      <name>shapeCodec</name>
      <value>mp4</value>
    </field>
    <file>
      <field>
        <name>filetitle</name>
        <value>demo</value>
      </field>
    </file>
  </shape>
</ItemSearchDocument>

Operators are also supported as part of a search criterion.

Find items that have metadata title = vidispine, or items that have metadata title = demo and contain shapes with shapeCodec=mp4:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="OR">
    <item>
      <field>
        <name>title</name>
        <value>vidispine</value>
      </field>
    </item>
    <operator operation="AND">
      <item>
        <field>
          <name>title</name>
          <value>demo</value>
        </field>
      </item>
      <shape>
        <field>
          <name>shapeCodec</name>
          <value>mp4</value>
        </field>
      </shape>
    </operator>
  </operator>
  <intervals>all</intervals>
</ItemSearchDocument>

Joins on search shapes

Find shapes belong to items that have metadata title = vidispine, and the shape should have a file with metadata filetitle = demo:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ShapeSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <item>
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
  </item>
  <shape>
    <field>
      <name>shapeCodec</name>
      <value>mp4</value>
    </field>
  </shape>
  <file>
    <field>
      <name>filetitle</name>
      <value>demo</value>
    </field>
  </file>
</ShapeSearchDocument>

Find shapes belong to items that have files with metadata filetitle = demo, and metadata title = vidispine:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ShapeSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <item>
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
    <file>
      <field>
        <name>filetitle</name>
        <value>demo</value>
      </field>
    </file>
  </item>
  <shape>
    <field>
      <name>shapeCodec</name>
      <value>mp4</value>
    </field>
  </shape>
</ShapeSearchDocument>

Joins on file search

Find files belong to items with metadata title=demo; it should also belongs to shapes with metadata shape_title=shape:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<FileSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <item>
    <field>
      <name>title</name>
      <value>demo</value>
    </field>
  </item>
  <shape>
    <field>
      <name>shape_title</name>
      <value>shape</value>
    </field>
  </shape>
</FileSearchDocument>

Joins on collection search

Not yet supported with Elasticsearch.

Find collections that have metadata title=vidispine or collections contains an item with metadata title=demo:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="OR">
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
    <item>
      <field>
        <name>title</name>
        <value>demo</value>
      </field>
    </item>
  </operator>
</ItemSearchDocument>

To find items with specific shapes or files, use a shape or file query as a subquery of the item query.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>title</name>
    <value>vidispine</value>
  </field>
  <item>
    <shape>
      <field>
        <name>shape_title</name>
        <value>demo</value>
      </field>
    </shape>
  </item>
</ItemSearchDocument>

Using an item subquery is only possible when the search interval is either generic or all. When using timed then no item subquery is allowed.

Find empty collections.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <operator operation="NOT">
    <item>
    </item>
  </operator>
</ItemSearchDocument>

Collection hierarchy joins

New in version 5.5.

Collection joins can be used to search for collections and items based on their relationships by using the <collection> element in the ItemSearchDocument. The <collection> element may only be used when searching explicitly for items or collections. To specify the relationship between the entities use the relation attribute with one of the attributes child, parent, ancestor or descendant. If no relation is specified the child relation is assumed.

Not supported with Elasticsearch.

Using a collection subquery is only possible when the search interval is either generic or all. When using timed no collection subquery is allowed.

Example

Find collections that have a child collection with title=vidispine.

PUT /collection
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <collection>
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
  </collection>
</ItemSearchDocument>

Example

Find collections that have a parent collection with title=collection1 or title=collection2 and a descendant collection containing an item with title=vidispine.

PUT /collection
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <collection relation="parent">
    <operator operation="OR">
      <field>
        <name>title</name>
        <value>collection1</value>
      </field>
      <field>
        <name>title</name>
        <value>collection2</value>
      </field>
    </operator>
  </collection>
  <collection relation="descendant">
    <item>
      <field>
        <name>title</name>
        <value>vidispine</value>
      </field>
    </item>
  </collection>
</ItemSearchDocument>

Example

The <collection> element may also be nested to search for complex relationships. Find collections that have a grandparent with title=vidispine.

PUT /collection
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <collection relation="parent">
    <collection relation="parent">
      <field>
        <name>title</name>
        <value>vidispine</value>
      </field>
    </collection>
  </collection>
</ItemSearchDocument>

Example

The <collection> subquery may also be used when searching for items. But only the parent and ancestor relations can be used (items can not contain collections). Find all items that have an ancestor collection with title=vidispine.

PUT /item
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ItemSearchDocument version="2" xmlns="http://xml.vidispine.com/schema/vidispine">
  <collection relation="ancestor">
    <field>
      <name>title</name>
      <value>vidispine</value>
    </field>
  </collection>
</ItemSearchDocument>

Highlighting

Highlighting can be enabled to determine which part of the metadata that matched the query.

Use the field element to enable highlighting for a certain set of fields only.

<highlight>
   <field>title</field>
</highlight>

Example

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <field>
      <name>title</name> <!-- Search for the words "interview" or "credits" within the title -->
      <value>interview</value>
      <value>credits</value>
   </field>
   <highlight> <!-- Having a highlight element will enable highlighting even if it is empty -->
      <matchingOnly>true</matchingOnly> <!-- Only highlight fields that actually matched the query. -->
      <prefix>[</prefix> <!-- A string that appears before the highlighted text -->
      <suffix>]</suffix> <!-- A string that appears after the highlighted text -->
   </highlight>
</ItemSearchDocument>

<ItemListDocument>
   <item id="VX-123" start="-INF" end="+INF"> <!-- Matches in the document were on the interval [-INF, +INF] -->
      <timespan start="-INF" end="100">  <!-- One match on [-INF, 100] -->
         <field>title</field>
         <value>[Interview] with the CEO.</value> <!-- The word "interview" is highlighted with the suffix and prefix -->
      </timespan>
      <timespan start="400" end="+INF"> <!-- Another match on [400, +INF] -->
         <field>title</field>
         <value>Closing [credits]</value> <!-- The word "credits" is highlighted with the suffix and prefix -->
      </timespan>
   </item>
</ItemListDocument>

When searching with cursor in non-generic timespans, only one timespan per item or collection contains highlighting information.

Sorting

Results can be sorted using sortable fields. Multiple fields can be used for sorting and are used in the order they are given.

It is also possible to sort by relevance by specifying _relevance as the field name.

Specify _type to sort by type. The type of an item is item and collection for collections, so if you want collections first in the results, then sort on _type in ascending order.

Any field can be used for sorting, it does not need to flagged as sortable. If a field contains multiple values: ascending order will compare with its minimum value and descending order will compare with its maximum value.

Example

Listing all items sorted according to length in descending order and format in ascending order.

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <sort>
      <field>durationSeconds</field>
      <order>descending</order>
   </sort>
   <sort>
      <field>originalFormat</field>
      <order>ascending</order>
   </sort>
</ItemSearchDocument>

Case-insensitive sorting

New in version 5.2.2.

Field values can be sorted case-insensitively if caseSensitiveSorting=false is configured in the metadata field definition:

<MetadataFieldDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <type>string-exact</type>
  <caseSensitiveSorting>false</caseSensitiveSorting>
</MetadataFieldDocument>

A reindex is required after the configuration change for it to take effect.

Faceting

Faceting is used to show number of matching items for one or more sub-constraints for a given result-set. You might for example be interested in displaying how many of the items returned from a search are of type video, how many are of type audio, and how many are of type data.

There are two types of operations that can be performed, counting and specifying ranges. Counting means that it will count the occurrences of each unique value. When specifying ranges, the number of occurrences within a certain range is counted. Both the start and the end of a range are inclusive and “*” can be used to represent minimum or maximum. Note that faceted search only can be used over non-timed metadata.

Example

item	category	price
`VX-251`	`tv`	100
`VX-252`	`radio`	200
`VX-253`	`tv`	300
`VX-254`	`phone`	400
`VX-255`	`radio`	500
`VX-256`	`radio`	100
`VX-257`	`phone`	200
`VX-258`	`phone`	300
`VX-259`	`phone`	200
`VX-260`	`phone`	300

Consider the items in the table above, together with their metadata on the fields my_category and my_price. A faceted search that should count the occurrences of each category and the occurrences of prices within the ranges [*, 199], [200, 399] and [400, *] might look like this:

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <facet count="false">
        <field>my_price</field>
        <range start="*" end="199"/>
        <range start="200" end="399"/>
        <range start="400" end="*"/>
   </facet>

   <facet count="true">
        <field>my_category</field>
   </facet>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <hits>13</hits>
   <item id="VX-248" start="-INF" end="+INF"/>
   <item id="VX-249" start="-INF" end="+INF"/>
   <item id="VX-250" start="-INF" end="+INF"/>
   <item id="VX-251" start="-INF" end="+INF"/>
   <item id="VX-252" start="-INF" end="+INF"/>
   <item id="VX-253" start="-INF" end="+INF"/>
   <item id="VX-254" start="-INF" end="+INF"/>
   <item id="VX-255" start="-INF" end="+INF"/>
   <item id="VX-256" start="-INF" end="+INF"/>
   <item id="VX-257" start="-INF" end="+INF"/>
   <item id="VX-258" start="-INF" end="+INF"/>
   <item id="VX-259" start="-INF" end="+INF"/>
   <item id="VX-260" start="-INF" end="+INF"/>
   <facet>
      <field>my_category</field>
      <count fieldValue="phone">5</count>
      <count fieldValue="radio">3</count>
      <count fieldValue="tv">2</count>
   </facet>
   <facet>
      <field>my_price</field>
      <range start="*" end="199">2</range>
      <range start="200" end="399">6</range>
      <range start="400" end="*">2</range>
   </facet>
</ItemListDocument>

Now assume we want to see how the prices are distributed for phones, we could filter the search in the following manner:

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <filter>
      <field>
        <name>my_category</name>
        <value>phone</value>
      </field>
   </filter>
   <facet count="false">
        <field>my_price</field>
        <range start="*" end="199"/>
        <range start="200" end="399"/>
        <range start="400" end="*"/>
   </facet>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <hits>5</hits>
   <item id="VX-254" start="-INF" end="+INF"/>
   <item id="VX-257" start="-INF" end="+INF"/>
   <item id="VX-258" start="-INF" end="+INF"/>
   <item id="VX-259" start="-INF" end="+INF"/>
   <item id="VX-260" start="-INF" end="+INF"/>
   <facet>
      <field>my_price</field>
      <range start="*" end="199">0</range>
      <range start="200" end="399">4</range>
      <range start="400" end="*">1</range>
   </facet>
</ItemListDocument>

The opposite is also possible, to see the distribution of the categories over a range of prices.

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">

   <filter>
        <field>
          <name>my_price</name>
          <range start="200" end="399"/>
        </field>
   </filter>

   <facet count="true">
        <field>my_category</field>
   </facet>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <hits>6</hits>
   <item id="VX-252" start="-INF" end="+INF"/>
   <item id="VX-253" start="-INF" end="+INF"/>
   <item id="VX-257" start="-INF" end="+INF"/>
   <item id="VX-258" start="-INF" end="+INF"/>
   <item id="VX-259" start="-INF" end="+INF"/>
   <item id="VX-260" start="-INF" end="+INF"/>
   <facet>
      <field>my_category</field>
      <count fieldValue="phone">4</count>
      <count fieldValue="radio">1</count>
      <count fieldValue="tv">1</count>
   </facet>
</ItemListDocument>

For counted facets, it is possible to supply a minCount, thereby excluding any fields that has a count lower than the specified minimum count.

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <facet count="true" minCount="3">
        <field>my_category</field>
   </facet>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <hits>13</hits>
   <item id="VX-248" start="-INF" end="+INF"/>
   <item id="VX-249" start="-INF" end="+INF"/>
   <item id="VX-250" start="-INF" end="+INF"/>
   <item id="VX-251" start="-INF" end="+INF"/>
   <item id="VX-252" start="-INF" end="+INF"/>
   <item id="VX-253" start="-INF" end="+INF"/>
   <item id="VX-254" start="-INF" end="+INF"/>
   <item id="VX-255" start="-INF" end="+INF"/>
   <item id="VX-256" start="-INF" end="+INF"/>
   <item id="VX-257" start="-INF" end="+INF"/>
   <item id="VX-258" start="-INF" end="+INF"/>
   <item id="VX-259" start="-INF" end="+INF"/>
   <item id="VX-260" start="-INF" end="+INF"/>
   <facet>
      <field>my_category</field>
      <count fieldValue="phone">5</count>
      <count fieldValue="radio">3</count>
   </facet>
</ItemListDocument>

By default, at most 100 facet counts will be returned. By using the maxResults attribute, this behaviour can be changed.

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <facet count="true" maxResults="1">
        <field>my_category</field>
   </facet>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <hits>13</hits>
   <item id="VX-248" start="-INF" end="+INF"/>
   <item id="VX-249" start="-INF" end="+INF"/>
   <item id="VX-250" start="-INF" end="+INF"/>
   <item id="VX-251" start="-INF" end="+INF"/>
   <item id="VX-252" start="-INF" end="+INF"/>
   <item id="VX-253" start="-INF" end="+INF"/>
   <item id="VX-254" start="-INF" end="+INF"/>
   <item id="VX-255" start="-INF" end="+INF"/>
   <item id="VX-256" start="-INF" end="+INF"/>
   <item id="VX-257" start="-INF" end="+INF"/>
   <item id="VX-258" start="-INF" end="+INF"/>
   <item id="VX-259" start="-INF" end="+INF"/>
   <item id="VX-260" start="-INF" end="+INF"/>
   <facet>
      <field>my_category</field>
      <count fieldValue="phone">5</count>
   </facet>
</ItemListDocument>

Facet exclusion

One or more search filters can be excluded from a facet using <exclude> tags. Facets can be named to make it possible to distinguish between different facets, for example when using multiple facets on the same field but with different excludes.

The facet exclusion is similar to how one can tag and exclude filters in Solr.

Example

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <filter name = "tvFilter">
     <field>
        <name>my_category</name>
        <value>tv</value>
     </field>
   </filter>
   <filter name = "priceFilter">
      <field>
        <name>my_price</name>
        <range start="200" end="399"/>
      </field>
   </filter>

   <facet count="true">
        <field>my_category</field>
   </facet>

   <facet name="excludeTv" count="true">
        <field>my_category</field>
        <exclude>tvFilter</exclude>
        <!-- <exclude>tvFilter2</exclude> Multiple exclusions -->
   </facet>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
   <item id="VX-253" start="-INF" end="+INF"/>
   <facet>
      <field>my_category</field>
      <count fieldValue="tv">1</count>
      <count fieldValue="phone">0</count>
      <count fieldValue="radio">0</count>
   </facet>
   <facet name="excludeTv">
      <field>my_category</field>
      <count fieldValue="tv">4</count>
      <count fieldValue="phone">1</count>
      <count fieldValue="radio">1</count>
   </facet>
</ItemListDocument>

Spell checking

Search terms can be checked against a dictionary. This enables “Did you mean…” types of searches. The dictionary used is built from the search index and updated periodically.

Example

Consider a user is intending to searching for the “original duration” but misspells both words:

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
    <text>orignal durraton</text>

    <suggestion> <!-- Enables spell checking -->
        <maximumSuggestions>2</maximumSuggestions> <!-- Optional: Specifies the maximum number of suggestions -->
        <accuracy>0.7</accuracy> <!-- Optional: A value between 0.0 (least accurate) and 1.0 (most accurate) of how accurate the spell check should be -->
    </suggestion>
</ItemSearchDocument>

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <hits>0</hits>
  <suggestion>
    <term>orignal</term> <!-- A misspelled search term -->

    <!-- A list of suggestions, with the most likely suggestion being first -->
    <suggestion>original</suggestion>
    <suggestion>ordinal</suggestion>
  </suggestion>
  <suggestion>
    <term>durraton</term>
    <suggestion>duration</suggestion>
  </suggestion>
</ItemListDocument>

Autocompletion

Text can be autocompleted against the search index.

Example

Assuming the user intends to type “original duration”. The user first starts typing “original”:

PUT /search/autocomplete
Content-Type: application/xml

<AutocompleteRequestDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <text>orig</text>
  <maximumSuggestions>3</maximumSuggestions>
</AutocompleteRequestDocument>

<AutocompleteResponseDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <suggestion>original</suggestion>
  <suggestion>origin</suggestion>
  <suggestion>originated</suggestion>
</AutocompleteResponseDocument>

Then the user continues to start typing “duration”:

<AutocompleteRequestDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <text>original dur</text>
  <maximumSuggestions>3</maximumSuggestions>
</AutocompleteRequestDocument>

<AutocompleteResponseDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <suggestion>original duration</suggestion>
</AutocompleteResponseDocument>

Autocomplete on metadata fields

You can also autocomplete on specific metadata fields. In order to make the autocompletion case insensitive, the metadata field should be set as <index>extend</index>.

Example:

A metadata field foo_bar with config:

<MetadataFieldDocument xmlns="http://xml.vidispine.com/schema/vidispine">
         <type>string-exact</type>
         <index>extend</index>
</MetadataFieldDocument>

and this filed contains multiple values: “Animal”, “Sky”, “Animal and Sky”, “animal and sky”

An auto-complete request with user input “animal a”:

PUT /search/autocomplete
Content-Type: application/xml

<AutocompleteRequestDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>foo_bar</field>
  <text>animal a</text>
</AutocompleteRequestDocument>

will give result:

<AutocompleteResponseDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <suggestion>Animal and Sky</suggestion>
  <suggestion>animal and sky</suggestion>
</AutocompleteResponseDocument>

Autocomplete within a search

It is also possible to get autocomplete suggestions while searching. The suggestions will then only match against the search result set.

Example

Let’s say you have a large number of assets, all of which are tagged with one or more tags. A user might want to filter down the result set, so in this example we search for any assets with category “stock_photo”, and at the same time we request autocomplete suggestions for the tag field matching the string “hi”.

PUT /item
Content-Type: application/xml

<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <field>
    <name>my_category</name>
    <value>stock_photo</value>
  </field>
  <autocomplete>
    <field>my_tag</field>
    <text>hi</text>
  </autocomplete>
</ItemSearchDocument>

The result might then look like:

<ItemListDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <hits>5</hits>
  <item id="VX-6934" start="-INF" end="+INF">
    <timespan start="-INF" end="+INF"/>
  </item>
  <item id="VX-3464" start="-INF" end="+INF">
    <timespan start="-INF" end="+INF"/>
  </item>
  <item id="VX-2658" start="-INF" end="+INF">
    <timespan start="-INF" end="+INF"/>
  </item>
  <item id="VX-7234" start="-INF" end="+INF">
    <timespan start="-INF" end="+INF"/>
  </item>
  <item id="VX-3723" start="-INF" end="+INF">
    <timespan start="-INF" end="+INF"/>
  </item>
  <autocomplete>
    <field>my_tag</field>
    <suggestion>highres</suggestion>
    <suggestion>hills</suggestion>
    <suggestion>history</suggestion>
  </autocomplete>
</ItemListDocument>

Search Boost

New in version 4.16.

It is also possible to boost some field values during a search.

To enable search boosting, either add the boost factor in the search document or as the default boost factor in the metadata field definition. Also, _score has to be used as the sorting field.

Example

To find items that have the word phoenix in either title_field or description_field. But the matches in title_field are more important.

PUT /item
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<ItemSearchDocument xmlns="http://xml.vidispine.com/schema/vidispine" version="2">
  <operator operation="OR">
    <field>
      <name>title_field</name>
      <value boost="10">phoenix</value>
    </field>
    <field>
      <name>description_field</name>
      <value>phoenix</value>
    </field>
  </operator>
  <sort>
    <field>_score</field>
    <order>descending</order>
  </sort>
</ItemSearchDocument>

Alternatively, the boost factor can also be set the the metadata field definition.

<MetadataFieldDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <name>title_field</name>
  <type>string</type>
  <boost>10.0</boost>
</MetadataFieldDocument>