Vidispine
System configuration [VC 21.3 GEN]
Indexing configuration
The indexing configuration contains the parameters that relate to search and indexing.
Where Vidispine can reach Solr or ZooKeeper.
When to commit or soft commit.
The Solr query request parameters.
The default field settings.
This configuration replaces the configuration properties listed under Search and indexing.
Example
Full text indexing could be disabled for all fields, unless explicitly specified for a field, using:
<IndexingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<solrPath>http://localhost:8088/solr</solrPath>
<fieldDefault>
<name>*</name>
<fullText>false</fullText>
</fieldDefault>
</IndexingConfigurationDocument>Metrics configuration
See StatsD on how to configure how metrics are sent to StatsD. The configuration resource is described at Metrics settings.
FTP pool configuration
By default jobs that need to read or write to an FTP server will establish, use and end separate connections to the server. By configuring a FTP connection pool you can change so that the jobs share and reuse FTP connections. This can reduce the time it takes to transfer files over high latency connections.
For example, to create a connection pool with the default settings:
PUT /configuration/ftp-pool
Content-Type: application/xml
<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<pool/>
<FtpPoolConfigurationDocument/>If no pool is specified then pooling will be disabled. Unless overridden, the pool will be unbounded, and connections will expire after 1 minute. That is, the above configuration is identical to:
PUT /configuration/ftp-pool
Content-Type: application/xml
<FtpPoolConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<pool>
<minSize>0</minSize>
<maxSize>-1</maxSize>
<evictionInterval>30000</evictionInterval>
<minIdleTime>60000</minIdleTime>
</pool>
</FtpPoolConfigurationDocument>The FTP pool configuration resource is described at FTP pool configuration.
Database purging
Vidispine supports mechanisms for purging old information in database tables. Especially four tables can grow quite large without purging enabled.
Updated method for purging configuration
The preferred way of updating the database purging configuration is by create a DatabasePurgingConfigurationDocument, see Database purging configuration.
Change-log table
The change-log table holds information about data that should be sent to other sites. If multi-site is disabled (disableSiteCrunching), this table grows forever.
To enable purging of the table, two configuration properties are used: changeLogPurgingTime and changeLogForcePurgingTime. The first one controls deletion of entries that have been processed, the other one controls deletion of entries regardless of state.
Sensible values are 43200 and 86400, corresponding to one and two months, respectively.
Configuration can also be controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<changeLog>
<age>43200</age>
<forceAge>86400</forceAge>
</changeLog>
</DatabasePurgingConfigurationDocument>Audit trail table
The audit trail table contains all API requests, see Audit trails.
To enable purging of the table, two configuration properties are used: auditTrailPurgingTime and auditTrailPurgingDirectory. Both must be set in order for purging to take place.
When purging is enabled, entries that are older than auditTrailPurgingTime minutes will be removed and put in a file inside the auditTrailPurgingDirectory folder.
A sensible value is 43200 or higher, corresponding to one month.
Configuration can also be controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<auditTrail>
<age>43200</age>
<uri>s3://key:secret@archival-bucket/requests/</uri>
</auditTrail>
</DatabasePurgingConfigurationDocument>By setting the compress element in the configuration document, entries will be stored using gzip compression. The default value is true.
The entries are stored in batches. The number of entries per batch is controlled by the batch element in the configuration document. The default value is 10000 entries. If there are less than 10000 entries that fulfill the time criteria, the purging of the audit trail table will pause.
By setting the body element in the configuration document, entries will also include the request bodies and response codes. The default value is false.
The directory can be a full URI, such as a S3 or FTP location.
Job table
To enable purging of the table, two configuration properties are used: jobPurgingTime and jobPurgingDirectory. Both must be set in order for purging to take place.
When purging is enabled, entries that are older than jobPurgingTime minutes will be removed and put in a file inside the jobPurgingDirectory folder.
Configuration can also be controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<job>
<age>43200</age>
<uri>s3://key:secret@archival-bucket/job/</uri>
</job>
</DatabasePurgingConfigurationDocument>By setting the compress element in the configuration document, entries will be stored using gzip compression. The default value is true.
The directory can be a full URI, such as a S3 or FTP location.
Transfer log table
Configuration is controlled via DatabasePurgingConfigurationDocument. For example:
<DatabasePurgingConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<transferLog>
<age>1440</age>
<uri>s3://key:secret@archival-bucket/transfers/</uri>
</transferLog>
</DatabasePurgingConfigurationDocument>Both the age and uri element has to be set. Entries than represents a finished transfer older than age minutes are exported to the destination.
The forceAge element controls exports of non-finished transfers - transfers that have disappeared for some reason. The default value of forceAge is the value of the age element.
By setting the compress element in the configuration document, entries will be stored using gzip compression. The default value is true.
The entries are stored in batches. The number of entries per batch is controlled by the batch element in the configuration document. The default value is 10000 entries. If there are less than 10000 entries that fulfill the time criteria, the purging of the transfer log table will pause.
Default job priority
New in version 5.2.1.
The default job priorities are configurable by type. For example, the following configuration document will make IMPORT jobs default to MEDIUM priority and EXPORT jobs default to HIGH priority:
<JobPriorityConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<job type="IMPORT">MEDIUM</job>
<job type="EXPORT">HIGH</job>
</JobPriorityConfigurationDocument>If no default priority has been specified for a given job type, the job will default to MEDIUM priority.
CORS configuration
New in version 4.15.
Vidispine can be configured to emit Cross-Origin Resource Sharing (CORS) headers.
The CORS configuration is set using the CORS configuration resource. The configuration document consists of a number of entries, each of them are checked for CORS evaluation. If an entry condition matches the incoming request, the CORS headers set in the entry are outputted, and no other entries are matched. For example:
PUT /configuration/cors<CORSConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<entry>
<request/>
<response>
<allowOrigin>*</allowOrigin>
</response>
</entry>
</CORSConfigurationDocument>The conditions in each entry can match HTTP method, the request path, the CORS origin, or any other header. For example:
<CORSConfigurationDocument xmlns="http://xml.vidispine.com/schema/vidispine">
<entry>
<request>
<pathRegex>API.*/item/.*</pathRegex>
<headerRegex>
<key>connection</key>
<value>.*aliv.*</value>
</headerRegex>
</request>
<response>
<allowOrigin>*</allowOrigin>
<allowMaxAge>86400</allowMaxAge>
</response>
</entry>
</CORSConfigurationDocument>Semantics are as follows.
Multiple methods can be set per entry. If one entry method matches the request method, the entry matches as far as for method. If no methods are set on the entry, the entry matches.
Origins can be specified both exact or via regular expressions. If one entry origin matches the request origin, the entry matches as far as for origin. If no origins are set on the entry, the entry matches.
Request paths can be specified via regular expressions. If one entry path matches the request path, the entry matches as far as for path. If no paths are set on the entry, the entry matches.
Request headers can be specified via regular expressions. All specified headers for the entry must match the request. If the HTTP request does not contain a value for a specified header, the entry does not match. If the HTTP request contains multiple values for a specified header, it is sufficient that only one value matched the entry header condition.
When the entry matches, the entry may contain which origin that is allowed, the max age for the access, which methods that are allowed, which HTTP headers. The entry can also contain other, arbitrary, headers. If methods are not specified, the methods are automatically deduced from the API methods.
Configuration properties
Configuration properties are used in Vidispine to control system-wide parameters.
Since 5.3, configuration properties can be used to control some system property settings, see below.
See also
See Create/modify configuration properties for more information about how to configure properties.
Some configuration properties are cached locally, and it may take up to 5 minutes until the new value is observed. These configuration properties are marked with “Cached: yes” below.
General
apiUri
URI to Application Server. Used by transcoder(s), so need to be a proper host if transcoder(s) run on another machine.
Mandatory: | Yes |
Example: |
|
apiNoauthUri
URI to Application Server, to use to access the no-auth API. Used by transcoder(s), so need to be a proper host if transcoder(s) run on another machine.
Example: |
|
clusterName
Optional alphanumerical identifier for the Vidispine installation/cluster. Must be set (to a unique identifier) if multiple Vidispine installations are to share a common set of transcoders.
Example: |
|
disableSiteCrunching
Do not build site replication packages. Recommended to be set to true for systems not running site replication.
Default: |
|
validatexml
Enable schema validation of the incoming and outgoing xml document.
Default: |
|
slaveLicenseProxy
Use a proxy for Connection to Vidispine Online Licensing System. Format is
http://IP:port/orsocks://IP:port/
Proxy authentication is not supported.
Default: | none |
defaultTranscoder
Default transcoder resource to use. Valid values are:
vidinet- To use the first available transcoder from Vidinet.A resource id - To use the transcoder with that id.
Example: |
|
Default: | none |
Search and indexing
Deprecated since version 4.2: The Solr and ZooKeeper properties are deprecated. Use Indexing configuration instead.
solrPath
URI ( not path! ) to Solr.
Mandatory: | Yes (No for SolrCloud) |
Example: |
|
elasticsearchPath
URI to Elasticsearch’s RESTful interface.
Mandatory: | Yes (if using Elasticsearch). Optional if |
Example: |
|
zkHost
For SolrCloud: A comma separated list of host:port pairs to the servers in the ZooKeeper ensemble.
Mandatory: | No (Yes for SolrCloud) |
Example: |
|
solrCollection
For SolrCloud: The collection in Solr to be used by Vidispine.
Mandatory: | No (Yes for SolrCloud) |
Example: |
|
solrQueryTimeout
The request timeout in milliseconds to use when querying Solr.
Default: |
|
solrGroupLimit
The maximum number of timespans to return per item or collection.
Mandatory: | No |
Example: |
|
solrPingAttempts
The number of times to ping a Solr node before aborting an active request.
Default: |
|
solrPingTimeout
The request timeout in milliseconds to use when checking if a Solr node. is alive
Default: |
|
solrCommitInterval
The interval (in milliseconds) of Vidispine sending hard commit to Solr.
Default: |
|
solrSoftCommitInterval
The interval (in milliseconds) of Vidispine sending soft commit to Solr.
Default: |
|
solrAutoSoftCommit
If Vidispine should send soft commit to Solr automatically.
Default: |
|
solrUpdateQueueSize
Number of documents Vidispine will send in batch to Solr.
Default: |
|
solrDeleteMergeSize
The number of delete queries to merge into one before sending to Solr.
Changed in version 5.6: The default was changed from -1 (do not merge any delete queries) to 100.
Default: |
|
elasticsearchWorkerCount
Number of worker threads to use when sending documents to Elasticsearch.
Default: |
|
Since: | 5.6 |
elasticsearchBulkBuffer
The document buffer size (in bytes) in the Elasticsearch worker thread.
Default: |
|
Since: | 5.6 |
indexFieldGroups
If metadata field groups should be indexed in Solr. Setting this to false can reduce the load and the size of the index if items have a large number of groups in the metadata, but will mean that no results will be available when searching for field groups.
Default: |
|
indexCollectionItemOrder
If the order of an item in a collection should be indexed in Solr. Settings this to false can greatly reduce the number of fields created in Solr and improve performance on systems with a lot of collections. This affects collection item retrieval. See also Retrieve the child-collections of a collection.
Recommended to be set to false for applications not relying on that feature. Requires a clean Solr index and a full re-index to take effect.
Default: |
|
indexTimespans
If time coded metadata should be indexed in Solr. Setting this to false can reduce the load and the size of the index if items/collections have a large number of timespans in the metadata, but will mean that no time coded metadata can be found.
Default: |
|
maxSearchResults
Maximum number of search results allowed to be returned (see Search items).
Default: |
|
legacyTransientFieldTypes
This setting controls the datatype of the transient metadata fields. If true then all transient fields will be of type string. If false the *_size and *_count fields will be of type integer, and the rest will have type string.
Default: |
|
skipLibraryIndexUpdates
If set to true, the auto-refreshing libraries won’t be updated after item metadata changes.
Default: |
|
indexDocumentMetadata
If document metadata should be indexed. Setting this to false can reduce the load and size of the index. Document metadata is not searchable if this property is is set to false.
Default: |
|
Since: | 5.0 |
indexQueueLimit
This setting throttles indexing to a maximum number of messages in the ActiveMQ queue. In a system where indexing (Solr/Elastic) is slower than the database, this setting avoids that too many messages are stored on the queue. Set this value to 0 to specify no limit. See also activeMQAdminUrl.
Default: |
|
Since: | 5.7.1 |
Cached: |
Metadata
disableMetadataSchema
If a metadata schema has been defined (see Metadata schema), allows metadata that does not comply to the schema.
Default: |
|
useAbsoluteSccTimeCode
If set to true Vidicore will not try to adjust the time codes in the SCC sidecar file to the usual relative (zero-based) timecodes for each item when imported.
Default: |
|
Since: | 5.6 |
Bulky Metadata
bulkyMetadataMigrationThreads
Number of threads to use for bulky metadata migration.
Default: |
|
Since: | 5.4.5 |
Authentication
passwordHashAlgorithm
The hash algorithm used to hash all user passwords. Note that changing this will make it impossible to authenticate with any existing user.
Default: |