Automatic import [VC 21.3 GEN]

A storage can be configured to automatically import new files/image sequences that are detected. Auto-import rules define what transcodes that should be performed as well as what metadata to be used if none can be found. Metadata can automatically be found if it shares the same filename and has the extension .xml, for example video.avi and video.xml.

Auto-import rules can also use Import settings to set up access control lists by setting the optional settingsId element.

Import using a specific transcoder resource

To use a specific transcoder resource during auto import, specify its resource id in the auto import rule using the resourceId element:

PUT /storage/VX-2/auto-import
Content-Type: application/xml

<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <tag>myflvtag</tag>
  <resourceId>VX-1</resourceId>
</AutoImportRuleDocument>

You may also specify which transcoder resource to use by setting the defaultTranscoder configuration property.

Setting a user for jobs started as a result of an auto import rule

The default behavior is that jobs started from an auto import rule will not have a user set. This can be changed by setting the user element in the rule:

PUT /storage/VX-2/auto-import
Content-Type: application/xml

<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <tag>myflvtag</tag>
  <user>angela@company.com</user>
</AutoImportRuleDocument>

Importing with a metadata file of an external format

Vidispine also supports auto imports with a metadata XML file that is of a different format than the native Vidispine MetadataDocument. This is achieved by associating a Metadata projections (XSLT transformation) with the auto import rule. First, create the projection, then set the auto import rule:

PUT /storage/VX-2/auto-import
Content-Type: application/xml

<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <tag>myflvtag</tag>
  <projection>myProjection</projection>
</AutoImportRuleDocument>

Where the projection element contains a Projection id.

Any auto imports from this storage will then first transform the supplied XML file using the specified projection.

Disable automatic import rules

Auto-import rules can be disabled by setting the enabled field to false, or by using the disable rule request. When a rule is disabled then no new auto-import jobs will be created for new files discovered on the storage.

Jobs will be created once the rule is enabled again. By default rules are enabled.

Example

If the admin only wants the user to be able to import during specific hours a program like cron could be used:

0 0 * * * curl -X PUT -uadmin:admin http://localhost:8080/API/storage/VX-1/auto-import/enable
0 5 * * * curl -X PUT -uadmin:admin http://localhost:8080/API/storage/VX-1/auto-import/disable

Sidecar auto import

If auto-import is enabled, sidecar files are by default identified by file extension and imported as metadata to files with matching filenames. See Importing sidecar files for the supported sidecar formats which are automatically identified. The AutoImportRuleDocument contains two fields determining how sidecar are handled by auto-import:

ignoreSidecarImport

False by default. If set to true, files with a sidecar file extension are not imported in any way.

disabledSidecarExtensions

Extensions can be specified here to be excluded from being treated as sidecar files, and may instead be imported as new items.

Since:

4.16

If there are several files in a storage with the same file name (prefix) with different extensions, only one will be imported as an item and others will be either imported as sidecars or ignored. This can be changed by setting the system-wide configuration property groupImportableFiles to false. Multiple files with the same prefix can then be auto-imported as individual items.

Multiple sidecar files can be imported to the same item. If groupImportableFiles is false, one sidecar file may be auto-imported multiple times as metadata for different items with the same file prefix.

Title as metadata

The AutoImportRuleDocument contains a field fileNameAsTitle. Setting this property to true means that the “title” fields of all single files imported form this storage will be set to their file names.

Applying file name filters to auto import rules

There are two kinds of filename filters that can be applied to auto import rules:

Exclusion filters

Used to exclude files from being auto imported. This can be useful when the OS creates files automatically, e.g. Thumbs.db on Windows or .DS_Store files on Mac OS. Note that the expression must match the entire path, not only a part of the path.

Shape tag filters

These can be used to transcode the imported file using a specific shape tag when a file name follows a certain pattern. You might want files ending in .tiff to be transcoded using the tag lowimage for example.

The filters are specified in the XML document you use to create/update the auto import rule.

Example

<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
  <metadata>
    <timespan start="-INF" end="+INF">
      <field>
        <name>title</name>
        <value>This is an auto-imported item.</value>
      </field>
    </timespan>
  </metadata>
  <tag>generictag</tag>
  <excludeFilter>
    <pattern>.*\.DS_Store</pattern>
  <excludeFilter>
  <shapeTagFilter>
    <pattern>.*\.tiff</pattern>
    <tag>lowimage</tag>
  </shapeTagFilter>
  <shapeTagFilter>
    <pattern>.*\.mxf</pattern>
    <tag>lowvideo</tag>
  </shapeTagFilter>
</AutoImportRuleDocument>

This rule will exclude any file ending with .DS_Store. Any files ending with .tiff will be imported with the shape tag lowimage, and any files ending in .mxf will be imported with the shape tag lowvideo. All files will be imported with the shape tag generictag.

Auto import of image sequences

Image sequences can be auto detected and imported if their file names match the predefined regex in AutoImportRuleDocument. The elements in the document are:

fileSequence

Defines the file name pattern, and it is mandatory.

sequenceMetadata

Defines the metadata file name pattern.

idGroup

The matching group in the regex should be used as the id of the file sequence.

numGroup

The matching group in the regex that should represent the position of a file in a sequence.

timeout

A sequence is considered as completed after a certain timeout (in seconds). The default timeout is 60 seconds.

Example:

<AutoImportRuleDocument xmlns="http://xml.vidispine.com/schema/vidispine">
        <tag>mp4</tag>
        <metadata>
                <timespan end="+INF" start="-INF">
                        <field>
                                <name>title</name>
                                <value>auto-imported item.</value>
                        </field>
                </timespan>
        </metadata>
        <sequenceDefinition>
                <sequenceMetadata>
                        <regex>(.*)-metadata.xml</regex>
                        <idGroup>1</idGroup>
                </sequenceMetadata>

                <fileSequence>
                        <regex>(.*)-([0-9]+).(dpx|tga|png|jpg)</regex>
                        <idGroup>1</idGroup>
                        <numGroup>2</numGroup>
                        <timeout>10</timeout>
                        <!-- seconds-->
                </fileSequence>
        </sequenceDefinition>
</AutoImportRuleDocument>

Given a storage with the above import rule, with the files:

foo-metadata.xml
foo-001.dpx
foo-002.dpx
foo-002.dpx

Then these would be recognized as a sequence foo with foo-metadata.xml as the metadata.