The following items on the list encompass breaking changes, features, and fixes that are relevant for the major release.

Release Notes are divided into the following sections:

Release 21.3

Setup and Operations

Component updates

The following central components were updated and will be shipped in the noted version:

Setting helm atomic flag per helm chart

The global helm chart atomic and roleback flags can now by overwritten per helm chart (item 180389).

Important deployment information

Features

New and Adapted Agents and Tasks

All tasks can be selected via VidiFlow’s Workflow Designer. The tasks and their input and output parameters are documented in the Workflow Designer itself after choosing a task.

If not stated differently, all changes are just implemented in the Linux/Docker version only.

List of New Tasks

CopyOriginalImageMetadataToItem

Images often contain metadata stored in IPTC, EXIF or other formats. While importing such a file, VidiCore reads this metadata and stores it at the videoComponent for the image in different metadata fields.

The new task CopyOriginalImageMetadataToItem can be used to copy all of this data to one metadata field called V3_Orignal_Image_Source_Metadata at the item, separated by newline character. This allows searching and showing for this metadata from the VidiCore index. Additionally, the id of the shape the data was read from is stored in the metadata field V3_Orignal_Image_Source_ShapeId.

Currently, only IPTC and EXIF data are supported.

WaitForJobState

When starting jobs in VidiCore it is important to be able to wait for specific job states reached, before continuing a VidiFlow workflow. This is simplified now by the new agent WaitForJobState. Given the Job ID, the state to wait for, a poll interval and a timeout value, this task wait for the job state to be reached. Currently allowed states are STARTED and FINISHED.

Flatten WorkflowObjects Data Structure

This new task is intented to parse a WorkflowObjects structure and flatten recursively all included collections to a list of items or partial items. These are returned in another WorkflowObjects.

Remove Item From All Collections

Items can be part of any number of collection. If users want to explicitely delete an item, this may not be possible as the collections may inherit deletion locks to the item. With this new task an item can be deleted from all collections containing it. Note that the task execution will be affected by access rights.

Additional new tasks

List of Adapted Tasks 

TrimMxfAgent

A new minor task version was implemented to support an additional optional parameter PartialItemPlatformUri. When given, the trimmed file will be attached as a shape to this item and no new item will be created.

FindLastItem

The task FindLastItem brings a new major task version which through an interrupting boundary event ItemNotFound to indicate in case no matching item can be found.

TranscoderS4M via Linux

As the media processing jobs are often resource consuming it may be needed to them to dedicated Kubernetes nodes that provide resources or even use a scalable service like AWS Fargate. In VidiFlow 21.1 the node affinity of the media processing job was set to the node affinity of the executing agent. In VidiFlow 21.3 this can be configured separately from the affinity of the executing agent.

Workflow documentation features

Users want to document their work on workflows in several sections. To support this, the following new features are part of VidiFlow 21.3:

• The workflow documentation field allows one to enter a detailed documentation for any workflow.

• Commit messages support saving a description when saving a workflow to document the changes in the new version.

• Input and output parameters can be documented new via description and examples.

For more information, please view Documenting Workflows in the Designer [ENT 21.3 OG] .

Rule Designer

The UI of the Rule Designer has been changed due to a new version of BPMN.io (DMN 1.3)

• It also comes with a new rule format, that is saved as a new version. UI always shows the new version

• Use '+' to create columns

• Some things have been simplified

• Please refer to the Camunda documentation for more details: https://bpmn.io/blog/posts/2020-dmn-js-8-0-0.html

Workflow Designer

Undo-redo and Copy-paste

The Workflow Designer now supports undo-redo and copy-paste functions. Important rules apply to these. For more information, please view Copy-Paste and Redo/Undo [ENT 21.3 OG] .

Highlight tasks with missing parameters or undefined boundary event names

Tasks with missing parameters or undefined boundary event names are often a cause for errors when saving or running workflows. Any task with missing parameters is highlighted now in red color. Defined boundary events without a a chosen name are highlighted in the same way.

For more information, please view Tasks with Missing Parameters and Undefined Boundary Event Names [ENT 21.3 OG].

Search for Task Names or IDs

Users can utilize Strg-F to search for Task Names, Tasks IDs or other aspects of the workflow.

Save animation

For longer lasting operations a UI animation is shown by WF Designer. The UI is blocked in the meanwhile for user interactions.

Improved error messages

Workflow Designer shows improved error messages now for many situations.

Workflow Monitor

VidiFlow 21.1 introduced a full new version of Workflow Monitor. The following improvements are available in the Workflow Monitor in VidiFlow 21.3.

More search results

Workflow Monitor 21.1 was limited to 1.000 results. Version 21.3 now brings up to 10.000 results in the result list. The performance was increased accordingly to allow for more results.

Filter to exclude workflow names

In order to filter for specific workflow names, you can exclude workflow names from the list by selecting the checkbox “Exclude selected workflows” on the filter dialog. For more information please view https://vidispine.atlassian.net/wiki/spaces/ITSFP/pages/2207781922#Filter-to-exclude-workflow-names.

Changes on time values

The format used for the column and filter “start time” adapts fully to the browsers language settings.

Task properties in the details view now also show seconds for the start and end time.

Filter for current user

It is now possible to filter for workflows executed by the currently logged in user in Workflow Monitor by pressing the button “current user” in the user filter page.

Support of freetext query parameter

The query parameter “?freeText=…” is now supported again.

Configure shown columns

Columns can be shown or hidden in Workflow Monitor. For more information, please view https://vidispine.atlassian.net/wiki/spaces/ITSFP/pages/2207781922#Configure-shown-columns.

Expand / collapse selected workflows

One can now select one or multiple workflows and expand or collapse all sub-workflows.

Prioritization of Workflows

VidiFlow 21.3 comes with new options for prioritizing workflows and tasks.

Prioritization offers one the possibility to set a workflow instance priority when starting a workflow or to change it on running workflows. This will not affect the priority of the workflows itself in Camunda, but it will use the current priority of a workflow to automatically adopt the priority for all tasks in the Rabbit MQ message bus. For more information please view https://vidispine.atlassian.net/wiki/spaces/ITSFP/pages/2207781922#Workflow-Prioritization .

Workflow Designer

The priority of a workflow instance can be accessed in the Workflow Designer via the variable WorkflowInstancePriority. This variable can be changed to another value from inside the workflow execution of from outside via the Workflow Monitor (see https://vidispine.atlassian.net/wiki/spaces/CKB/pages/2207781687#Workflow-Monitor.1).

For more information, please view Prioritizing Workflows in the Designer [ENT 21.3 OG].

Migration

All Rabbit MQ agent queues need to be migrated in order to support priority queues. There’s a migration script that needs to be manually invoke with the agents shut down in order to ensure all the messages are migrated correctly. If queues still have consumers when the script runs, those queues are skip, mentioning the found consumers. The script can be called subsequently.

Please refer to the setup guide for details of the migration script.

Limitations

• Priority can only be set for Linux based agents, all service fabric tasks have the same priority.

• Changes in fetch-and-lock mechanism in the Camunda Broker will lead to larger log files.

• The feature can lead to higher load in Camunda Broker in different situations, but should smoothen over time.

Cron Workflows

Cron workflows allow one to trigger workflows recurringly.

For the configuration of Cron workflows, please refer to Functional Use Case Category Cron [ENT 21.3 OG].

Cron Workflows are executed by the new service CronWorkflowScheduler. They can be monitored in the new Hangfire Dashboard Cron Workflow Scheduler Service Dashboard.

VidiCore Libraries

Libraries are a way to collect different items with the ability to have the library content dynamically updated based on a query (see https://apidoc.vidispine.com/latest/item/library.html). For more information please view Global Configuration Libraries [ENT 21.3 OG] .

In Workflow Designer a library’s items can be accessed by the new task “Get Item IDs Of Library“, using the libraries configured in ConfigPortal.

Access to multiple metadata fields in workflows

Before release 21.3 it was possible to read either one metadata field or alternatively all metadata fields from VidiCore objects. Now the following additional options are available:

Task Get Object Metadata Values

Reads metadata values for fields provided in a list for a given VidiCore object.

Task Get Key-Value Metadata Values

Reads key-value metadata values for a given VidiCore object.

Task Get Metadata Values using Content Path

Gets metadata values for a VidiCore item using the content path syntax. Support object and technical metadata.

Performance and stabilization improvements

VidiFlow has experienced various performance and stabilization improvements in key components.

Changed retry strategy for REST calls

The retry strategy for REST calls was changed in the following aspects:

• For most services, a retry will be done automatically independently of the returned error code. This is because many services do not provide a perfect HTTP error code handling. For Vidispine’s own services a different strategy is implemented which does not perform any retries for HTTP 4xx codes (with exception of 408, 429).

• A de-escalating retry strategy is now implemented which increases the timeout between consecutive retries each time. General retry time is about 4.5 minutes as before, but with much fewer calls.

Task contracts show Rabbit queue

All VidiFlow tasks inform now automatically about the used message bug queue name. This can be seen in the Agent’s Contract overview:

Agent logging supports analysis of specific task executions

All log entries from agents that belong to one task have now a unique ID (“external task ID”. That allows filtering for all messages from one task in Kibana to see them in chronological sequences.

DotNet Diagnostic Tools bundled

All VidiFlow agents and services container images will now contain the following Microsoft .NET diagnostic tools for performance troubleshooting and diagnosing. The tools are available in the folder /tools.

• dotnet-trace

• dotnet-counters

• dotnet-dump

Additional improvements

ConfigPortal Improvements

The following improvements were done in ConfigPortal:

Support of MaxConcurrentConnection for CP API

CP API supports now the Kestrel MaxConcurrentConnection (item 188179) to prevent thread starvation issue that caused the application to became unresponsive with too many concurrent requests. Values can be changed in CP API appsettings.json. The default value is set to 0 (no limitation).

Configure VidiCore thumbnail options

ConfigPortal offer now (item 163924) the ability to configure the different option for saving thumbnails (see https://apidoc.vidispine.com//latest/system/thumbnails.html#how-thumbnails-are-saved-on-disk).

Additional improvements for ConfigPortal

Fixes