Overview

The Camunda BPMN engine is used for executing workflows modelled in BPMN. The VidiFlow workflow designer used for modelling the workflows in BPMN is built on Camunda’s browser-based designer. VidiFlow provides services task to Camunda to execute MAM-specific functionality within a BPMN workflow.

Definition

Since VidiFlow uses a contract-based design for all its service tasks and a workflow is some sort of high-level “service” task, it makes sense to apply the same design for workflows as well. Unfortunately, a “pure” BPMN definition focusses more on the execution paths a workflow shall take rather then which input and output variables it needs or provides. Therefore, a VidiFlow workflow consists of two entities

• the BPMN definition

• the workflow contract

Note that the workflow contract will be created by the workflow designer and will be (internally) required to start a VidiFlow workflow.

Design Time

The workflow designer is accessible via ConfigPortal. There you can add new workflows and edit existing ones. The workflow’s contract and BPMN are not only stored in Camunda for execution but also in ConfigPortal which enables staging of workflows from one environment to another.

The BPMN definition of the changed workflow is deployed to Camunda twice:

• First, for validating the BPMN definition. This Camunda deployment is deleted directly afterwards.

• Next, the workflow is saved to ConfigPortal to obtain the correct workflow version information.

• Last, the workflow with the correct version information is deployed to Camunda for later execution. This deployment is done via the Workflow Deployment Service to have a deployment being consistent with the ConfigPortal Staging scenario describe below in the next section.

ConfigPortal Staging

When a new workflow is not created or modified via the Workflow Designer, the new BPMN definitions or their updates will not end up in Camunda directly. Therefore, ConfigPortal is sending a notification to the Workflow Deployment service via the RabbitMQ Message Bus [ENT 22.1 ARC]. The Workflow Deployment service will fetch all workflow contracts and BPMN definitions from ConfigPortal and applies all changes to the Camunda engine. This will ensure that all BPMN workflows deployed to Camunda are in sync with ConfigPortal.

Start Time

When starting a workflow, the contract is fetched from ConfigPortal. Default values of optional input variables are applied before starting the workflow on Camunda.

Execution Time

The workflow is being executed by the Camunda engine. The VidiFlow Camunda broker uses a set of REST-based long-polling calls (see https://vidispine.atlassian.net/wiki/spaces/IST/pages/2194707442/Camunda+Broker#Task-Partitioning) to wait for notifications on service tasks to be executed from the Camunda engine. The broker sends a message to the message queue corresponding to the contract of each service task.

The broker receives up to 10 Camunda service tasks per call. Once the corresponding messages have been sent to the RabbitMQ message bus, the broker will fetch the next 10 tasks from Camunda via another long-polling REST call to the Camunda engine.

The VidiFlow agent responsible for this queue fetches the message from the bus and executes the task. Upon task completion a task result message is sent by the agent via the message bus back to the broker. Only after this messages has been submitted successfully, the original task message is marked as completed and removed from the queue.

Marking the message as complete only after successful task execution ensures that messages will be resent by the bus in case the agent crashes or is forcefully restarted during task execution.

For each task an entry in the workflow index is created and updated during task. The index allow the workflow monitor to fetch and search this information in a performant way.