Vidispine
BPMN Workflows [ENT 21.3 ARC]
- Andreas Schmidt
Overview
The Camunda BPMN engine is used for executing workflows modelled in BPMN. The VidiFlow workflow designer used for modelling the workflows in BPMN is built on Camunda’s browser-based designer. VidiFlow provides services task to Camunda to execute MAM-specific functionality within a BPMN workflow.
Design Time
The workflow designer is accessible via ConfigPortal. There you can add new workflows and edit existing ones. The workflow’s contract and BPMN are not only stored in Camunda for execution but also in ConfigPortal which enables staging of workflows from one environment to another.
The BPMN definition of the changed workflow is deployed to Camunda twice:
First, for validating the BPMN definition. This Camunda deployment is deleted directly afterwards.
Next, the workflow is saved to ConfigPortal to obtain the correct workflow version information.
Last, the workflow with the correct version information is deployed to Camunda for later execution. This deployment is done via the Workflow Deployment Service to have a deployment being consistent with the ConfigPortal Staging scenario describe below in the next section.
In VidiFlow prior to 21.1 the BPMN was only saved to the Camunda engine for validation and then immediately deleted. The actual deployment to Camunda happened when starting the workflow on Camunda.
ConfigPortal Staging
When a new workflow is not created or modified via the Workflow Designer, the new BPMN definitions or their updates will not end up in Camunda directly. Therefore, ConfigPortal is sending a notification to the Workflow Deployment service via the RabbitMQ https://vidispine.atlassian.net/wiki/spaces/CKB/pages/2207985116. The Workflow Deployment service will fetch all workflow contracts and BPMN definitions from ConfigPortal and applies all changes to the Camunda engine. This will ensure that all BPMN workflows deployed to Camunda are in sync with ConfigPortal.
Start Time
When starting a workflow, the contract is fetched from ConfigPortal. Default values of optional input variables are applied before starting the workflow on Camunda.
In VidiFlow prior to 21.1 the BPMN was deployed to Camunda when starting the workflow on Camunda.
Execution Time
The workflow is being executed by the Camunda engine. The VidiFlow Camunda broker uses a REST-based long-polling call to wait for notifications on service tasks to be executed from the Camunda engine. The broker sends a message to the message queue corresponding to the contract of each services task.
The broker receives up to 10 Camunda service tasks in parallel. Once the corresponding messages have been sent to the RabbitMQ message bus, the broker will fetch the next 10 tasks from Camunda via another long-polling REST call to the Camunda engine.
The VidiFlow agent responsible for this queue fetches the message from the bus and executes the task. Upon task completion a task result message is sent by the agent via the message bus back to the broker. Only after this messages has been submitted successfully, the original task message is marked as completed and removed from the queue.
Marking the message as complete only after successful task execution ensures that messages will be resent by the bus in case the agent crashes or is forcefully restarted during task execution.
For each task an entry in the workflow index is created and updated during task. The index allow the workflow monitor to fetch and search this information in a performant way.