Document toolboxDocument toolbox

Vidispine

Multimedia Content Delivery with VidiStream [ENT 21.1 ARC]

Owned by Andreas Schmidt
May 31, 2021
6 min read

Introduction

The VidiStream application (formerly named “Streaming Server”) can deliver DASH, HLS and RMFS streams to clients. It currently comes in form of different installation packages:

  • Worker-based VPMS Module (deprecated)

  • Windows Service Fabric Service (deprecated)

  • Standalone Windows Service (deprecated)

  • Linux Docker Container (pre-packaged in Kubernetes deployment)

Except for the Linux Docker Container all other deployment methods are obsolete and will not be developed further. As of now the containerised Linux VidiStream offers all features of the other three variants and will be the only one for which new features will be developed.

Operational Modes

In general three distinguishable modes of operation are available with VidiStream which will be highlighted next. While preview/proxy streaming is technically relatively simple and rendering much more complex, both variants produce DASH conformant output that is exposed at the /dash endpoint. The third mode of operation here encompasses all features that are not related to DASH streaming.

1. Repackaging of single files for DASH streaming (preview/proxy streaming).

In this mode VidiStream serves single files as a DASH stream. Normally a server capable of DASH streaming relies on pre-segmented files with a fixed package duration. These pre-segmented smaller files are often available in various qualities enabling on-the-fly switching of the delivery quality in case of available bandwidth changes. This process of adjusting the visual quality on-the-fly based on network throughput measurements is sometimes called adaptive streaming.

A USP of VidStream is its capability of creating and packaging DASH segments on-the-fly from single files as opposed to the dependency on pre-segmented small DASH file segments which streaming servers typically require. This makes it very easy to integrate the Streaming Server into environments where no specially pre-segmented DASH files need to be available, any single file MP4 or MXF is directly streamable. Hence the customer can benefit from convenient file handling together with DASH conformant streaming without having to additionally segmentize all proxy files for streaming.

In case the streamed files are streamed unmodified “as are” with a lower resolution quality, one Streaming Server instance is capable of delivering streams to 100-200 clients simultaneously making it a solution low on required resources. A recent evaluation of preview streaming via VidiStream on a production-grade system showed that the optimum utilization was at ~160 parallel users per instance, with more than 200 users deteriorating effects set in so that the upper limit per instance is estimated to be around 200 parallel users. Tests were conducted with three standalone Windows VidiStream instances having 12 logical cores each.

While it is possible to do on-the-fly transcoding to different bitrates with the MediaFramework Streaming Server as well (mirroring available segments with different QOS) it puts a high amount of strain on the server and hence reduces the number of clients which can be served by a single instance by factor 10. Choosing to allow on-the-fly transcoding to lower qualities can however be an option in environments where bandwidth is not guaranteed to be sufficient to serve the files inherent quality.

2. Live streaming of complex timelines using DASH (rendering)

Besides segmenting files on-the-fly and delivering the repackaged contents unchanged, VidiStream can also perform advanced tasks involving processing the source material. In this mode a more or less complex timeline can be submitted and VidiStream renders the timeline on-the-fly and again creates DASH segments from the dynamically processed input. This can be used to serve relatively simpler usecases such as:

  • deliver a sequence of files “back-to-back”, meaning without any noticeable interruption of playback when transitioning between two files.

  • deliver only particular audio tracks to the client

However the main usecase for the rendering mode is to serve the proxy timeline preview of the VidiEditor. With the rough-cut editing application VidiEditor the user can create a timeline with multiple video and audio tracks and even apply audio and video effects such as blending to the timeline. VidiStream combines all the layers present in the timeline and presents DASH segments that reflect the timeline in terms of modeled visual and audio content .

In proxy render mode, only a significantly lower number of clients can be served by one VidiStream instance, the exact number depending on the available underlying resources of the host machine or the Docker container respectively.

Since the development of this feature is currently ongoing, as of now only a subset of the planned functionality is available now but the amount of available effects is constantly increasing.

Note that the same timeline can technically also be rendered in typical Hires output formats such as XDCAM HD or AVCintra. This is however not intended for live preview with VidiStream and is normally offloaded to a backend task for the Transcoder. The transcoder understands the same timeline structure as the StreamingServer (due to the underlying shared MediaFramework architecture) and can render the timeline in high resolution quality.

3. Non-DASH based additional features

While it can be combined with any DASH compatible client, VidiStream works best when the client integrates the MediaPlayer package that is produced by Vidispine as well. This Javascript based client player library is created in collaboration with the VideoEngine team and supports additional features VidiStream offers outside the scope of MPEG-DASH streaming. Clients implementing this library benefit from availability of the whole feature set of VidiStream.

Additional non-DASH compliant features that the Streaming Server offers are:

TCP proprietary streaming (EditMate)

In the context of EditMate it is possible to import media into Premiere projects by streaming them. The Premiere Importer component is in this use-case responsible to retrieve the media used in the timeline from VidiStream and convert it to the expected technical format Adobe Premiere needs to work with the media files. In this sense the Premiere Importer acts as a client to VidiStream.

To meet the high performance editing demands when delivering streams to an EditMate client (integrated into Adobe Premiere) there exists a performance optimized proprietary streaming protocol for this. It is based on TCP communication to avoid the HTTP header overhead, delivering data fast and efficiently to enable near to real-time jumping, scrubbing and playback with varying speed.

HTTP streaming based on the same proprietary communication protocol is also offered for less performance critical scenarios. This has the advantage of better integration into client or cloud networks but comes at the cost of reduced performance.

Trick Play

With the trick play mode VidiStream offers an additional API and endpoint at /trickplay. At this endpoint clients can request individual video and audio frames and get them delivered in a very performant fashion avoiding the overhead of the DASH streaming protocol for this special usecase.

Trick play for example can be activated on mouse-down when scrubbing through a timeline in VidiEditor or on hovering over a preview image of a video in a gallery view in MediaPortal. In these cases the trick play delivery mode very fastly delivers I-Frames based on the scrub or hover position. In consequence, browsing through the video contents at this level feels much more fluent than would be possible with DASH where only larger video segments are packageable. While trick play can be used to request any arbitrary video or audio frame, in these use cases it is sufficient to deliver only I-Frames because due to the limited amount of pixels only a coarse position can be navigated to anway. Accessing arbitrary frames is put to use in the dynamic hard cut preview in VidiEditor where a hard cut is placed between two video clips and trickplay provides a live preview of the end- and startframe while the user shifts the cut point back and forth.

No prerequisites need to be matched to use trick mode in clients because the Streaming Server can deliver the frames directly from the streamed source files.

Index Files

In combination with the Vidispine transcoder backend module it is possible to pre-create indexing information for files which are to be streamed at some point by VidiStream. Given the availability of these index files, file access time is vastly improved because byte positions are easily obtained from the index when browsing through the proxy file in comparison with searching the video files structure every time.This information has solely technical meaning to the streaming server and is stored on disk next to the files to stream.

Volume Curve Data

In combination with the Vidispine transcoder backend module audio volume curves can be generated in the backend and delivered to the client by the streaming server. In contrast to the index files however the volume curve data is not stored on disk side-by-side with the file to stream but is stored in the VidiCore backend. Since it visualizable information this has the advantage that it can be accessed for visualisation purposes by any client.

Metrics

The Streaming Server is capable of exposing metrics in the Prometheus format, by default they are accessible at the streaming/metrics endpoint.