Curator Live File Transcode Mechanics

Updated on December 9th, 2022

Overview

Curator’s high-performance proxy transcode engine (XCode) can read live growing files (of an appropriate format, e.g., MXF wrapped) and create a viewable proxy in Curator while the high-resolution file is still ingesting or being written to storage. This provides immediate access to a community of users, including remote workers.

Note that at the time of writing, this isn't compatible for object storage including S3.

Essential Mechanics Explained

Syntax: file://<filename>

Parameters:

Filemode = <value>

Possible values are normal and growing (the default is normal). This is only relevant for input URLs and enables special behaviour in the case where we are reading a file which is still being written. In normal mode, seeing physical end of file causes the input connection to be closed; in growing mode, the file is polled for a maximum of timeout seconds, waiting for more data to be written. The input connection is closed when a timeout period completes without any further changes to the file. See also the Headroom parameter.

Timeout = <value>

The value here will be a decimal number giving the growing timeout in seconds (default 10).  Only relevant for filemode=growing. Growing is FSTAT total size in bytes changing within the timeout period. Once the timeout triggers the splitter (IPV file reader) drains the remaining data until the EoF is reached, or bad data which under either trigger the proxy is closed and read terminated.

Headroom = <value>

This will be a decimal number, optionally followed by k or m, giving the file's safety headroom in bytes, kilobytes or megabytes (default 0). A non-zero headroom value means that the file reader will lag the physical end of file by the amount given, only returning (draining) the headroom area at the point that timeout expires. This function is provided for the case where the file creator is known to write periodic "footer" blocks, then seeking backwards before overwriting them with more valid data. The headroom value should be set so that is at least as high as the biggest footer block. Only relevant for filemode=growing.

Was this article helpful?