The AWS S3 bucket acts as a virtual pipeline allowing your proxies to flow securely through the cloud-based (AWS) Curator deployment without being exposed to end applications or the browser itself, using authentication tokens to ensure the streaming process is secure when requests are sent by the browser. The web application acquires the specific token, allowing for the proxy to be streamed and cached by the browser for playback efficiency.

Minimum Curator Software version requirements:

Curator Release 3.2, with the addition of:

• Curator Gateway 1.3.1.67
• Curator Clip Link 3.1.1.18
• Curator Clip Select 2.13.1.21
• Curator for Adobe 2.5.1.28 (Win) 2.5.14 (Mac) – uses configuration file version 3.2.0.4
• Curator Logger 2.2.1.41

or Curator Release 3.2.1 (Update 1) which includes the above or newer applications.

Installation Steps

The following installation steps can be followed for Curator 3.2 or Curator 3.2 Update 1 installations. These steps include Mediastores and Metadata information for the S3 proxies, as shown below.

Mediastores

The HI-RES Mediastore should be configured with a Device Director profile set as OutputProfile: AWS S3 and Path: [a path referring to a folder within this bucket], and the archive is also in S3 with Source: S3-ARCHIVE. Mediastores configuration steps can be found below:

Metadata Names

Additional metadata names are required for S3 proxy and Hi-Res in the S3 storage. 

NOTE: The S3WebProxyPath and S3HiResPath metadata need both to be set to data type text – on the existing system, this might require a change in the database from data type URL and needs to be followed by a re-index. This is because the definition of URL does not allow for the aws: protocol specification we need to store for S3 objects.

*Read permissions should be provided to the above fields within relevant workgroups. Typically this will be All Users.  

**The Path metadata should only have permission set for System Administrators in order to obfuscate the S3 storage details from the end users.

The metadata name and display name can be changed to match customer schemas. However, if any names are changed, you will need to change the Mediastores entries as well.

Curator Gateway Configuration for Secure Proxy Streaming

Proxy routing must be set up for /proxies/{catchall}. This can be accessed via Curator Gateway>API Routes.

NOTE: The /proxies as a route name is not interchangeable with another route name at this time, so please use this as given.

Create a new API route with an Upstream Path Template of /proxies/{catchAll}, providing the name of the S3 proxy bucket for the streaming, as well as details of the bucket and the folder with the proxies.

Set Authentication to Enabled (ticked) with the Authentication Resource Curator Proxies.

Further, you must ensure that for the authorization and cookie options under the Header Information section, Remove is ticked:

In the above example, the Downstream Path Template is set to the /Top-folder/WebProxies folder in the S3 bucket called AWS-bucket, which will be available as CuratorGateway/proxies (set in Upstream settings).

Curator Client Application setting for Secure Proxy Streaming

The client applications require that Curator Proxies are added to the Allowed API Resources (scopes) in Client settings in Curator Gateway for each of the Web Apps, so that Curator for Adobe (and the Curator for Adobe Importer/Exporter) can stream and import the secured proxies. As of Curator 3.3, the Curator for Adobe clients are separated for each Adobe software type, and each will need to have the setting added if not present.

Playback Configuration in the client

The client application requires Video Server configuration to point to the Curator Gateway server and ProxyPathprepend set to CuratorGateway/proxies, the routed replacement for the folder in the bucket. Remember to leave the DefaultVideoConfig with an empty value to avoid defaulting to the NFX proxies. Use CuratorGateway/proxies (case-sensitive) for the ProxyPathPrepend.

    <add key="DefaultVideoConfig" value="" />
    <add key="VideoServerAddress" value="https://MyGatewayServer.com" />
    <add key="ProxyPathPrepend" value="CuratorGateway/proxies" />
    <add key="ProxyRootFolder" value="" />
    <add key="ProxyPathAppend" value="/{name}.m3u8" />
    <add key="ThumbnailVttLocation" value="{videoUptoExtension}_thumbnail.vtt" />
    <add key="ThumbnailVttLocationIfUsingNfxVttFiles" value="{nfxVttFilesShadowingVideoUptoExtension}_thumbnail.vtt" />

For Curator for Adobe, equivalent settings can be found in the HLS Playback Settings part of the configuration for Curator for Adobe 2.5. Please ensure that you are using schema 3.2.0.4. If that schema is not already present import it from the distribution zip or from the installation path of the IPV Curator for Adobe.

Alternative Streaming for the Web Apps using secured NFX

The web apps (not including Curator for Adobe) can also be set up in an alternative way using a direct secured NFX streaming either as the default with DefaultVideoConfig set to nfxvid or with this left blank to allow the application to probe for the presence of NFX (usually for stored proxies) vs Video Server provided (usually for Live proxies). Several NFX locations are possible to configure for NFX by pipe separating the locations.

To secure NFX connection

A separate application pool for applications using NFX streaming needs to be set up in order to avoid 404 Not found errors, and to allow the encryption of the secure key (querystring_protect_cu).

• In Advanced Settings>Process Model>Identity set up an account with rights to the folder such as administrator for the host or IIS_IUSR (if the password is known) in the Application Pool Identity dialog.

• Set up the web applications which are to use the NFX streaming to the same Application Pool.
• Obtain the current user encrypted value of the secure key string by navigating to one of the websites you have placed in the above Application Pool, appended with:

nfx/util/querystring_protect_cu_debug?awsSecretKey=HerePlaceSecretKeyObtainedFromTheAWS

replacing the HerePlace with the secure key, e.g.,:

https://MyCurator.ipv.com/CuratorClipLink/nfx/util/querystring_protect_cu_debug?awsSecretKey=bnAukbGcFD/yRGIcf5CDBqTjnQoHN/L9l8gtj/AB 

NOTE: Usually Amazon secret keys do not contain special characters like the plus sign (+) that need escaping in URLs. However, if you do see one, the rules of browsers and URLs are that + (for example) will be treated as space, so if your Amazon secret key to encrypt has a + you must escape the URL to replace it in the URL with %2B (and so on for other problem characters).

Collect the encrypted string corresponding to the secret key from the returned page under the [protectedOutput] heading.

To set up the web app to use the secure NFX connection with secret encrypted as above.

To default the streaming to NFX, set the DefaultVideoConfig to nfxvid.

Enter the mounted footage value using the json5 format, setting a kind of S3 and setting keyPrefix to the folder within the bucket that contains proxies (in this case, this is: top-folder/WebProxies).

Identify the bucket, setting the bucketName and region and set useAcceleratedEndpoint to true if this was set up when configuring the bucket (typically needed for performance).

Provide the awsAccessKey and set the awsSecretKeyFormat to qsprotect_cu and awsSecretKey to the encrypted value obtained as above.

It is possible, but not recommended for security reasons, to use plain text secure key entry with key format set to plaintext.

   <add key="NfxVideoServing.RootPathsPipeSeparatedList[mountedFootage]" value=
"json5: { kind: 's3', keyPrefix: 'top-folder/WebProxies', bucketName: 'My-AWS-Bucket', region: 'ap-southeast-2', useAccelerateEndpoint: true, awsAccessKey: 'AKIAQAMNOAXZBY6ZQJBV', awsSecretKeyFormat: 'qsprotect_cu', awsSecretKey: 'AQAAANCMnd8BFdERjHoAwE_Cl-sBAAAAV0v_gj207UatIGb1JPYn5AAAAAACAAAAAAAQZgAAAAEAACAAAADGeOst1afxIxn8pe2kcT6sRfEjW4mhBzzE1jeqJv4zGAAAAAAOgAAAAAIAACAAAAD3DCMLyfAZP7uEwJGm8-f6rFX4o5jTJ1gtIT2L5aUXaUAAAADxJRXLV2drU77-cdNLWzPsHznjXTTfbPnqv_LEELxB28IMuX3c-tfTYw0FvfH6pqeBUANuPJx9ggsBtL4YjRK3QAAAAA5oB-2D8lY_GM5vkZ8RRCd9KWZKg0dqsHdOnCOAkYPm8iJ6OwW6hZ_lChT1LZykz8BRKvBMox2M_Pvj479mij0' }" />
    <add key="DefaultVideoConfig" value="nfxvid" />

The Json5 structure has been formatted for ease of reading:

{
     kind: 's3',
keyPrefix: 'top-folder/WebProxies',
bucketName: 'My-AWS-Bucket',
region: 'ap-southeast-2',
    useAccelerateEndpoint: true,
     awsAccessKey: 'AKIAQAMNOAXZBY6ZQJBV',
awsSecretKeyFormat: 'qsprotect_cu',
awsSecretKey: 'AQAAANCMnd8BFdERjHoAwE_Cl-sBAAAAV0v_gj207UatIGb1JPYn5AAAAAACAAAAAAAQZgAAAAEAACAAAADGeOst1afxIxn8pe2kcT6sRfEjW4mhBzzE1jeqJv4zGAAAAAAOgAAAAAIAACAAAAD3DCMLyfAZP7uEwJGm8-f6rFX4o5jTJ1gtIT2L5aUXaUAAAADxJRXLV2drU77-cdNLWzPsHznjXTTfbPnqv_LEELxB28IMuX3c-tfTYw0FvfH6pqeBUANuPJx9ggsBtL4YjRK3QAAAAA5oB-2D8lY_GM5vkZ8RRCd9KWZKg0dqsHdOnCOAkYPm8iJ6OwW6hZ_lChT1LZykz8BRKvBMox2M_Pvj479mij0'
 }

Dummy Proxies Setup

In addition, your dummy proxy and placeholder proxies should be updated to S3. This can be done using the AWS CLI client, which can be downloaded directly from the AWS Website.

Once downloaded and installed on your client machine, open a Command Prompt window and carry out the following :

---

C:\Windows\system32>aws configure
 AWS Access Key ID [None]: <YOURKEYID>
 AWS Secret Access Key [None]: <YOURSECRETACCESSKEY>
Default region name [None]: us-east-1 (or equivient region)
 Default output format [None]: text
C:\Windows\system32>cd..
C:\Windows>cd..
C:\>cd /d d:\ (or equivient storage location)
d:\>cd webproxies
d:\WebProxies>cd dummyproxy
d:\WebProxies\DummyProxy>aws s3 sync . s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy --exclude="Export" --exclude="StagingVTT" --sse --storage-class STANDARD_IA --exact-timestamps

---

The result should look something like this:

 upload: DummyProxy\dummyproxy_audio0.m3u8 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_audio0.m3u8
 upload: DummyProxy\dummyproxy_thumbnail_0.jpg to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_thumbnail_0.jpg
 upload: DummyProxy\dummyproxy_thumbnail.vtt to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_thumbnail.vtt
 upload: DummyProxy\dummyproxy_video.m3u8 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_video.m3u8
 upload: DummyProxy\dummyproxy_subtitle.vtt to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_subtitle.vtt
 upload: DummyProxy\dummyproxy_audio1.m3u8 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_audio1.m3u8
 upload: DummyProxy\dummyproxy.m3u8 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy.m3u8
 upload: DummyProxy\dummyproxy_video.mp4 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_video.mp4
 upload: DummyProxy\dummyproxy_audio1.mp4 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_audio1.mp4
 upload: DummyProxy\dummyproxy_audio0.mp4 to s3://<YOURS3BUCKETNAME>/webproxies/dummyproxy/DummyProxy/dummyproxy_audio0.mp4

NOTE: Proxies are case-sensitive when it comes to streaming from S3. So you might find that you need to rename your dummy proxy to match up with what is attempted to be streamed by Curator from S3. If this is required, rename all the dummy proxy files appropriately and then amend all the references within the m3u8 files before uploading the folder again.