Introduction
Curator File Service is used to remotely abstract files across multiple client applications from the underlying storage model (File system or AWS S3 with scope to extend to other cloud storage models e.g. Azure, Google Cloud). File Server also provides for the migration of BLOBs from the Curator Server Database, including thumbnails, PFR indexes, raw AI data, and .prproj files.
Curator Server remains the single source of truth for metadata and therefore file storage location. File Server synchronizes required metadata from Curator Server.
File Service uses a resumable streaming protocol allowing you to upload/download files to different client applications and get progress on those uploads from a single location. By collating all of the file-related configuration logic into a single place, it makes it easy to manage files from different applications without needing to duplicate configuration information for each application.
File Service also provides remote access to files*, securely bridging the gap for remote users who don't have direct access to the underlying storage without exposing files to the outside world.
In addition to this, File Service provides an audit trail, containing information on upload and download frequency along with other file-related activity. (See the Viewing User Activity section below for more information on this).
Using File Service, you can specify a preferred upload/download location for files that exist in multiple locations (See the Adding Selection Criteria section below for more information on this).
*Supported file types in version 1.1 are: Files including PSDs, Illustrator files, images, and BLOBs
Curator File Service Deployment
Refer to the Curator File Service installation guide to import Curator File Service using Web Deploy.
When File Service is deployed on an IIS server separate from Curator Gateway, follow instructions for IIS and Registry settings
from IIS and Registry Config. When Curator File Service is sharing the IIS server with Curator Gateway, make sure you have followed
the Installation step Add Application Pool in the instructions for the Curator File Service installation guide.
Note: The Gateway configuration of hosts (guide here) & API routing (guide here) should be completed before
completing any of the below steps.
Curator File Service securely fetches HiRes material for Curator applications. The configuration requires you to provide the
details of the access to the storage (Shared folder or S3 bucket as appropriate) and the corresponding file location metadata
so Curator File Service and metadata remain synchronized. Synchronization is performed by use of rabbitMQ (a prerequisite
for Curator Server and Curator File Service). In systems where Ingest and or file moves have happened prior to installation of Curator
File Service 1.1, typically where File Service is added during an upgrade, the Migration step detailed below is necessary where
initial metadata values are translated to storage description in Curator File Service.
Adding Stores
The Stores section of File Service displays a list of all the stores you have added. There are two different types of storage locations
that you can configure - File System (data is stored in a shared folder of your choosing) or S3 (data is stored in a specified S3 bucket).
| Navigate to Curator File Service | From Curator Gateway, select Curator File Service:
|
 |
|
| Select Stores, then click the plus icon | From the left-hand navigation bar, select Stores. Click the plus icon to add a store.
|
 |
|
| Select Type | Select the Type (this can be either File System or S3) from the drop-down and click Next:
|
 |
|
|
For File System store: Create a folder for CuratorFileStore |
If you have selected the File System Store type: In the main Curator file storage location, create a new folder that will be the storage location for Curator File Service.  Note the network path for this folder - this will be the Root in the next step. |
 |
|
|
For File System store: Enter Name and Root |
If you have selected the File System Store type: Enter the Name and Root. Note: The Root is the network path for the folder created in the previous step.  Click Create. NOTE: By default, Create Store will build a directory on storage using the local IIS USER, in some cases, this is not possible due to storage requiring a domain account or a different access level. As an example, you may receive an access error when attempting this step preventing you from progressing. To change the user which creates the directory, navigate to IIS > AppPools > Advanced Settings on your CuratorFileService App pool > Update account which is marked against the 'Identity' Value >OK. Once updated try creating the store again. |
 |
|
|
For S3 store:
Enter connection details to the S3 bucket.
|
If you have selected the S3 Store type: You will need to enter connection details for the S3 bucket as shown below:  Click Create. |
 |
|
| Confirm Creation | A confirmation message will be shown. Confirm creation of your store by clicking Stores on the left-hand navigation bar to verify details. |
Adding Selection Criteria
From the Selection Criteria screen, you can create upload selection criteria and download selection criteria for your stores.
At least one, default, Upload Selection criterion must be created. Upload selection criteria allow you to select default criteria to be
used when files are uploaded to each of your specified stores.
Download selection criteria will direct a download to use a specific preferred store, which is useful in cases where a HiRes file has
been uploaded to more than one location. If download selection criteria has not been specified, the first served file will be provided
by default.
| Access the Selection Criteria section | In the left-hand navigation bar, select Selection Criteria.  From this screen, you will see two types of selection criteria listed: Upload Selection Criteria, and Download Selection Criteria. |
 |
|
| Create default Upload Selection Criteria for the Store |
To add upload selection criteria: Click the plus icon to the right of the Upload Selection Criteria table. At least one default Upload selection criterion has to be created, pointing to the default Store. Select the above created Store (see the Adding Stores configuration steps) from the Store drop-down. A "default" name is acceptable. The expected label to use for the default store is hi-res  Click Create. |
 |
|
| Create Download Selection Criteria for the store |
To add download selection criteria: Click the plus icon to the right of the Download Selection Criteria table. Select the above created Store (see the Adding Stores configuration steps) from the Store drop-down. A "default" name is acceptable.  Click Create. |
Migrating Assets
Migration, on pre-existing Curator systems, transfers information about the storage of the HiRes files from chosen metadata to a record in the File Server so that the files are available through this route. This metadata would then normally be selected for further synchronization - for more information on synchronization, see the Synchronizing Metadata Names configuration steps below.
NOTE: On systems with many assets where there are many (~10,000+) metadata values to be migrated, you need to set IIS to not recycle the process worker or migration might never complete. This is recommended in order to prevent issues with RAM usage.
| Set up IIS if more than 10^4 values need to be migrated | To stop the IIS from recycling the worker during the migration, you will need to set the Idle Time-Out (minutes) to 0. To do this, In IIS manager, select Application Pools from the Connections menu (left-hand side). Select the pool that was created for File Service and in which File Service is running from the central list and select Advanced Settings... from the Actions > Edit Application Pool menu (right-hand side). From the pop-up, scroll down to the Process Model and replace the value for Idle Time-out (minutes) with 0 (the default is 20 minutes): 
|
 |
|
| Select Migration | In the left-hand navigation bar, select Migration: Click the wrench icon.
|
 |
|
| Complete top fields |
Enter appropriate details for the database server that data will be migrated from (either FQDN or IP address of database server), Curator Server's database name from which the metadata values will be migrated, and the database's user and password. Note: Data will be migrated from the curatordb database to the CuratorFileService database previously created during the installation process.  Select Start.
|
 |
|
| Migration runs | Depending on the number of assets the migration will vary in the time required to complete. Click the refresh icon in the top right to see the current status of the migration. 
|
 |
|
| Migration completes |  |
 |
|
| Set IIS back to recycle File Server webpage worker | If you want to preserve as much of the available resources as possible for the other pages running from that IIS server, you will need to revert to previous values after all of the migration is completed. To do this, set the value for the Idle Time-Out (minutes) back to the value it was before migration (default is 20). |
Synchronizing Metadata Names
From File Server 1.1 onwards, it is possible and necessary to synchronize file paths with S3 metadata, allowing you to synchronize metadata
names between Curator Server and Curator File Service. If metadata declared as synchronized are updated, File Service will
automatically pick up on these.
| Select Names | From the left-hand navigation bar, select Names. Click the plus icon at the top right to add metadata names which will be synchronized between the two servers. |
|
|
| Create Name | Add a Metadata Name, and any associated Labels (if required). Click Create. |
Viewing Synchronisation Failures
The Failures section allows you to view a list of all synchronization failures that have occurred. This section is intended to help
with troubleshooting the system storage configuration.
| Select Failures | From the left-hand navigation bar, select Failures. This includes the metadata Name, Value, Asset Id, Message (failure message), Date/Time, User, Client, and Remote IP Address. The Retry icon to the right of each entry allows you to retry synchronization. This may be necessary if you have not configured your bucket correctly prior to attempting some ingest. If the bucket has subsequently been configured, clicking Retry should fix the issue. |
Viewing Files
The Files section allows you to view your synchronized assets' information. From here, you can view an asset's Store, its Source,
Created Date/Time, Updated Date/Time, Version, Size, and use the Download icon on the right-hand side to download the file
directly from File Service.
| Select Files | From the left-hand navigation bar, click on Files. Select an asset's Filename to view further information on that asset. |
Fixing Broken File Links
For emergency cases in which file links are broken and require a direct replacement, you can upload a file directly from File Service
to replace a broken link. This is not a substitute for the ingest. Uploading a file from here will directly associate it with the selected
asset ID.
NOTE: This should not be used as a general method for uploading files. As a minimum, the synchronized location metadata will
need to be separately checked where it would typically appear in Curator (e.g., Clip Link or Curator for Adobe Panel) to ensure
that it is correctly linked to the file in the appropriate storage, typically using the file:// or aws:// URL format.
| Select Upload | From the left-hand navigation bar, click on Upload to upload a file directly from File Service.
|
Viewing User Activity
The Activity section allows you to view when each of your assets was accessed and by whom, including the activity Type (giving
details of whether the asset was updated or downloaded), Store (showing where the asset was accessed from), Date/Time, User,
Client, and Remote IP Address.
| Select Activity |
From the left-hand navigation bar, click Activity.
|