Workflow Agent Installation Guide

Updated on December 9th, 2022

Table of Contents

X

Workflow Agent Installation Guide

This document describes how to install Workflow Agent and the required configuration in the Curator system. Some connectors may require further configuration steps.

Workflow Agent jobs are initiated using the transfer asset architecture in the Curator workflows. Typically utilising the Send to Destination plug-in, "Open with" webpage exercising a Send To action via the Gateway or a scavenge to initiate a transfer. This will start the standard transfer workflows, which can then be configured to use the Send to Workflow Agent workflow to trigger a Device Director job, which will send the asset id and MediaStore configuration as parameters to the Workflow Agent.

The Workflow Agent is deployed on Linux instance of Ubuntu Server version 20.04.3 (newer versions are not currently recommended) and the deployment utilises Docker available in Ubuntu 20.04. The Workflow Agent integrates to the rest of the Curator system through Device Director using a Generic Device. Instructions for deploying images for the working Workflow Agent can be found in this document. An example of configuration can be seen for the Workfront connector: see Workfront Connector - Workflow Agent configuration and the Workfront Connector - Curator configuration.

Pre-requisites

Required prerequisites
- an instance with open ports: 
- Outbound: All traffic allowed
Inbound:  Custom TCP  8080, 1880; SSH  22, HTTP 80, HTTPS 443 

- a basic ability to work in a Linux environment - see our Linux Primer 

- Ubuntu Server 20.04.3

Some general knowledge of Linux, including understanding key concepts, components, and the command line are all useful. For a good starting point, please refer to our Linux Primer article, which contains information relevant to installing Workflow Agent.

As of writing, Workflow Agent has been installed on various machines, including an AWS instance and a Hyper-V virtual machine on Windows 10. For testing or trial purposes you might consider using a virtual machine to run this locally, although for production you will likely be using a server.

Ubuntu Server must be installed on your server. Version 20.04.3 of Ubuntu Server is recommended - more recent versions of Ubuntu Server are not currently recommended. Ubuntu Server is available here. It should also be available on AWS when you configure an instance. All outbound traffic should be allowed from the instance and these Inbound Ports must be open on the instance: Custom TCP   8080, 1880; SSH  22, HTTP 80, HTTPS 443. In AWS these can be configured by setting permission rules in launch wizard which will create a security group with these entries.

If you install Ubuntu Server manually, you will have the option to install other software such as Docker. We recommend you do not install Docker at this point but instead do it later (as described below). 

You should have a version of Workflow Agent available in Dropbox or a similar hosting service. Workflow Agent needs to be added to the server. There are various ways to do this, including sharing a folder / directory (SMB) and downloading directly to the server (Wget). These methods are detailed in the Linux Primer article linked above.

Before continuing please have Ubuntu Server running. Ensure you have the ability to control it via the command line and have the Workflow Agent files on the server before continuing.

Delete

Installing Docker

First, you need to create an installation script. To do this, follow the steps detailed below.

Navigate to where you want to make the script. From here, you can use a command like this: > sudo nano DockerScriptName.sh which should open Nano and create an empty file with that name. Copy and paste the following into the empty file:  

> #!/bin/bash

apt-get update
apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
    
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
  
apt-get update
apt-get install docker-ce

Press CTRL+X to exit (with save check). Select Y and press Enter. Give permission to the script using this command: > chmod +x DockerScriptName.sh. You can then run it with > sudo bash DockerScriptName.sh. You can also run the individual lines if you want using sudo before them.

You can avoid the need to add sudo before each command by adding a user to a Docker group. To do this, use the following command: sudo adduser <user> docker , then log on again.

For further reading on installing this on 20.04, please see here. Please make sure the installation is compatible with your particular version of Ubuntu Server.

Delete

Downloading Workflow Agent

You will need a means of securely copying the compressed files to the Linux server. WinSCP can be used if you are copying from a Windows location; Linux command scp will work between Linux servers.

The installation comes in two formats (tar.gz and zip), both containing the same files. For Linux hosts, tar.gz may be easier to use as the tar command to extract it is native to the Linux installation.

Once the file is downloaded, it should be extracted into the shared location; it will have a directory inside such as WorkflowAgent-v1.0.0.0.

The following steps outline how this can be done using WinSCP, which can be found here: https://winscp.net/eng/download.php.

With WinSCP installed, complete the following:

  1. Open WinSCP.
  2. Select New Session.

    From here, change the file protocol to SCP, enter the host name (this will be the IP address of the Linux Server), enter the port number (this should be 22), then enter a username and password and click Login.
  3. You will be prompted with a warning.Select Yes.
  4. A confirmation message will appear when you have successfully connected to the Linux Server.
  5. On your local machine, copy the Workflow Agent zip or .tar.gz file to the system's root folder (e.g. C:\ or /root).
  6. On the Linux Server, you will need to create a new folder within SCP called Installers in /home/[username]/.
    To do this, right-click on WinSCP on Linux Server and select New. Then select Directory. Name your new folder Installers. Then select OK to confirm.
    NOTE: Permissions may need to be changed or you may need to run sudo commands in order to make this work correctly.
  7. Open the Installers folder and drag/move the Workflow Agent.zip or .tar.gz file from the root folder of your local machine to the Installers folder you just created. This will copy the installer from the local machine to the Linux Server.

To decompress and unpackage the downloaded files for the tar.gz use the command: 

     tar xvzf file.tar.gz -C /path/to/sharedlocation 

where you substitute file with the actual name of the install file and the /path/to/sharedlocation with your shared location on the Linux server.

For the .zip file you may need to install the unzip program if not already present:

     sudo apt-get install unzip

After installing the unzip utility, run:

     unzip file.zip -d /path/to/sharedlocation

where you substitute file with the actual name of the install file and the /path/to/sharedlocation with your shared location on the Linux server.

Delete

Installing Workflow Agent

You will need to update the configuration settings. To do this, navigate to and open workflow-agent.config using a command like > sudo nano workflow-agent.config.  

The URLs used for Curator Server and Curator Gateway must be changed, as shown below:  

# Curator Gateway URL
IPV_WA_CURATOR_GATEWAY_URL=https://yourURLhere.ipv.com/CuratorGateway
# Curator Server URL
WORKFLOW_ENGINE_CURATOR_SERVER_URL=https://yourURLhere.ipv.com:11210/CuratorServer/

If this is intended as development mode, where flows are not overwritten on startup, change the following to be true instead:  

IPV_WA_DEVELOPMENT_MODE=true

If you are using Self Signed certificates, then you need to change this value to assume you have trusted certificates:  

IPV_WA_AUTH_TYPE=trusted

Create a directory to install Workflow Agent using the following command: > sudo mkdir NameOfInstallationDirectory

Navigate into this directory using one of these commands: > cd NameOfInstallationDirectory   or > cd Path/To/NameOfInstallationDirector. 

You must run the installation script from within the installation directory otherwise there may be issues. 

Run a script like this: > sudo sh ../path/to/workflow-agent-update.sh

Workflow Agent should begin installing, including starting the Docker containers.

Delete

Updating and Changing Configuration

To install a new version, simply repeat the installation process but with the path to the updated version. This will stop the existing docker images, and install new ones instead.
The same procedure can also be used to change configuration settings by re-copying them from the shared location.
Note that when updating, a backup file will be created of the old files with a date-time extension. These backups can be safely removed if not wanted.
If you need to remove the Workflow Agent, the Workflow Agent Removal article may be helpful.

Delete

Troubleshooting

The following error may appear when trying to reinstall after upgrading an instance:

ERROR: for ipv-workflow-agent-mysql  Cannot start service ipv-workflow-agent-mysql: driver failed programming external connectivity on endpoint ipv-workflow-agent-mysql (09fd2b67c1a1546d4b1f709c6fcaf007a4a80b1982a6e36d01ff09fbae7b1295): Bind for 0.0.0.0:13306 failed: port is already allocated.
This can be fixed by doing a Docker service restart. 

If the error persists, double-check workflow-agent.config and the URLs defined there. Verify that they are correct: a common issue can be found where 'http' and 'https' are mixed up. Other potential issues include container instances running old or incorrect configuration. If that is the case, you may find the Workflow Agent Removal article helpful, followed by reinstallation. This should be a quick process. 

Delete


Before continuing to configure the Workflow Agent in Curator, check if your integration has specific instructions. To do this, browse the Subcategories for Curator configuration. The configuration steps below are the minimum that will allow Workflow Agent to communicate with Curator - your integration may have additional required configuration or specific naming requirements.


Configuration in Curator Device Director

To enable the integration using Workflow Agent to communicate with the rest of the Curator system, you should set up a profile for the Workflow Agent and an integration specific device placed in the integration specific group in the Device Director.

If your version of Device Director does not already have this, you will need to create a Generic Device type Profile with the name Workflow Agent. You will need to create the Generic Device type Device named for your integration and with the appropriate authorization setting and enter it to a Generic Device type Group named after the Integration.

Go to the Profiles tab. Select the Generic Device profile type and continue the process, naming it Workflow Agent. Click Create to save. 

Go to the Devices tab and begin the process. Select the Generic Device type, enter a name specific to integration e.g. Workfront Agent and the address of the instance of the Workflow Agent with port and path to the API e.g.: http://example.workflowagent.server:8080/api/ipv/v1 

Under Authorization Token Behavior, select Always send Device Director Service authorization and select Create.








Go to the Groups tab and create a new Group of type Generic Device, enter a name specific to your integration e.g.Workfront and select Create

In the next screen, under Actions select Add Devices, then tick the device created in the previous step and select Add Selected Devices.

Delete

Configuration in Process Engine

Required prerequisites
- on Curator 3.3.1, installed module Proces Engine API 1.2.0.0
- other process Engine modules that are required by the integration

Make sure you have the required Process Engine Modules installed in Process Engine. Process Engine API is required for integrations and is included by default from Curator 3.4, although some older versions do not include it. Other modules may be needed for your integration that utilises the Workflow Agent and there may be a Process Engine Module for the integration itself. 
Log in to Process Engine Monitor and check that the required modules are present and import the remaining.

The integrations are typically used with the Send-To (plug-in or a website) and scavenge functionality. As a minimum, you need to create or check the configuration of the MediaStore declaring the Process Engine workflow delivering the action from the Send-To to the Workflow Agent and its parameters including the name of the Workflow Agent workflow.

Go to the MediaStores tab in Process Engine Monitor and find to configure or create a new MediaStore with a name in the form INTEGRATION_NAME-ACTION-PURPOSE (e.g. WORKFRONT-UPDATE-PRODUCTION) with a Custom template. Add the keys and values as below, the entries in [.] need to be set to values specific to the integration.  

DeviceProfile     Workflow Agent
DeviceGroup [name of the Device Group, specific to integration, as created above]
MediaStoreDescription     This is a transfer destination for the update Workfront plugin
SendCollection     [True or False]
SourceRequired [True or False]
StoreType Dynamic
Workflow Spawn - Send to Workflow Agent
WorkflowAgentWorkflowName [name of the Workflow Agent workflow to be executed] 


Delete




Was this article helpful?