Please note: This Service is coming soon and the information herein is provisional – please contact Sales if you are interested in this Engine

IPV is developing a new Workflow environment with an Orchestration Engine (Process Engine) and worker services (Workflow Agent). The Workflow Agent is the first stage and is able to run in parallel with the current Workflow framework. This engine is designed to be horizontally scalable and provide workflow execution to communicate with third-party devices and data processing. An example of this is an integration with Veritone, where the orchestration layer (PE) initiates a job to upload some content to Veritone and the Workflow Agent processes the results to create sub-clips based on the AI data that Veritone generates. 

Working in conjunction with Curator Device Director, the Node-RED Workflow Agent is a new workhorse for future integration touchpoints and workflow execution, providing flow-based workflow programming for rapid and easy workflow development and deployment. With its "low-code" visual designer approach, Node-RED provides rapid and easy workflow development and deployment, even by non-developers.

Workflow Agent architecture and deployment

A conceptual architecture diagram

Designed as a containerized system and based on Docker images for simplicity and ease of deployment, Node-RED instances consist of three different containers:

• A Workflow Agent controller
• A Node-RED execution engine
• A persisting layer

Benefits of deploying in this way include:

• Easy to debug, manage and monitor
• Horizontally scaling, using the Device Director to manage nodes and instances
• Extensibility with options coming to provide elastic compute
• Rapid workflow and device extensions with a library of Nodes

All three containers will usually be deployed together on a single system using Docker-compose and are treated as a single service.

An example workflow using a Workflow Agent for Speech-to-Text AI transcription

The following diagram showcases an example Node-RED flow for a Speech-to-Text AI transcription.

The high-level flow is as follows:

• Media assets are found in Curator and a plug-in is used to send jobs to Device Director
• Using the generic Device driving an IPV Node-RED-powered Workflow Agent, Node-RED can be used to build a workflow that communicates with the AI engine

Minimum machine specification for the Workflow Agent

The Node-RED engine runs on a Linux platform and is delivered in a Docker Container.

IPV's minimum recommendation for machines powering Workflow Agent workflows is a work in progress, however for a machine to run 5 concurrent workstreams the minimum consideration should be:

• 8 CPU cores
• 64GB memory
• 250GB HD

Why Node-RED?

After thorough testing and comparisons, IPV selected Node-RED for its workflow execution engine for the following reasons:

• Node-Red is a lightweight system so is light on system resources
• It is easy to install and easy to use, even by non-developers, enabling businesses to create custom workflows much more easily and rapidly
• Node-RED is a mature project, open-sourced in September 2013 and is one of the founding projects of the JS Foundation in October 2016 and has regular releases. The public repository on GitHub has a lot of contributions, including the engine Nodes, Dashboards, etc., making it easy for businesses to re-use existing workflow for even faster roll-out
• The community is well supported with channels such as a Forum, Slack, Google Groups based mailing list and Stack Overflow and several areas for learning more about Node-RED, user guides, tutorials, API reference, etc., adding to the ease with which businesses can get up to speed with developments and deployments
• Node-RED allows for persistence to be added to workflows, such as restarting, tracking, etc.
• Today there are more than 2,880 nodes and > 1,750 flow types. Ready-made integrations include:
  • IoT
  • SQL and NoSQL databases
  • Raspberry Pi
  • Salesforce
  • Google Cloud Text To Speech
  • GCP Storage
  • etc..
• In tests, Node-RED has proven to very performant and it’s easy to create new nodes using node.js projects