Overview

Metadata Central provides the service managing the API and graph database supplying context dependent values to Curator services, including large data sets. Most frequently used in selecting the correct sets of data for Sports logging based on selected properties of games/meetings. 

Metadata Central can be configured to provide a polling Extract Transform and Load (ETL) service for the updates of the rosters and schedules from the supported external providers. The ETL Scheduler works by polling and then ingesting known sources for schedules and for rosters updates. The sources and schemas for data interpretation are defined in the C:\Program Files (x86)\IPV\Teragator\Service\ETLSources.config configuration file for the ETL Service.

The full configuration consists of:

• Setting up the database provision (Neo4J, requires JDK version specific to the version of Neo4j)
• During upgrade only where previously SQLSERVER  database was used - one-off data conversion to Neo4J format (performed inhouse, talk to Development)
• During an upgrade between major versions, e.g. where 3.5 Neo4j was used and 4.3 Neo4j is to be used a proper dump-upgrade-restore process as prescribed for Neo4j is required
• (Previously required triplestores configuration is not required anymore, beyond selecting the type of installation with endpoint and basic configuration of ETL (on/off and timing) within the installer,
• Configuring the etlSource in the ETLSources.config which must refer to a known 'schema' as listed
  NOTE: Any unknown service that is not listed in the list of the recognised 'schema' as present in the latest version of the ETLSources.config will require development work to add the logic to obtain the appropriate data from the feed.

Steps for the installation and configuration of the MDC Teragator and the ETL scheduler for external update service or file drop

The following steps are valid from Teragator 2.0.4:

• Choose the version of Neo4j and run prerequisite installation and configuration (JDK) and Neo4J installation and setup on the server on which Teragator will be installed, remembering to switch the need for authentication off in the \[installation path]\conf\neo4j.conf
  • if you have run a conversion of older data to the Neo4J, make sure you have installed that version on your server, place the converted datafiles in the [neo4j installation]/data folder
  • if you are converting Neo4j from 3.5 family to 4.3 family (available from Teragator 2.0.4) you will need to update to JDK 11, dump and restore the data using Neo4j off-line backup process, and set dbms.allow_upgrade=true in the \[installation path]\conf\neo4j.conf
• Run the provided Teragator msi installer, setting the endpoint
  • in the screen following the acceptance of the license:
• 
  • replace the localhost with the FQDN of the host server. The http protocol for Service address is correct. 
  • select the type of configuration (usually MetadataConfiguration_GenericMultiTeamSportsLoggerConfig) - this replaces the formerly needed step of manually creating "triplestores",
  • select whether ETL is to be run and its polling frequency.
• Complete by selecting Next and Install on the next screen.
  NOTE: Make sure the port for Teragator Endpoint (11190) is open on the server on which MDC Teragator is installed so that web apps can reach the endpoint via the Resource setting. The port is not automatically open for IPV Teragator Service.
• Configure the etlSource entries in the ETLSources.config which must refer to a known 'schema' as listed there.
  NOTE: Any unknown service that is not listed in the list of the recognised 'schema' as present in the latest version of the ETLSources.config will require development work to add the logic to obtain the appropriate data from the feed.
• Start the MDC 'IPV Teragator Service' with Local System as the logon account. This will also start the ETL polling is test to true.

Changes in Configuration

If at a later point ETL scheduler should be stopped or its polling frequency changed.

• Locate C:\Program Files (x86)\IPV\Teragator\Service\ETLSources.config 
• Alter appropriately 
• Locate Teragator.Host.Rest.WindowsService.exe.config and set the appropriate period for ETL detection ( in seconds) and ETL to be active:
  <add key="Scheduler_Period_In_Seconds" value="600"/> 
  <add key="Scheduler_Is_Active" value="true"/> 
• Restart the MDC 'IPV Teragator Service'