Upgrade the Semarchy xDI Runtime

This article describes how to upgrade Semarchy xDI Runtime.

When you upgrade your runtime from the main software package, you should generally follows these steps, though the order may vary:

  1. Prepare yourself for the upgrade.

  2. Install the runtime to a new location.

  3. Copy data to the new runtime.

  4. Perform any version-specific upgrade steps.

  5. Clean up.

Read the entire document before starting the process, especially the version-specific steps.

Pre-upgrade steps

Review the release notes

Read the release notes to get information about the latest Semarchy xDI release, such as new features, bug fixes, breaking changes, and removals.

System requirements

Review the system requirements for the new release. Take special note of supported Java versions, as you may need to install a newer one depending on which Runtime version you are upgrading from.

Stop the Runtime

If it is running, stop the active Runtime.

Back up the Runtime

Back up the active runtime installation, including:

Upgrade steps

Get and install the Runtime

  1. Download Semarchy xDI Runtime from the Semarchy website.

  2. Decompress the downloaded file and move the content to a new installation directory.

  3. Check and set the execute permission on all the .bat (Windows) and .sh (Linux or macOS) scripts located at the root of the new installation directory.

  4. Replicate the read/write permissions for other files from the previous installation directory. In particular, check and set write permissions for the following subdirectories:

    • ./temp

    • ./build

    • ./sessions

  5. Copy the content of the following directories from the previous Runtime installation to the new installation:

    • ./build/deliveries

    • ./build/packages

    • ./scheduler (if you are using the Runtime’s scheduler)

    • ./sessions (if you use the Runtime’s internal log database)

    • ./temp (if your Processes store specific files in this directory)

Update modules

Copy the modules from the previous installation to the modules folder in the new installation directory. By default, modules are in the ./modules subdirectory where Runtime is installed.

Check that the modules have all their needed files. If your integration flows were updated in a newer version of xDI Designer, you may need to replace module files with newer versions.

Configure the Runtime

Configure the new Runtime installation based on your previous configuration. Some options may carry over as-is, but there may be changes between versions.

Perform version-specific steps

Depending on which version you upgrade from, you may need to follow more steps to make the new runtime work. See the next section for these steps.

Version-specific upgrade steps

This section lists extra steps to take depending on the Semarchy xDI version you upgrade from.

Version-specific upgrade steps are grouped with the relevant product versions, listed from newer to earlier versions. Perform the steps in the other order, from earliest to newest. Check all entries before you continue, as you may need to perform multiple steps.

For example, upgrading from S17 to 2023.1 LTS requires extra steps. Follow the instructions for versions S20, 5.3.x, and 2023.1.x in that order.

If your version does not appear in this document, you can skip to the post-upgrade steps.

2023.3.0 and earlier to 2023.4.0 and later

xDI Runtime version 2023.4.0 introduce the following changes that may require upgrade steps:

  • The xDI Runtime scheduler now saves its settings in the main runtime configuration file by default. The legacy scheduler properties file is still supported, but not available by default.

  • The internal database has changed from H2 to HSQL.

Move the scheduler configuration

Previous versions of Runtime used the file named engineScheduler.properties to store the scheduler configuration. Runtime version 2023.4.0 can still read this file, but scheduler settings should be migrated when possible.

If you choose to migrate the file now, follow these steps:

  1. Open the properties/engineScheduler.properties file from the previous Runtime installation.

  2. Open the properties/engineParameters.xml file from the updated Runtime installation.

  3. In engineParameters.xml, find the section enclosed in <scheduler></scheduler> tags.

  4. Add your engineScheduler.properties file to the properties directory in the Runtime installation directory.

For more information, see the relevant section of the runtime scheduler documentation.

Migrate the internal database contents

xDI Runtime switches to using HSQL instead of H2 as its internal database. If you used the default internal database, you must decide whether to keep the data or not.

  • If you do not wish to carry over the internal database, skip these migration steps.

  • If you were using the internal database, and want to keep using it, you must migrate your data to HSQL. Follow the instructions in this section.

The built-in runtime database is, by default, only accessible from the runtime itself. It is not recommended for production use.

For production environments, you should use a certified database server instead.

The tool used for the migration has the following limitations:

  • Scheduler: The source and target schemas must have the default name SCHEDULER.

  • SesssionLogs: The source and target schemas must have the default name LOGS.

In the following guide, "source runtime" refers to the runtime with the H2 database, and "target runtime" refers to the updated version with HSQL.

First, back up and prepare the source runtime. Navigate to the source runtime installation path, then back up the scheduler and sessions directories.

Open xDI Designer, and create an H2 module:

  1. Find and download version 1.2.140 of the H2 JDBC driver.

  2. Create a module of type H2 named h2_legacy_driver.

  3. Add the downloaded H2 JDBC driver to the module.

Prepare the migration tool:

  1. Download the H2 to HSQL migration tool to the system with the target runtime.

  2. Unzip the migration tool to a location of your choice.

  3. Copy the h2_legacy_driver module directory to the migration tool’s modules subdirectory.

  4. Copy the source scheduler and sessions directories to the migration tool’s source_data subdirectory.

Prepare the target runtime:

  1. Copy the h2_legacy_driver module directory to the target runtime’s modules subdirectory.

  2. Find the sessionLogs_migration.deliv and sessionLogs_migration.deliv files in the migration tool’s deliveries.

  3. Copy the two .deliv files to the target runtime’s build\deliveries subdirectory.

  4. Make sure the target runtime configuration (in engineParameters.xml) is set the following way:

    1. The temporary folder must be the default temp.

    2. There must be an admin user with admin-password as its password.

Open a command prompt in the target runtime directory, and run the startMigration.bat (Windows) or startMigration.sh (Linux) script. The migration tool takes the following arguments:

  1. Common arguments:

    Arguments Description

    runtimePath <path>

    The target runtime path. This argument is mandatory.

    The target runtime path must not contain any spaces or special characters.

    deliveryLogLevel <logLevel>

    Set the log level used by the migration deliveries. The default value is 0 if this parameter is not set.

  2. Scheduler database arguments:

    Arguments Description

    schedulerMigration

    When this argument is defined, the script migrates the scheduler database from H2 to HSQL.

    Only one of the schedulerMigration or sessionLogsMigration argument must be used at the same time.

    deleteHsqlScheduler

    Deletes the target HSQL scheduler database prior to the migration. Use this argument if you want to ensure that the target database is emptied before the migration.

    This argument can be used only when migrating the scheduler database with schedulerMigration.

    h2SchedulerLogin <login>

    The login of the source H2 scheduler database. To specify an empty value, omit this argument.

    h2SchedulerPassword <password>

    The password of the source H2 scheduler database. To specify an empty value, omit this argument.

    hsqlSchedulerLogin <login>

    The login of the target HSQL scheduler database. To specify an empty value, omit this argument.

    hsqlSchedulerPassword <password>

    The password of the target HSQL scheduler database. To specify an empty value, omit this argument.

    schedulerInstanceId <instanceId>

    The target scheduler instance identifier. It must be equal to the org.quartz.scheduler.instanceId parameter of the engineParameters.xml` file.

    The default value is RUNTIME_HSQL_STD if this parameter is not set.

    schedulerInstanceName <instanceName>

    The target scheduler instance name. It must be equal to the org.quartz.scheduler.instanceName parameter of the engineParameters.xml`file.

    The default value is RUNTIME_HSQL_STD if this parameter is not set.

  3. Sessions logs database arguments:

    Arguments Description

    sessionLogsMigration

    When this argument is defined, the script migrates the session log database from H2 to HSQL.

    Only one of the schedulerMigration or sessionLogsMigration argument must be used at the same time.

    deleteHsqlSessionLogs

    Deletes the target HSQL session logs database prior to the migration. Use this argument if you want to ensure that the target database is emptied before the migration.

    This argument can be used only when migrating sessions logs with sessionLogsMigration.

    h2SessionLogsLogin <login>

    The login of the source H2 session logs database. To specify an empty value, omit this argument.

    h2SessionLogsPassword <password>

    The password of the source H2 session logs database. To specify an empty value, omit this argument.

    hsqlSessionLogsLogin <login>

    The login of the target HSQL session logs database. To specify an empty value, omit this argument.

    hsqlSessionLogsPassword <password>

    The password of the target HSQL session logs database. To specify an empty value, omit this argument.

Session logs database migration example:

Windows:

startMigration.bat -runtimePath C:\Semarchy\runtime -sessionLogsMigration -deleteHsqlSessionLogs -h2SessionLogsLogin sa -hsqlSessionLogsLogin backend-user -hsqlSessionLogsPassword backend-password

Linux:

bash startMigration.sh -runtimePath /Semarchy/runtime -sessionLogsMigration -deleteHsqlSessionLogs -h2SessionLogsLogin sa -hsqlSessionLogsLogin backend-user -hsqlSessionLogsPassword backend-password

Scheduler database migration example:

Windows:

startMigration.bat -runtimePath C:\Semarchy\runtime -schedulerMigration -deleteHsqlScheduler -h2SchedulerLogin sa -hsqlSchedulerLogin backend-user -hsqlSchedulerPassword backend-password -schedulerInstanceId RUNTIME_HSQL_STD -schedulerInstanceName RUNTIME_HSQL_STD

Linux:

bash startMigration.sh -runtimePath /Semarchy/runtime -schedulerMigration -deleteHsqlScheduler -h2SchedulerLogin sa -hsqlSchedulerLogin backend-user -hsqlSchedulerPassword backend-password -schedulerInstanceId RUNTIME_HSQL_STD -schedulerInstanceName RUNTIME_HSQL_STD

2023.1.x and earlier to 2023.2.0 and later

The thread count for the scheduler thread pool changed from 50 to 3. When copying your settings, make sure you are not using the old value.

This setting is controlled by the org.quartz.threadPool.threadCount parameter in the engineParameters.xml configuration file.

5.3 and earlier to 2023.1.0 and later

xDI Runtime version 2023.1.0 introduces the following changes that may require upgrade steps:

  • The default Jython third-party library has been updated.

  • Support for RMI connections to the runtime has been removed.

  • The scheduler database structure has been updated.

Update Jython scripts

The Jython library update replaces the javos module with the os module. Scripts that use the javos module should replace it with os accordingly.

The Runtime jythonVersion parameter is now commented out by default so that the runtime use the included version by default.

Switch from RMI to HTTP

If you are using RMI connections to the runtime, you must change them to use the HTTP protocol instead:

  1. Replace all instances of rmi://<hostname>:<port> or rmis://hostname:port with http://hostname:port or https://hostname:port.

  2. Replace the RMI port with the HTTP port. The new default port is 42200.

  3. For instances where you were accessing the runtime without specifying the protocol, change the port only.

You may need to change connection information in the following locations:

Update the scheduler database

Scheduler data structures were changed in 2023.1.0. If you were using the xDI Runtime scheduler, you must update them using an update script before starting the updated runtime:

  1. Make sure all runtime instances are stopped.

  2. Create a backup of the runtime database, if you haven’t already.

  3. In the runtime installation directory, go to the /scripts/scheduler/upgrade/2023.1.0/ directory.

  4. Find the SQL script that corresponds to the database you use.

  5. Run the corresponding SQL script on your database to update the scheduler data. To run the script, you can use xDI Designer or an external tool.

S17 to S20 or 5.3 and later

xDI Runtime version S20 and 5.3 introduce the following changes that may require upgrade steps:

  • Java 8 or Java 11 are required to run the application.

  • The application now supports modules to manage and use third-party libraries. Modules are found in the runtime installation directory, under modules.

  • RMI connections can use a custom certificate.

  • SOAP endpoints have changed.

Connecting to a runtime using the RMI protocol is no longer possible as of version 2023.1.0. Switch to HTTP connections by following the upgrade steps for version 2023.1.0 and later.

Update Java

Make sure either Java 8 or Java 11 are installed. Java 11 is recommended.

Set up modules

Use Semarchy xDI Designer to create modules, then copy them over to the runtime:

  1. Find and install or update xDI Designer to match the runtime version you are upgrading to.

  2. Launch xDI Designer.

  3. Create modules for all technologies your integration flows work with.

  4. Add any other third-party libraries you need to the modules.

  5. Update your metadata, mappings and processes to recognize the modules.

  6. Copy all module directories from Designer to Runtime.

    • Copy them from the runtime/modules subdirectory of the Designer installation path.

    • Copy them to the modules subdirectory of the Runtime installation path.

Set up JDBC libraries

If you store runtime logs or schedules in a database other than the default one, you must add the necessary drivers to the default module.

  1. Find the JDBC drivers for the database(s) you use for logging and scheduling.

  2. Add the drivers to the modules\default subdirectory of the Runtime installation path.

Update SOAP endpoints

xDI Runtime has two SOAP endpoints: a WSI-compliant endpoint, and a non-compliant endpoint. Before version S20, these used the same URL. Version S20 changes URLs to make the two endpoints more distinct, and correct inconsistent behaviors.

If you ran version S19 and earlier, and exposes SOAP or WSDL web services from a runtime, update all endpoints in xDI resources such as metadata. You must match the endpoints shown by the runtime at startup.

Endpoints may vary based on your configuration. Here is an example:

SOAP Endpoint (local url): http://localhost:42200/wsi/DeliverableService?WSDL
SOAP Legacy "Non WSI-Compliant" Endpoint (local url): http://localhost:42200/nonwsi/DeliverableService?WSDL

Post-upgrade steps

When you finish all the upgrade steps, start the runtime and test your integration flows. If errors occur, make sure that you copied over settings correctly, and that your modules contain the correct libraries.

After validating the upgraded installation, you can start using it.