Upgrade procedure steps

This page offers guidance on the steps involved in the Semarchy xDM upgrade process.

Make sure to follow the steps corresponding to the upgrade path that you have selected.

Stop all server instances and connections

Before upgrading, you must stop all applications and user connections to Semarchy xDM. During the upgrade:

No application should access the xDM services and APIs.
No user should access the xDM user interfaces for design-time or data management operations.
No user or application should access the repository or data locations' database schemas.

You should contact your database administrator and middleware administrator to make sure that the unavailability of xDM during the upgrade process is correctly managed.

Stop all xDM servers. This is done either by:

Stopping xDM from your application server administration console.
Stopping the application server if it is dedicated to xDM only.

Back up your installation

Before upgrading, make sure that:

The schema(s) containing the xDM repositories are backed up; use the database utilities for performing such backup operations.
The schema(s) containing the data locations are backed up; use the database utilities for performing such backup operations.
The xDM installation folders in the application servers are backed up.
Any other relevant content (plugin JAR files, etc.) is also backed up.

Before going further, make sure that all the required backups are done, and that the entire xDM environment is stopped and not accessed by any user or application.

Duplicate database schemas (out-of-place upgrade)

If you have decided to perform an out-of-place upgrade, you must replicate (i.e., copy) all the schemas (repositories, data locations, etc.) using the database utilities.
The new product version, considered as a separate installation, will point to this copy of the schemas.

Use a clear naming convention for these new schemas to be able to identify them easily. For example, if the original schemas are called SEM_REPOSITORY, SEM_DLOC1, etc., the new ones may be called SEM_5_3_REPOSITORY, SEM_5_3_DLOC1, etc.

With PostgreSQL, when replicating the databases, make sure that the search_path parameter is correctly set to $user, public, extensions in the database copies.

Install the new server version

Perform the following operation for every application server into which xDM has been deployed.

When installing a new server version for an upgrade, pay attention to the versions of the application server and additional libraries.

Some libraries are compatible with a certain version of Java or the application server and will work with your current configuration. Do not systematically upgrade the application server and the libraries when upgrading Semarchy to the latest version. If you decide to upgrade these components to newer versions, make sure to review and test the compatibility and configuration.

To eliminate the need for configuring authentication at the application server level, Semarchy xDM version 5.3 introduced built-in support for third-party identity providers, enabling single sign-on (SSO) with LDAP, Google, Active Directory, Microsoft Entra ID (formerly known as Azure Active Directory), etc.
Consequently, the previous authentication libraries, such as Tomcat tools for authentication and role mapping (specifically, the com.semarchy.tool.jee.tomcat-<tomcat_version>.jar file) were no longer required.

However, to address issues caused by the removal of the JNDI Connection Factory, this object has been restored with a new naming format. The outdated JNDI Connection Factory JAR file (com.semarchy.tool.jee.tomcat-<tomcat_version>.jar) must be replaced with com.semarchy.tool.jee.tomcat_<xdm_version>.<build-date>-<git_commit>.jar to ensure seamless operation and prevent JMS notification server failures.

In-place upgrade

To install the new version as part of an in-place upgrade, deploy the new xDM web archive (WAR) file on top of the previous version.

For more information about re-deploying applications on top of previous installations, see your application server’s documentation. The instructions for Apache Tomcat are provided below.

To perform an in-place upgrade for Apache Tomcat.

Upload the new semarchy.war file to a temporary directory in the Tomcat server, for example /tmp (Unix/Linux) or c:\temp (Windows).
Make a copy of the existing configuration file (<tomcat>/conf/Catalina/localhost/semarchy.xml) to the same directory on the server.
Connect to the Apache Tomcat Manager (http://<tomcat_host>:<tomcat_port>/manager/).
In the applications list, note the context Path of the existing xDM application, and then click the Undeploy button to un-deploy it.
In the Deploy directory of WAR file located on the server section, type in the context path and enter the file paths to the semarchy.xml and semarchy.war files.
For example:
- Context Path (required): /semarchy.
- XML Configuration file URL: /tmp/semarchy.xml (Unix/Linux) or c:\temp\ semarchy.xml (Windows)
- WAR or Directory URL: /tmp/semarchy.war (Unix/Linux) or c:\temp\ semarchy.war (Windows)
Click the Deploy button.

Out-of-place upgrade

To install the new version as part of an out-of-place upgrade, deploy the new version xDM WAR file as a new deployment, in a new application server instance, separated from the previous version.

For more information on the deployment procedure, see your application server’s documentation.

When performing an out-of-place update, make sure to deploy the new version as a new deployment in a different application server instance. It is not possible to deploy multiple xDM applications in the same application server instance.

Configure the new server

Perform the following operation for each deployment of the new xDM version.

In-place upgrade

For in-place upgrades, the updated deployment reuses the configuration already in place. Check this configuration in the application server console or configuration files. The following points should be checked:

The startup configuration is set up for Semarchy xDM to connect the repository schema.

If upgrading from a version before 5.3, create the startup configuration and apply it to each of the servers where xDM is deployed.
If upgrading from a version before 5.3, the SEMARCHY_SETUP_TOKEN environment variable is set with the value that will be requested to authenticate and perform the repository upgrade.
If upgrading from a version before 5.3, make sure to remove unnecessary libraries used for SSO authentication and replace the outdated JNDI Connection Factory JAR file (com.semarchy.tool.jee.tomcat-<tomcat_version>.jar) with com.semarchy.tool.jee.tomcat_<xdm_version>.<build-date>-<git_commit>.jar.
The JavaMail session servers are configured and working.

Out-of-place upgrade

To configure the new version as part of an out-of-place upgrade:

Configure the startup configuration in the new deployment to connect to the copy of the original repository schemas (e.g., SEM_5_3_REPOSITORY).

If upgrading from a version before 5.3, make sure to create the startup configuration and apply it to each of the new servers where xDM is deployed.
If upgrading from a version before 5.3, set the SEMARCHY_SETUP_TOKEN environment variable to a value that will be requested to authenticate and perform the repository upgrade.
If upgrading from a version before 5.3, make sure to remove unnecessary libraries used for SSO authentication and replace the outdated JNDI Connection Factory JAR file (com.semarchy.tool.jee.tomcat-<tomcat_version>.jar) with com.semarchy.tool.jee.tomcat_<xdm_version>.<build-date>-<git_commit>.jar.
Make sure that the existing JavaMail session resource is configured or available for the new deployment.

At that stage, the copy of the original environment should be fully functional.

Start a new server instance

If upgrading from a version before 5.3.0, make sure that:

The Semarchy xDM startup configuration is set as required to connect to the repository.
The SEMARCHY_SETUP_TOKEN environment variable is set to a value that will be requested to authenticate and perform the repository upgrade.

Perform this operation for a single new deployed version instance. If you have multiple repositories, you should repeat this operation for each repository.
Start your application server or connect to your application server and start the deployed version.

Upon the initial startup, inspect the application logs for potential issues related to the startup configuration. Any misconfiguration will hinder xDM from starting, as it will not be able to connect to the repository schema.

Upgrade the repository

A repository upgrade is required only for major and minor versions. For patches, upgrading the repository is not necessary and the Repository Upgrade dialog does not appear when you connect.

Upgrading the repository is an operation that cannot be undone. Make sure you have made a backup of the repository schema before proceeding.

Before upgrading the repository

Versions before 5.3.0

If upgrading from a version before 5.3.0, or if upgrading to a release that includes changes to the identity management framework built in xDM, you will be prompted for a setup token to log in. The setup token is an arbitrary string that must be provided in a SEMARCHY_SETUP_TOKEN environment variable as part of the startup configuration.

Repository stored in Oracle

The Oracle database versions 11g and 12c have introduced a new feature called automatic column group detection that automatically creates extended statistics for column groups. These extended statistics prevent the upgrade process from renaming certain columns in the repository, causing an ORA-54032: column to be renamed is used in a virtual column expression error to appear during the repository upgrade process.

Before upgrading a repository hosted in an Oracle 11g or 12c instance, it is required to drop these statistics. They will be automatically re-created by the database as needed.

To drop unwanted extended statistics for repositories hosted in Oracle 11g and 12c:

Connect to the database hosting the repository schema with a query tool (SQL Developer, SQL*Plus), using the repository schema user.
Run the script provided below:

begin
  for xrec in (
    select
      TABLE_NAME, EXTENSION
    from
      USER_STAT_EXTENSIONS
    where
      TABLE_NAME like 'MTA%'
      and (
        EXTENSION like '%FROMEDITION%'
        or EXTENSION like '%TOEDITION%'
      )
  )
  loop

    DBMS_STATS.DROP_EXTENDED_STATS (
      ownname=>null,
      tabname=>xrec.table_name,
      extension=>TO_CHAR(xrec.extension)
    );

  end loop;
end;

Once the unwanted extended statistics are dropped, proceed with the repository upgrade.

Repository upgrade

To upgrade the repository:

Connect to the new instance you have started, using a user account with the SemAdmin role or using the setup token configured in the startup configuration.
At the first connection, the Semarchy xDM Upgrade dialog appears.
Review the dialog.
If upgrading from a version before 5.3, or if upgrading to a release that includes changes to the identity management framework built-in Semarchy xDM, need to provide an administrator username and password. This user is created or updated by the upgrade process with this password and granted the SemAdmin role.
Select I have made a backup of the repository and data location schemas and then click Upgrade Instance.

The repository upgrade starts.

The repository upgrade process can only be triggered by a user with the SemAdmin role, or when logged in with a setup token.

Upgrade data locations

A data location upgrade is required only for major and minor versions. For patches, a data location upgrade is not needed.

When a data location needs or is upgrading:

Users cannot access applications in the data location. Applications using the data location will appear disabled on the Semarchy xDM Welcome page.
Model designer cannot deploy model editions in the data location.

To reduce application downtime, it is recommended to plan the data location upgrade immediately after the repository upgrade.

The data location upgrade is an operation that cannot be undone. Make sure you have made a backup of the data location schemas before proceeding.

Development data locations upgrade is possible only if the current (latest) model state is deployed and if this model is valid. Before upgrading a development data location, make sure that the model does not raise any design-time validation errors.

Before upgrading data locations

Versions before 5.3.0

Semarchy xDM introduces a new mechanism to connect the backend databases, including those hosting data locations. Datasources are no longer configured at the application server level, but in the repository.

The repository upgrade automatically creates empty datasources for the components that refer to application server datasources. However, it does not recover the datasource configurations from the application server.

Referenced JNDI datasources are upgraded as empty platform datasources with normalized names. The normalized names are derived from the JNDI names by performing the following operations:

The java:comp/env/jdbc/ prefix is removed.
The remaining part is normalized by converting the characters that are not alphanumeric or underscore to _.

For example, in a data location, a reference to a JNDI datasource named java:comp/env/jdbc/DATA_LOCATION_1 is upgraded to a platform datasource named DATA_LOCATION_1. That datasource is not configured.

Before upgrading the data locations, administrators must reconfigure their datasources in xDM.

For a Tomcat installation, the original datasources configuration should be retrieved from the semarchy.xml file.

When re-configuring data location datasources, make sure to set the following technology-specific driver properties:

For Oracle datasources, the oracle.jdbc.J2EE13Compliant property must be set to true.
For PostgreSQL datasources, the defaultRowFetchSize property should be set to 2000.

Data locations stored in Oracle

Before upgrading a data location hosted in an Oracle 11g or 12c instance, it is required to drop the extended statistics that may prevent column renaming. For more information, see Before upgrading the repository.

To drop unwanted extended statistics, for each data location:

Connect to the database hosting the data location schema with a query tool (SQL Developer, SQL*Plus), using the data location schema user.
Run the script provided below:

begin
  for xrec in (
    select
      TABLE_NAME, EXTENSION
    from
      USER_STAT_EXTENSIONS
    where (
        TABLE_NAME like 'GI_%'
        or TABLE_NAME like 'GD_%'
        or TABLE_NAME like 'SD_%'
  or TABLE_NAME like 'MI_%'
      )
      and (
        EXTENSION like '%B_IS_VALIDATED%'
        or EXTENSION like '%B_STATUS%'
  or EXTENSION like '%B_HASSUGGMERGE%'
      )
  )
  loop

    DBMS_STATS.DROP_EXTENDED_STATS (
      ownname=>null,
      tabname=>xrec.table_name,
      extension=>TO_CHAR(xrec.extension)
    );

  end loop;
end;

Once the unwanted extended statistics are dropped, proceed with the upgrade.

Data location upgrade

To upgrade the data locations:

Connect to the new instance you have started, using a user account with the SemAdmin role or having a role with the Application Management privilege.
On the Welcome page, open the Application Builder.
Select the Management perspective.
A warning message appears in the Management view to give the list of data locations that need an update.
Click the link under this list to perform the upgrade.
The Data Location Upgrade dialog appears.
Review the dialog.
Select the checkbox and then click the Upgrade button.
The upgrade starts. When it completes, the window is refreshed, and the warning message disappears.

Restart all server instances

The upgrade process is complete.
You may now restart all instances of xDM in the new version and open access through the user interface, services, APIs, and database tables.