Semarchy xDI 2024.1 release notes

This document provides the latest information about Semarchy xDI version 2024.1, including new features and bug fixes.

Support information

Semarchy xDI version 2024.1 is a long-term support (LTS) release.

LTS product versions are released annually, and include all changes introduced since the last LTS release. They are maintained and supported for an extended period with patches and fixes.

Alternatively, mainstream support (MS) versions offer the latest improvements and features, and are released more frequently.

For more details about our releases and support policies, see our Support and maintenance terms.

Feature highlights

Semarchy xDI 2024.1 consolidates features and fixes from prior Mainstream Support versions, and adds new features and bug fixes as well. For more information about updates in versions from this past year, refer to these release notes:

Read on for features and changes specific to version 2024.1.

Serializers

xDI 2024.1 introduces serializers, which aid users to transform and integrate complex hierarchical data structures into single text columns. Like with deserializers, users will find an intuitive drag-and-drop interface to add serializers to mappings, and fluid prompts to configure how the serializer functions.

Version 2024.1.0 comes with the JSON as RDBMS Strings serializer, with more to come soon.

New deserializer templates

Semarchy xDI expands on deserializers with dedicated templates to deserialize XML data contained in Greenplum and PostgreSQL database columns.

Runtime metrics, monitoring, and health check

xDI Runtime adds the ability to monitor the Runtime with metrics and health information. This feature gives administrators an overview of the xDI infrastructure health, letting them detect bottlenecks and potential issues.

Expressions with target data

xDI Designer now allows you to use columns from target datastores in mapping expressions when their execution location is set to Target. This feature allows for data manipulation and processing after data has reached its target, removing some previous processing limitations.

New connectors

Azure Service Bus

Semarchy xDI brings initial support for Azure Service Bus, a cloud messaging system for connecting apps and devices across public and private clouds.

Microsoft Azure Cosmos DB

Semarchy xDI adds support for Azure Cosmos DB, allowing you to integrate Microsoft’s multimodel database service with any other supported source or target technology.

This new connector supports the MongoDB and PostgreSQL data models.

Updated connectors

Snowflake - Snowpipe Streaming API

The Snowflake connector has been updated with support for the Snowpipe Streaming API, making it possible to continuously stream rows of data into Snowflake tables.

Elasticsearch version 8

Elasticsearch version 8 is now supported.

Other notable changes

System requirements

Semarchy xDI 2024.1 adds official support for the latest Long Term Support version of Java, Java 21 LTS. Java 17 LTS remains supported.

Support has been added for the following Runtime and Analytics backend databases:

  • Oracle Database 21c

  • PostgreSQL version 15

  • PostgreSQL version 16

  • Microsoft SQL Server 2022.

The following versions are no longer supported as Runtime and Analytics backend databases:

  • Oracle Database 18c and below

  • PostgreSQL versions 11 and below

  • Microsoft SQL Server 2017.

Review Semarchy xDI system requirements here.

Improved sub-metadata support

Sub-metadata support has been expanded in connectors. The Couchbase connector now supports sub-metadata at the cluster and scope level, and the HTTP Rest connector supports it for root and path nodes.

Runtime IPv6

The xDI Runtime now supports IPv6 connections out-of-the-box. This change also applies to access restriction configurations, which now accept IPv6 in single address or CIDR format.

Java JMX now on by default

Java Management Extensions (JMX) are now re-enabled by default. See Configure Java options.

OAuth2 improvements

OAuth2 code has undergone a refactoring to speed up its implementation in other connectors.

Deprecated features

  • This release deprecates some environment variables from older xDI versions. They should be replaced with their current XDI-prefixed equivalents. See Deprecated environment variables.

  • The option to disable certificate checking in Amazon S3 tools has been deprecated, as it is no longer exposed in the latest Amazon SDK.

Removed features

Support for AUTONOMOUS SQL transactions has been removed due to unreliable SQL behavior. In older projects that are still configured to use this transaction type, xDI will interpret those transactions as AUTOCOMMIT instead.

Changelog

2024.1.11

Semarchy xDI Production Analytics

Bug fixes

  • DI-10848: Fixed an issue preventing xDI Analytics from using a custom security key through the REST API.

  • DI-10851: Fixed occasional package upload failures when using the PATCH method with the REST API.

  • DI-10915: Fixed an issue preventing xDI Analytics from using a custom keystore when keystore and key passwords do not match.

Semarchy xDI Designer

Feature improvements

  • DI-10325: Updated internal labels in mappings so Semarchy Data Intelligence can correctly capture asset names and compute siblings.

Bug fixes

  • DI-10092: Fixed an issue that created incomplete metadata definitions when reverse-engineering Vertica tables.

Semarchy xDI Runtime

Bug fixes

  • DI-9303: Template INTEGRATION RDBMS to Json Lines: fixed the No Data Behavior parameter so its createEmptyFile setting works.

  • DI-9957: Process action Get Files with SCP: fixed an issue that prevented the action from populating the ind_session_file_op_lst table.

  • DI-10193: Fixed an issue that caused SQL actions to ignore error tolerance mode with some database technologies.

  • DI-10260: Process action SQL Operation: fixed an issue that blocked current and future sessions after running a SELECT operation without a bind link.

  • DI-10541: Updated Spring Framework third-party libraries.

  • DI-10547: Fixed incorrect statistics when using Kafka Raw metadata in transactional mode.

  • DI-10824: Updated Spring Framework third-party libraries.

  • DI-10941: Fixed a java.lang.IllegalAccessException error with HDFS tools.

License service

Bug fixes

  • DI-10543: Updated Spring Framework third-party libraries.

2024.1.10

Semarchy xDI Designer

Bug fixes

  • DI-9884: Fixed an issue causing the Google Sheets wizard to fail.

  • DI-9901: Fixed an issue that caused process errors when using md:formatDate in a metadata field to format date strings.

  • DI-10310: Fixed an issue which prevented integration flows from writing files when the file name or path contains ampersand (&) characters.

Semarchy xDI Runtime

Bug fixes

  • DI-10227: Updated Apache Tomcat third-party libraries.

  • DI-10296: Fixed an issue that sent incorrect status display information for integration flows, whether or not they were successful.

  • DI-10297: Fixed an issue preventing the runtime configuration parameter soapServerThreadPoolSize from working. A parameter concurrentSessionsLimit has been added as an alias, deprecating soapServerThreadPoolSize.

  • DI-10317: Fixed a NullPointerException error when producing messages with empty field values in a Kafka topic.

  • DI-10472: Fixed an issue that caused target datastores to have missing data when using Serializers with complex, nested JSON structures.

2024.1.9

Semarchy xDI Production Analytics

Bug fixes

  • DI-9959: Updated README.txt files with the latest URLs.

  • DI-10152: Fixed a javaNullPointer exception when migrating a Deployment Manager to a Delivery Project.

Semarchy xDI Designer

Bug fixes

  • DI-9754: Fixed an issue that prevented reading source RDBMS tables when writing to SAP.

  • DI-9888: Fixed an error that occurred when consulting data from a MariaDB table.

  • DI-9959: Updated README.txt files with the latest URLs.

  • DI-9960: Fixed an issue that erased notes and layouts when copying and pasting mappings that were imported from legacy software versions.

Semarchy xDI Runtime

Known issues

  • DI-9901: Using the md:formatDate expression in a metadata field causes a process to fail with an error.

Bug fixes

  • DI-9578: Fixed an error when running the startdelivery script with Java version 21.

  • DI-9932: Fixed an issue that prevented Execute Delivery process actions from working when memory mode was off.

  • DI-9959: Updated README.txt files with the latest URLs.

2024.1.8

Semarchy xDI Designer

Bug fixes

  • DI-9282: Fixed an issue with the metadata search dialog which prevented categories from appearing.

  • DI-9732: Fixed an issue where deleting projects did not actually remove all their files on disk.

Semarchy xDI Runtime

Known issues

  • DI-9901: Using the md:formatDate expression in a metadata field causes a process to fail with an error.

Bug fixes

  • DI-9891: Fixed an issue that stopped scripting actions from working when using a JavaScript string replace function.

  • DI-9899: Process action Operating System Command: fixed an issue causing a parameter to be mandatory although it should be optional.

  • DI-10120: Fixed an issue that prevented CLOB fields from loading properly in processes.

2024.1.7

Semarchy xDI Designer

Bug fixes

  • DI-9307: Fixed query generation when a datastore alias is a reserved keyword. The generated delivery now adds a prefix to the alias, and generates a working query.

  • DI-9971: Fixed an issue that generated incorrect SQL queries due to truncated field names in stages.

Semarchy xDI Runtime

Bug fixes

  • DI-9302: Template INTEGRATION Rdbms to JSON: fixed an issue that generated invalid output when concatenating multiple JSON objects into a JSON array.

  • DI-9646: Fixed SOAP request authentication with some SSL configurations.

Semarchy xDI Production Analytics

Feature updates

  • DI-6363: Added the ability to specify a username and password when initializing a database with the autoinit button.

Bug fixes

  • DI-8770: Fixed an issue which prevented multiple packages from being imported at the same time.

  • DI-9006: Fixed Open API definition to mention required authentication.

  • DI-9543: Updated Apache Commons third-party libraries.

2024.1.6

Semarchy xDI Designer

Bug fixes

  • DI-9501: Fixed an issue where the Last Build menu entries did not respect the build folder location.

  • DI-9669: Fixed an issue that hid project impacts and dependencies until after a full cache rebuild.

  • DI-9744: Fixed an issue that kept a process’s default web service invocation path active after defining a custom path.

  • DI-9748: Fixed an issue preventing arrow icons from appearing when performing an impact analysis on a table.

  • DI-9758: Fixed an issue that prevented the CLI from building packages on a first attempt.

  • DI-9895: Improved performance when importing or using a workspace from legacy software versions.

Semarchy xDI Runtime

Feature updates

  • DI-9638: Template LOAD File: added the ability to ignore empty lines at the end of a file.

Bug fixes

  • DI-9521: Fixed an issue that prevented the Raise Error action from working when executing a delivery.

  • DI-9627: Fixed an issue that prevented files with wildcards in their name from being loaded.

  • DI-9642: Fixed an issue where xDI Runtime child sessions showed the wrong status until the main session finished.

  • DI-9647: Fixed an issue that prevented an error code from showing when the runtime log database was unavailable.

  • DI-9744: Fixed an issue that kept a process’s default web service invocation path active after defining a custom path.

  • DI-9784: Updated Jython third-party libraries.

  • DI-9796: Updated Spring third-party libraries.

  • DI-9857: Template INTEGRATION FILE (FileExport): fixed an issue preventing null values from being processed correctly.

  • DI-9915: Fixed an issue with bind links that caused processes to run indefinitely.

  • DI-9952: Template INTEGRATION Rdbms to BigQuery: fixed an issue that stopped Snowflake to BigQuery mappings with a ClassNotFoundException error.

License server

Bug fixes

  • DI-9670: Fixed an issue that used up too many license validation tokens, when using Java versions 17 and 21 at the same time on the machine running xDI Designer.

  • DI-9863: Updated Spring third-party libraries.

2024.1.5

Semarchy xDI Designer

Bug fixes

  • DI-8789: Fixed an issue preventing the Show Used Metadata setting from working.

  • DI-9457: Fixed an issue which made metadata link selector buttons open a process list instead.

  • DI-9488: Fixed an issue which cleared reference nodes when doing a reverse engineer operation on existing metadata.

  • DI-9545: Fixed an issue with mappings where stages with multiple sets did not generate the necessary SQL clauses.

  • DI-9648: Fixed an issue that corrupted processes when merging them with the EMF Compare tool.

  • DI-9652: Fixed incorrect SQL queries or unreported SQL syntax errors due to incorrectly parsed filters.

Semarchy xDI Runtime

Bug fixes

  • DI-6426: Fixed an issue with looping SQL actions which made statistics cumulative instead of having separate statistics per step.

  • DI-8904: Updated amqp-client third-party libraries.

  • DI-9519: Updated Apache Commons Compress third-party library.

  • DI-9549: Fixed an issue that prevented the buildDelivery script from working when the name of the configuration file contains spaces.

  • DI-9652: Fixed incorrect SQL queries or unreported SQL syntax errors due to incorrectly parsed filters.

  • DI-9660: Fixed an issue that prevented BLOB fields from loading properly in processes.

  • DI-9666: Updated Spring third-party libraries.

2024.1.4

Semarchy xDI Designer

Bug fixes

  • DI-7854: Fixed an issue which prevented HTTP REST or WSDL authentication with some SSL configurations such as Mutual SSL authentication.

  • DI-9417: Template INTEGRATION File: Fixed an issue which prevented Designer from creating new files when the Truncate Target File option was enabled.

  • DI-9554: Fixed an issue which slowed down prepare or build operations when extracting metadata from Semarchy xDG.

Semarchy xDI Runtime

Bug fixes

  • DI-7854: Fixed an issue which prevented HTTP REST or WSDL authentication with some SSL configurations such as Mutual SSL authentication.

  • DI-9507: Fixed an issue which prevented integration flows from writing files when the file name or path contains ampersand (&) characters.

  • DI-9537: Action Execute SSH commands: fixed an issue which made the action succeed when the SSH login information was incorrect.

  • DI-9548: Updated Spring Web third-party libraries.

License server

Bug fixes

  • DI-9547: Updated Spring Web third-party libraries.

2024.1.3

Semarchy xDI Runtime

Bug fixes

  • DI-9546: Fixed a performance regression on the SQLFileExport process action.

2024.1.2

Semarchy xDI Designer

Bug fixes

  • DI-9272: Fixed an issue that made jobs fail when publishing multiple deliveries at the same time.

  • DI-9274: Fixed an issue preventing XPath expressions from working in a process with a repeated query.

  • DI-9308: Fixed an issue which showed an error when creating an OAuth2 entry for login.microsoftonline.com.

  • DI-9385: Added the ability to disable hostname verification with later Java versions when communicating with the license server.

  • DI-9423: Fixed an issue which prevented multiple deliveries from being built with the CLI when they require multiple user-defined functions.

  • DI-9445: Fixed the error Linked workspace operation failed when renaming Mappings.

  • DI-9476: Fixed an issue which prevented processes from referencing temporary objects.

Semarchy xDI Runtime

Features and improvements

  • DI-9347: Added the ability to escape additional characters in string data, allowing you to process strings with characters such as commas or newlines.

Bug fixes

  • DI-8922: Template Load XML to Hive: fixed an error when string data contains special characters.

  • DI-8932: Removed sensitive technical Java information from the API error response.

  • DI-9422: When access over HTTP is enabled, fixed an issue which blocked connections to the runtime when one allowed host was unreachable.

  • DI-9459: Fixed an issue which added a header to an exported SQL file when a header was not requested.

  • DI-9475: Updated Spring Framework third-party libraries.

Semarchy xDI Production Analytics

Bug fixes

  • DI-9451: When running Analytics in Docker, fixed a java.io.FileNotFoundException error when opening the schedule editor.

License server

Features and improvements

  • DI-9514: Added the ability to stop Java from checking certificate SNI hosts. You can use the disableSNIHostCheck parameter to control this behavior.

Bug fixes

  • DI-9464: Fixed an issue preventing license validation when running multiple instances of xDI Designer on a single machine.

  • DI-9472: Updated Spring Framework third-party libraries.

2024.1.1

xDI Designer

Bug fixes

  • DI-9273: Fix execution errors when installing to a path that contains spaces.

  • DI-9276: Fix java.lang.ClassNotFoundException: com.indy.engine.uberbuilder.executor.UberRunner$Executor error when executing an uber jar file.

  • DI-9278: Fix CLI builds of multiple deliveries at once with multiple user-defined functions.

  • DI-9309: Fix issue with OAuth2 redirect URL when generating a token.

  • DI-9327: Fix CLI package extraction and error logging.

xDI Runtime

Bug fixes

  • DI-8318: Fix loading rows with the RDBMS Replicator when the source is a multi-record file.

  • DI-9242: Stop reporting irrelevant session errors.

  • DI-9251: Support PPK v3 private key files in SSH server metadata.

  • DI-9262: Fix string delimeters for empty values in the SQL FILE EXPORT process action.

  • DI-9276: Fix java.lang.ClassNotFoundException: com.indy.engine.uberbuilder.executor.UberRunner$Executor error when executing an uber jar file.

  • DI-9335: Fix 403 error when calling a delivery with the /nonwsi SOAP endpoint.

2024.1.0

xDI Analytics

Features and improvements

  • DI-8203: Suppress overly verbose information from Tomcat startup logs.

  • DI-8426: Certify PostgreSQL versions 15.x and 16.x for use as Semarchy xDI backend databases.

  • DI-8600: Upgrade Eclipse target platform.

  • DI-8711: Add support for Java 21 LTS.

  • DI-8767: Certify Microsoft SQL Server 2022 for use as a Semarchy xDI backend database.

  • DI-8963: Certify Oracle version 21c for use as a Semarchy xDI backend database.

Bug fixes

  • DI-8831: Upgrade JSON third-party libraries.

  • DI-9036: Fix page access over OpenAPI 3 URL.

xDI Designer

Features and improvements

  • DI-3419: Add support for field expressions that reference target columns when the execution location is set to Target.

  • DI-5353: Add support for Elasticsearch 8.

  • DI-7621: Implement serializer functionality in Designer. Use the serializer in mappings to parse structured data from a source and flatten it to string data.

  • DI-8601: Upgrade eclipse target platform.

  • DI-8710: Add support for Java 21 LTS.

Bug fixes

  • DI-4858: Fix which icons are displayed when a connector technology inherits properties from another one.

  • DI-5757: Fix dragging and dropping inside a metadata reference field on Linux and macOS.

  • DI-7328: Fix inability to create a stage or deserializer when dragging and dropping a schema into a mapping, on Linux instances of Designer.

  • DI-7973: Fix the Metadata Process Palette so the Process Reference parameter selector displays processes instead of metadata only.

  • DI-8441: Refactor HTTP proxy handling; fixes various HTTP errors.

  • DI-8522: Fix removing or replacing output structure in a deserializer.

  • DI-8544: Correctly display only available metadata when opening the metadata creation wizard.

  • DI-8730: Fix error logged when selecting a Deserializer field.

  • DI-8737: xDI Designer freezes when moving items in the project explorer.

  • DI-8783: Handle special characters correctly when using the "WKS:/" prefix to select files in CLI mode.

  • DI-8816: Fix invalid query generated for a mapping when a datastore or stage alias is a reserved word.

  • DI-8872: Executing an Uber runtime does not generate a mapping execution log.

  • DI-8916: Handle directory paths correctly when they contain escape characters such as \t.

  • DI-8966: Missing element IDs may cause errors when copying a mapping.

  • DI-9029: Show an error when a datasource staging area is defined in a mapping, but the component is not installed.

  • DI-9184: Fields and descriptors are deleted when unmapping an expression on a stage.

xDI Runtime

Features and improvements

  • DI-7889: Allow parameters in the scheduler configuration file to properly use environment variables.

  • DI-7945: Add new connector for Azure Service Bus, a cloud messaging as a service platform.

  • DI-8107: Enable IPv6 support in the Runtime by default.

  • DI-8426: Certify PostgreSQL versions 15.x and 16.x for use as Semarchy xDI backend databases.

  • DI-8478: Update Rhino third-party library.

  • DI-8713: Add support for Java 21 LTS.

  • DI-8738: Generate correct URL to access Runtime Swagger/OpenAPI definitions behind a load balancer.

  • DI-8767: Certify Microsoft SQL Server 2022 for use as a Semarchy xDI backend database.

  • DI-8963: Certify Oracle version 21c for use as a Semarchy xDI backend database.

  • DI-8964: Add health and status monitoring capabilities to xDI Runtime.

Bug fixes

  • DI-7370: Remove AUTONOMOUS SQL transaction type, as it has no effect.

  • DI-8451: Remove unused metadata parameter from the file driver JDBC URL.

  • DI-8524: Failed processes do not always close their database connections properly.

  • DI-8681: Delivery cannot be deleted if the internal scheduler is not started.

  • DI-8702: Update Spring third-party library.

  • DI-8703: Fix unclosed statement in exception block.

  • DI-8733: When a runtime is configured to use synchFirstLog mode with log level 0, Java session threads cannot close.

  • DI-8741: Executing a command with the operating system command action takes longer than executing the same command externally.

  • DI-8791: Fix connection pooling error due to SQL To Parameters action failure.

  • DI-8806: Fix runtime connection leak when using autocommit connections.

  • DI-8817: Re-enable JMX by default on the xDI Runtime. This allows xDI Runtime monitoring in environments such as Docker.

  • DI-8835: Correctly parse configuration files with multi-line properties when generating a delivery.

  • DI-8908: JDBC connection properties are cached beyond their expected expiration period.

  • DI-9030: Fix mapping build failure when using a WSDL metadata as target.

  • DI-9053: Update Spring Boot third-party libraries.

  • DI-9054: Update Spring Framework third-party libraries.

License Server

Features and improvements

  • DI-8712: Add support for Java 21 LTS.

Bug fixes

  • DI-8687: Update Jetty third-party library.

  • DI-8865: Update Spring Boot third-party libraries.

  • DI-9049: Update Spring Framework third-party libraries.