Semarchy xDM 3.3 release notes

Feature highlights

Semarchy xDM 3.3 introduces new features for companies to design and use value-driven master data applications. This version enhances the core MDM platform capabilities with matching on related entities, enrichment-validation-matching as a service, continuous loads, and SemQL ID generation.

Matching on child and parent entities

This version improves the match and merge capabilities of Semarchy. With this update to the matcher, designers can automate matching records using related (parent and child) records. For example, match 2 stores using fuzzy logic on the name, parent company and manager name, or match two contacts when one of their emails matches exactly.

With this major enhancement, Semarchy xDM lets you use all the relationships from the hub to increase the accuracy of your match and merge rules.

Enrichment, validation, matching as a service

The web service interface for integration, generated from the logical model in Semarchy xDM, makes complete use of the rules stored in the model and enables integration architects to run enrichment, validation and matching rules against data that is being integrated into the MDM hub. The web service also supports validating records in dry runs, without actually modifying data in the hub.

This guarantees the perfect quality of the data integrated into the MDM hub through the web service interface. Via this feature, a third-party system can leverage the MDM platform as a provider of data enrichment, standardization, quality and de-duplication services.

Continuous loads

Continuous loads enable integration developers to push data into the MDM hub in a continuous way without having to take care of Load Initialization or Load Submission. At regular intervals, Semarchy xDM automatically processes data loaded in the continuous loads.

This new feature dramatically simplifies and accelerates low-latency data integration flows targeting the MDM hub.

SemQL ID generation

The generation of the source and golden IDs is now reorganized and enhanced to support sequences, UUIDs, manual data entry as well as IDs generated from a SemQL expression.

Using this feature, designers have the flexibility to provide meaningful and customized identifiers that perfectly fit the requirements of the business.

Design-time

  • The new Lookup Enricher performs a data lookup on any mapping table stored in a local or remote database.

  • The Plugin Enrichers Aggregation (enabled with job parameters) enables designers to run subsequent plugin enrichers in a single in-memory processing chain, avoiding successive database read/writes.

  • A new Change Indicator appears in the stewardship application to indicate when a record from an ID matching entity is being updated, or if a data entry workflow contains an obsolete version of that record.

  • A new queryLoadInformation web service operation to query loads and batches.

  • The post-consolidation validation can now use child records data.

  • The reference picker SemQL filter is enhanced to support the reference value being set as well as attributes from the referenced master record when the referenced entity is fuzzy matching.

  • The SemQL order by clauses for custom ranking and additional order by now support the NULLS LAST and NULLS FIRST syntax.

Changelog

Version 3.3.10

Bug fixes

  • MDM-9112: Version 3.3.9 fails to deploy on Oracle Weblogic 12.1.x.

  • MDM-9044: Job log purge fails with a NullPointerException

Version 3.3.9

Bug fixes

  • MDM-8672: Version 3.3.8 incorrectly requires a JDK 8 to run.

  • MDM-8862: A memory leak causes performance issues when a large number of users access the welcome page.

  • MDM-7826: A model edition switch causes 100% CPU usage and fails for large data hubs.

Version 3.3.8

New features

  • MDM-6960: Google Maps Enricher plug-in now supports using an API Key in addition to the Client ID and Signature.

  • MDM-8214: New Usage Limit parameter for the Google Maps Enricher limits the number of requests per second.

Version 3.3.7

Bug fixes

  • MDM-6170: Job deployment fails with a java.sql.SQLException: ORA-01461 exception when custom hints (PARAM_CUSTOM_HINTS) length exceeds 4000 characters

Version 3.3.6

New features

  • MDM-5940: Golden integration records are navigable from the post integration (golden error) records in SemQL.

Bug fixes

  • MDM-5719: Hint generated in SQL code at an incorrect position in the presence of specific characters.

  • MDM-5921: Google Translate Enricher plugin fails to run if Source Language is not set.

  • MDM-6073: Golden Record Masters Count value becomes inconsistent with actual number of master records under certain conditions.

Version 3.3.5

New features

  • MDM-4843: Support for table functions in SemQL conditions. For example:`CustomerId in MY_TABLE_FUNCTION(X, Y)`.

Bug fixes

  • MDM-4811: Engine fails to restart after an OutOfMemoryError caused by an enricher during a job execution.

  • MDM-5089: "IllegalArgumentException: Multiple entries with same key" error while generating a job definition with a match rule having an attribute used multiple times for binning.

  • MDM-4838: Seemingly random "Graphic is disposed" error when trying to view model or workflows diagrams.

  • MDM-5210: User logout does not remove sessions objects from memory on Oracle WLS 12c.

Version 3.3.4

New features

  • MDM-4678: Model variables can be set as job parameters and used in enricher, validation, match and consolidation SemQL expressions.

Bug fixes

  • MDM-4750: Application loading time optimization.

  • MDM-4186: Manually suspended jobs cannot be canceled or restarted when a notification is configured.

  • MDM-4613: Matching process of fuzzy-matching entities is broken if an entity uses inheritance.

Version 3.3.3

New features

  • MDM-4529: Generic JNDI/JMS Connector for the Tomcat Server.

  • MDM-4271: Lookup enricher now supports using the Lookup Value as the Fallback Value.

Bug fixes

  • MDM-4443: Typo ("Tast") in Workflow Task and Workflow Task Duration dashboards.

  • MDM-4395: In the Overview page, the Model Design and Data Stewardship blocks are incorrectly positioned.

  • MDM-4380: Upgrade fails with "SWTException: Device is disposed" when the client session is closed.

  • MDM-4547: Person Name Enricher suggests incorrect first/last name inversions.

  • MDM-4446: Running Pulse Profiling fails with NoClassDefFoundError on spring ConnectionCallback

  • MDM-4582: New continuous load’s Submit Interval is set to 10 and cannot be modified.

  • MDM-4588: Plugin Enricher fails when aggregation is enabled and a condition is set.

Version 3.3.2

Bug fixes

  • MDM-3925: Data location upgrade to version 3.2 is very slow.

Version 3.3.1

Bug fixes

  • MDM-4306: "Align merged Exclusion Groups Ids" task merges too many exclusion groups.
    These exclusion groups cause large number of records to be automatically added to duplicate management workflows. This fix changes the integration jobs generation to prevent excessive merges of exclusion groups. For this fix to be effective, you must re-deploy the model in order to force the re-generation of the integration jobs.

  • MDM-4293: War deployment fails in Oracle WebLogic server with an "java.lang.IllegalStateException: Proxy servlet not properly initialized. httpServiceRuntimeImpl is null" error.

  • MDM-4327: Unable to browse an ID matching entity with an ID attribute using UUID. A "com.semarchy.commons.sql.exception.SQLCoreException: Error while binding parameter xxx with value …​" is raised.

  • MDM-4330: Clicking actions in the Workbench cause "java.lang.RuntimeException: General Error: Actions and ActionDefs are out of sync in the cache…​" errors.

Version 3.3.0

Known issues

  • MDM-4230: Repository, data location or Pulse Metrics warehouse upgrade fails with an ORA-54032: column to be renamed is used in a virtual column expression error.
    This error is caused by a new feature in Oracle version 11g and 12c called Automatic Column Group Detection that automatically creates extended statistics, preventing the upgrade process to rename certain columns in the schemas. Before upgrading a repository, data location or Pulse Metrics warehouse hosted in an Oracle 11g or 12c, it is required to drop these statistics. The procedure for removing these statistics is explained in the upgrade process.

Breaking changes

  • MDM-3826: Apache Geronimo dropped from the certified application servers list.

  • MDM-4011: Integration web service generateXXXIds operation now requires a LoadID.

New features

  • MDM-1025: Matching on referenced attributes.

  • MDM-1179: Consecutive plug-in enrichers can be aggregated for single pass execution.

  • MDM-1407: Documentation includes a naming convention.

  • MDM-1588: Plug-in enricher for data lookup.

  • MDM-1843: Documentation includes chapter for advanced datasources configuration for application servers.

  • MDM-2447: Documentation links to the index from every documentation page.

  • MDM-2526: HAVE and HAS are treated synonyms in SemQL ANY/ALL clauses.

  • MDM-2874: Change indicator appears on records in the data entry workflow when the record is modified in another workflow or external load (ID matching entity).

  • MDM-3080: New queryLoadInformation web service operation to query loads and batches.

  • MDM-3254: Source ID generation supports SemQL expression for data entry, import and integration web services.
    For fuzzy matching entities, a different ID generation scheme can be specified for source and golden IDs.

  • MDM-3480: Generated physical model documentation includes a toggle to show/hide internal (non-API) tables and columns.

  • MDM-3565: Matching on child records.

  • MDM-3567: Certification Process changes from a "by entity" to a "by Phase" execution scheme.

  • MDM-3570: Post-consolidation validation can use child records data.

  • MDM-3767: Default value for "Auto-confirm golden record" is now set to 99.

  • MDM-3771: Reference Picker Filter allows using the reference value being set.

  • MDM-3799: Reference Picker Filter supports Referenced.MasterRecord when the referenced entity is Fuzzy Matching.

  • MDM-3880: SemQL order by clauses for custom ranking and additional order by support NULLS (LAST | FIRST) syntax.

  • MDM-3888: High-availability section of the installation guides clarifies the need for sticky sessions for AWS setup.

  • MDM-3947: Fuzzy matching entities support sequences in addition to UUID for generated Source IDs.

  • MDM-3949: Integration web services support triggering enrichers, validations and duplicate detection and optional data persistence.

  • MDM-3951: Continuous loads enable pushing data in a continuous way without load initialization or submission.

  • MDM-3964: Administration guide explains logging (Log4J) configuration.

  • MDM-3965: New section in developer’s guide explains the model development lifecycle

  • MDM-4165: New DupsCheckoutCause built-in attribute in duplicate management workflows to filter records checked out by the user and the system.

  • MDM-4249: Validation and Enricher Plug-in editor only shows mandatory parameters and inputs by default.

Bug fixes

  • MDM-3594: A suspended job does not appear in red automatically in the Administration Console without "moving" the focus to another line.

  • MDM-3758: Test configuration values incorrectly set in the job notification policy test configuration.

  • MDM-3761: "Unknown BusinessObjectViewId" error raised on workflow transition between two tasks that use different business object views.

  • MDM-3762: Business object views disappear from the data steward application in Chrome on Mac OS.

  • MDM-3807: When moving the last publisher upwards in the "Add Publisher Ranking" wizard, the last Publisher disappears from the list.

  • MDM-3938: Client error in the data steward application when reordering columns in the "Select Displayed Attributes" dialog.

  • MDM-4027: Only a single email is sent when a data location has multiple job notification policies.

  • MDM-4028: Validation error when the PARAM_CC parameter is removed from a notification policy.

  • MDM-4052: A FDN link to an entity which display name contains a pipe (|) is not displayed properly in a form view : the link label is truncated.