Enable Data Quality

Data Quality in Semarchy xDI is enabled through constraints that are defined on datastores and applied at run-time.

Define Constraints on Metadata

Constraints in Semarchy xDI are defined on datastores and include:

Constraints are automatically added by Reverse Engineering and can be added or modified manually.

To view and edit constraints on a datastore:

  1. Navigate to the concerned datastore from the Project Explorer or the metadata’s editor.

  2. When a column is selected, the Standard finger tab of the Properties view contains the data quality constraints that apply to this column:

Property

Description

Type

Datatype of the column.

Size

Maximum size. Unit depends on the value of Type (bits for numeric data types, digits for VARCHAR, etc.).

Decimal Digits

Maximum number of decimal digits. Automatically set to 0 for non-decimal numeric data types.

Is Nullable

Defines the default behavior if the column contains a null value. If this option is selected and if Reject Null Value is set to default for this datastore on a mapping, null values for this column will cause a reject.

Nullable Security Level

Defines if a null value should raise a Fatal error, a Reject or a Warning.

Primary, alternate, and foreign keys, as well as conditions, defined for the datastore are displayed at the same level as columns. For more details, please refer to the sections below.

Primary keys

Primary Keys uniquely identify each record in a datastore.

To add a Primary Key constraint to a datastore, proceed as follows:

  1. Right-click the datastore in the metadata’s editor and then select New > Primary Key.

  2. Enter a Name for the constraint.

  3. Optionally, set the following properties:

    • User Message: Defines the specific message to return if this constraint is violated.

    • Reject Code: Defines the specific code to return if this constraint is violated.

    • Severity Level: Defines whether a violation of this constraint should raise a Fatal error, a Reject or a Warning.

The columns that compose a primary key are defined using Column Reference child objects. To add a Column Reference to a new or existing Primary Key constraint:

  1. Right-click the new node added under the datastore for the new constraint and select New > Column Reference.

  2. In the Properties view, select the Primary Column you want to add to the primary key.

  3. Save the editor.

Foreign Keys

Foreign Keys define a referential integrity constraint on a datastore column towards another datastore.

To add a Foreign Key constraint to a datastore:

  1. Right-click the datastore in the metadata’s editor and then select New > Foreign Key.

  2. Enter a Name for the constraint.

  3. In the Referenced Datastore section, set the Schema and the Datastore targeted by this foreign key.

  4. Optionally, set the following properties:

    • User Message

    • Reject Code

    • Severity Level

The columns referenced by a Foreign Key are defined using Column Relation child objects. To add a Column Relation to a new or existing Foreign Key:

  1. Right-click the new node added under the datastore for the new constraint and select New > Column Relation.

  2. In the Properties view, select the Column from the current Datastore.

  3. Select the Referenced Column from the referenced datastore.

  4. Save the editor.

Alternate Keys

Alternate Keys define secondary keys that are composed of one or more columns. By definition, Alternate Keys imply a unicity constraint.

To add an Alternate Key constraint to a datastore:

  1. Right-click the datastore in the metadata’s editor and then select New > Alternate Key.

  2. Enter a Name for the constraint.

  3. Optionally, set the following properties:

    • User Message

    • Reject Code

    • Severity Level

The columns that compose an alternate key are defined using Column Reference child objects. To add a Column Reference to a new or existing Alternate Key constraint:

  1. Right-click the new node added under the datastore for the constraint and select New > Column Reference.

  2. In the Properties view, select the Primary Column to add to the alternate key.

  3. Save the editor.

Conditions

Conditions describe a custom constraint defined by a condition expression that will be evaluated at run-time.

To add a Condition constraint to a datastore:

  1. Right-click the datastore in the metadata’s editor and then select New > Condition.

  2. Enter a Name for the constraint.

  3. Optionally, set the following properties:

    • User Message

    • Reject Code

    • Severity Level

  4. Enter a Condition Expression.

Nullable values

Nullable values describe the behavior of Semarchy xDI when the field has a null value.

To add nullable constraints to a column:

  1. Select the column in the metadata’s editor.

  2. In the Standard finger tab of the Properties view, set the following properties:

    • Nullable User Message: Defines the message to return if this column is null.

    • Nullable Reject Code: Defines the specific code to return if this column is null.

    • Nullable Severity Level: Defines whether a null value should raise a Fatal error, a Reject or a Warning.

Configure And Execute Rejects

By default, constraints do not generate rejects. This must be configured on mappings or Processes.

Configure Rejects in Mappings

Rejects are configured on mappings to send to a Rejects table the records that do not meet the constraints defined on a target datastore. For more details, refer to Rejects Management.

Execute Rejects in processes (Standalone Mode)

Rejects that are activated on a mapping apply every time this mapping is executed in a data flow. Additionally, it is possible to check constraints and generate rejects in processes without loading data into the target datastore. This procedure is designated as standalone mode and is enabled by adding a Reject step to processes.

For more details, rejer to Work with Processes