Develop your plugin

This page provides step-by-steps instructions to develop an xDM plugin.

Create the plugin project

A Semarchy xDM plugin is developed as part of a plugin project, into which you declare extension points for each plugin contained in this project.

Create a new plugin project

A plugin project contains one or more Semarchy xDM plugins and can be deployed as a single jar.

To create a new plugin project:

  1. In Eclipse, select File > New > Others….

  2. In the Wizards filter, enter Plugin. The list shows the Plugin Project wizard.

  3. Select the Plugin Project wizard and then click Next

  4. In the Project Name, enter the name of your project (e.g., com.acme.phoneStandardizer). Leave the other fields as is.

  5. Click Next. In the Content page, change the following fields:

    • Version: version of your plugin (e.g., 1.0.0)

    • Name: Visible Name for your plugin (e.g., Phone Standardizer)

    • Provider: Your company’s name (e.g., ACME Corp.)

    • Un-select the Generate an activator, This plugin will make contributions to the UI and Enable API analysis options.

    • Make sure the Would you like to create a rich client application? option is set to No.

  6. Click Next

  7. Un-select the Create a plugin using one of the templates option.

  8. Click Finish

  9. If asked, click Yes to switch to the development perspective.

Add the extensions to the project

To define Semarchy xDM extension points, you need to add these extension points' types to the project.

To add the Semarchy xDM extension points to the project:

  1. In the plugin editor, select the Dependencies tab

  2. In the Required Plugins, click Add and then select the com.semarchy.engine.extensionpoint.
    The extension point is added to the dependencies.

  3. The extension point is added with a minimum version required equal to the version of the plugin SDK. For example, if you have installed the SDK in version 3.1.0, the minimum server version required is 3.1.0.
    To make your plugin compatible with previous minor versions of the server (for example, to install the plugin on a 3.0.0 server), you must modify this dependency.

    1. Select the extension point in the list and then click the Properties… button.

    2. In the com.semarchy.engine.extensionpoint dialog, edit the Minimum Version field (e.g., replace 3.0.1 with 3.0.0 to make this plugin dependent on any server version after 3.0.0).

    3. Click OK to close the dialog.

  4. Select the Overview tab.

  5. In the Overview tab, click the Extensions hyperlink. The editor switches to the Extensions tab.

  6. In the Extensions tab, click Add…, and select the extension point corresponding to the type of plugin you want to deliver in this project:

    • com.semarchy.integration.extensionpoint.rowTransformer: This extension point type is used to create enrichers.

    • com.semarchy.integration.extensionpoint.rowValidator: This extension point type is used to create validations.

  7. Repeat the previous step to add both types of extensions if you want to mix enrichers and validations in your project.

Define the plugin

A project can contain several Semarchy xDM plugins (for example, a plugin to enrich telephone numbers, and another one to validate email addresses). Each plugin is declared as an extension endpoint of the rowTransformer or rowValidator endpoint type.

An extension endpoint is declared with all the plugin metadata: The name, version, provider, as well as the parameters, input and output fields definition.

Declare an extension endpoint

For the rest of this example, you will focus on creating an enricher (Row Transformer). Validation plugins follow the same development path.

To create an extension endpoint:

  1. Right-click the com.semarchy.integration.extensions.rowTransformer extension you just added, right-click and select New > Row Transformer.

  2. In the Extension Element Details section, enter the following values:

    • Id: ID of the enricher/validation (e.g., com.acme.phoneStandardizer.intlPhoneStandardizer). Note this ID as it will be used later in the test cases.

    • Name: text name of the enricher or validation (e.g., International Phone Standardizer)

    • Provider Name: your company’s name (e.g., ACME Corp.)

    • isThreadSafe: set this option to true if you know that your code is thread-safe. Otherwise, set it to false. For more information about this option, see Parallel execution and thread safety.

  3. Click the Class link.

  4. In the New Java Class wizard, enter the following:

    • Package: name of the package containing the class for your enricher/validation (e.g., com.acme.phonestandardizer)

    • Name: name of the class for your enricher/validation (e.g., IntlPhoneStandardizer)

  5. Click Finish.
    The code editor for your class opens.

Parallel execution and thread safety

Semarchy xDM supports parallel processing of records in plugins, provided that the plugin is designed and declared as thread-safe. By setting the isThreadSafe option to true in a rowTransformer or rowValidator extension point, you declare that the code of the plugin is thread-safe and can be executed with several parallel threads.

It is the plugin developer’s responsibility to review and test the plugin to make sure of its thread safety before setting the isThreadSafe option to true.

Declare the parameters and input/output fields

Parameters and input/output types

Plugins use declared types for the parameters, inputs and outputs. When setting the values for the parameters and inputs, and when mapping the outputs, appropriate typing allows design-time validation and prevents run-time issues.

In the plugin code, when managing inputs and parameters, each declared type will correspond to a given Java type. Make sure to define and cast your variable according to the type of the input/output or parameter that they will map to.

The table below provides the correspondence between the plugin types and Java types.

Plugin type Corresponding Java type

BINARY

java.lang.Byte[]

BOOLEAN

java.lang.Boolean

BYTE_INTEGER

java.lang.Byte

DATE

java.util.LocalDate

DATETIME (deprecated, use DATE instead)

java.util.Date

TIMESTAMP

java.util.Date

DOUBLE

java.lang.Double

FLOAT

java.lang.Float

INTEGER

java.lang.Integer

LONG_INTEGER

java.lang.Long

SHORT_INTEGER

java.lang.Short

STRING

java.lang.String

Declare the parameters

Parameters allow customizing the behavior of the plugin.

To declare the plugin parameters:

  1. In the plugin editor, select the Extensions tab

  2. Select the parameters node under your Extension Point, right-click and select New > param.

  3. In the Extension Element Details section, enter the following values:

    • Name: name of the parameter (e.g., NULLIFYONERROR)

    • Type: type of the parameter (e.g., BOOLEAN)

    • Label: user-facing label for this parameter (e.g., Nullify on Error)

    • shortDescription: user-facing description for this parameter (e.g., If set to 'true', returns null for a phone number that cannot be processed. Otherwise returns the original phone number.)

    • isMandatory: set to true to make this parameter mandatory.

  4. Repeat the two previous steps to add more parameters.

Declare the input fields

Input fields are the values received by an enricher or validation plugin.

To declare the plugin input fields:

  1. In the plugin editor, select the Extensions tab

  2. Select the inputFields node under your Extension Point, right-click and select New > field.

  3. In the Extension Element Details section, enter the following values:

    • Name: name of the field (e.g., INPUTPHONE). This name should only contain uppercased alphanumerics and should not be longer than 30 characters.

    • Type: type of the field (e.g., STRING)

    • Label: user-facing label for this field (e.g., Input Phone)

    • shortDescription: user-facing description for this field (e.g., Phone Number to be standardized)

    • isMandatory: set to true to make this field mandatory.

  4. Repeat the two previous steps to add more input fields.

Declare the output fields

Output fields are the values returned by an enricher plugin.

To declare the plugin output fields:

  1. In the plugin editor, select the Extensions tab

  2. Select the outputFields node under your Extension Point, right-click and select New > field.

  3. In the Extension Element Details section, enter the following values:

    • Name: name of the field (e.g., STANDARDIZEDPHONE). This name should only contain uppercased alphanumerics and should not be longer than 30 characters.

    • Type: type of the field (e.g., STRING)

    • Label: user-facing label for this field (e.g., Standardized Phone)

    • shortDescription: user-facing description for this field (e.g., Standardized Phone Number)

    • isMandatory: set to true to make this field mandatory (a value is always returned).

  4. Repeat the two previous steps to add more output fields.

Make sure that the input and output field names only contain uppercased alphanumerics and are no longer than 30 characters.
Make sure to use uppercase for the parameter and field names. These names are not visible to the end-users.

Implement the plugin code

Now that the plugin is declared, it is possible to implement the specified behavior.

To implement the plugin code:

  1. Switch to the class editor.

  2. If needed, Use the Eclipse quick fix feature to add the interface methods currently unimplemented in the code.

  3. Implement the required methods for your plugin:

    • For a rowTransformer: the setUp, tearDown and transform Methods.

    • For a rowValidator: the setUp, tearDown and isValid Methods.

Download the plugin source code: IntlPhoneStandardizer.java.

The following sections describe the methods implemented in the plugin code.

Setup/teardown methods

These methods are respectively used to initialize the plugin - typically by setting the value of the parameters - and to release the resources used by the plugin.

When a plugin is used in Semarchy xDM, the SetUp and TearDown methods are not called if no record is passed to the plugin for processing.

In this case, the only parameter is NULLIFYONERROR. This parameter’s value is stored in a private Boolean variable called nullifyOnError.

Note the use of the getParameterAsBoolean method to retrieve a boolean corresponding to the type set while declaring the parameters and input/output fields. The same methods exist for other supported data types.

private boolean nullifyOnError;
...
@Override public void setUp(IRowTransformerSetupInfo pSetupInfo) {
	nullifyOnError = pSetupInfo.getParameterAsBoolean("NULLIFYONERROR");
}

@Override public void tearDown() {

}
String referring to parameter names should always be in uppercase in the Java code.
The IRowTransformerSetupInfo object can be used to retrieve the data location’s datasource, using the getTargetDataSource() method.

Transform method (enricher plugins)

Enrichers plugins enrich a list of data rows by calling the transform method. This method returns the list of enriched data rows.
In our example, this method simply calls a private method called transformOneRow to process one of these data rows.

@Override public List<IDataRow> transform(List<IDataRow> pDataRowsBatch) {
        ArrayList<IDataRow> outputDataRowList = new ArrayList<IDataRow>();
        for (IDataRow inputRow : pDataRowsBatch) {
                outputDataRowList.add(transformOneRow(inputRow));
        }
        return outputDataRowList;
}

The transformOneRow method transforms an input data row (IDataRow) into an output data row.

  • The input data row is passed to the transform method in the pDataRow variable, and contains the input fields passed to the enricher. You access each field value using the getValue method.

  • The output data row is created and returned by this method, with the output fields set using the setValue method.

In the example below, the input data row only contains the INPUTPHONE field and the output data row is built only with the STANDARDIZEDPHONE field.
The core of the transformation takes place in the private method normalizePhoneNumber.

An example of this method is provided below. Review the complete code of this method in the IntlPhoneStandardizer.java source code.

private IDataRow transformOneRow(IDataRow pDataRow) {
        // First, create the returned IDataRow.
        DataRow outputDataRow = new DataRow();

        // Make sure to set a null value for each output field in the
        // outputDataRow.
        outputDataRow.setValue("STANDARDIZEDPHONE", null);

        // The transformation is done below. It uses the normalizePhoneNumber
        // method defined in the class.
        if (pDataRow.getValue("INPUTPHONE") != null) {
                outputDataRow.setValue("STANDARDIZEDPHONE",
                                normalizePhoneNumber(pDataRow.getValue("INPUTPHONE")
                                                .toString()));
        }
        return outputDataRow;
}
It is recommended to set all the mandatory output field values to null immediately after creating the output DataRow. This avoids returning partially complete data row as the output. This is done by issuing the following code for each output field outputDataRow.setValue("<output_field_name>", null);.
String referring to input/output field names should always be in uppercase in the Java code.

Isvalid method (validation plugins)

The isValid method is required for a validation plugin.
This method is similar to the transform method. It takes as an input a list of data rows and returns a list of boolean values indicating whether the input data rows comply with the validation specified in the plugin. Typically, this method uses an isValidOneRow private method that validates a single data row.