Serialize source data

This document describes how to use serializers in xDI Designer mappings.

Overview

Serializers read textual, structured data such as a JSON file, and processes the data to embed it in a field within a target datastore. It is useful in situations where you want to process your structured data from within a database to take advantage of RDBMS tools and workflows.

Mappings offer you serializer templates based on the computed input format. For example, when the input is in JSON format, Designer offers a JSON serializer.

Refer to Templates for the current list of supported formats and templates.

Create a serializer

To create and configure a serializer in a mapping:

  1. Choose a database schema to use as a backend engine to process the data.

  2. Drag and drop the database schema into a mapping.

  3. From the menu that appears, select the Create Serializer option.

    serializer create from schema

  4. Add your input data.

  5. Set the output data.

  6. Add fields (Optional).

  7. Map target datastores.

Configure a serializer

Add input

The input is a reverse-engineered metadata representing a data structure, such as a JSON metadata, to serialize.

To define the serializer input, find a reverse-engineered metadata representing structured data, and drag and drop it from the Project Explorer onto the input field.

serializer add input

Set output

The Output is a datastore field containing embedded semi-structured data.

When you execute the mapping, the process serializes the input structure into string data, then loads it into the output structure. The data from the output is loaded into the mapped fields in the target datastore.

To work properly, you must set the type of the Output field by following these steps:

  1. Click on the Output field.

  2. In the Properties > Standard view, click Enable Datatype+

    serializer output properties 1

  3. In the Properties > Structure view, activate the Type field and select the datatype that corresponds to the mapped field in the target datastore.+

    serializer output properties 2

Add fields

Serializer fields are optional containers to store source column data alongside the output structure data. Create fields when you need to load and process data from other datastores, or when you need to join two serializers.

To add serializer fields:

  1. In the upper left corner of the serializer box, click the create field icon.

    serializer field

  2. Drag and drop a field from the source datastore onto the one you created.

    serializer field mapping

  3. Select one of the Map options.

  4. Repeat the procedure for any other fields you need.

You can now use the fields the created as a source to map a target datastore. You can also use them to join the serializer with other datastores.

Join serializers

You may have more than one serializer in your mapping. In such a scenario, you can join them using fields.

To join two serializers:

  1. Select a serializer and add the fields containing the data to use for the join. See Add fields.

  2. Repeat the procedure for the other serializer.

  3. Drag and drop the field from one serializer on the field in the other serializer.

The two serializers are joined. Use them as a source to load target datastores.

Mapping examples

In this sample, different source datastores are joined, and their fields are mapped onto a JSON structure inside the Serializer input. The output is then mapped onto a single field in the HOTEL_MANAGEMENT database schema, where it is stored as a single string.

serializer sample mapping

Templates

The serializer provides two generic templates to serialize data to an RDBMS string:

  • SERIALIZER JSON as RDBMS String, to serialize JSON data.

  • SERIALIZER XML as RDBMS String, to serialize XML data.

It also provides technology-specific templates optimized for certain database providers.

Specific JSON deserializers

  • SERIALIZER Hsql JSON as String for HSQL targets.

  • SERIALIZER JSON as Snowflake VARIANT for Snowflake targets, where the target field is a Snowflake VARIANT datatype.

Specific XML deserializers

  • SERIALIZER Hsql XML as String for HSQL targets.

Limitations

Semarchy xDI does not support XML namespaces when serializing XML data.