This tutorial will guide you through the process of retrieving data from Semarchy xDM using the REST API.
Knowing how to query data using the REST API is essential to building a successful MDM program. While typical approaches involve middleware tools or custom programs to query and load data from xDM, some business cases require integration developers to connect Semarchy xDM to an external system via the REST API. This approach facilitates near real-time queries and contrasts with batch loading, which is better handled through SQL.
In this tutorial, you will learn to use simple queries and named queries.
This tutorial is based on the Customer B2C Demo application, and provides hands-on experience with retrieving fuzzy-matching entities and understanding the functionality of the REST API.
This tutorial is part of the Data Publishing & Consumption track, which is composed of SQL-based and REST-based tutorials.
Before following this tutorial, you must have set up Semarchy xDM and completed the Customer B2C demo tutorial. Additionally, you must have configured a REST client, which is described in the Set up a REST client for Semarchy xDM.
If you have not completed these prerequisites, return to the Tutorials menu.
Otherwise, enjoy this tutorial!
The REST API is available at the following base URL:
http://[host]:[port]/semarchy/api/rest
In the rest of the tutorial, this URL may be referred to as the [base_url]
.
Each operation in the REST API is available under this base URL. The minimum structure for a request is [base_url]/query/[data_location_name]/[entity_name]/[view_type]
.
Parameters definition:
[data_location_name]
: name of the data location to query (e.g., CustomerB2CDemo
)[entity_name]
: name of the entity to query (e.g., Person
)[view_type]
: type of data view to query (e.g., GD
for golden data)The URL may also take additional parameters.
Before we begin constructing the REST call, take a look at the REST API documentation that Semarchy xDM generates for you automatically. This is a fantastic resource that will help you construct your REST URLs and minimize the risk of errors.
You can find the REST API documentation at the following URL:
http://[public_dns_or_ip_address]:[port]/semarchy/api/rest/api-ui/index.html
On a typical on-premises instance of xDM, the REST documentation URL is http://localhost:8088/semarchy/api/rest/api-ui/index.html
All entities from the data location are listed in the left menu.
In this section, you will learn how to retrieve all records from a table using the REST API. You will query the Source Data
(SD), Master Integration
(MI) data, and Golden Data
(GD) tables for one entity.
Source Data
(SD), Master Integration
(MI) data, and Golden Data
(GD) tables using the REST API.You will retrieve all the customer records in the Person entity from the CustomerB2CDemo
data location. The template request you need is:
[base_url]/query/[data_location_name]/[entity_name]/
[view type]
/[record_ID]
For golden data records, the value for the view_type
parameter is GD
.
[base_url]/semarchy/api/rest/query/CustomerB2CDemo/Person/GD
You need to modify the request to use your host and port:
localhost
, and the port 8088
:http://localhost:8088/semarchy/api/rest/query/CustomerB2CDemo/Person/GD
80
, you will not need to include it in your request URL.To test this first request:
GET
method. If you have correctly set up the authorization with your credentials, you should see results returned. These are all the customer golden data records returned in JSON format.
http://:/semarchy/api/rest/query/CustomerB2CDemo/Person/MI
http://:/semarchy/api/rest/query/CustomerB2CDemo/Person/SD
Note the difference of results for GD, MI and SD tables.
Great job! You successfully queried xDM via REST API. To summarize:
This is just a preview of what you can do with xDM REST API. In the next section, you will learn how to get a record with its identifier.
In this section, you will learn how to retrieve a record using its golden-record identifier. This is useful for example, if your organization has a call center application that needs to retrieve from Semarchy xDM up-to-date data for a customer known by their ID.
For the needs of this tutorial, you must retrieve the ID of the customer golden record Mary-Ann Markes. You will use this ID within your REST request.
The value for this ID is specific to your environment as it depends on the order in which records were processed by Semarchy xDM.
To retrieve this golden-record ID, proceed as follows:
In the example below, the golden record has the ID 10008.
As indicated by the REST API documentation, use the GET
method to retrieve a single customer record. This is the skeleton of the REST call for searching by customer ID:
[base_url]/query/[data_location_name]/[entity_name]/
[view type]
/[record_ID]
In practice, you will use the view GD to query golden records. Your request will look like:
http://[host]:[port]/semarchy/api/rest/query/CustomerB2CDemo/Person/GD/[record_id]
To send this request in Postman:
GET
.This is the result you can expect:
{ "ID": 10008, "FirstName": "Mary-Ann", "LastName": "Markes", "PhoneticFirstName": "MRN", "PhoneticLastName": "MRKS", "NormalizedFirstName": "MARY-ANN", "NormalizedLastName": "MARKES", "Nickname": "mary-ann", "DateOfBirth": "1899-12-31", "SourceEmail": "maryann@markes.com", "CleansedEmail": "maryann@markes.com", "ValidEmailDomain": "1", "SourcePhone": "203-967-5612", "StandardizedPhone": "(203) 967-5612", "PhoneGeocodingData": "Stamford, CT", "ValueStatus": "HIGH", "PersonType": "CUSTOMER", "Address": "1200 Summer St #-103 Stamford CT 06905 US", "Address.Street": "1200 Summer St #-103", "Address.City": "Stamford", "Address.State": "CT", "Address.PostalCode": "06905", "Address.Country": "US" }
Note how the information differs from the data you can see in the application UI and the SQL query results observed in the Load data via the SQL API tutorial.
You learned how to construct a request URL using a record's ID.
In the next section, you will learn a more advanced request URL to look up a customer record based on first names and no IDs. This involves using a SemQL expression to search through all customer records.
In the previous section, you learned how to retrieve a customer record using the customer's account ID. However, customers frequently find themselves without their account ID readily available. Customer service representatives will frequently ask customers for other identifying details, such as their first and last name or address.
Following this business case, we will build on the previous request URL using the customer's name instead of account number.
The REST API allows external systems in your organization to retrieve records in xDM using filters. In this section, you will use the GET
method to retrieve a single customer record by its name, using a SemQL filter.
In our environment, we selected the customer golden record of John Edelman to retrieve data. We will retrieve their record using their last name, Edelman.
This is the skeleton of the REST call using a filter:
[base_url]/query/[data_location_name]/[entity_name]/
[view type]
?$f=[SemQL_filter]
In practice, this is what request URL looks like on a typical on-premises quick-start instance:
http://localhost:8088/semarchy/api/rest/query/CustomerB2CDemo/Person/GD?$f=LastName%20LIKE%20%27Edelman%27
The SemQL filter originally started as LastName LIKE 'Edelman'
.
Remember to substitute the spaces in the URL with %20
and the single quotes with %27
. For more information about percent encoding (also called URL encoding), see the "Percent-encoding" Wikipedia page.
To run this request in Postman:
GET
.Note that the $f
parameter was correctly interpreted by Postman as a query parameter (in the Params tab).
This is the result you can expect:
{ "records": [ { "ID": 11303, "FirstName": "John", "LastName": "Edelman", "PhoneticFirstName": "JN", "PhoneticLastName": "ETLM", "NormalizedFirstName": "JOHN", "NormalizedLastName": "EDELMAN", "Nickname": "jack", "DateOfBirth": "1976-09-03", "SourceEmail": "john@jedelman.com", "CleansedEmail": "john@jedelman.com", "ValidEmailDomain": "1", "SourcePhone": "415.381.9176", "StandardizedPhone": "(415) 381-9176", "PhoneGeocodingData": "Mill Valley, CA", "ValueStatus": "NORMAL", "PersonType": "CUSTOMER", "Address": "591 Redwood Hwy #-1200 Mill Valley CA 94941 US", "Address.Street": "591 Redwood Hwy #-1200", "Address.City": "Mill Valley", "Address.State": "CA", "Address.PostalCode": "94941", "Address.Country": "US" } ] }
Note how the information differs from the data you can see in the application UI and the SQL query results observed in the Load data via the SQL API tutorial.
In this section, you have built an advanced request URL using a filter that allowed you to filter on the customer's last name with a SemQL filter instead of account number.
To summarize:
Next, you will learn about named queries and how to build a custom data structure to create a customized REST endpoint.
In the previous section, you learned how to retrieve a customer record using a SemQL filter that queries the customer's last name.
So far in this tutorial, you have used predefined REST endpoints, such as retrieving a single record and using filters. There will be situations where you need a custom REST endpoint because your systems require a special data structure.
In this section, you will learn how to create a named query in the model design. Then you will use the named query in Postman to retrieve the custom data structure.
To create a named query in the model designer:
PersonByIDWithProducts
Person [Entity]
Customer
and then click Next.Products
.Value Expression
Product.ProductName
Product
Value Expression
PurchaseDate
PurchaseDate
Value Expression
RegistrationDate
RegistrationDate
We will call the named query to view the results. This is the skeleton of the REST call using a named query:
[base_url]/named-query/[data_location_name]/[named_query]/
[view type]
In practice, this is what our request looks like on an on-premises quick-start xDM instance:
http://localhost:8088/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD
To run this request in Postman:
GET
.The result you should see is all the customer golden records with their corresponding products:
{ "records": [ { "ID": 10001, "CleansedEmail": "angelica@aldaba.com", "NormalizedFirstName": "angelica", "NormalizedLastName": "miller", "PersonType": "CUSTOMER", "StandardizedPhone": "(503) 228-3027", "ValueStatus": "NORMAL", "Products": [ { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-12-23", "RegistrationDate": "2017-03-13" }, { "Product": "Streamliner Saddle", "PurchaseDate": "2015-11-04", "Registrationdate": "2015-12-14" } ] }, { "ID": 10002, "CleansedEmail": "saraj@yahoo.com", "NormalizedFirstName": "sara", "NormalizedLastName": "johnson", "PersonType": "CUSTOMER", "StandardizedPhone": "(415) 472-3289", "ValueStatus": "NORMAL", "Products": [ { "Product": "Foxy Elite Cycling Shoes", "PurchaseDate": "2016-10-12", "Registrationdate": "2016-10-16" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-05-09", "Registrationdate": "2016-05-11" }, { "Product": "Streamliner Saddle", "PurchaseDate": "2016-04-12", "Registrationdate": "2016-05-10" } ] }, ...(continued) ] }
Note that the list of products linked to each customer is returned by the named query with the Product
, PurchaseDate
and RegistrationDate
properties.
You have successfully created a named query. Named queries are useful for creating custom data structures to support your organization's specific requirements when integrating Semarchy xDM with other systems.
To summarize:
Next, you will build on the basic named query you created by adding query parameters that allow you to filter results.
In the previous section, you learned how to create a basic named query. This resulted in a custom REST endpoint for retrieving a customer record along with their associated products.
In this section, you will build upon the named query by incorporating more advanced functionalities. You will add query parameters that allow you to filter results with a SemQL expression, similar to the filter you used in ⓺ Get a customer record by last name.
In a customer support call center scenario, the customer service representative needs to retrieve customer details from the call center system. They will typically request information like email address or account ID, which both can serve as identifiers.
To submit this identifier information to xDM through the REST API endpoint, we use query parameters.
To create a query parameter in the named query:
IDENTIFIER
QUERY_PARAM_IDENTIFIER
String
Start by testing that your named query is still working: use the same request that you used in the previous section in Postman to call the named query.
http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD
You should get the same results as before adding the parameter.
Adding query parameters is the first operation required to filter the records xDM retrieves. The second operation involves setting up a filter that uses these query parameters. The filter is a SemQL expression that compares the incoming information with the existing data in xDM.
To create the filter and see how it works with query parameters:
SEM_NUMBER_TO_CHAR( ID ) = :QUERY_PARAM_IDENTIFIER
OR CleansedEmail = :QUERY_PARAM_IDENTIFIER
OR REGEXP_REPLACE( StandardizedPhone, '[^[:digit:]]', '' ) = REGEXP_REPLACE( :QUERY_PARAM_IDENTIFIER, '[^[:digit:]]', '' )
SEM_NUMBER_TO_CHAR( ID ) = :QUERY_PARAM_IDENTIFIER OR CleansedEmail = :QUERY_PARAM_IDENTIFIER OR REGEXP_REPLACE( StandardizedPhone, '[^[:digit:]]', '', 'g' ) = REGEXP_REPLACE( :QUERY_PARAM_IDENTIFIER, '[^[:digit:]]', '', 'g' )
To use the named query, include the parameters in the request URL. In this section, you will learn how to update your request URL to include the query parameters and their corresponding values (i.e., the arguments).
These are the three identifiers that this named query can accept:
The IDENTIFIER
query parameter is set up to only accept strings. However, the filter is able to check for account IDs, email addresses, and phone numbers. It converts an account ID into a string for comparison with the ID in xDM and removes non-digit characters from a phone number for comparison with the phone number in xDM.
This is the skeleton of the REST call using a named query:
[base_url]/named-query/[data_location_name]/[named_query]/
[view type]
?IDENTIFIER=[customer_ID_or_cleansed_email_or_phone_number]
After the IDENTIFIER
query parameter, you can provide the customer's account ID, email address, or phone number.
First, start with the customer's account number (golden record ID).
Picture a scenario where customer Toni Mattos contacts the customer support helpline regarding a warranty issue. The customer support representative requests Toni's account number to access her information. Toni provides her account number (i.e., her golden record ID): 11299. The following is an example of the request that the call center system can make to retrieve Toni's record from xDM:
http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD?IDENTIFIER=11299
To run this request in Postman:
GET
.The result you should see is Toni Mattos' information and the corresponding products she has purchased and registered:
{ "records": [ { "ID": 11299, "CleansedEmail": "toni@mattos.com", "ValueStatus": "HIGH", "StandardizedPhone": "(321) 727-6512", "NormalizedFirstName": "TONI", "PersonType": "CUSTOMER", "NormalizedLastName": "MATTOS", "PersonProducts": [ { "Product": "Carbonite Century ", "PurchaseDate": "2016-07-15", "RegistrationDate": "2016-08-04" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2014-09-17", "RegistrationDate": "2014-11-26" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-08-20", "RegistrationDate": "2016-10-08" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-10-09", "RegistrationDate": "2016-10-29" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2015-06-16", "RegistrationDate": "2015-07-23" }, { "Product": "Streamliner Saddle", "PurchaseDate": "2015-03-24", "RegistrationDate": "2015-05-10" } ] } ] }
Notice that the attributes related to the person are not in a user-friendly order. In the next section, you will reorder the attributes so that the first name and last name attributes come after the ID.
To reorder the attributes so that they are in a logical order:
In case the customer does not have her account number, here is how you could retrieve her customer account information based on her email:
http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD?IDENTIFIER=toni%40mattos.com
To run this request in Postman:
GET
.The result you should see is Toni Mattos' information and the corresponding products she has purchased and registered:
{ "records": [ { "ID": 11299, "NormalizedFirstName": "TONI", "NormalizedLastName": "MATTOS", "StandardizedPhone": "(321) 727-6512", "CleansedEmail": "toni@mattos.com", "PersonType": "CUSTOMER", "ValueStatus": "HIGH", "Products": [ { "Product": "Carbonite Century ", "PurchaseDate": "2016-07-15", "RegistrationDate": "2016-08-04" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2014-09-17", "RegistrationDate": "2014-11-26" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-08-20", "RegistrationDate": "2016-10-08" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-10-09", "RegistrationDate": "2016-10-29" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2015-06-16", "RegistrationDate": "2015-07-23" }, { "Product": "Streamliner Saddle", "PurchaseDate": "2015-03-24", "RegistrationDate": "2015-05-10" } ] } ] }
In case the customer does not have her account number or email, here is how you could retrieve her customer account information using her phone number (321) 727-6512:
http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD?IDENTIFIER=3217276512
To run this request in Postman:
GET
.The result you should see is Toni Mattos' information and the corresponding products she has purchased and registered. Here is an excerpt of what you can expect:
{ "records": [ { "ID": 11299, "NormalizedFirstName": "TONI", "NormalizedLastName": "MATTOS", "StandardizedPhone": "(321) 727-6512", "CleansedEmail": "toni@mattos.com", "PersonType": "CUSTOMER", "ValueStatus": "HIGH", "Products": [ { "Product": "Carbonite Century ", "PurchaseDate": "2016-07-15", "RegistrationDate": "2016-08-04" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2014-09-17", "RegistrationDate": "2014-11-26" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-08-20", "RegistrationDate": "2016-10-08" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2016-10-09", "RegistrationDate": "2016-10-29" }, { "Product": "Renaud Martin Skin Suit", "PurchaseDate": "2015-06-16", "RegistrationDate": "2015-07-23" }, { "Product": "Streamliner Saddle", "PurchaseDate": "2015-03-24", "RegistrationDate": "2015-05-10" } ] } ] }
You have learned how to retrieve data in xDM using an advanced named query, including:
Well done! You have run your first REST GET
requests to retrieve data from Semarchy xDM.
In the next unit of the Data Publishing & Consumption track, you will learn how to load data using the REST API.
To explore other resources, return to the Tutorials menu.
Thank you for completing this tutorial.