Consume data using the REST API

The REST API supports querying data stored in a data hub. In addition, the named query feature provides customized REST endpoints to consume data with predefined structures and query parameters.

Query records

Query multiple records

Method

GET

Base URL

http://<host>:<port>/semarchy/api/rest/

URL

[base_url]/query/[data_location_name]/[entity_name]/[view_type]
[base_url]/query/[data_location_name]/[entity_name]/[view_type]([load_id])
[base_url]/query/[data_location_name]/[entity_name]/[view_type]([batch_id or as_of_date])

URL structure

The URL is configured using the following values:

  • [data_location_name]: name of the data location to query.

  • [entity_name]: name of the entity to query.

  • [view_type]: type of data view to query. For more information, see View types.

  • [load_id]: ID of the load containing the source records. This value works with certain view types only. You can only query data in external loads, not in continuous loads.

  • [batch_id or as_of_date]: ID or date of the batch into which the records were created, modified or deleted. This value works with certain view types only. The date must be provided in ECMAScript standard format (YYYY-MM-DDTHH:mm:ss.sssZ).

Supported parameters

Pagination, query expressions, filtering, sorting

Response format

The response contains the list of records with the query expressions.

Multiple records query: sample response
{
  "records": [
	{
  	"CustomerName": "Gadgetron",
  	"City": "Honolulu"
	},
	{
  	"CustomerName": "StayPaft",
  	"City": "Honolulu"
	}
  ]
}

Query a single record

Method

GET

Base URL

http://<host>:<port>/semarchy/api/rest/

URL

[base_url]/query/[data_location_name]/[entity_name]/[view_type]/[record_id]
[base_url]/query/[data_location_name]/[entity_name]/[view_type]([load_id])/[record_id]
[base_url]/query/[data_location_name]/[entity_name]/[view_type]([batch_id or as_of_date])/[record_id]

URL structure

The URL is configured using the following values:

  • [data_location_name]: name of the data location to query.

  • [entity_name]: name of the entity to query.

  • [view_type]: type of data view to query. For more information, see View types.

  • [load_id]: ID of the load containing the source records. This value works with certain view types only. Note that you can only query data in external loads, but not in continuous loads.

  • [batch_id or as_of_date]: ID or date of the batch into which the record was created, modified, or deleted. This value works with certain view types only. The date must be provided in ECMAScript standard format (YYYY-MM-DDTHH:mm:ss.sssZ).

  • [record_id]: ID of the record to query.

Record ID structure

The record_id value depends on the view_type used. The IDs for the various view types are listed below:

  • GD, GH4B, GI, GDWE, SA4L, and SA4LK: <id_attribute>

  • MD, MH4B, MI, and UM:

    • Fuzzy-matched entity: PublisherID.SourceID

    • ID-matched entity: PublisherID.<id_attribute>

  • GH: FromBatchID.<id_attribute>

  • MH:

    • Fuzzy-matched entity: FromBatchID.PublisherID.SourceID

    • ID-matched entity: FromBatchID.PublisherID.<id_attribute>

  • GX: BatchID.<id_attribute>

  • MX:

    • Fuzzy-matched entity: BatchID.PublisherID.SourceID

    • ID-matched entity: BatchID.PublisherID.<id_attribute>

  • SD and SDWE:

    • Fuzzy-matched entity: LoadID.PublisherID.SourceID

    • ID-matched entity: LoadID.PublisherID.<id_attribute>

  • SA and SAWE: LoadID.<id_attribute>

  • GE: <id_attribute>$ConstraintName.ConstraintType

  • SE:

    • Fuzzy-matched entity: PublisherID.SourceID$ConstraintName.ConstraintType

    • ID-matched entity: PublisherID.<id_attribute>$ConstraintName.ConstraintType

  • AE: <id_attribute>$ConstraintName.ConstraintType

  • SA: LoadID.<id_attribute>

For example, to query a source data record (SD view), for matching entities, you must provide the concatenation of LoadID.PublisherID.SourceID for the record_id.

Supported parameters

Query expressions

Response format

The response contains the list of query expressions.

Single record query: sample response
{
    "CustomerName": "Gadgetron",
    "City": "Honolulu",
    "UPPER(CustomerName)": "GADGETRON"
}

Count records

Method

GET

Base URL

http://<host>:<port>/semarchy/api/rest/

URL

[base_url]/count/[data_location_name]/[entity_name]/[view_type]
[base_url]/count/[data_location_name]/[entity_name]/[view_type]([load_id])
[base_url]/count/[data_location_name]/[entity_name]/[view_type]([batch_id or as_of_date])

URL structure

The URL is configured using the following values:

  • [data_location_name]: name of the data location to count.

  • [entity_name]: name of the entity to count.

  • [view_type]: type of data view to query. For more information, see View types.

  • [load_id]: ID of the load containing the source records. This value works with certain view types only. Note that you can only count records in external loads, but not in continuous loads.

  • [batch_id or as_of_date]: ID or date of the batch into which the records were created, modified or deleted. This value works with certain view types only. The date must be provided in ECMAScript standard format (YYYY-MM-DDTHH:mm:ss.sssZ).

Supported parameters

Filtering

Response format

The response contains the count of records.

Record count query: sample response
{
  "count": "42"
}

Use named queries

Named queries provide customized REST endpoints to consume data with a predefined structure and query parameters.

For more information about designing named queries, see Configure named queries.

Query multiple records with a named query

Method

GET

Base URL

http://<host>:<port>/semarchy/api/rest/

URL

[base_url]/named-query/[data_location_name]/[named_query]/[view_type]
[base_url]/named-query/[data_location_name]/[named_query]/[view_type]([load_id])
[base_url]/named-query/[data_location_name]/[named_query]/[view_type]([batch_id or as_of_date])

URL structure

The URL is configured using the following values:

  • [data_location_name]: name of the data location to query.

  • [named_query]: name of the named query.

  • [view_type]: type of data view to query. For more information, see View types.

  • [load_id]: ID of the load containing the source records. This value works with certain view types only. Note that you can only query data in external loads, but not in continuous loads.

  • [batch_id or as_of_date]: ID or date of the batch into which the records were created, modified or deleted. This value works with certain view types only. Note that the date must be provided in ECMAScript standard format (YYYY-MM-DDTHH:mm:ss.sssZ).

Supported parameters

Pagination, filtering, sorting

In addition, query parameters are passed to the query using their name. For example: ?param1=value1&param2=value2.

Sorting applies before the sort expression defined in the name query itself. Pagination and filtering only apply to the root object of the named query.

Response format

The response contains a hierarchy of objects corresponding to the named query definition.

Count records in a named query

Method

GET

Base URL

http://<host>:<port>/semarchy/api/rest/

URL

[base_url]/named-query-count/[data_location_name]/[named_query]/[view_type]

URL structure

The URL is configured using the following values:

  • [data_location_name]: name of the data location to query.

  • [named_query]: name of the named query.

  • [view_type]: type of data view to query. For more information, see View types.

Supported parameters

Filtering

In addition, query parameters are passed to the query using their name. For example: ?param1=value1&param2=value2

Response format

The response contains the count of records returned by the named query.

REST API query parameters

View types

Certain REST endpoints use a view type as part of their URL. This view type, identified by an alias (GD, MD, etc.) defines the type of data (golden, master, errors, etc.) queried for the entity, using the endpoint.

Example: Query employee golden records using the GD view
/api/rest/query/HR/Employee/GD
Example: Query employee master records using the MD view
/api/rest/query/HR/Employee/MD

Some of these view types must be contextualized to list records for a given load ID or a given point in history (identified by a batch ID or as-of date).

Example: Query employee golden record history using the GH view as of batch 2565
/api/rest/query/HR/Employee/GH(22565)
Other view types ignore this contextualization syntax. You must use filtering to query a subset of the records.
Example: Query employee golden data, filtering only the records changed in batch 22565
/api/rest/query/HR/Employee/GD?$f=BatchID=22565

The following table lists the most common view types.

View type URL contextualization Description

GD

Golden data consolidated and certified records.

GE

Errors detected after master data consolidation (post-consolidation), for matching entities.

GDWE

Golden data consolidated and certified records, filtered to only show those with errors (matching entities only)

GH

Golden data history.

GH4B

batch_id or as_of_date

Golden data history as of batch or golden data if not historized. This view type requires a batch_id or as_of_date value in the URL.

GX

Deleted golden records logs (and data for soft delete).

MD

Enriched, validated, and cleansed master records.

MH

Master records history.

MH4B

batch_id or as_of_date

Master data history as of batch or master data if not historized. This view type requires a batch_id or as_of_date value in the URL.

MX

Deleted master records logs (and data for soft delete).

SD

Source records from finished loads.

SE

Errors on source records, returned with the data from the erroneous record.

SDWE

Source records from finished loads, filtered to only show those with errors.

SD4L

load_id

Source data loaded by the publishers for a given load. This view type requires a load_id value in the URL.

SA

Source data authored by users.

AE

Errors detected on source authoring data.

SAWE

Source data authored by users with errors.

SA4L

load_id

Source data authored by users for a given load. This view type requires a load_id value in the URL. This view automatically resolves references to source/golden records (for basic entities) or source/master records (for matching entities), as performed when authoring the records in the applications.

For more information about the views and their aliases, see Data access views, attributes, and lineage.

Pagination

When calls return a large number of records, it is recommended to page the record set. By requesting smaller subsets of data, you will get a response much faster than when requesting the entire, potentially large, record set.

Parameter Default value Description

$offset

0

Defines the record offset for pagination.

$limit

100

Defines the maximum number of returned records.
Example: Request Customer records between positions 20 and 29
/query/CustomerAndFinancialMDM/Customer/GD?$offset=20&$limit=10

Query expressions

Query expressions define which values are returned by the query for each record.

Parameter Default value Description

$baseExprs

USER_ATTRS

Defines the set of base attributes to include in the records, in addition to those specified using the expr parameter. Possible values are:

  • NONE: no attributes.

  • USER_ATTRS: all entity attributes, except the built-in attributes and references.

  • VIEW_ATTRS: all entity attributes, except references, including built-in attributes.

$expr

N/A

Expression to include to the record in addition to, or to remove from, the base attributes (baseExprs). You can put as many expressions as you need as query parameters. These expressions may be one of the following:

  • [semql_expression]: a SemQL expression to add to the base attributes. The expression string is used as the alias for the value.

  • [alias]:[semql_expression]: a SemQL expression to add to the base attributes, preceded by its alias.

  • $exclude:[semql_expression]: a SemQL expression to remove from the base attributes.
Example: Query all Customer attributes, except the built-in ones, and add Creator and CreationDate built-in attributes
/query/CustomerAndFinancialMDM/Customer/GD?$baseExprs=USER_ATTRS&$expr=Creator&$expr=CreationDate
Example: Query only CustomerName (in uppercase, aliased as 'Name') and TotalRevenue (aliased as 'Revenue')
/query/CustomerAndFinancialMDM/Customer/GD?$baseExprs=NONE&$expr=Name:Upper(CustomerName)&$expr=Revenue:TotalRevenue
Example: Query all Customer attributes, except the built-in ones and TotalRevenue
/query/CustomerAndFinancialMDM/Customer/GD?$baseExprs=USER_ATTRS&$expr=$exclude:TotalRevenue

Sorting

Sort expressions define the order of the records in the response.

Parameter Default value Description

$orderBy

None

SemQL ORDER BY clause defining the order of the records in the response.
Example: Query customers sorted by total revenue and name
/query/CustomerAndFinancialMDM/Customer/GD?$orderBy=TotalRevenue%20DESC,CustomerName%20ASC

Filtering

Use filters to request only specific records that match your specified criteria.

Parameter Default value Description

$f

SemQL filter. It is a SemQL condition applied to filter records. Only records matching this condition are returned. You can combine multiple SemQL filters in the same query.
Example: Query contacts whose first name starts with 'Joe'
/query/CustomerAndFinancialMDM/Contact/GD?$f=FirstName%20LIKE%20%27Joe%25%27