Install Semarchy xDM on Snowflake

Semarchy xDM is available as a Native App on the Snowflake Marketplace, which allows you to deploy and manage your xDM application directly within your Snowflake environment.

Prerequisites

Before deploying Semarchy xDM on Snowflake, ensure you have:

  • A Snowflake account with the privileges required to install and manage the Semarchy xDM Native App.

  • Snowflake roles configured to allow users to access the Native App.

    While administrators may already have the necessary permissions through their existing Snowflake roles, non-admin users must be granted the application role exposed by the Native App in order to sign in. For guidance on managing roles for Native App consumers, see the official Snowflake documentation.

  • A Snowflake database schema for storing your master data. For more information on how to create a Snowflake database schema, see Configure the database schemas.

Architecture

Semarchy xDM on Snowflake deploys as a Native App with two primary containers:

  • The xDM container: a virtualized environment that encapsulates and runs the xDM application with all its features.

  • The repository container: a PostgreSQL database that stores xDM metadata (persisted in a block storage volume).

The actual master data managed by xDM is stored in a Snowflake database schema, separating application management from data storage. This architecture allows for optimal performance and scalability while maintaining data governance within your Snowflake environment.

Known limitations

The following limitations apply to Semarchy xDM in Snowflake:

Import action limitations

Due to constraints inherent to Snowflake hybrid tables, synchronous operations such as Import actions may time out when processing more than a few hundred rows. The exact threshold varies depending on the model complexity. For bulk data loading on Snowflake, it is strongly recommended to use dedicated ETL/ELT tools and the integration layer.

Enricher and validator limitations

When using Snowflake hybrid tables as a data location, Java Plugin Enrichers, REST Enrichers, and Plugin Validators are not suitable for large batch processing due to Snowflake JDBC transaction limits.

Plugin enrichers and validators execute one UPDATE statement per processed row. Snowflake hybrid tables enforce a maximum number of DML statements per transaction. When processing large data volumes, this limit is exceeded, causing integration jobs to fail or hang. Typical symptoms include:

  • Processing succeeds with small datasets (~5,000 records or fewer)

  • Jobs fail, hang, or run for several hours with larger datasets (~25,000 records or more)

  • Error message: "The maximum number of DML statements has been exceeded by transaction"

For large-scale data enrichment on Snowflake hybrid tables, use SemQL enrichers instead, which are optimized for set-based operations and do not encounter this limitation.