As the focal point of governance, a Data Hub is a critical piece in all data management and data transformation initiatives. It is, most of all, the enabler for data flows between diverse endpoints, including operational applications (ERP, CRM, etc), data warehouses, data lakes, and even external parties. Being able to implement repeatable and scalable integration from and to the data hub, quickly and at a reasonable cost is critical to the success of the data hub strategy.
Semarchy’s Unified Data Platform supports out-of-the-box integration in batch and streaming mode. Semarchy xDM version 5.3 enhances these integration capabilities with new features to simplify end-to-end integration of the data hub, for both analytical and operational use cases.
Streaming Integration with Data Notifications
Near real-time propagation of data, changes are key for operational efficiency. When information consolidates from multiple sources using match and merge capabilities (consolidation pattern), or when new data is authored into the hub (centralized pattern), it must be propagated to downstream systems that require high-quality and well-governed information to pursue business processes.
Data Notifications, introduced in xDM 5.3, enable designers to easily automate data and change propagation from the data hub to consuming applications.
Data Notifications are delivered – on schedule or data change – to event streaming platforms (such as Apache Kafka) or to applications via Node JS REST API calls and message services. Their payload is fully customizable, using Semarchy xDM’s Named Queries, to accommodate the requirements of the consumers. Notifications may leverage the comprehensive audit trail available in the hub to propagate golden data, master data, validation errors, history, etc.
With Data Notifications, designers and integration teams can simplify data integration and change propagation patterns from the data hub to downstream systems.
Model-Driven SQL APIs
The new Database Views feature allows designers to create logical views in the model and deploy them in the data hub schema as physical SQL views, where they can be consumed by external applications in batch, by business intelligence, reporting, and visualization tools, including the platform xDM Dashboards module.
Database Views use the full power of SemQL transformations, functions, and expressions to define columns’ content and data filters based on Semarchy’s semantically complete model.
Using database views, developers can quickly design and deploy SQL APIs with their data hubs, fully aligned with the model structure and lifecycle.
Other Integration Enhancements
Data Notifications and Database Views are the integration highlights of version 5.3. Many other features have been added to make integration faster and simpler, including Delayed Continuous Loads to handle large batch integrations, Mass Update and Integration Summary in the REST API, Named Query performance enhancements, etc.
To learn more about Semarchy xDM 5.3 new features, review the detailed release notes. If you have not read it yet, head to the previous blog post in this series to learn more about the integrated configuration feature highlight.
You can also get started with xDM on-premises or in the cloud today!