Kafka Component

Overview

The Kafka Component is used in Semarchy xDI to integrate with this technology and produce integration flows.

Install the Kafka Component

If it is not available yet in your Semarchy xDI Designer, install the Kafka Component from Designer using the component installation process.

Supported Features

Feature Description

Metadata

Kafka producers and consumers can be designed in Metadata.

Consume messages

Reading messages from Kafka consumers is supported.

The following common features are supported:

  • Reading messages keys

  • Reading messages values

  • Reading messages headers

The following advanced features are supported:

  • Reading messages properties such as the topic it is coming from, the corresponding partition, offset, …​

  • Reading messages from multiple topics at the same time

  • Reading multiple messages in parallel

  • Transactional mode to read messages in a transaction

When reading messages, you can write the received messages in any database, file, …​

Produce messages

Writing data into Kafka through Kafka Producers is supported.

The following common features are supported when producing messages:

  • Defining message key

  • Defining message value

  • Defining message headers

The following advanced features are supported when producing messages:

  • Defining message partition

  • Producing messages in multiple topics at the same time

  • Producing multiple messages in parallel

  • Transactional mode to produce messages in a transaction

You can produce messages from any database, file, …​

Data consultation

Consulting and consuming messages directly from Semarchy xDI Designer is supported.

Kafka versions

Supported Kafka versions: 2.6 to 3.2