Apama 10.15.0 | Connecting Apama Applications to External Components | Standard Connectivity Plug-ins | The Kafka Transport Connectivity Plug-in | Configuring the connection to Kafka (dynamicChainManagers)
 
Configuring the connection to Kafka (dynamicChainManagers)
You configure one or more dynamicChainManagers to connect to different Kafka brokers. For example:
dynamicChainManagers:
kafkaManager:
transport: kafkaTransport
managerConfig:
channelPrefix: "kafka:"
bootstrap.servers: "localhost:9092"
Connection-related configuration is specified in the managerConfig stanza on the dynamicChainManagers instance. The following configuration options are available for managerConfig:
Configuration option
Description
bootstrap.servers
This is the only option in the Kafka configuration for which you must specify a value. You can either set it under managerConfig (to be used as the default for all consumers and producers) or in the configuration of a specific consumer or producer (which overrides any default given in the parent chain manager).
When this option is set under managerConfig, it is used as the default. It needs to be enclosed in quotation marks. Example:
bootstrap.servers: "localhost:62618"
Type: string.
channelPrefix
Prefix for dynamic mapping. If the prefix ends with a colon (:), it needs to be enclosed in quotation marks (see also Using YAML configuration files).
When the channel is mapped to a Kafka topic, the prefix is not used. For example, if the prefix is "kafka:", then the channel kafka:test/a maps to the Kafka topic test/a.
Type: string.
Default: "kafka:".
consumerConfig
Keys and values of the consumer configuration options in Kafka. See the Kafka documentation at https://kafka.apache.org/documentation/ for detailed information on the consumer configs.
Some default values are provided by the Kafka transport, but you can override them by specifying different values.
The default values are:
*group.id: A unique identifier for every instance.
*session.timeout.ms: "30000"
*key.deserializer: "org.apache.kafka.common.serialization.StringDeserializer"
*value.deserializer: "org.apache.kafka.common.serialization.StringDeserializer"
Type: map.
producerConfig
Keys and values of the producer configuration options in Kafka. See the Kafka documentation at https://kafka.apache.org/documentation/ for detailed information on the producer configs.
Some default values are provided by the Kafka transport, but you can override them by specifying different values.
The default values are:
*linger.ms: 0
*key.serializer: "org.apache.kafka.common.serialization.StringSerializer"
*value.serializer: "org.apache.kafka.common.serialization.StringSerializer"
Type: map.
Kafka allows clients to connect over SSL. You use the consumerConfig and producerConfig configuration options of the Kafka transport to specify the SSL configuration. See the Kafka documentation at https://kafka.apache.org/ for detailed information on how to configure Kafka clients to use SSL.