Adapter for Apache Kafka 9.6 | webMethods Adapter for Apache Kafka Documentation | webMethods Adapter for Apache Kafka Installation and User’s Documentation | Adapter for Apache Kafka Connections | Configuring an Adapter for Apache Kafka Consumer Connection
 
Configuring an Adapter for Apache Kafka Consumer Connection
When you configure Adapter for Apache Kafka , you specify information that Integration Server uses to connect to an Apache Kafka system. You can configure Adapter for Apache Kafka connections manually using the Integration Server Administrator screen.
*To configure Consumer connection
1. In the Adapters menu of Integration Server Administrator's navigation area, click webMethods Adapter for Apache Kafka.
2. On the Connections screen, click Configure New Connection.
3. On the Connection Types screen, click Apache Kafka Consumer Connection to display the Configure Connection Type screen.
4. In the webMethods Adapter for Apache Kafka section, use the following fields:
Field
Description/Action
Package
The package in which to create the connection. You must create the package using Designer before you can specify the package using this parameter. For general information about creating packages, see the webMethods Service Development Help for your release.
Note:
Configure the connection in a user-defined package rather than in the adapter's package. For other important considerations when creating packages for Adapter for Apache Kafka, see Overview of Package Management
Folder Name
Specifies the folder in which you create the connection.
Connection Name
Specifies the name you want to give to the connection. Connection names cannot have spaces or use special characters reserved by Integration Server and Designer. For more information about the use of special characters in package, folder, and element names, see the webMethods Service Development Help for your release.
5. In the Connection Properties section, use the following fields:
Field
Description/Action
Bootstrap Servers
The list of servers with the combination of host and port used for establishing the initial connection to the Apache Kafka cluster. This list should be in the form, host1:port1,host2:port2 and so on.
groupId
Defines a unique string that identifies the consumer group that this consumer connection belongs to.
Client ID
Defines the ID string to pass to the Kafka server when a request is made.
Key Deserializer Class
Defines the deserializer class for key that implements org.apache.kafka.common.serialization.Deserializer interface.
Value Deserializer Class
Defines the deserializer class for value that implements the org.apache.kafka.common.serialization.Deserializer interface. The default value is, com.wm.adapter.wmkafka.idata.IDataDeserializer
Enable Auto Commit
Select one of the following values for this field to enable the auto commit.
*If the value is set to true, the consumer’s offset is periodically committed.
*If the value is set to false, the Adapter for Apache Kafka commits the consumer’s offsets.
Auto Commit Interval(ms)
Specifies the frequency in milliseconds that the consumer offsets are auto-committed to Kafka, if enable.auto.commit is set to true.
Session TimeOut(ms)
Specifies the timeout used to detect consumer failures while using the Kafka's group management facility.
Auto Offset Reset
Select one of the following values if no initial offset value is set in Apache Kafka:
*earliest Automatically reset the offset to the earliest offset
*latest Automatically reset the offset to the latest offset
*none Throws exceptions to the consumer if no previous offset is found for the consumer's group
Kafka Version
Specify the Apache Kafka version that the consumer needs to connect to.
The following are the supported values:
*v9 This is equivalent to Apache Kafka 0.9.0.x version.
*v10 This is equivalent to Apache Kafka 0.10.x.x version.
Other Properties
Specifies the Kafka related properties for additional configuration. Use the following format:
propertyName1=value;propertName2=value
6. In the Connection Management Properties section, use the following fields:
Field
Description/Action
Enable Connection Pooling
Enables the connection to use connection pooling. For more information about connection pooling, see Adapter Connections .
Note:
If you plan to enable connection pooling in a clustered environment, consider the connection pool size.
Minimum Pool Size
If connection pooling is enabled, this field specifies the number of connections to create when the connection is enabled. The adapter will keep open the number of connections you configure here regardless of whether these connections become idle.
Maximum Pool Size
If connection pooling is enabled, this field specifies the maximum number of connections that can exist at one time in the connection pool.
Pool Increment Size
If connection pooling is enabled, this field specifies the number of connections by which the pool will be incremented if connections are needed, up to the maximum pool size.
Block Timeout
If connection pooling is enabled, this field specifies the number of milliseconds that Integration Server will wait to obtain a connection with the database before it times out and returns an error. For example, you have a pool with Maximum Pool Size of 20. If you receive 30 simultaneous requests for a connection, 10 requests will be waiting for a connection from the pool. If you set the Block Timeout to 5000, the 10 requests will wait for a connection for 5 seconds before they time out and return an error. If the services using the connections require 10 seconds to complete and return connections to the pool, the pending requests will fail and return an error message stating that no connections are available. If you set the Block Timeout value too high, you may encounter problems during error conditions. If a request contains errors that delay the response, other requests will not be sent. This setting should be tuned in conjunction with the Maximum Pool Size to accommodate such bursts in processing.
Expire Timeout
If connection pooling is enabled, this field specifies the number of milliseconds that an inactive connection can remain in the pool before it is closed and removed from the pool. The connection pool will remove inactive connections until the number of connections in the pool is equal to the Minimum Pool Size. The inactivity timer for a connection is reset when the connection is used by the adapter.
If you set the Expire Timeout value too high, you may have a number of unused inactive connections in the pool. This consumes local memory and a connection on your backend resource. This could have an adverse effect if your resource has a limited number of connections.
If you set the Expire Timeout value too low, performance could degrade because of the increased activity of creating and closing connections. This setting should be tuned in conjunction with the Minimum Pool Size to avoid excessive opening/closing of connections during normal processing.
Startup Retry Count
The number of times that the system should attempt to initialize the connection pool at startup if the initial attempt fails. The default value is 0.
Startup Backoff Timeout
The number of seconds that the system should wait between attempts to initialize the connection pool.
7. Click Save Connection.
The connection you created appears on the adapter's Connections screen and in Designer.
You can enable a connection only if the parameters for the connection are valid.