API Gateway 10.11 | Administering API Gateway | Deployment | Deployment Configurations | Connecting to an External Elasticsearch
 
Connecting to an External Elasticsearch
This section explains the changes that you must make in the config.properties file to enable API Gateway to communicate with the external Elasticsearch.
You can also connect to an external Elasticsearch through externalized configurations. For more details, see Externalizing Configurations .
Note:
If you use an external Elasticsearch with same version as API Data Store, then you can use the Kibana or dashboard that is shipped with API Gateway, else they have to be configured separately. If you have configured Elasticsearch externally, then you have to configure Kibana externally. To know the compatible Kibana and Filebeat (Beats) versions for your Elasticsearch, see https://www.elastic.co.
Important:
When you use an external Elasticsearch, you must use the Elasticsearch plugin mapper size. Without this plugin, API Gateway does not start. To install the mapper size plugin, use the command sudo bin/elasticsearch-plugin install mapper-size. If you have deployed a clustered environment and have configured external Elasticsearch on multiple nodes, you must install this plugin on all the nodes that use external Elasticsearch.
*To connect to an external Elasticsearch
1. Navigate to WmAPIGateway/config/resources/elasticsearch/config.properties
The config.properties file contains all the properties and Elasticsearch configurations.
2. Configure the following properties:
Property and Description
pg.gateway.elasticsearch.autostart
This property specifies whether the Elasticsearch starts automatically. If an external Elasticsearch is configured it has to be manually started. This property needs to be set to false to avoid API Data Store starting automatically.
Default value: true
pg.gateway.elasticsearch.client.http.response.size
This property specifies the response size, in MB, for API Gateway Elasticsearch client.
Default value: 100
pg.gateway.elasticsearch.config.location
This property specifies the location of the config file if you want to read port details from some other Elasticsearch config file
pg.gateway.elasticsearch.hosts
Mandatory
This property lists Elasticsearch hosts and ports. The values are comma separated.
Default value: localhost:9240
Note:
Once a host is added to this property, this is the value that is used to connect to Elasticsearch and the host configured in gateway-es-store.xml is not considered.
pg.gateway.elasticsearch.http.keepAlive
Mandatory
This property creates the persistent connection between client and server.
Default value: true
pg.gateway.elasticsearch.http.connectionTimeout
Mandatory
This property specifies the time, in milliseconds, after which the connection times out.
Default value: 10000
pg.gateway.elasticsearch.http.socketTimeout
Mandatory
This property specifies the wait time, in milliseconds, for a reply once the connection to Elasticsearch is established after which it times out.
Default value: 30000
pg.gateway.elasticsearch.http.maxRetryTimeout
Mandatory
This property specifies the wait time, in milliseconds, for retries after which it times out.
Default value: 100000
It is advisable to set max retry time for a request to (number of nodes * socketTimeOut )+connectionTimeout
pg.gateway.elasticsearch.http.keepAlive.maxConnections
Mandatory
This property specifies the maximum number of persistent connections that can be established between an API Gateway and Elasticsearch cluster.
Default value: 50
pg.gateway.elasticsearch.http.keepAlive.maxConnectionsPerRoute
Mandatory
This property specifies the maximum number of persistent connections that can be established per HTTP route to an Elasticsearch server.
Default value: 15
pg.gateway.elasticsearch.http.username
This property specifies the user name to connect to Elasticsearch using basic authentication.
pg.gateway.elasticsearch.http.password
This property specifies the password to connect to Elasticsearch using basic authentication.
pg.gateway.elasticsearch.https.keystore.filepath
This property specifies the Keystore file path for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.truststore.filepath
This property specifies the truststore file path for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.keystore.password
This property specifies the Keystore password for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.keystore.alias
This property specifies the Keystore alias for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.truststore.password
This property specifies the truststore password for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.enabled
This property specifies whether you want to enable or disable the HTTPS communication with Elasticsearch.
Default value: false
If this property is set to false none of the above properties related to HTTPS are respected.
pg.gateway.elasticsearch.outbound.proxy.enabled
This property specifies whether you want to enable or disable outbound proxy communication.
Default value: true
pg.gateway.elasticsearch.outbound.proxy.alias
This property specifies the outbound proxy alias name used to connect to Elasticsearch.
pg.gateway.elasticsearch.https.enforce.hostname.verification
This property enforces the host name verification for SSL communication.
Default value: false
pg.gateway.elasticsearch.sniff.enable
Mandatory
This property enables sniffers to add the other nodes in an Elasticsearch cluster to the client so that the client can talk to all nodes.
Default value: true
This configuration must be set to false if you are changing the network when API Gateway or Elasticsearch is running.
pg.gateway.elasticsearch.tenantId
This property allows you to specify a tenant name of your choice. This value must be same across all nodes.
The default value of this property is the Integration Server instance name. So, ensure that you provide same name for all Integration Server nodes in a cluster.
If you modify this value, you must edit the value in all nodes and restart the API Gateway server for the change to take effect.
pg.gateway.elasticsearch.sniff.timeInterval
Mandatory
This property enables adding the newly added Elasticsearch cluster nodes to existing REST client in a specified time interval in milliseconds.
Default value: 60000
3. Restart API Gateway for the HTTP client to take effect.
Note:
If hosts and ports are changed for Elasticsearch then you have to update the appropriate Elasticsearch configuration for Kibana separately and restart the Elasticsearch server as well as Kibana.
You can also externalize the Elasticsearch tenant ID and configuration by using a master configuration file. For details, see Externalizing Configurations .