API Gateway 10.5 | Configuring API Gateway | API Gateway Configuration | Connecting to an External Elasticsearch
 
Connecting to an External Elasticsearch
 
API Gateway, Elasticsearch, Kibana Compatibility Matrix
API Gateway uses Elasticsearch as its primary data store to persist different types of assets such as APIs, Policies, and Applications apart from runtime events and metrics. By default, all assets are stored in the default Elasticsearch. But, you can configure API Gateway to use an external Elasticsearch to store the API Gateway assets. For information about the Elasticsearch version that is compatible with your API Gateway version, refer API Gateway, Elasticsearch, Kibana Compatibility Matrix.
When you configure external Elasticsearch you can have one of the following configurations:
*External Elasticsearch to store only the analytics.
This is achieved by configuring the external Elasticsearch as a destination store the analytics data in the configured destination. For details about the supported destinations and their configuration, see Destination Configuration. In this case the core configurations (such as APIs, Applications, Policies, Plans, Packages, Administration Settings, Security Configurations (Keystores/Trustores) and Tokens (OAuth/API Keys)) are stored in the internal default Elasticsearch.
*External Elasticsearch to store all API Gateway assets.
You can configure this in one of the following ways:
*Specifying the appropriate properties in the configurations config.properties file, which is explained in this section.
*Using externalized configuration files. For details, see Using the Externalized Configuration Files.
This section explains the changes that you must make in the config.properties file to enable API Gateway to communicate with the external Elasticsearch.
The configurations specified in the config.properties file override the values that are configured in gateway-es-store.xml during runtime and the values in gateway-es-store.xml are not changed. During the first start-up of API Gateway, default values from gateway-es-store.xml are automatically copied to config.properties. From the next start-up of API Gateway, values from config.properties are used. Once the host is specified in config.properties the value is not over-written from gateway-es-store.xml.
Note:
If you use an external Elasticsearch with same version as API Gateway Data Store, then you can use the Kibana or dashboard that is shipped with API Gateway, else they have to be configured separately. To know the compatible Kibana and Filebeat (Beats) versions for your Elasticsearch, see https://www.elastic.co/es/support/matrix#matrix_compatibility.
*To connect to an external Elasticsearch
1. Navigate to WmAPIGateway/config/resources/elasticsearch/config.properties
The config.properties file contains all the properties and Elasticsearch configurations.
2. Configure the following properties:
Property and Description
pg.gateway.elasticsearch.autostart
This property specifies whether the Elasticsearch starts automatically. If an external Elasticsearch is configured it has to be manually started. This property needs to be set to false to avoid API Gateway Data Store starting automatically.
Default value: true
pg.gateway.elasticsearch.client.http.response.size
This property specifies the response size, in MB, for API Gateway Elasticsearch cient.
Default value: 100
pg.gateway.elasticsearch.config.location
This property specifies the location of the config file if you want to read port details from some other Elasticsearch config file
pg.gateway.elasticsearch.hosts
Mandatory
This property lists Elasticsearch hosts and ports. The values are comma separated.
Default value: localhost:9240
Note:
Once a host is added to this property, this is the value that is used to connect to Elasticsearch and the host configured in gateway-es-store.xml is not considered.
pg.gateway.elasticsearch.http.keepAlive
Mandatory
This property creates the persistent connection between client and server.
Default value: true
pg.gateway.elasticsearch.http.connectionTimeout
Mandatory
This property specifies the time, in milliseconds, after which the connection times out.
Default value: 10000
pg.gateway.elasticsearch.http.socketTimeout
Mandatory
This property specifies the wait time, in milliseconds, for a reply once the connection to Elasticsearch is established after which it times out.
Default value: 30000
pg.gateway.elasticsearch.http.maxRetryTimeout
Mandatory
This property specifies the wait time, in milliseconds, for retries after which it times out.
Default value: 100000
It is advisable to set max retry time for a request to (number of nodes * socketTimeOut )+connectionTimeout
pg.gateway.elasticsearch.http.keepAlive.maxConnections
Mandatory
This property specifies the maximum number of persistent connections that can be established between an API Gateway and Elasticsearch cluster.
Default value: 50
pg.gateway.elasticsearch.http.keepAlive.maxConnectionsPerRoute
Mandatory
This property specifies the maximum number of persistent connections that can be established per HTTP route to an Elasticsearch server.
Default value: 15
pg.gateway.elasticsearch.http.username
This property specifies the user name to connect to Elasticsearch using basic authentication.
pg.gateway.elasticsearch.http.password
This property specifies the password to connect to Elasticsearch using basic authentication.
pg.gateway.elasticsearch.https.keystore.filepath
This property specifies the Keystore file path for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.truststore.filepath
This property specifies the truststore file path for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.keystore.password
This property specifies the Keystore password for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.keystore.alias
This property specifies the Keystore alias for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.truststore.password
This property specifies the truststore password for establishing HTTPS communication with Elasticsearch.
pg.gateway.elasticsearch.https.enabled
This property specifies whether you want to enable or disable the HTTPS communication with Elasticsearch.
Default value: false
If this property is set to false none of the above properties related to HTTPS are respected.
pg.gateway.elasticsearch.outbound.proxy.enabled
This property specifies whether you want to enable or disable outbound proxy communication.
Default value: true
pg.gateway.elasticsearch.outbound.proxy.alias
This property specifies the outbound proxy alias name used to connect to Elasticsearch.
pg.gateway.elasticsearch.https.enforce.hostname.verification
This property enforces the host name verification for SSL communication.
Default value: false
pg.gateway.elasticsearch.sniff.enable
Mandatory
This property enables sniffers to add the other nodes in an Elasticsearch cluster to the client so that the client can talk to all nodes.
Default value: true
This configuration must be set to false if you are changing the network when API Gateway or Elasticsearch is running.
pg.gateway.elasticsearch.sniff.timeInterval
Mandatory
This property enables adding the newly added Elasticsearch cluster nodes to existing REST client in a specified time interval in milliseconds.
Default value: 60000
3. Restart API Gateway for the HTTP client to take effect.
Note:
If hosts and ports are changed for Elasticsearch then you have to update the appropriate Elasticsearch configuration for Kibana separately and restart the Elasticsearch server as well as Kibana.
You can also externalize the Elasticsearch tenant ID and configuration by using a master configuration file. For details, see Externalizing Configurations .