Configuring Elasticsearch Cluster

Before you start, ensure that the Elasticsearch is not started after API Gateway installation.

1. If you have started API Gateway before setting up the Elasticsearch cluster configuration, perform the following steps before proceeding with the configuration:

Delete the nodes folder from the Installation Location\InternalDataStore\data folder.

2. Open elasticsearch.yml from SAG_root/InternalDataStore/config/elasticsearch.yml in any node that you want to cluster.

The following configuration is a sample of how the configuration appears initially.

cluster.name:"SAG_EventDataStore"
node.name: node1
path.logs: SAG_root\InternalDataStore/logs
network.host:0.0.0.0
http.port:9240
discovery.seed_hosts: ["node1:9340"]
transport.tcp.port:9340
path.repo:['SAG_root\InternalDataStore/archives']
cluster.initial_master_nodes:["node1"]

discovery.seed_hosts. You provide a list of nodes to the Elasticsearch that it should try to contact. Once the node contacts a member of the unicast list, it receives a full cluster state that lists all nodes in the cluster. It then proceeds to contact the master and join the cluster.

path.repo. This is a mandatory configuration for performing backup and restore. This is the location where the Elasticsearch writes the snapshots to. Hence, it is important to have a location that is accessible to all the nodes. The location must be network file system, S3 or Azure in the clustered setup.

cluster.initial_master_nodes. This parameter must be set so that when you start a cluster for the first time cluster bootstrapping is performed. The parameter must contain the names of the master-eligible nodes in the initial cluster and must be defined on every master-eligible node in the cluster. This setting helps prevent split-brain, the existence of two masters in a single cluster.

Elasticsearch provides an option to configure the locations where you would want to store your data and logs. Ensure that you specify the locations that are accessible and have enough disk space. It is also important to monitor and ensure basic house keeping of the data location by planning an effective data retention strategy. You can change the defaults using the following configuration:

path.data: /var/lib/elasticsearch
path.logs: <IS\_Installed\_Location>/InternalDataStore/logs
// These values can be changed

Elasticsearch, by default, binds to loop back address and hence it is important to change it for a production deployment. For more details on this configuration, see https://www.elastic.co/guide/en/elasticsearch/reference/7.13/modules-network.html

Nodes with same cluster names form a cluster. That is, if there are three nodes in the cluster, the value in the cluster.name property must be same across all three nodes. In other words, Elasticsearch forms a cluster with nodes that have the same cluster.name.

4. Provide the names of all participating nodes, as seen in the node.name property, and the ports they use, as seen in the http.port property, in the discovery.seed_hosts property in the following format:

If there are three nodes in the cluster, the value in the discovery.seed_hosts property is as in the following example:

The names of all nodes appear in the cluster.initial_master_nodes property. The node name displayed in this property is same as seen in the node.name property.

Sample configuration of a node is as follows:

cluster.name:"SAG_EventDataStore"
node.name: node1
path.logs: SAG_root\InternalDataStore/logs
network.host:0.0.0.0
http.port:9240
discovery.seed_hosts: ["hostname1:9340","hostname2:9340","hostname3:9340"]
transport.tcp.port:9340
path.repo:['SAG_root\InternalDataStore/archives']
cluster.initial_master_nodes:["node1","node2","node3"]