Upgrading API Gateway cluster
Note:
This procedure is applicable only for on-premise installation.
Prerequisites:
Install latest fixes in both source and target versions.
Note:
The version of target API Gateway must be higher than the source API Gateway instance. Supported source API Gateway versions are 10.1 and above.
If custom keystore or truststore files are used in the source
API Gateway installation, copy the files to the same location in the target installation.
The following image illustrates the different stages in upgrading an API Gateway cluster:
The API Gateway migration utility provides the support to:
Separately migrate Elasticsearch and
API Gateway file configurations.
Migrate data from externally configured Elasticsearch.
Log all the migration data in a single file, that is migrationLog.txt.
Revert in case of failure in Elasticsearch data migration.
Note:
In the API Gateway versions 10.2 and above, the folder name EventDataStore is changed to InternalDataStore. Throughout this section, the source API Gateway installation directory is referred as <SOURCE> and the target API Gateway installation directory is referred as <TARGET>.
To upgrade an
API Gateway cluster
1. Configure the path.repo property in the source elasticsearch.yml file located at <SOURCE>\InternalDataStore\config. Make sure that the path.repo is a shared network folder and is accessible for all the Elasticsearch nodes in the cluster.
2. Configure the reindex.remote.whitelist property in the target elasticsearch.yml file for re-indexing the data in the target Elasticsearch. This YAML file is located at <TARGET>\InternalDataStore\config. The reindex.remote.whitelist property copies the documents from the <SOURCE> to <TARGET> Elasticsearch instances.
Syntax for reindex.remote.whitelist property in the target elasticsearch.yml file is as follows:
reindex.remote.whitelist: <source_host>:<source_port>
Note:
The
reindex.remote.whitelist value in the target elasticsearch.yml file must match the value of the
pg.gateway.elasticsearch.hosts property present in the config.properties file located at
<SOURCE>\IntegrationServer\instances\default\packages\WmAPIGateway\config\resource\elasticsearch.
For managed Elasticsearch instances, you do not have to configure the
reindex.remote.whitelist property.
An example for the reindex.remote.whitelist property in the elasticsearch.yml file is as follows:
cluster.name: SAG_EventDataStore
node.name: SAG-1YVHZY21633236119876
path.logs: C:\SoftwareAG_1011\InternalDataStore/logs
network.host: 0.0.0.0
http.port: 9240
discovery.seed_hosts: ["hostname1:9340","hostname2:9340","hostname3:9340"]
transport.tcp.port: 9340
path.repo: ['C:\SoftwareAG_1011\InternalDataStore/archives']
cluster.initial_master_nodes: ["node1","node2","node3"]
reindex.remote.whitelist: localhost:9240
3. Optional. Configure the source Elasticsearch host and port details in the config.properties file located at <SOURCE>\IntegrationServer\instances\default\packages\WmAPIGateway\config\resources\elasticsearch.
Syntax for Elasticsearch host and port details in the source config.properties file is as follows:
pg.gateway.elasticsearch.hosts=<source_host>:<source_port>
Configure the source Elasticsearch host and port details for all the API Gateway nodes in the cluster.
Note:
This step is applicable only if the source API Gateway version is 10.1. From versions 10.2 and above, the Elasticsearch host and port values are populated automatically.
4. Configure the basic authentication credentials to connect to the source Elasticsearch, if the source Elasticsearch is protected.
To configure the basic authentication credentials, add the following properties to the config.properties file located at <SOURCE>\IntegrationServer\instances\default\packages\WmAPIGateway\config\resources\elasticsearch:
pg.gateway.elasticsearch.http.username=<username>
pg.gateway.elasticsearch.http.password=<password>
Configure the basic authentication credentials for all the API Gateway nodes in the cluster.
5. Configure the certificates to connect to the source Elasticsearch.
If the source Elasticsearch is protected with HTTPS, add the source certificates (public key) into the target Elasticsearch JVM's truststore. For example, in case of API Data Store, add the public keys to the truststore cacerts file located at <TARGET>\jvm\jvm\jre\lib\security. Do for all the target API Gateway nodes.
Note:
If, external Elasticsearch is used for the target API Gateway, then the certificates must be imported to its corresponding JVM.
Sample command to import the truststore of the source Elasticsearch into the target Elasticsearch JVM is as follows:
<TARGET>\jvm\jvm\bin\keytool -import -keystore <TARGET>\jvm\jvm\jre\lib\security\cacerts -file <truststore.jks> -alias <alias>
Property | Description |
truststore.jks | Truststore used in <SOURCE> Elasticsearch. Provide the full path of the truststore. For 10.1, it is be available at <SOURCE>\WmAPIGateway\config\resources\bean\gateway-es-store.xml and the property is <prop key="searchguard.ssl.http.truststore_filepath"\>sagconfig/root-ca.jks\</prop> For 10.2 and above, it is available at <SOURCE>\WmAPIGateway\config\resources\elasticsearch\config.properties and the property is pg.gateway.elasticsearch.https.truststore.filepath. Example: sagconfig/root-ca.jks |
alias | Alias used in <SOURCE> Elasticsearch. Example: wm.sag.com |
6. Start both the source and target Elasticsearch instances and make sure that API Gateway (Integration Server) instances are not started.
Note:
Avoid port conflict. If the source and target API Gateway instances are running in the same machine, then you might not be able to start both the source and target Elasticsearch instances in parallel with the default port configurations. In this case, the target Elasticsearch instance port can be changed temporarily for performing the migration. Both, HTTP and TCP ports must be changed. To change the target Elasticsearch instance ports:
a. Open the elasticsearch.yml file located at <TARGET>/InternalDataStore/config. Change the value of the HTTP port in the http.port property, and TCP port in the transport.tcp.port property.
b. Open the config.properties file located at <TARGET>/IntegrationServer/instances/default/packages/WmAPIGateway/config/resources/elasticsearch and find the pg.gateway.elasticsearch.hosts property. If the property is set to changeOnInstall, then do not make further changes to property. If a port is configured, then update it to a new value.
Revert the temporary target Elasticsearch instance ports to the default port configurations, after completing the migration and the target Elasticsearch instance is shutdown.
If you are reverting the elasticsearch ports in the elasticsearch.yml file after the migration, then you need to specify the same ports in the config.properties file located at <TARGET>/IntegrationServer/instances/default/packages/WmAPIGateway/config/resources/elasticsearch.
7. Migrate the API Gateway assets and the analytics data. Go to <TARGET>\IntegrationServer\instances\default\packages\WmAPIGateway\bin\migrate and run the following migration utility command.
Note:
Before running the migration utility command, set up the target Elasticsearch cluster and make sure that all the source and target Elasticsearch instances in the clusters are up and running. Also, do not start the API Gateway instances on any of the nodes.
migrate.bat datastore -dstoreSrc <full path to source Elasticsearch config.properties>
Note:
You should run the migrate.bat datastore command in only one target API Gateway node of the cluster.
Property | Description |
dstoreSrc | If the source and target API Gateway instances are running on the same machine, provide the location where <SOURCE> config.properties file is located. Sample command is as follows: migrate.bat datastore -dstoreSrc<SOURCE>\IntegrationServer\instances\default\packages\
WmAPIGateway\config\resources\elasticsearch\config.properties |
If the source and target instances are running on different machines, then the source installation directory or the Elasticsearch config.properties file must be shared in the network. Otherwise, copy and paste the source config.properties file to the shared location. Sample command is as follows: migrate.bat datastore -dstoreSrc\chebackup01\installations\source\IntegrationServer\instances\default\
packages\WmAPIGateway\config\resources\elasticsearch\config.properties |
8. Migrate the platform configurations.
Ignore this step, if you choose to set up the target platform configurations manually. Alternatively, if you choose to migrate the platform configurations using externalized configurations, you can still use the migrate.bat apigateway command to migrate the platform configurations. To use the migrate.bat apigateway command, go to <TARGET>\IntegrationServer\instances\default\packages\WmAPIGateway\bin\migrate and run the following command.
migrate.bat apigateway -srcDir <SOURCE> -instanceName <source instance name> -newInstanceName <target instance name>
Note:
You should run the migrate.bat apigateway command in all the target API Gateway nodes of the cluster.
Property | Description |
srcDir | If the source API Gateway instance is installed on the same machine, provide the source API Gateway installation directory. Sample command is as follows: -srcDir C:\installations\source |
If the source API Gateway instance is installed on a different machine, then share the entire installation folder. Sample command is as follows: -srcDir \chebackup01\source |
instanceName | Optional. Provide the <SOURCE> instance name that you want to migrate. If you do not provide any name, then the default instance is used. Sample command is as follows: -instanceName test |
newInstanceName | Optional. Provide the <TARGET> instance name that you want to migrate to. If you do not provide any name, then the default instance is used. If you have created a new instance other than the default in Integration Server and you want to migrate to the new instance then provide its name. Sample command is as follows: -newInstanceName prod |
9. Perform these steps after migrating the platform data and platform logs.
a. Shutdown the target Elasticsearch instance.
b. Start the target API Gateway server.
The API Gateway upgrade is complete.
You can find the logs at the target directory
<TARGET>/install/logs/migrationLog.txt.
You can find the
API Gateway migration reports at
<TARGET>\install\logs\APIGW_Migration_Reports_<date_time>.
