How do I Secure API Data Store Communication using HTTPS with Search Guard Plugin ?
You can secure API Data Store (a simple Elasticsearch instance), one of the components in an API Management setup, to communicate securely over HTTPS. This section explains how to secure Elasticsearch using Search Guard, an Elasticsearch plugin, that offers encryption, authentication, and authorization to protect data from attackers and other misuses. Search Guard secures Elasticsearch by exposing it over HTTPS, and enables basic authentication by configuring users.
Before you begin
Ensure that you have:
A basic understanding of
API Gateway and its communication with API Data Store for storing data.
A basic understanding of Kibana and its communication with API Data Store for rendering the dashboards in
API Gateway.
To secure API Data Store communication using HTTPS
1. Install and initialize Search Guard plugin.
a. Shutdown API Gateway.
b. Open the command prompt to the location Installation_Dir/InternalDataStore/bin
c. Run the following command:
elasticsearch-plugin.bat install Search_Guard_plugin
file_location_in_the_file_system
d. Type y when the installation procedure prompts for additional required permissions it requires.
You should see a procedure completion message Installed search-guard-7 on successful installation.
e. Copy the folder sagconfig from Installation_Dir\IntegrationServer/instances\Instance_name\packages\WmAPIGateway\config\resources\elasticsearch to Installation_Dir\InternalDataStore.
f. Copy the certificates node-0-keystore.jks and truststore.jks from Installation_Dir\InternalDataStore\sagconfig to Installation_Dir\InternalDataStore\config.
g. Navigate to Installation_Dir\InternalDataStore\config\ and open the file elasticsearch.yml.
h. Delete all the properties that start with searchguard, if present, and add the Search Guard properties as follows:
searchguard.ssl.transport.keystore_type: JKS
searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks
searchguard.ssl.transport.keystore_alias: cn=node-0
searchguard.ssl.transport.keystore_password: a362fbcce236eb098973
searchguard.ssl.transport.truststore_type: JKS
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_alias: root-ca-chain
searchguard.ssl.transport.truststore_password: 2c0820e69e7dd5356576
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.transport.enable_openssl_if_available: true
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_type: JKS
searchguard.ssl.http.keystore_filepath: node-0-keystore.jks
searchguard.ssl.http.keystore_alias: cn=node-0
searchguard.ssl.http.keystore_password: a362fbcce236eb098973
searchguard.ssl.http.truststore_type: JKS
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_alias: root-ca-chain
searchguard.ssl.http.truststore_password: 2c0820e69e7dd5356576
searchguard.ssl.http.clientauth_mode: OPTIONAL
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"]
searchguard.authcz.admin_dn:
- "CN=sgadmin"
i. Optional. If you are using trial version of Search Guard, add the following entry.
searchguard.enterprise_modules_enabled: false
j. Save and close the file.
k. Run Installation_Dir\InternalInternalDataStore\bin\enable_ssl.bat,
This installs the Search Guard plugin and starts the API Data Store.
l. Shutdown and restart the API Data Store.
m. Navigate to Installation_Dir\InternalDataStore\plugins\search-guard-7\tools and run the following command to initialize the API Data Store.
sgadmin.bat -cd ..\..\..\sagconfig\
-ks ..\..\..\sagconfig\sgadmin-keystore.jks
-kspass 49fc2492ebbcfa7cfc5e -ts ..\..\..\sagconfig\truststore.jks
-tspass 2c0820e69e7dd5356576 -nhnv -p 9340 -cn SAG_EventDataStore
If you are using the Linux command it would be as follows:
sgadmin.sh -cd ../../../sagconfig\
-ks ../../../sagconfig/sgadmin-keystore.jks
-kspass 49fc2492ebbcfa7cfc5e -ts ../../../sagconfig\truststore.jks
-tspass 2c0820e69e7dd5356576 -nhnv -p 9340 -cn SAG_EventDataStore
2. Add users for basic authentication.
a. Navigate to Installation_Dir\InternalDataStore\sagconfig and open the sg_roles_mapping.yml file.
b. Add the username (for example, TestUser) in the users list as follows:
sg_all_access:
reserved: true
users:
"TestUser"
backend_roles:
"admin"
c. Generate the hash code for your password.
a. Run the command as follows:
Installation_Dir\InternalDataStore\
plugins\search-guard-7\tools>hash.bat.
b. Type the password.
c. Press Enter.
This generates the hash code.
d. Navigate to Installation_Dir \InternalDataStore\sagconfig and open the file sg_internal_users.yml.
e. Add the username and password as follows:
#keys cannot contain dots
#if you have a username with dots then specify it with username: xxx
Administrator:
hash: "$2a$12$sm2AEpQx6QNq6YRSYHGCnetiRWKMWrQY/udSSI0dDFZ1r3qo51bzK"
TestUser:
hash: "$2a$12$Ua1gUiWaW5/b8ohgDqTfg.ruEDNOCsuV9RexlTigNf65TvSn6/Loy"
f. Shutdown and restart the API Data Store.
API Data Store now runs on a secure channel on the HTTPS port and requests the basic authentication details.
3. Change the Kibana configuration to connect to Elasticsearch.
a. Navigate to Installation_Dir\profiles\IS_default\apigateway\dashboard\config\ and open the file, kibana.yml.
b. Uncomment the following properties and update them as follows:
elasticsearch.username:
TestUserelasticsearch.password:
TestUser@123elasticsearch.ssl.verificationMode:
certificateelasticsearch.ssl.certificateAuthorities:
file path of your root-ca.pem certificate elasticsearch.hosts:
https://hostname: 9240
Note:
The port in elasticsearch.hosts is protocol-dependent. If you are using versions of Kibana prior to 7.13.0, and the elasticsearch.url setting in the configuration file depended on an unspecified port, append :9200 to the URL in the elasticsearch.hosts setting.
Sample kibana.yml file
c. Open the uiconfiguration.properties file located at Installation_Dir\profiles\IS_default\apigateway\config and set apigw.kibana.autostart to false.
4. Change the API Gateway configuration to connect to Elasticsearch.
a. Navigate to Installation_Dir\IntegrationServer\instances\default\packages\WmAPIGateway\config\resources\elasticsearch and open config.properties file.
b. Uncomment the following properties and update them as follows:
pg.gateway.elasticsearch.http.username=TestUser
pg.gateway.elasticsearch.http.password=TestUser@123
pg.gateway.elasticsearch.https.truststore.filepath=Installation_Dir/InternalDataStore
/sagconfig/truststore.jks
pg.gateway.elasticsearch.https.truststore.password=2c0820e69e7dd5356576
pg.gateway.elasticsearch.https.enabled=true
c. Start the API Data Store manually.
d. When API Data Store is up and running, start the Kibana server manually by running the kibana.bat file located at Installation_Dir\profiles\IS_default\apigateway\dashboard\bin.
e. Start API Gateway.
You can now log on, create APIs, and access the Analytics page with the user credentials.
Configuring Search Guard with self-generated certificates
As an API Provider, if you want to generate your own certificates to use with Search Guard instead of the default certificates that are shipped with API Gateway, you can configure Search Guard with user generated certificates as Step 5. Search Guard provides an offline TLS tool. Use the tool to generate the required certificates for running Search Guard in a production environment.
1. Configure Search Guard with user generated certificates.
b. Create a YAML file at Tool Installation Directory\config
When you run the TLS tool command, it reads the node and certificate configuration settings from this YAML file, and places the generated files in a configured directory.
Sample YAML file
c. Run the following command to generate the required certificates.
Tool Installation Directory/tools/sgtlstool.bat
-c ../config/Demo.yml -ca -crt
The generated certificates are placed in the Tool Installation Directory/tools/out folder.
d. Copy the certificates listed below from the folder Tool Installation Directory/tools/out to the Installation_Dir/EventDataStore/config folder.
test-node-1.key
test-node-1.pem
test-node-1_http.pem
test-node-1_http.key
test-client.pem
test-client.key
root-ca.pem
root-ca.key
e. Configure the generated certificates in the API Data Store elasticsearch.yml file.
f. Start API Data Store manually.
A log message warns that the Search Guard is not initialized after API Data Store is up because the Search Guard is not initialized with the latest certificates.
g. Open a command prompt and change the directory to Installation_Dir\EventDataStore \plugins\search-guard-7\tools
h. Run the command
sgadmin.bat -cd ..\sagconfig -nhnv -icl -cacert
..\..\..\config\root-ca.pem -cert ..\..\..\config\test-client.pem
-key ..\..\..\config\test-client.key
-keypass your certificate password -p 9340
Done with success log message appears.
i. Shut down and restart API Data Store.
API Data Store now uses the generated certificates for SSL communication.
