Filebeat
API Gateway uses Filebeat to monitor the log files and forwards them to either API Data Store or external ElasticSearch. Filebeat is shipped along with API Gateway and its configuration can be found in the following location:
SAG_Install_Dir/profiles/IS_Instance_Name/apigateway/filebeat/filebeat_apigateway.yml
When log aggregation is enabled, API Gateway initiates the Filebeat process in the background to monitor the log files and forwards the log files to either API Data Store or external Elasticsearch.
API Gateway + API Data Store + Terracotta Server
In case, if you also want to aggregate Terracotta server logs that are stored in different location, perform the following steps:
If Terracotta Server and API Gateway is installed in the same machine, perform the following steps:
1. In the Administration > General > Application logs > Log aggregation section, disable the log aggregation.
Note:
By default, the Terracotta logs are available in the (user.home)/terracotta/server-logs location. You can change this folder by specifying the custom log data location in the tc-config.xml.
2. Update the filebeat_template.yml file in the SAG_Install_Dir\profiles\IS_Instance_Name\apigateway\filebeat location with the following content. Make sure you place the following content below the DashboardLogs configuration:
- type: log
# Change to true to enable this prospector configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- C:/Users/alice/terracotta/server-logs/terracotta-server.log
fields:
node: ${NODE}
fileType: TerracottaServerLogs
timezone: ${TIMEZONE}
fields_under_root: true
multiline.pattern: (([\s]+)20[0-9]{2}-)|20[0-9]{2}-
multiline.negate: true
multiline.match: after
3. In the Administration > General > Application logs > Log aggregation section, enable log aggregation.
4. In the Analytics > Application Logs section, check whether the Terracotta Server logs are aggregated.
If Terracotta Server and API Gateway are installed in different machine, perform the following steps:
1. Edit the tc-config.xml in Terracotta Server with the following content to store the logs in a network location and start the Terracotta Server:
<?xml version="1.0" encoding="UTF-8"?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config">
<servers>
<server host="localhost" name="%h">
<data>C:/install/TerraCottaServer/Terracotta/server/bin/data</data>
<logs>//worstation01/share/tcserverlog</logs>
<offheap>
<enabled>true</enabled>
<maxDataSize>512m</maxDataSize>
</offheap>
</server>
<restartable enabled="true"/>
</servers>
</tc:tc-config>
2. In the Administration > General > Application logs > Log aggregation section, disable log aggregation.
3. Update the filebeat_template.yml file in the SAG_Install_Dir\profiles\IS_Instance_Name\apigateway\filebeat location with the following content. Make sure you place the following content below the DashboardLogs configuration:
- type: log
# Change to true to enable this prospector configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- //mckmut02/share/tc-server-logs/terracotta-server.log
fields:
node: ${NODE}
fileType: TerracottaServerLogs
timezone: ${TIMEZONE}
fields_under_root: true
multiline.pattern: (([\s]+)20[0-9]{2}-)|20[0-9]{2}-
multiline.negate: true
multiline.match: after
4. In the Administration > General > Application logs > Log aggregation section, enable the log aggregation.
5. In the Analytics > Application Logs section, check whether the Terracotta Server logs are aggregated.
Configuring Log for Elasticsearch Client in API Gateway
API Gateway uses Elasticsearch REST client to connect to API Data Store or External Elasticsearch. By Default, the logs that are created by these REST clients are ignored to avoid over logging. To troubleshoot or diagnose elasticsearch calls from API Gateway, you can manually configure the log configuration for Elasticsearch REST client using the following steps:
1. Open the log4j2.properties located at SAG_Install_Dir\profiles\IS_Instance_Name\configuration\logging.
2. Add the following configuration at the end of the file:
logger.10.name=org.elasticsearch.client
logger.10.additivity=false
logger.10.level=info
logger.10.appenderRef.lar.ref=ESRestClient
logger.11.name=org.apache.http
logger.11.additivity=false
logger.11.level=info
logger.11.appenderRef.lar.ref=ESRestClient
logger.12.name=org.apache.http.wire
logger.12.additivity=false
logger.12.level=info
logger.12.appenderRef.lar.ref=ESRestClient
logger.13.name=org.apache.http.impl.conn
logger.13.additivity=false
logger.13.level=info
logger.13.appenderRef.lar.ref=ESRestClient
appender.esrestclient.name=ESRestClient
appender.esrestclient.type=RollingFile
appender.esrestclient.fileName=<INSTALL_LOCATION>/IntegrationServer/instances/default/logs/ESRestClient.log
appender.esrestclient.filePattern=<INSTALL_LOCATION>/IntegrationServer/instances/default/logsESRestClient.log.%i
appender.esrestclient.layout.type=PatternLayout
appender.esrestclient.layout.pattern=%d [%t] %-5p %c %x - %m%n
appender.esrestclient.policies.type=Policies
appender.esrestclient.policies.size.type=SizeBasedTriggeringPolicy
appender.esrestclient.policies.size.size=10MB
appender.esrestclient.strategy.type=DefaultRolloverStrategy
appender.esrestclient.strategy.max=10
#This is a custom Platform provided filter which matches all log messages that contain OSGi data in MDC (bundle.id, component.name, etc.)
appender.esrestclient.filter.osgi.type=LogServiceFilter
appender.esrestclient.filter.osgi.onMatch=DENY
appender.esrestclient.filter.osgi.onMismatch=NEUTRAL
3. Restart API Gateway.
You can see the ESRestClient.log at the SAG_Install_Dir\IntegrationServer\instances\instance_name\logs location. You can see the request sent to particular elasticsearch node and what is the response status code in the ESRestClient.log