Removing User Data

Data protection laws and regulations, such as the General Data Protection Regulation (GDPR) might require specific handling of user data, even after a user profile is removed. Additionally, employees or other clients with user accounts on API Gateway may request that any user identifying information such as user name, email addresses, or client IP addresses be removed from API Gateway. To comply with data protection requirements and user requests, in addition to deleting the user account, you may need to complete activities such as deleting or masking the user data.

Note:
API Gateway can optionally capture the runtime transaction logs, which contain the API request and response data (customer-defined) that flow through to the API Gateway. Though there are options to purge or clean up these data, this section does not define procedure for the same as the customer-defined data is out of the scope of this functionality.

This consists of APIs, policies, applications, aliases, packages, plans, administration configurations, users, and groups. This is stored in API Data Store and Integration Server store.

This consists of the transaction events, monitoring events, error events, policy violation events, threat protection events, lifecycle events and performance metrics. This information can be stored in API Data Store or external destinations.

This consists of UI, server, Elasticsearch, Kibana, filebeat, and Platform logs . This is stored in filesystem and API Data Store.

This consists of the runtime transactions that are captured when you trace the API. This information can be stored in API Data Store.

API Gateway strongly recommends the use of internal or technical user for policy configurations like Authorize user, Invoke IS Service and Outbound Authentication that has user credentials. This avoids the life cycle of actual or real user object from impacting the API Gateway configurations.

In case, real users are used in policy configurations, then the API Provider or API Administrators should take the responsibility of changing the configurations once the user is deleted from the system.

In API Gateway, as you increase the log level to Debug or Trace, there can be messages that include the userid. So when a user has to be deleted from the system, Administrators have to clean up this data. The application logs are stored in file system till API Gateway version 10.2. From API Gateway 10.3, API Administrators have an option to persist the logs additionally in API Data Store.

The following is a sample query run to mask the user data in the application log stored either in the API Data Store or an external data store.

curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json'
http://hostname:port/gateway_default_log/doc/_update_by_query -d ' {
"script": { "id": "findAndReplace", "params": {
"find": "user123", "replace": "******"} } }

hostname:port refers to the host name and port of the system where the API Data Store or the external data store that contains the data resides.

The field Replace contains the string that is used to mask the data mentioned in the Find field and replaces the data with the string provided in the logs.

Logs are always persisted in a file system. You can perform a search and replace using text editing tools. For example, you could search all server.log files for the id of the user to be deleted and replace it with anonymous or blank string. Following logs need to be cleaned up.

Install Directory\Integration Server\instances\Instance_Name\logs\server.log. For details to cleanup other Integration Server related logs, see webMethods Integration Server Administrator’s Guide.

If enabled, audit logs would have information about user actions. This contains user reference and has to be cleaned up after user deletion. You can achieve the cleanup in the following ways:

As an Administrator, you can clean up the audit logs persisted in API Data Store by running the following curl query to replace or mask the desired data.

curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json'
http://hostname:port/gateway_default_audit_auditlogs/_update_by_query -d ' {
"script": { "id": "findAndReplace", "params": {
"find": "user123", "replace": "******"} } } '

The field Replace contains the string that is used to mask the data mentioned in the Find field and replaces the data with the string provided in the logs.

As an Administrator, you can use the extended setting saveAuditlogsWithPayload to not store the request payloads in the audit logs. Though this does not completely eliminate the user information in audit logs it certainly minimizes the occurrences. Software AG recommends you to use this property with caution as turning on this option might lead to less audit data being captured.

If you enable the tracer, the data that API Gateway captures might have user-specific information. You can either mask or remove the user data from the server log trace span, mediator trace span, and request response trace span indices. You can either mask full or partial text in the user data.

The following sample query is used to mask the user data in the server log trace span:

curl -X POST http(s)://elasticsearch_hostname:elasticsearch_port/
gateway_{tenant}_serverlogtracespans/_doc/_update_by_query?pretty -H 'Content-Type: application/json' -d'
{
"script": {
"id": "findAndReplaceInTracerData",
"params": {
"find": "textToFind",
"replace": "textToReplace"
}
}
}'

The following sample query is used to mask the user data in the mediator trace span:

curl -X POST http(s)://elasticsearch_hostname:elasticsearch_port/
gateway_{tenant}_mediatortracespan/_doc/_update_by_query?pretty -H 'Content-Type: application/json' -d'
{
"script": {
"id": "findAndReplaceInTracerData",
"params": {
"find": "textToFind",
"replace": "textToReplace"
}
}
}
'

The following sample query is used to mask the user data in the request response trace span:

curl -X POST http(s)://elasticsearch_hostname:elasticsearch_port/
gateway_{tenant}_requestresponsetracespans/_doc/_update_by_query?pretty -H 'Content-Type: application/json' -d'
{
"script": {
"id": "findAndReplaceInTracerData",
"params": {
"find": "textToFind",
"replace": "textToReplace"
}
}
}
'

The field Replace contains the string that is used to mask the data mentioned in the Find field and replaces the data with the string provided in the server log trace span, mediator trace span, and request response trace span indices.