Removing User Data

Data protection laws and regulations, such as the General Data Protection Regulation (GDPR) might require specific handling of user data, even after a user profile is removed. Additionally, employees or other clients with user accounts on API Gateway may request that any user identifying information such as user name, email addresses, or client IP addresses be removed from API Gateway. To comply with data protection requirements and user requests, in addition to deleting the user account, you may need to complete activities such as deleting or masking the user data.

Note:
API Gateway can optionally capture the runtime transaction logs, which contain the API request and response data (customer-defined) that flow through to the API Gateway. Though there are options to purge or clean up these data, this section does not define procedure for the same as the customer-defined data is out of the scope of this functionality.

This consists of APIs, policies, applications, aliases, packages, plans, administration configurations, users, and groups. This is stored in API Gateway Data Store and Integration Server store.

This consists of the transaction events, monitoring events, error events, policy violation events, threat protection events, lifecycle events and performance metrics. This information can be stored in Internal Data Store or external destinations.

This consists of UI, server, Elasticsearch, Kibana, filebeat, and Platform logs . This is stored in filesystem and Internal Data Store.

API Gateway strongly recommends the use of internal or technical user for policy configurations like Authorize user, Invoke IS Service and Outbound Authentication that has user credentials. This avoids the life cycle of actual or real user object from impacting the API Gateway configurations.

In case, real users are used in policy configurations, then the API Provider or API Administrators should take the responsibility of changing the configurations once the user is deleted from the system.

In API Gateway, as you increase the log level to Debug or Trace, there can be messages that include the userid. So when a user has to be deleted from the system, Administrators have to clean up this data. The Application logs are stored in file system till API Gateway version 10.2. From API Gateway 10.3, API Administrators have an option to persist the logs additionally in Internal Data Store.

The following is a sample query run to mask the user data in the application log stored either in the Internal Data Store or an external data store.

curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json'
http://hostname:port/gateway_default_log/doc/_update_by_query -d ' {
"script": { "id": "findAndReplace", "params": {
"find": "user123", "replace": "******"} } }

hostname:port refers to the host name and port of the system where the Internal Data Store or the external data store that contains the data resides.

The field Replace contains the string that is used to mask the data mentioned in the Find field and replaces the data with the string provided in the logs.

Logs are always persisted in filesystem. You can perform a search and replace using text editing tools. For example, you could search all server.log files for the id of the user you want to delete and replace it with anonymous or blank string. You have to clean up the following logs:

SAG_InstallDirectory\Integration Server\instances\Instance_Name\logs\server.log. For details to cleanup other Integration Server related logs, see webMethods Integration Server Administrator’s Guide.

If enabled, Audit logs would have information about user actions. This contains user reference and has to be cleaned up after user deletion. You can achieve the cleanup in the following ways:

As an Administrator, you can clean up the Audit logs persisted in API Gateway Data Store by running the following curl query to replace or mask the desired data.

curl -X POST -H 'Accept: application/json' -H 'Content-Type: application/json'
http://hostname:port/gateway_default_audit/auditlogs/_update_by_query -d ' {
"script": { "id": "findAndReplace", "params": {
"find": "user123", "replace": "******"} } } '

hostname:port refers to the host name of the system where the Internal Data Store resides and the corresponding port.

The field Replace contains the string that is used to mask the data mentioned in the Find field and replaces the data with the string provided in the logs.

As an Administrator, you can use the extended setting saveAuditlogsWithPayload to not store the request payloads in the audit logs. Though this does not completely eliminate the user information in audit logs it certainly minimizes the occurrences. Software AG recommends you to use this property with caution as turning on this option might lead to less audit data being captured.