Storage Services
Use Storage services to insert, retrieve, update, and remove entries from a data store.
When using the storage services, keep in mind that the short-term store is not intended to be used as a general-purpose storage engine. Rather, it is primarily provided to support shared storage of application resources and transient data in Integration Cloud. It is recommended not to use the short-term store to process high volumes, large data records, or to permanently archive records.
Note:
User specific data which may be considered as personal data will be stored and retained till the retention period defined in Execution Results.
Note:
These services are a tool for maintaining state information in the short-term store. It is up to the developer of the Integration to make sure that the Integration keeps track of its state and correctly handles restarts.
Locking Considerations
The following sections describe in general how the storage services handle locking requests.
Entry Locking
To maintain data integrity, the short-term store uses locking to ensure that multiple threads do not modify the same entry at the same time. For insertions and removals, the short-term store sets and releases the lock. For updates, the client must set and release the lock. Using locking improperly, that is, creating a lock but not releasing it, can cause deadlocks in the short-term store.
The following guidelines can help you avoid short-term store deadlocks:
Release locks in the thread through which they were set. In other words, you cannot set a lock in one thread and release it in another. The safest way to do this is to release each lock in the Integration that acquired it.
Unlock entries before the Integration completes. Entries remain locked until released using a put or an explicit unlock. To accomplish this, always pair a call to get or lock with a call to put or unlock so that every lock is followed by an unlock. In addition, use a try-catch pattern in your Integration so that an exception does not prevent the Integration from continuing and releasing the lock.
Data Store Locking
When a storage service locks an entry, the service also implicitly locks the data store in which the entry resides. This behavior prevents another thread from deleting the entire data store and the entries it contains while your thread is working with the entry. When the locked entry is unlocked, the implicit lock on the data store is also released.
Be careful when explicitly unlocking data stores. Consider the following example:
1. User_A locks an item. This creates two locks: an explicit lock on the entry, and an implicit lock on the data store.
2. User_A later unlocks the data store explicitly while still holding the lock on the entry.
3. User_B locks, then deletes the data store, including the entry locked by User_A in the first step.
When User_A explicitly unlocked the data store in step 2, User_B was able to delete the entry the User_A was working with.
Automatic Promotion to Exclusive Lock
If a storage service tries to acquire an exclusive lock on an object, but finds a shared lock from the same thread already in place on the object, the service will try to promote the lock to an exclusive lock.
If a storage service that requires an exclusive lock encounters a shared or exclusive lock held by another thread, it will wait until the object becomes available. If the object remains locked for the period specified by the waitlength parameter passed by the service, the service will fail.
Sample Integration for Checkpoint Restart
The following diagram shows how to create checkpoint restarts into your Integrations. It explains the logic of an Integration and shows where the various storage services are used to achieve checkpoint restarts.
The following Storage services are available:
Element | Package and Description |
| Inserts a new entry into a data store. |
| Deletes a data store and all its contents. Any data in the data store is deleted. If the data store does not exist, the service takes no action. |
| Retrieves a value from a data store and locks the entry and the data store on behalf of the thread that invoked the service. |
| Obtains a list of all the keys in a data store. |
| Locks an entry and/or data store on behalf of the thread invoking this service. |
| Inserts or updates an entry in a data store. If the key does not exist in the data store, the entry is inserted. |
| Removes an entry from a data store. |
| Unlocks an entry or a data store. |
