External versus Internal Datasets
This figure also illustrates how datasets are stored in in-memory stores:
When Presto Analytics stores data in an In-Memory Store, it appends the data to any existing data by default. You can also clear existing data before storing new data in the <storeto> statement.
Other systems can also store data in In-Memory Stores. This is external data and the store is sometimes called an external in-memory store.
In both cases, the system that first stores the dataset is the system that creates the In-Memory Store.
To allow Presto Analytics to work with external data, external memory stores must be configured with:
The name of the
cache manager that manages memory for the in-memory store.
Note: | BigMemory does not require cache manager names, but they are a best practice for caches used as Presto Analytics In-Memory Stores. Cache manager names prevent potential name collisions for stores. |
Search attributes for the dataset that identify the columns in the dataset and the datatype of the data in each column.
Configuration for declared stores specifies the cache manager, search attributes and connection information for the store.
For an example, see Declare
BigMemory
Stores for
Presto Analytics.With dynamic stores that hold external data, the external system must set a name for the cache manager and define search attributes programmatically when the external system creates the store, using the
BigMemory API. See
BigMemory Documentation for more information and examples.
Configuration defined in Presto for the dynamic store allows Presto Analytics to retrieve this configuration information.
