Declared Versus Dynamic In-Memory Stores
Each In-Memory Store holds one dataset with any number of rows, also known as entries, within the memory constraints defined for the store. In previous releases you could store many different datasets in one store with a dataset being one entry in the store.
As shown in the following figure, there are two ways to create In-Memory Stores: declared and dynamic.
Dynamic In-Memory Stores
Dynamic stores are created programmatically on the fly when:
MashZone NextGen Analytics first stores a dataset to a store that does not yet exist.
By default, MashZone NextGen Analytics sets the overflowToOffHeap property to true for dynamic stores. All other properties use their default values.
You can set additional configuration properties to use with new dynamic stores created by
MashZone NextGen Analytics in the
dynamiccache.xml configuration file in the
MashZoneNG-config/classes folder. For more information on these properties, see
BigMemory documentation.
In addition, each column in the datasets for dynamic In-Memory Stores created by MashZone NextGen Analytics is defined as searchable with the known or derived datatype for that column. For XML or CSV datasets, columns may all be considered string unless a schema is provided for the dataset. This also uses the MashZone NextGen Analytics class de.rtm.adapters.bigmemory.RecordMetaDataAttributeExtractor.
An external system creates a store dynamically in
BigMemory. These stores are also known as
dynamic external stores because the external system is also the system the stores data in the store.
Note: | Access to dynamic external stores requires BigMemory be installed as a server or server array. MashZone NextGen must also have access to the BigMemory license. |
To connect to dynamic external stores, a MashZone NextGen administrator must define connection configuration. The Terracotta Management Console (TMC) must also be running to successfully connect to a dynamic external in-memory store.
MashZone NextGen Analytics uses the connection configuration to retrieve configuration for the store from the Terracotta Management Console (TMC) that manages the BigMemory host for this store. This includes search attribute information that is required to allow MashZone NextGen Analytics to work with the columns in this dataset.
One common example is the use of a dynamic external store to allow MashZone NextGen to work with datasets from distributed stores in the Apama MemoryStore.
Declared In-Memory Stores
Declared stores are defined in BigMemory configuration files (ehcache.xml files) that MashZone NextGen administrators add to the MashZone NextGen Server to allow you to work with these stores. S
Declared In-Memory Stores can contain data loaded by MashZone NextGen or data loaded by external systems. With declared in-memory stores, the producer system that stores data in the store is the system that creates the store. MashZone NextGen Analytics is always the consumer system that retrieves data from the store for analysis.
Declaring stores before use allows you to:
Customize store properties for each store, providing better memory management and data retention.
Define search attributes for the columns in the dataset in each store.
Column search attributes are required to allow MashZone NextGen Analytics to work with datasets stored by external systems. It is optional, but a best practice for datasets stored by MashZone NextGen Analytics.
Defining search information also allows MashZone NextGen Analytics to delegate filtering and sorting to the In-Memory Store which provides better performance.
