BigMemory Max driver details
Note:
Support for using BigMemory Max for queries is deprecated and will be removed in a future release. It is recommended that you now use Terracotta's TCStore for queries. For queries and applications that do not involve caching, the TCStore driver is now the recommended driver for the distributed MemoryStore.
Apama continues to support the BigMemory Max driver. This is still the recommended driver for caching use cases, but no longer for queries.
Using BigMemory Max from the Apama MemoryStore
For reference, the following table maps Apama MemoryStore terminology to BigMemory Max classes; this may be useful when referring to the BigMemory Max documentation:
MemoryStore Event Object | BigMemory Max Class |
Store | CacheManager |
Table | Cache |
Row | Element |
By default, a distributed MemoryStore Store uses the BigMemory Max default cache manager. To specify the use of a different cache manager, specify the name property on the configuration bean. For example:
<property name="configuration.name" value="myCacheManager"/>
In a cluster, if one correlator calls subscribeRowChanged() for a given MemoryStore table, then all correlators in that cluster that modify the entries in that table must also call subscribeRowChanged() on that table even if they do not consume the events.
Iterating over a table may require pulling the entire table into memory. It may fail if the table is being modified.
If accessing a BigMemory Max table from Apama and non-Apama applications, clients will need the correct cache configuration (available from the Terracotta Management Console) and have the appropriate Apama classes available on their classpath (available in the distmemstore and ap-distmemstore-bigmemory.jar files) in order to access the cache.
Configuring the BigMemory Max driver
You can create configuration files for BigMemory Max when using Apama in Software AG Designer. The providerDir property should be set to the location of the BigMemory Max installation whose client libraries will be used, which is typically in the same location as Apama.
The driver for BigMemory Max is configured as follows:
You can set BigMemory Max driver properties (described in the table below) in the
storeName-spring.xml configuration file. Alternatively, you can specify many of these properties in an
ehcache.xml configuration file and then specify the path for that file in the
storeName-spring.xml configuration file using the
ehcacheConfigFile property. If this is done, many of the properties in the
storeName-spring.xml configuration file will be ignored; the settings derived from the
ehcache.xml file will be used instead.
Use the
storeName-spring.properties file to set configuration properties for the BigMemory Max driver.
Using off-heap storage requires setting
-XX:MaxDirectMemorySize=. Specify this in the command line for starting the correlator as
-J-XX:MaxDirectMemorySize=. The BigMemory Max documentation provides recommendations for specifying the value of this property. When you add a correlator to a correlator launch configuration in
Software AG Designer, you can select the
Maximum Java off-heap storage in MB option. See
Correlator arguments.
When using the BigMemory Max driver, all correlators accessing the same data in a BigMemory Max cluster must have the same configuration.
Driver bean properties with equivalents in ehcache.xml
The following properties can be set either in the driver properties, or using a
ehcache.xml configuration file. See the
BigMemory Max documentation for more details. For more information on the Ehcache types mentioned below, see the Ehcache Javadoc and search for the required type such as
CacheConfiguration.
Property Name | Description |
cacheConfiguration | Type: CacheConfiguration Ehcache CacheConfiguration bean, shared by all caches (Tables). Typically used as a compound bean name, for example, cacheConfiguration.overflowToOffHeap. |
cacheConfiguration.eternal | Type: boolean Disables expiration (removing old, unused values) of entries if true. Set to true in the default storeName-spring.xml configuration file. |
cacheConfiguration.
maxEntriesLocalHeap | Type: int The number of entries for each table. This is the maxEntriesLocalHeap entry in the .properties file. |
cacheConfiguration.
overflowToOffHeap | Type: boolean Whether to use off-heap storage. For scenarios where data is fast changing and being written from multiple correlators, the cache may perform better if this is disabled. This is the cacheConfiguration.overflowToOffHeap entry in the .properties file. |
cacheDecoratorFactory | Type: String Name of a class to use as a cacheDecoratorFactory. The named class must be on the classpath and must implement Ehcache's CacheDecoratorFactory interface. |
cacheDecoratorFactoryProperties | Type: Properties Properties to pass to a cacheDecoratorFactory. Allows use of the same class for many caches. |
clusterName | Type: String Comma-separated list of host:port identifiers for the servers, or a tc-config.xml file name. Best practice is to list all Terracotta Server Array (TSA) nodes. |
configuration | Type: Configuration Ehcache Configuration bean. Typically used as a compound bean name, for example, configuration.monitoring. |
configuration.name | Type: String By default, the BigMemory Max default cache manager is used. Use this property to specify the use of a different cache manager. |
maxMBLocalOffHeap | Type: long Number of MB of local off-heap data. Total across all tables, per correlator process. |
pinning | Type: String Either an attribute value of "inCache" (default) or "localMemory" or a <null/> XML element (that is, <property name="pinning"><null/></property>.) Pinning prevents eviction if the cache size exceeds the configured maximum size. Recommended if the cache is being used as a system of record. |
terracottaConfiguration | Type: TerracottaConfiguration Ehcache TerracottaConfiguration bean. Typically used as a compound bean name, for example, terracottaConfiguration.consistency. |
terracottaConfiguration.
clustered | Type: boolean Whether to use a TSA. Set to true in the default storeName-spring.xml configuration file. |
terracottaConfiguration.
consistency | Type: String Either 'STRONG' or 'EVENTUAL'. STRONG gives MemoryStore-like guarantees, while EVENTUAL is faster but may have stale values read. This is the terracottaConfiguration.consistency entry in the .properties file. |
terracottaConfiguration.
localCacheEnabled | Type: boolean Whether to cache entries in the correlator process. Set to true in the default storeName-spring.xml configuration file. |
terracottaConfiguration.
synchronousWrites | Type: boolean If true, then data is guaranteed to be out of process by the time a Row.commit() action completes. Disabling this can increase speed. This is the terracottaConfiguration.synchronousWrites entry in the .properties file. |
Driver bean properties with no equivalents in ehcache.xml
You can set the following BigMemory Max driver properties in the storeName-spring.xml configuration file, but not in the ehcache.xml configuration file as they modify how the driver accesses the BigMemory Max cache.
Property Name | Description |
backupCopies | Type: int Ignored. Not supported. The number of backups is governed by the TSA topology defined in the
BigMemory Max documentation and used to configure the TSA nodes. |
converterConfig.default | Type: RowKeyValueConverter bean Specifies the default converter to be used for all tables which do not explicitly specify a converter. A converter is a bean of type RowKeyValueConverter. It is used to convert non-Apama format Ehcache key/value pairs to Apama format key/value pairs. If not specified, then Apama's default converter is used which assumes that the key/value pairs in Ehcache are in Apama format. For example: <property name="converterConfig.default" ref="MyDefaultConverter"/> See the
API Reference for Java (Javadoc) and the samples in the samples\distmemstore\bigmemory\converters directory of your Apama installation for more information about RowKeyValueConverter which can be found in the com.apama.correlator.memstore package. |
converterConfig.byTable | Type: Map(String, RowKeyValueConverter) Per-table converter configuration. If a converter is not specified for a table, the default converter specified by the converterConfig.default property is used. For example: <property name="converterConfig.byTable">
<map>
<entry key="SensorTable">
<bean class="SensorDataConverter" />
</entry>
</map>
</property> |
ehcacheConfigFile | Type: String Path to an ehcache.xml configuration file. Note:
If this is specified, any other properties listed in this table will be ignored. |
exposeSearchAttributes | Type: boolean Enable exposing search attributes. If true, then the MemoryStore schema columns are exposed as BigMemory Max search attributes and are indexed, so that other clients of BigMemory Max can perform searches on the data set. If exposeSearchAttributesSet is non-empty, then only the named columns are exposed as BigMemory Max search attributes. See notes below about non-Apama applications accessing the data in a BigMemory Max cluster. |
exposeSearchAttributesSet | Type: Set(String) Limits the set of columns in each table that should be exposed as search attributes. Entries are in the form tableName.columnName. If empty, all schema columns are exposed as search attributes. There is an incremental cost per column that is exposed, so for performance, only expose the columns which need to be used in searches. For example, to expose only the Surname and FirstName columns of myTable: |
initialMinClusterSize | Type: int The minimum cluster size (number of correlators) that must be connected for prepare to finish. |
logLevel | Type: String The log level. |
rowChangedOldValueRequired | Type: boolean Whether to expose old values in rowChanged events. Must be set to false. Note:
BigMemory Max currently sends notifications for rows removed by a Table.clear() operation, but this may change in a future release. |
useCompareAndSwap | Type: boolean Whether to use compare and swap (CaS) operations or just put/remove. Some versions of BigMemory Max support only CaS in Strong consistency. Important:
For correct behavior of Apama queries, you must leave the useCompareAndSwap property in its default (true) setting. |
useCompareAndSwapMap | Type: Map(String, Boolean) Per-table (cache) configuration for whether to use CaS or put/remove. |