TCStore (Terracotta) driver details
Apama provides a MemoryStore driver for Terracotta. This driver uses the TCStore API to allow Apama to read and write records in TCStore datasets, which may also be read and written by other Apama correlators and non-Apama components such as other Software AG products or custom clients written against the TCStore API.
The TCStore client libraries are installed automatically with the Apama server. So there is no need to explicitly select them at installation time, or to specify their location in the driver configuration.
Note:
Apama's TCStore driver only supports clustered mode. Currently, there is no support for embedded TCStore, for performing searches from Apama, or for the Ehcache API.
BigMemory Max is the recommended driver for caching use cases (see also
BigMemory Max driver details). For Apama queries and applications that do
not involve caching (such as store and "system of record" use cases), TCStore is the recommended MemoryStore driver.
Using TCStore from the Apama MemoryStore
The following table maps Apama MemoryStore terminology to TCStore classes; this may be useful when referring to the
Terracotta documentation:
Apama MemoryStore Event API Name | TCStore Class |
Store | DatasetManager |
Table | Dataset |
Row | Record |
Field value of a Row | Cell |
Apama schemas and TCStore
Apama applications using the MemoryStore always specify a schema indicating the names and types of the fields (cell definitions) when preparing a table for use.
TCStore does not have a concept of schemas, as records in a given table are not required to have the same cell definitions. Row fields (cells) named in the schema during table preparation can always be accessed from Apama. It is not necessary for every field named in the schema to exist in a given row (record). Apama simply returns a default value (for example, an empty value, 0 or false) when reading any field listed in the schema that is not present in a given row. It is also possible to access extra fields that are present in an individual row (record) but not listed in the schema, using any of the getXXX actions or toDictionary(). If a getXXX action is called with a key that is not present in the schema and is also not present as an extra field on the row, it will throw an exception. You can call getKeys() to see a list of all the fields (schema and non-schema) accessible.
Apama can therefore make use of non-homogenous data written to TCStore by other clients. If there is no strictly defined schema for a dataset, it is possible to not specify any fields in the schema, subject to the above restrictions. It is also possible for different Apama applications or different versions of the same application to interact with a single TCStore table using different schemas. When developing Apama applications, you should remember that all rows in a table do not necessarily contain the same fields, and ensure that all clients are using the same types for each field.
Apama can also dynamically add and remove additional non-schema fields. Any set method on a Row for a new field will create a new field on the row. Non-schema fields can be removed either with the removeNonSchema method or by setting the field to an empty any object.
Note:
All clients using a Dataset should agree on using the same types for a given cell name. The behavior when multiple clients read and write a row field (cell) with the same field name but different types is undefined, and may change in future releases of Apama.
Apama can only access cells of the Java types Long (maps to integer in EPL), Double (maps to float in EPL), String (for string or decimal), and Boolean. The decimal type is not supported by TCStore, so the driver converts to or from a string automatically for decimal fields defined in the schema. For non-schema fields, the EPL application needs to convert between string and decimal manually.
Configuring the TCStore driver
You can create configuration files for the TCStore driver using Software AG Designer. The only required property is the list of Terracotta servers to connect to.
The following table lists the properties that can be configured in the storeName-spring.xml file for the TCStore driver. The most commonly used properties also have ${varname.storeName} substitution variables in the storeName-spring.properties file to make it easy to change the values by editing the properties file manually or using the Distributed MemoryStore editor in Software AG Designer.
Property Name | Description |
servers | Type: string A comma-separated list of host1:tsa-port1,host2:tsa-port2,... values for each server from one of the stripes in the cluster, that is, all the servers defined in the Terracotta server configuration. Servers from other stripes are found automatically. A terracotta:// URI prefix can optionally be added to the first host:port, but is not required. At least one server must be specified. If high availability is required, multiple servers should be specified. See the Terracotta documentation for detailed information about cluster architecture. |
useCompareAndSwap | Type: boolean Specifies whether to provide safe atomic updating of rows by multiple clients. This means that a commit only succeeds if the value last seen by the client before it made the change matches the current value at the time of the commit. The default is true, which is the recommended setting for most use cases. Compare and swap can be disabled if each TCStore client (and thread) is known to have exclusive access to a given row, or if best effort rather than fully reliable behavior is required, which is likely to result in improved performance. If compare and swap is disabled, MemoryStore commit() operations do not fail as a result of writes made by other clients. Important:
For correct behavior of Apama queries, you must leave the useCompareAndSwap property in its default (true) setting. |
classpath | Type: string Specifies the path of the driver. This property should not be changed from its default value. |
connectionTimeoutSecs | Type: integer Specifies the timeout used when establishing the initial connection to the Terracotta server while preparing the store. The default is 0, which uses the standard timeout supplied by TCStore. See the Terracotta documentation on withConnectionTimeout for further details. |
logLevel | Type: string Specifies the verbosity of log messages from the Terracotta client. It does not affect the verbosity of the Apama TCStore driver. The default value for this is to remain unset, which uses the same log level for Terracotta as the correlator's main log level. |
clusterName | Type: string Not recommended for use. Specifies a display name that is logged at startup. This setting has no impact on behavior, and does not have to match the name of the connected Terracotta cluster. The default is hardcoded: Terracotta. |
enforceApamaSchema | Type: string Not recommended for use. Specifies whether Apama should attempt to detect use of conflicting schemas by multiple Apama clients using the same Terracotta cluster. The default is false, as for typical TCStore use cases this checking is not useful. |
securityRootDirectory | Type: string Specifies the path to the security root directory, if using TLS. For more information, see the Terracotta documentation. |
The following properties are configured in the same files, but are used only when Apama creates a new dataset (table) as a result of trying to open a dataset that does not yet exist. These settings have no effect if the dataset was already created, whether by Apama or another TCStore client.
These settings apply to all datasets created by a given Apama store. If it is necessary to create datasets with different settings on the same Terracotta cluster, you have to configure separate Apama stores for each group of datasets needing the same configuration, or use another tool or client to create the datasets. To change the dataset settings after creation, you have to delete any data directories and restart the cluster. See the Terracotta documentation for more information.
The dataset creation configuration properties are:
Property Name | Description |
offheapResourceName | Type: string Specifies the name of a resource defined in the Terracotta server configuration to be used for off-heap storage of the dataset contents. This is a required setting when creating datasets from Apama. |
enableDatasetPersistence | Type: boolean Specifies whether dataset contents are persisted on disk. The default is false, indicating that dataset contents should be held only in memory. |
persistenceDataRootName | Type: string Specifies the logical name of a data directory defined in the Terracotta server configuration. Note that this is a logical name; it is not a directory path. This property only has an effect if enableDatasetPersistence is true. |
The following standard configuration properties are not used by this driver and are either ignored or rejected if set:
backupCopiesinitialMinClusterSize clusterName rowChangedOldValueRequired (cannot be set to
true for this driver).
Note:
TCStore currently sends notifications for rows removed by a Table.clear() operation, but this may change in a future release.
Note:
The standard enforceApamaSchema property is set to false by default for TCStore, which is the recommended setting in almost all cases.