Configuration files for distributed stores
The configuration for a distributed store consists of a set of .xml and .properties files. Each distributed store in a project will have the following files:
storeName-spring.xmlstoreName.properties A distributed store is configured using a bean element in the Spring XML configuration file. The bean element has the following attributes:
id – The unique name for this distributed store, which must match the name used in calls to
Storage.prepareDistributed() and
Storage.open() in EPL.
class – The name of the
StoreFactory implementation used by this distributed store.
When the correlator is started with the --distMemStoreConfig configDir argument, it will load all XML files matching *-spring.xml in the specified configuration directory, and also all *-spring.properties files in the same directory. (Note, the correlator will not start unless the specified directory contains at least one configuration file.)
When using Apama Studio, these files are generated automatically. New storeName-spring.xml and storeName.properties files are created when a Store is added to a project. The most commonly used settings can be changed at any time using the Distributed MemoryStore editor (which rewrites the .properties file whenever the configuration is changed). In addition, the storeName-spring.xml files can be edited manually in Apama Studio to customize more advanced configuration aspects. To edit the XML, open the Distributed MemoryStore editor and in the Configuration Files section, click the name of the file to open it in the appropriate editor. Once the editor for an XML file has been opened, you can switch between the Design and Source views using the tabs at the bottom of the editor window.
Some property values usually need to be changed when a development and testing configuration is deployed to a different environment such as one for production use. Making use of substitution variables is the best way to maintain different bean property values in different environments, as you can use the same XML file, with a different
.properties file for each environment. For more details on using substitution variables to specify configuration properties, see
Substitution variables. For more information on modifying property values when moving from a test environment to a production environment, see
Changing bean property values when deploying projects.
XML configuration file format
The configuration files for a distributed store use the Spring XML file format, which provides an open-source framework for flexibly wiring together the different parts of an application, each of which is represented by a bean. Each bean is configured with an associated set of properties, and has a unique identifier which can be specified using the id attribute.
It is not necessary to have a detailed knowledge of Spring to configure a distributed store, but some customers may wish to explore the
Spring 3.0.5 documentation to obtain a deeper understanding of what is going on and to leverage some of the more advanced functionality that Spring provides.
The Apama distributed MemoryStore configuration will load any bean that extends the Apama AbstractStoreFactory class.
Setting bean property values
Most bean properties have primitive values (such as string, number, boolean) which are set like this:
<property name="propName" value="my value"/>
However, it is also possible to have properties that reference other beans, such as a configuration bean defined by the third-party distributed cache provider. These property values can be set by specifying the id of a top-level bean as in the following example (where it is assumed that myConfig is the id of a bean defined somewhere in the file):
<property name="someConfigProperty" ref="myConfig"/>
Any top-level bean may be referenced in this way, that is, any bean that is a child of the <beans> element and not nested inside another bean. Referencing a bean that is defined in a different configuration file is supported.
Instead of referencing a shared bean, it is also possible to configure a bean property by creating an 'inner' configuration bean nested inside the property value like this:
Note, advanced users may want to exploit Spring's property inheritance by using the parent attribute on an inner bean to inherit most properties from a standard top-level bean while overriding some specific subset of properties or by type-based 'auto-wiring'.
You can use the Spring syntax for compound property names to set the value of a property held by another property. For example to set a property stringProp on a bean held by the property beanProp, use the following:
<property name="beanProp.stringProp" value="myValue"/>
Or, to set the value of the key myKey in a property that holds a Map called mapProp, use the following:
<property name="mapProp[myKey]" value="myValue"/>
Substitution variables
Substitution variables in the form ${varname} can be used to specify bean property values. Instead of specifying bean property values directly in an XML configuration file, you use ${varname} substitution variables in the XML file and specify the values of those variables in a .properties file inside the configuration directory. This makes it possible to edit the variable values in Apama Studio and to use different values during deployment to a production environment using the Apama Ant macros.
Although .properties and -spring.xml files often have similar names, there is no explicit link between them, so any properties file can define properties for use by any -spring.xml file. Although in some cases it may be useful to share a single substitution variable across multiple XML files, this is not normally the desired behavior, and therefore the recommendation is that all properties follow the naming convention ${varname.storeName}.
In addition to the standard substitution variables shared by most drivers, you can add your own substitution variables for important or frequently changed properties specific to the driver specific to the cache integrated with your application. This is especially important when changing from a development environment to a production environment.
It is also possible to provide property values at runtime as Java system properties, such as specifying -J-Dvarname=value on the correlator command line.
The special variables ${APAMA_HOME} and ${APAMA_WORK} are always available.
Substitution variables are evaluated recursively, so a substitution variable can refer to another substitution variable, for example, classpath=${installDir}/foo.jar.
Standard configuration properties
The following four standard properties are supported by Apama distributed cache drivers. These properties should be supported by customer-developed implementations as well.
clusterName – This is a required property. It is a provider-specific string. For BigMemory Max, this is a comma-separated list of
host:port pairs that identify the servers in the Terracotta Server Array. Some other caches use this as just a name, used to group together distributed store nodes that communicate with each other and share data. Store objects with the same
clusterName values should operate as a single cluster, sharing data between them. Most providers require this property and will fail to start if it is not set. Care must be taken to ensure that different clusters, and thus
clusterName values, are used for development/testing and production environments, as serious errors would be introduced if the production and testing nodes were able to communicate with each other. Apama's BigMemory Max driver makes it easy to avoid this pitfall since it requires a list of
host:port pairs. However, if you are using another driver, then for this reason, as well as whatever firewalls may exist between development/testing and production, the recommendation is to explicitly add a suffix such as
_testing or
_production to the
clusterName to indicate clearly which environment it belongs to.
logLevel – This is an optional property; the default is provider-specific, but typically is the same as the correlator log level. The
logLevel property is an Apama log level string (compatible with
com.apama.util.Logger) such as
ERROR,
WARN,
INFO,
DEBUG which will be used to set the log level for the provider if possible (some providers will write to the main correlator log file, through log4j or the Apama Logger, but others may write to a separate file). If not specified, the default log level is determined by the author of the driver, based on the criteria of avoiding the correlator log or stdout being filled with third party distributed store messages while logging a small number of the most important messages.
backupCopies – This is an optional property; the default is 1. The
backupCopies property specifies the number of additional redundant nodes that should hold a backup copy of each key/value data element. The minimum value for this property is 0 (indicating no redundancy, that is, all data is held by a single node). Note, some providers may allow customizing the backup count on a per-table basis, in which case this property specifies an overridable default value for tables that do not specify it explicitly. For BigMemory Max, this setting has no effect. The number of backup copies is determined by the Terracotta Server Array configuration, which is separate from the Apama configuration.
initialMinClusterSize – This is an optional property. It specifies the minimum number of nodes a cluster must have before the
Finished event is sent in response to a call to
prepareDistributed. This provides a way to make sure that a cluster is fully ready for correlator nodes to request and process data. The default is
1, which specifies that a
Finished event is sent without waiting for additional nodes when preparing the distributed store.
rowChangedOldValueRequired - Indicates whether the old value is required when there is a notification that a row has changed. If set to false, the value of
oldFieldValues is empty for
RowChanged.changeType.UPDATE events. If true, the previous value is available. This currently cannot be set to true for BigMemory Max. The default is true.
If all four standard properties were set, the bean configuration would look like: