How to Calculate File Descriptor and Mapped Memory Requirements per Store
Calculating the minimum requirements for file descriptors and mapped memory regions per store enables you to better control and allocate resources in your system. On Linux, you use the
vm.max_map_count and
ulimits parameters to do so. For more information about these parameters, see
Tuning the Linux Operating System.
When you create a store of type Persistent, Universal Messaging creates a store_uniqueID folder in the Software AG_directory \UniversalMessaging\server\instance_name\data directory. Each folder contains two types of mapped memory files:
state.mem. A single file per store. Contains one file descriptor and one mapped memory region.
spindle.mem. The number of spindle files depends on the
Events Per Spindle property configured for the store. Contains two file descriptors, one for read operations and one for write operations, and one mapped memory region.
In addition, for persistent queues and channels with durables (shared and serial), Universal Messaging creates a store_name_durable folder that contains the following types of mapped memory files:
*.idx. The number of index pages depends on the number of published events on the store. Contains one file descriptor and one memory mapped buffer per 4 096 events.
*.txn. The number of transaction pages depends on the number of published events on the store. Contains one file descriptor and one memory mapped buffer per 4 096 events.
*.nsb. Only for durables. Contains a single file descriptor and one memory mapped region per durable.
To calculate the requirements for file descriptors and mapped memory regions, you also consider the number of published events on the store, the events per spindle, the number of spindles, and in case of a channel with durables, the number of durables.
Example: Simple Channel Store
You have a simple channel with the following parameters:
Event count (N): 100 000
Events per spindle (S): 10
Number of spindles: 10 000
Inside the store_uniqueID folder:
1 state.mem file containing:
1 file descriptor
1 mapped memory region (32 byte)
10000
spindle.mem files containing:
2 file descriptors per file or 20 000 in total
1 mapped memory region (S * 16 byte) per file or 10 000 in total
As a result, the total for minimum file descriptors and mapped memory regions for the channel is:
Minimum file descriptors: 1 + 20 000 = 20 001
Minimum mapped memory regions: 1 + 10 000 = 10 001
Example: Queue
You have a queue with the following parameters:
Event count (N): 100 000
Events per spindle (S): 10
Number of spindles: 10 000
Inside the store_uniqueID folder:
1 state.mem file containing:
1 file descriptor
1 mapped memory region (32 byte)
10000
spindle.mem files containing:
2 file descriptors per file or 20 000 in total
1 mapped memory region (S * 16 byte) per file or 10 000 in total
Inside the queue_name_durable folder:
*.idx file containing:
1 file descriptor per file or 25 in total (N/4 096)
1 mapped memory buffer per 4 096 events or 25 in total (N/4 096)
*.txn file containing:
1 file descriptor per file or 25 in total (N/4 096)
1 mapped memory buffer per 4 096 events or 25 in total (N/4 096)
As a result, the total for minimum file descriptors and mapped memory regions for the queue is:
Minimum file descriptors: 1 + 20 000 + 25 + 25 = 20 051
Minimum mapped memory regions: 1 + 10 000 +25 + 25 = 10 051
Example: Channel with Durables
You have a channel with durables with the following parameters:
Event count (N): 100 000
Events per spindle (S): 10
Number of spindles: 10 000
Number of durables (D): 10
Inside the store_uniqueID folder:
1 state.mem file containing:
1 file descriptor
1 mapped memory region (32 byte)
10000
spindle.mem files containing:
2 file descriptors per file or 20 000 in total
1 mapped memory region (S * 16 byte) per file or 10 000 in total
Inside the store_name_durable folder:
*.nsb file containing:
1 file descriptor
1 mapped memory region per durable or 10 in total
*.idx file containing:
1 file descriptor per file or 250 in total: (N/4 096) * D
1 mapped memory buffer per 4 096 events or 250 in total: (N/4 096) * D
*.txn file containing:
1 file descriptor per file or 250 in total: (N/4 096) * D
1 mapped memory buffer per 4 096 events or 250 in total: (N/4 096) * D
The formulas to calculate minimum file descriptors and mapped memory regions for a channel with durables are:
Minimum file descriptors: 1 + 2 * (N/S) + 1 + (N/4 096) * D + ( N/4 096) * D
Minimum mapped memory regions: 1 + N/S + D + (N/4 096) * D + ( N/4 096) * D
As a result, the total for minimum file descriptors and mapped memory regions for the channel in the example is:
Minimum file descriptors: 1 + 20 000 + 1 + 250 + 250 = 20 502
Minimum mapped memory regions: 1 + 10 000 + 10 + 250 + 250 = 10 511