File Partitioning


When is Partitioning Required?

File partitioning may be useful for any of the following reasons:

  • data volumes are too large to be managed within a single file;

  • separate files need a consolidated view as well as an individual view;

  • data archiving is needed with an efficient, high-performance recall capability.

Implementing Partitioning in the Application

The burden of partitioning has traditionally fallen on the application, making them extremely complex. For example, the application often has to determine the partitions targeted by an access or update, or to consolidate results from queries against several partitions.

Apart from the increased complexity, the more significant problem is that the usual level of application independence from the physical data model is lost. The cost of development and ongoing maintenance in these situations can be enormous.

Implementing Partitioning using Adabas Vista

Adabas Vista allows the application data to be partitioned into separate Adabas files without having to re-engineer the applications that access this data. This means that the application continues to refer to an Adabas file as though it is a single logical entity, even though the physical data may be partitioned and distributed across multiple files, databases or machines.

Typical Adabas Vista Deployment Scenarios

This section shows typical deployments of Adabas Vista:

Very Large Files

Very large files present an obvious use for partitioning. Although Adabas can hold a massive number of records within a file, the difficulties in managing such large amounts of data remain. The key to managing mass data is the ability to divide the data into more easily managed units.

In a distributed system, partitioning large files can bring benefits in terms of performance or resource utilisation. Partitioning data across multiple machines can help with load balancing and localisation of data. Overall data availability can also be improved because Adabas Vista has a partition outage feature that allows the application to continue working even though some of the partitions may not be available.

Consolidation

Applications and their data often change over time. Sometimes it becomes necessary to merge multiple files into a single file. This can present difficulties if the files are different or if the ability to access the original files must be retained.

Adabas Vista has a consolidation feature that allows a single file image to be imposed on multiple, previously unrelated files. Direct access to the individual files is maintained, and even though the files may be different, they support the same consolidated view.

Archiving

Many organisations have a requirement to keep data for a specified period of time. The sheer amount of data will often force an organisation to archive in order to keep the volume to a manageable level. If there is a need to access the archived data, then a process must also be established to recall it.

Most archiving systems access old data, copy it to archive, and delete it from the current file. This can take a considerable amount of time and may interfere with normal operation or access to the file.

Adabas Vista offers a simple and flexible approach to archiving and recalling data. Partitioning by date means that new data can be directed towards new partitions. Partition criteria might be based, for example, on handling a complete month or year’s worth of data. Old partitions can be backed-up and removed completely or access to them reduced by altering the partition definitions. Should access to the older data be required, the old file can simply be restored and the partition definition altered to make it available to the application again.