| Sun StorEdge SAM-FS Storage and Archive Management Guide |    
|
| Sun StorEdge SAM-FS Storage and Archive Management Guide |
|
| C H A P T E R 5 |
|
Releasing |
Releasing is the process by which the releaser makes disk cache space available by identifying archived files and releasing their disk cache copy. This makes room for other files to be created or staged from archive media. The releaser can release only archived files. Releasing the file results in a file without any data on the disk cache.
The Sun StorEdge SAM-FS file systems automatically invoke the releaser process when a site-specified disk threshold is reached. In contrast, you can use the release(1) command to release a file's disk space immediately or to set releasing parameters for a file. For more information about the releaser process, see the sam-releaser(1M) man page.
The releaser contains features that allow you to specify that files be released immediately after archiving, that files never be released, or that files be partially released. The partial release feature is particularly useful because some applications, such as filemgr(1), read only the beginning of the file. With partial release, a portion of the file remains on the disk cache and the remainder of the file is released. Reading the first part of the file still on disk cache does not necessarily trigger the staging of the rest of the file back to disk cache from the archive media. These features, and many others, are described in this chapter.
This chapter contains the following topics:
When file system utilization exceeds its configured high watermark, the file system management software invokes the releaser. First, the releaser reads the releaser.cmd file and collects the directives that control the release process. Next, it scans the file system and collects information about each file. Finally, after scanning the entire file system, the releaser begins releasing files in priority order.
The releaser continues to release files as long as the file system remains above the configured low watermark. Typically, the releaser frees enough space to allow the file system to drop below the low water mark. If the releaser cannot find any files to release, it exits. The releaser runs later when more files can be released. While above the high watermark, the file system starts the releaser every one minute.
The high and low watermarks are set with the high=percent and low=percent file system mount options. For more information about these mount options, see the mount_samfs(1M) man page.
A file system can contain thousands of files. Keeping track of the release priority for all the files can be wasteful because releasing only several large files might return the file system to its low watermark. However, the releaser must examine the priority of each file or risk missing the best candidates for release. The releaser handles this condition by identifying only the first 10,000 candidates.
After identifying the first 10,000 candidates, the releaser discards subsequent candidates if they do not have a priority greater than the lowest-priority candidate among the first 10,000.
After the releaser has determined the priority of the first 10,000 candidates, it selects the files with the highest priority for release. After releasing each file, the releaser checks to see if the file system cache utilization is below the low watermark. If so, the releaser stops releasing files. If not, the releaser continues releasing the files in priority order.
If the releaser has released all 10,000 candidates and the file system is still above the low water mark, it starts over and identifies 10,000 new candidates.
The releaser exits if it cannot find any viable candidates. This can occur, for example, if files do not yet have archive copies. The Sun StorEdge SAM-FS file systems start the releaser again after one minute has elapsed.
This section explains terms used throughout this chapter.
The age concept refers to the amount of elapsed time from a given event until now. A file's inode keeps track of the following times that are used by the releaser:
You can view these times by using the sls(1) command with the -D option. Each time has a corresponding age. For example, if it is 10:15 a.m., a file with a modify time of 10:10 a.m. has a data-modified age of five minutes. For more information about the sls(1) command, see the sls(1) man page.
A candidate is a file that is eligible to be released. The reasons why a file would not be a candidate are as follows:
A priority is a numeric value that indicates the rank of a candidate file based on user-supplied weights that are applied to numeric attributes of that candidate. The overall priority is the sum of two types of priority: age priority and size priority.
Candidate files with numerically larger priorities are released before candidates with numerically smaller priorities.
The weight is a numeric value that biases the priority calculation to include file attributes in which you are interested and to exclude file attributes in which you are not interested. For example, the size attribute of a file is excluded from the priority calculation if the size weight is set to zero. Weights are floating-point values from 0.0 to 1.0.
A file can be partially released by specifying that a beginning portion of the file remain in disk cache while the rest of the file is released. For example, partial release is valuable when using utilities like filemgr(1) that read the beginning of a file.
Releasing and staging are complementary processes. Files can be completely released from online disk cache after they are archived, or a site can specify that the beginning of a file (the stub) remain in disk cache while the remainder of the file is released. This ability to partially release a file provides immediate access to data in the file stub without staging the file.
A system administrator can specify both the default partial release size and the maximum size of the stub to remain online when a file system is mounted. The system administrator can set these on the mount(1M) command, as follows:
A user can specify the default stub size for a file by specifying the -p option on the release(1) command or the p option on the sam_release(3) library routine. To specify different-sized file stubs for different types of files or different applications, a user can specify the -s option on the release(1) command or the s option on the sam_release(3) library routine. The -s and s values must be less than the -o maxpartial value used on the mount(1M) command when the file system was mounted.
Another mount option, -o partial_stage=n, allows a system administrator to establish how much of a partial release stub must be read before the rest of the file is staged. That is, reading past the -o partial_stage=n size specification initiates the stage of the file.
By default, the -o partial_stage=n option is set to the size of the partial release stub. This value can be configured, though, and it affects file staging as follows:
Example. Assume that the following options are in effect:
The filemgr(1) program is being used, and it reads the first 8 kilobytes of a file. The file is not staged. A video-on-demand program reads the same file, and the file is staged after it reads past the first 16 kilobytes of the file. The application continues reading the 2 gigabytes of disk data while the archive tape is mounted and positioned. When the video-on-demand program reads past 2 gigabytes of file data, the application reads immediately behind the staging activity. The application does not wait because the tape mounting and positioning is done while the application reads the partial file data.
Several command line options affect whether a file can be marked for partial release. Some options are enabled by the system administrator, and others can be enabled by individual users. The following sections describe the release characteristics that can be set by the various types of users.
The system administrator can change the maximum value and default value for partial release when the file system is mounted. The mount(1M) options in TABLE 5-1 affect partial release. For more information about the mount(1) command, see the mount_samfs(1M) man page.
The system administrator sets maximum and default values for the size of a file stub that can remain in disk cache after the file is released. The system administrator also determines whether or not the partial release feature is enabled for a particular file system.
By using the release(1) command and the sam_release(3) library routines, however, a user can set other release attributes and can specify the files to be marked for partial release. The command and library options that determine partial release attributes are shown in TABLE 5-2. For more information about the release(1) command, see the release(1) man page. For more information about the sam_release(3) library routine, see the sam_release(3) man page.
The /etc/opt/SUNWsamfs/releaser.cmd file consists of directive lines that specify site-specific releasing actions. The releaser.cmd file can contain directives for setting the release priority, specifying a log file, and other actions.
The following sections describe the releaser.cmd directives:
For more information about these directives, see the releaser.cmd(4) man page.
Files are released from a file system using a priority order determined by directives defined in the releaser.cmd file. Both file age and file size are considered. By default, sites release the largest, oldest files first, leaving the smallest, newest files on disk. The following sections show how the releaser considers a file's age and size when determining the release priority of files in a file system.
For additional information about releaser directives, see the releaser.cmd(4) man page.
The releaser considers the following possible ages when determining the age-related component of a file's release priority:
In some cases, you might want the access age of a file to take precedence over the modification age. In other cases, a simple age derived from the most recently accessed time, modified time, and residence-changed time is preferred.
By default, the age of a file is the more recent of the file's three ages:
You can use directives to specify that a weighted age priority be used when calculating the release priority for a file.
CODE EXAMPLE 5-1 shows the age priority directives' formats.
weight_age = float weight_age_access = float weight_age_modification = float weight_age_residence = float |
float 1.0. By default, float = 1.0.
This directive cannot be specified in conjunction with the weight_age_residence, weight_age_modify, or weight_age_access directives.
float 1.0. By default, float = 1.0.
These directives cannot be specified in conjunction with the weight_age directive.
If the weight_age_residence, weight_age_modify, and weight_age_access directives are used, the age-related priority for a file is calculated based on a combination of all three ages. First, file age data is gathered for each file's possible age. Secondly, the file age data is multiplied by the weighting factors specified in the releaser.cmd file. Finally, the file's age-related priority is calculated by summing the product of the age data multiplied by each weighting factor, as shown in the equation in CODE EXAMPLE 5-2.
Example. CODE EXAMPLE 5-3 shows lines in a releaser.cmd file that specify that only the file's residence age be considered (and that the modification age and the access age be ignored) when calculating the release priority of a file.
weight_age_residence = 1.0 weight_age_modify = 0.0 weight_age_access = 0.0 |
After a file's age-related priority is calculated, it is multiplied by the file's size-related priority. The size-related priority is calculated as shown in the following section.
The releaser considers a file's size when determining the size-related component of a file's release priority. The size of the file (in 4-kilobyte blocks) is multiplied by the weight specified for the weight_size directive to obtain the size-related component of a file's release priority.
The format of the weight_size directive is as follows:
For float, specify a floating-point number in the following range: 0.0 float 1.0. By default, float = 1.0.
Example. CODE EXAMPLE 5-4 shows a releaser.cmd file that specifies that when calculating a file's release priority, a file's size is to be ignored for all files in the samfs1 and samfs2 file system.
You can use the fs = family_set_name directive in the releaser.cmd file to indicate that the directives that follow the fs = directive apply only to the named file system. This directive has the following format:
For family_set_name, specify the name of a Family Set in the mcf file.
Directives preceding the first fs = directive are global and apply to all file systems. Directives following the fs = directive override global directives. The directives described in this chapter can be used as either global directive or as directives specific to one file system.
The releaser.cmd(4) man page includes examples of the fs = directive.
The no_release and display_all_candidates directives can be useful when tuning or debugging the releaser. These directives are as follows:
These directives are helpful when debugging because the releaser writes the names of release candidates to the log file, but it does not physically release them from the file system.
The min_residence_age directive enables you to specify the minimum amount of time that a file must reside in a file system before it becomes a candidate for release. This directive has the following format:
For time, specify a time in seconds. The default time is 600, which is 10 minutes. There is no practical minimum or maximum time setting.
If a logfile directive is specified in the releaser.cmd file, the releaser either appends its activity log to the indicated file name, or the releaser creates the file name if it does not exist. This directive has the following format:
For filename, specify the name of a log file.
CODE EXAMPLE 5-5 shows a sample log file (note that some lines have been wrapped to fit on the page).
The releaser(1M) man page describes the information contained in the log file. Because the size of the log increases with each releaser run, be sure to allow for decreasing the size of the log, or omit the logfile keyword.
CODE EXAMPLE 5-6 shows the mathematical relationships that exist among the statistics shown under the ---after scan--- line:
total_inodes = wrong_inode_number + zero_inode_number + zero_mode + not_regular + extension_inode + zero_arch_status + already_offline + damaged + nodrop + archnodrop + too_new_residence_time + too_small + negative_age + total_candidates released_files = total_candidates |
By default, files marked for rearchiving are released. If the rearch_no_release directive is specified in the releaser.cmd(4) file, the releaser does not release the files marked for rearchiving. This directive has the following format:
You can use the list_size directive to specify the number of releaser candidates. If you notice that the releaser makes multiple file system scans before it releases the number of files needed to get to the low water mark, you might want to consider raising this value to a level greater than the default of 10,000. This might be true in a file system that contains many small files. You can get information about releaser activities from the releaser log file. This directive has the following format:
For number, specify an integer such that 10 number 2,147,483,648.
Most directives in the archiver.cmd file affect archiving, but the archive set assignment directive allows you to specify release attributes that apply to all files in an archive set.
The archive set assignment directive has the following format:
TABLE 5-3 shows the directives that pertain to releasing.
For more information about these and the other archiver.cmd directives, see Archiving.
It is necessary to decide the characteristics of files in cache for your site. It is wasteful to load a tape if you are staging only a few kilobytes, so you may want to bias your system to retain small files in cache. CODE EXAMPLE 5-7 shows the directives to use in the releaser.cmd file to release the largest files first.
Alternately, you may want to retain recently modified files in cache since a recently modified file might be modified again soon. This avoids the overhead created when the file is staged to enable modification. In this case, use the second set of age weights. CODE EXAMPLE 5-8 shows the directives to use in the releaser.cmd file to weight files in strict order starting with the oldest modified to the most recently modified.
weight_size = 0.0 weight_age_access = 0.0 weight_age_modify = 1.0 weight_age_residence = 0.0 |
However, as the following examples demonstrate, most situations are not this straightforward.
Example 1. Assume that you want to release the largest files first. There are hundreds of small files that are the same size, and there are several large files. The cumulative size of the small files might exceed the size of the single, largest file. Eventually, the releaser releases all the large files. If weight_age = 0.0 is specified, the releaser releases the small files in essentially random order because they are all the same size and have the same release priority.
In this scenario, you could set weight_age = 0.01 as a tiebreaker. The releaser would release the older of two equally sized files first.
Example 2. This example presents a better method to specify how to release the largest files first.
Set weight_size = 1.0 and weight_age = 0.01.
These directives violate the largest-first policy by counting smaller, less recently accessed files as better candidates than larger, more recently accessed files. You can make this effect as small as you want by making weight_age smaller than weight_size. For example, based on the previous settings, a 4-kilobyte file that staged 100 minutes ago and an 8-kilobyte file that just staged both have the same release priority.
The releaser randomly chooses a file to release. If it chooses a 4-kilobyte file, it violates the largest-first intent. Setting weight_age considerably smaller (for example, to 0.001) reduces this effect. If a 4-kilobyte file staged 1,000 minutes ago, it has the same priority as the 8-kilobyte file that just staged.
You can use the no_release and display_all_candidates directives and run the releaser manually to obtain a list of candidates in priority order for use in adjusting the priority weights.
From time to time, you might want to run the releaser manually. For this, you need to know the mount point of the file system and the low watermark the releaser should attempt to reach.
For example, to release files in the /sam1 file system until it reaches 47 percent full, log in as root and type the following:
The final argument, weight-size, is overridden by a weight_size command in the releaser.cmd file. As the releaser runs, it writes information to your screen and to the releaser log file (if specified in the releaser.cmd file.) For more information, see the sam-releaser(1M) man page.
There can be several reasons for the releaser to not release a file. Some possible reasons are as follows:
| Sun StorEdge SAM-FS Storage and Archive Management Guide | 817-4093-10 |
|
Copyright © 2004, Sun Microsystems, Inc. All rights reserved.