| Sun StorEdge SAM-FS Storage and Archive Management Guide |    
|
| Sun StorEdge SAM-FS Storage and Archive Management Guide |
|
| C H A P T E R 4 |
|
Archiving |
Archiving is the process of copying a file from a Sun StorEdge SAM-FS file system to a volume that resides on a removable media cartridge or on a disk partition of another file system. Throughout this chapter, the term archive media is used to refer to the various cartridges or disk slices to which archive volumes are written. The Sun StorEdge SAM-FS archiving capabilities include many features, such as those that you can use to specify that files be archived immediately, to specify that files never be archived, and to perform other tasks.
This chapter describes the archiver's theory of operations, provides general guidelines for developing archive policies for your site, and explains how to implement policies by creating an archiver.cmd file.
The following topics are presented:
The archiver automatically writes Sun StorEdge SAM-FS files to archive media. Operator intervention is not required to archive and stage the files. Files are archived to a volume on the archive media, and each volume is identified by a unique identifier called a volume serial name (VSN). Archive media can contain one or more volumes. To identify an individual volume, the media type and VSN must be specified.
The archiver starts automatically when a Sun StorEdge SAM-FS file system is mounted. You can customize the archiver's operations for your site by inserting archiving directives into the following file:
/etc/opt/SUNWsamfs/archiver.cmd
The archiver.cmd file does not need to be present for archiving to occur. In the absence of this file, the archiver uses the following defaults:
The following sections describe the concept of an archive set and explain the operations performed during the archiving process.
An archive set identifies a group of files to be archived. Archive sets can be defined across any group of file systems. Files in an archive set share common criteria that pertain to the size, ownership, group, or directory location. The archive sets control the destination of the archive copy, how long to keep the copy archived, and how long to wait before archiving the data. All files in an archive set are copied to the volumes associated with that archive set. A file in the file system can be a member of one and only one archive set.
As files are created and modified, the archiver copies them to archive media. Archive files are compatible with the standard UNIX tar(1) format. This ensures data compatibility with the Sun Solaris operating system (OS) and other UNIX systems. This format includes the file access data (inode information) and the path to the file. If a complete loss of your Sun StorEdge SAM-FS environment occurs, the tar(1) format allows file recovery using standard UNIX tools and commands. The archiving process also copies the data necessary for Sun StorEdge SAM-FS file system operations. This data consists of directories, symbolic links, the index of segmented files, and archive media information.
In the remainder of this section, the term files refers to both file data and metadata. The terms file data and metadata are used only when a distinction is required. The term file system refers to a mounted Sun StorEdge SAM-FS file system.
Archive set names are determined by the administrator and are virtually unlimited with the following exceptions:
The no_archive archive set is defined by default. Files selected to be in this archive set are never archived. Files in a temporary directory, such as /sam1/tmp, for example, might be included in the no_archive archive set.
The allsets archive set is used to define parameters that apply to all archive sets.
By default, the archiver makes one copy of each archive set, but you can request up to four archive copies for each archive set. An archive set and a copy number become a synonym for a collection of volumes. The archive copies provide duplication of files on separate volumes.
To ensure that files are complete before archiving, the archiver waits a specified period of time after the file is modified before archiving it. As mentioned previously, this period of time is called the archive age.
The data in a file must be modified before the file is considered to be a candidate for archiving or rearchiving. A file is not archived if it is only accessed. For example, issuing a touch(1) or an mv(1) command on a file does not cause it to be archived or rearchived. Issuing an mv(1) command alters the file name but not the file data, and this can have ramifications in a disaster recovery situation if you are restoring from tar(1) files. For more information on disaster recovery, see the Sun QFS, Sun SAM-FS, and Sun SAM-QFS Disaster Recovery Guide.
Files are selected for archiving based on their archive age. The archive age can be defined for each archive copy.
Users can change the default time references on their files to values far in the past or future by using the touch(1) command. This can cause unexpected archiving results, however. To avoid such problems, the archiver adjusts the references so that they are always in the following range:
creation_time < time_ref < time_now
The following sections describe the steps taken by the archiver from the initial file scan to the file copy process.
There is a separate sam-arfind process for each mounted file system. The sam-arfind process monitors each file system to determine the files that need archiving. The file system notifies its sam-arfind process whenever a file is changed in a manner that would affect its archival state. Examples of such changes are file modification, rearchiving, unarchiving, and renaming. When notified, the sam-arfind process examines the file to determine the archive action required.
The sam-arfind process determines the archive set to which the file belongs by using the file properties descriptions. The characteristics used for determining a file's archive set include the following:
If the archive age of the file for one or more copies has been met or exceeded, sam-arfind adds the file to one or more archive requests for the archive set. The archive request is the collection of files that all belong to the same archive set. Separate archive requests are used for files being rearchived. This allows scheduling to be controlled independently for files not yet archived and for those being rearchived. The archive request is a file that resides in the following directory:
/var/opt/SUNWsamfs/archiver/file_sys/ArchReq
The files in this directory are binary files, and you can display them by using the showqueue(1M) command.
The archive request is sometimes referred to as an ArchReq.
If the archive age of the file for one or more copies has not been met, the directory in which the file resides and the time at which the archive age is reached is added to a scan list. Directories are scanned as the scan list times are reached. Files that have reached their archive age are added to archive requests.
If a file is offline, the sam-arfind process selects the volumes to be used as the source for the archive copy. If the file copy is being rearchived, the sam-arfind process selects the volume containing the archive copy that is being rearchived.
If a file is segmented, only those segments that have changed are selected for archival. The index of a segmented file contains no user data, so it is treated as a member of the file system archive set and is archived separately.
The archive priority is computed from file property characteristics and from file property multipliers associated with the archive set. Essentially, the computation is as follows:
archive_priority = the sum of (file_property_value * property_multiplier)
Most file_property_value numbers are 1 or 0, as the property is TRUE or FALSE. For instance, the value of the property copy 1 is 1 if archive copy 1 is being made. The values of copy 2, copy 3, and copy 4 are, therefore, 0.
Others, such as archive age and file size, can have values other than 0 or 1.
The property_multiplier values are determined from the -priority parameters for the archive set. Various aspects of a file, such as age or size, can be given values so that your site can alter the archive request's priority. For more information on the -priority parameter, see the archiver.cmd(4) man page.
The archive_priority and the property multipliers are floating-point numbers. The default value for all property multipliers is 0.0. The archive request is set to the highest file priority in the archive request.
There are two methods by which files are marked for archiving: continuous archiving and scanning. With continuous archiving, the archiver works with the file system to determine which files need to be archived. With scanning, the archiver periodically peruses the file systems and selects files for archiving. The following sections describe these methods.
Continuous archiving is the default archiving method (examine=noscan). With continuous archiving, you can specify scheduling start conditions for an archive set by using the -startage, -startcount, and -startsize parameters. These conditions allow you to optimize archive timeliness versus archive work done.
When any of the scheduling start conditions are reached, the sam-arfind process sends each archive request to the archiver daemon, sam-archiverd, to be scheduled for file copying to archive media.
As an alternative to continuous archiving, you can specify examine=scan to direct sam-arfind to examine files for archival using scanning. Files needing archival are placed into archive requests. The sam-arfind process scans each file system periodically to determine which files need archiving. The first scan that sam-arfind performs is a directory scan. During this scan, sam-arfind descends recursively through the directory tree. Each file is examined, and the file status flag archdone is set if the file does not need archiving. During successive scans, the .inodes file is scanned. Only those inodes with the archdone flag not set are examined.
When the file system scanning has been completed, the sam-arfind process sends each archive request to the archiver daemon, sam-archiverd, to be scheduled for file copying to archive media. The sam-arfind process then sleeps for the duration specified by the interval=time directive. At the end of the interval, the sam-arfind process resumes scanning.
When archive requests are received by the sam-archiverd daemon, they are composed. This step describes the composition process.
All the files in an archive request might not be archived at one time. This can be caused by the capacity of the archive media or by the controls specified in the archiver command file. Composing is the process of selecting the files to be archived from the archive request at one time. When the archive copy operation has been completed for an archive request, the archive request is recomposed if files remain to be archived.
The sam-archiverd daemon orders the files in the archive requests according to certain default and site-specific criteria. The default operation is to archive all the files in an archive request to the same archive volumes in the order that they were found during the file system scan. The site-specific criteria allow you to control the order in which files are archived and how they can be distributed on volumes. These criteria are called archive set parameters, and the order in which they are evaluated is as follows: -reserve, -join, -sort, -rsort (performs a reverse sort), and -drives. For more information on these parameters, see the archiver.cmd(4) man page.
If the archive request belongs to an archive set that has -reserve owner specified, the sam-archiverd daemon orders the files in the archive request according to the file's directory path, user name, or group name. This action is controlled by the -reserve parameter for the archive set. The files belonging to the first owner are selected for archiving. The remaining files are archived later.
If the archive request belongs to an archive set that has the -join method specified, the sam-archiverd daemon groups the files together according to the -join method specified. If a -sort or -rsort method is also specified, then the sam-archiverd daemon sorts the files within each group according to the -sort or -rsort method. The archive request is joined and sorted.
Each group of joined files is treated as if it were a single file for the remainder of the composing and scheduling processes.
If the archive request belongs to an archive set that has a -sort or -rsort method specified, the sam-archiverd daemon sorts the files according to the sort method specified on the -sort or -rsort parameter. Depending on the sort method, the sam-archiverd daemon tends to keep files together based on the sort method, age, size, or directory location. By default, the archive requests are not sorted, so the files are archived in the order in which they are encountered during the file system scan.
The sam-archiverd daemon determines whether the files are online or offline. If both online and offline files are in the archive request, the online files are selected for archiving first.
If the archive request was not required to be joined or sorted by a sort method, the offline files are ordered by the volume upon which the archive copies reside. This ensures that all files (within each archive set) on the same volume are staged at the same time in the order in which they were stored on the media. When more than one archive copy of an offline file is being made, the offline file is not released until all required copies are made. All the files to be staged from the same volume as the first file are selected for archiving.
Note that using the -join, -sort, or -rsort parameters can have a negative effect on performance when archiving offline files. This is due to the possiblity that the order of the files to be archived does not match the order of the volumes needed for the offline files. It is recommended that you use the -join, -sort, or -rsort parameters only for the first archive copy to be made. Other copies will most likely maintain the order of the first copy if enough archive media is available when the copies are started.
The archive requests are entered in the sam-archiverd daemon's scheduling queue.
The scheduler in the sam-archiverd daemon executes on demand when the following conditions exist:
The archive requests in the scheduling queue are ordered by priority. Each time the scheduler executes, all archive requests are examined to determine if they can be assigned to a sam-arcopy process to have the files copied to archive media.
There must be drives available to use for making file copies. There must be volumes available that can be used by the archive set and have sufficient space to hold the files in the archive request.
If the archive set has the -drives parameter specified, the sam-archiverd daemon divides the selected files in the archive request among multiple drives. If the number of drives available at this time is less than that specified by the -drives parameter, the smaller number is used.
If the total size of files in the archive request is less than the -drivemin value, only one drive is used. The -drivemin value is either the value specified by the -drivemin parameter or it is the archmax value.
The archmax value is specified by the -archmax parameter or the value defined for the media. For more information on the -archmax parameter and the archmax= directive, see the archiver.cmd(4) man page.
If the total size of files in the archive request is more than the -drivemin value, then the following value is computed: drive_count = total_size / drivemin. If drive_count is less than the number of drives specified by the -drives parameter, then drive_count becomes the number of drives to use.
Drives can take differing amounts of time to archive files. You can use the -drivemax parameter to obtain better drive utilization. The -drivemax parameter requires you to specify the maximum number of bytes to be written to a drive before rescheduling that drive for more data.
There must be a volume, or volumes, with enough space to hold at least some of the files in the archive request. The volume that has most recently been used for the archive set is used if there is enough space. Also, the volume must not be in use by the archiver.
If a volume usable for the archive set is presently busy, another is selected. This is true unless the -fillvsns parameter is specified. In this case, the archive request is not schedulable.
If an archive request is too big for one volume, the files that can fit on the volume are selected to be archived to the volume. If the archive request contains files that are too big to fit on one volume, and volume overflow for the archive request is not selected, the files cannot be archived. An appropriate message for this condition is sent to the log.
You can specify volume overflow for the archive set (by using the -ovflmin parameter) or for the media (by using the ovflmin= directive). For more information on the -ovflmin parameter and the ovflmin= directive, see the archiver.cmd(4) man page. This specification, ovflmin, determines the minimum size for files to overflow media. An ovflmin specified for the archive set takes precedence over a media-defined ovflmin. If the size of the files is less than ovflmin, the files cannot be archived. An appropriate message for this condition is sent to the log.
If the size of the files is more than ovflmin, then additional volumes are assigned as required. The additional volumes are selected in order of decreasing size in order to minimize the number of volumes required for the file.
If no usable volumes can be found for the archive request, the archive request waits.
Certain properties, such as whether or not the file is online or offline, are used in conjunction with the archive priority (computed in Step 1) when determining the scheduling priority for a particular archive request. For more information on customizing the property multiplier, see the -priority parameters described on the archiver.cmd(4) man page.
For each archive request, the sam-archiverd daemon computes the scheduling priority by adding the archive priority to multipliers associated with various system resource properties. These properties are associated with the number of seconds that the archive request has been queued, whether or not the first volume to be used in the archiving process is loaded into a drive, and so on.
Using the adjusted priorities, the sam-archiverd daemon assigns each ready archive request to be copied.
When an archive request is ready to be archived, the sam-archiverd daemon steps through each archive request to mark the archive file (tarball) boundaries so that each archive file's size is less than the -archmax target_size specification. If a single file is larger than target_size, it becomes the only file in an archive file.
For each archive request and each drive to be used, the sam-archiverd daemon assigns the archive request to a sam-arcopy process to copy the files to the archive media. If a single file is larger than target_size, it becomes the only file in an archive file. The archive information is entered into the inode.
If archive logging is enabled, an archive log entry is created.
If the file was staged, the disk space is released. This process continues until all files in the list have been archived.
A variety of errors and file status changes can prevent a file from being successfully copied. This can include read errors from the cache disk and write errors to the volumes. Status changes include modification since selection, file open for write, and file removed.
When the sam-arcopy process exits, the sam-archiverd daemon examines the archive request. If any files have not been archived, the archive request is recomposed.
CODE EXAMPLE 4-1 shows sample output is from running the archiver(1M) -l command.
The sam-archiverd daemon schedules the archiving activity. The sam-arfind process assigns files to be archived to archive sets. The sam-arcopy process copies the files to be archived to the selected volumes.
The sam-archiverd daemon is started by sam-fsd when Sun StorEdge SAM-FS activity begins. The sam-archiver daemon executes the archiver(1M) command to read the archiver.cmd file and builds the tables necessary to control archiving. It starts a sam-arfind process for each mounted file system; likewise, if a file system is unmounted, the associated sam-arfind process is stopped. The sam-archiverd process then monitors sam-arfind and processes signals from an operator or other processes.
The sam-arfind and sam-arcopy processes produce a log file that contains information about each archived or automatically unarchived file. The log file is a continuous record of archival action. You can use the log file to locate earlier copies of files for traditional backup purposes.
This file is not produced by default. You can use the logfile= directive in the archiver.cmd file to specify that a log file be created and to specify the name of the log file. You determine the name of this file. For more information on the log file, see the The archiver.cmd Directives in this chapter and see the archiver.cmd(4) man page.
The archiver logs warnings and informational messages in the log file using the syslog facility and archiver.sh.
CODE EXAMPLE 4-2 shows sample lines from an archiver log file with definitions for each field.
Reading left to right, the fields in the previous listing have the content shown in TABLE 4-1.
The archiver.cmd file controls the archiver's behavior. By default, the archiver runs whenever sam-fsd is started and a Sun StorEdge SAM-FS file system is mounted. The default actions that the archiver takes are as follows:
It is likely that you will customize the actions of the archiver to meet the archiving requirements of your site. These actions are controlled by directives located in the archiver command file (archiver.cmd).
|
1. Decide whether you want to edit the archiver.cmd file or if you want to edit a temporary archiver.cmd file. (Optional)
Perform this step if you have an /etc/opt/SUNWsamfs/archiver.cmd file and your system is already archiving files. Consider copying your archiver.cmd file to temporary location where you can edit and test it before putting it into production.
2. Use vi(1) or another editor to edit your archiver.cmd file or the temporary file.
Add the directives you need in order to control archiving at your site. For information on the directives you can include in this file, see The archiver.cmd Directives and Disk Archiving.
3. Save and close the archiver.cmd file or the temporary file.
4. Use the archiver(1M) -lv command to verify the correctness of the file.
Whenever you make changes to the archiver.cmd file, you should check for syntax errors using the archiver(1M) command. Specifying the archiver(1M) command as follows evaluates an archiver.cmd file against the current Sun StorEdge SAM-FS system:
The preceding command lists all options and writes a listing of the archiver.cmd file, volumes, file system content, and errors to the standard output file (stdout). Errors prevent the archiver from running.
By default, the archiver(1M) command evaluates file /etc/opt/SUNWsamfs/archiver.cmd for errors. If you are working with a temporary archiver.cmd file prior to putting it into production, you can use the -c option on the archiver(1M) command and supply this temporary file's name.
5. Repeat Step 2, Step 3, and Step 4 until your file is free from errors.
You must correct all errors before you move onto the next step. The archiver does not archive any files if it finds errors in the archiver.cmd file.
6. Move the temporary file to /etc/opt/SUNWsamfs/archiver.cmd. (Optional)
Perform this step only if you are working with a temporary file.
7. Save and close the archiver.cmd file.
8. Use the samd(1M) config command to propagate the file changes and restart the system.
The archiver.cmd file consists of the following types of directives:
The directives consist of lines of text read from the archiver.cmd file. Each directive line contains one or more fields separated by spaces or tabs. Any text that appears after the pound sign character (#) is treated as a comment and is not examined. Lines can be continued onto the next line by ending the line with a backslash (\).
Certain directives in the archiver.cmd file require you to specify a unit of time or a unit in bytes. To specify these units, use one of the following letters in The archiver.cmd File Directive Units as a suffix to the number that signifies the unit.
CODE EXAMPLE 4-3 shows a sample archiver.cmd file. The comments at the right indicate the various types of directives.
The following sections explain the archiver.cmd directives. They are as follows:
Global directives control the overall archiver operation. These global directives in an archiver.cmd file can be identified either by the equal sign (=) in the second field or by the absence of additional fields. These directives allow you to optimize archiver operations for your site's configuration.
Global directives must be specified prior to any fs= directives in your archiver.cmd file. The fs= directives are those that pertain to specific file systems. The archiver issues a message if it detects a global directive after an fs= directive.
The archivemeta directive controls whether or not file system metadata is archived. If files are often moved around and there are typically many changes to the directory structures in your file system, you would want to archive your metadata. If, however, the directory structures are very stable, you can disable metadata archiving and reduce the actions performed by your removable media drives as cartridges are loaded and unloaded to archive metadata. By default, metadata is archived.
This directive has the following format:
For state, specify either on or off. The default is on.
Metadata archiving differs depending on whether you are using a Version 1 or a Version 2 superblock, as follows:
The archmax directive specifies the maximum size of an archive file. User files are combined to form the archive file. No more user files are added to the archive file after the target_size is met. Large user files are written in a single archive file.
To change the defaults, use the following directive:
There are advantages and disadvantages to setting large or small sizes for archive files. For example, if you are archiving to tape and archmax is set to a large size, the tape drive stops and starts less often. However, when writing large archive files, there is the possibility that when an end-of-tape is reached prematurely, a large amount of tape can be wasted. As a rule, archmax should not be set to more than 5 percent of the media capacity. For example, you can use the following archmax directive for a 20 gigabyte tape:
The archmax directive can also be set for an individual archive set.
By default, a file being archived is copied to archive media using a memory buffer. You can use the bufsize directive to specify a nondefault buffer size and, optionally, to lock the buffer. These actions can improve performance, and you can experiment with different buffer_size values.
This directive has the following format:
For example, this directive can be specified in the archiver.cmd file in a line like the following:
You can specify a buffer size and a lock on an archive set basis by using the -bufsize and -lock archive set copy parameters. For more information, see Archive Set Copy Parameters.
By default, the archiver uses all of the drives in an automated library for archiving. To limit the number of drives in an automated library used by the archiver, use the drives directive.
This directive has the following format:
|
The Family Set name of the automated library as defined in the mcf file. |
|
Also see the -drivemax, -drivemin, and -drives archive set copy parameters described in Specifying the Number of Drives for an Archive Request: -drivemax, -drivemin, and -drives.
New files and files that have changed are candidates for archiving. The archiver finds such files by implementing one of the following methods:
The examine directive controls whether the archiver performs continuous or scan-based archiving, as follows:
Specify one of the keywords shown in TABLE 4-6 for method.
The archiver executes periodically to examine the status of all mounted Sun StorEdge SAM-FS file systems. The timing is controlled by the archive interval. The archive interval is the time between scan operations on each file system. To change the time, use the interval directive.
|
Note - The interval directive is effective only if the examine=scan directive is also specified in the archiver.cmd file. |
This directive has the following format:
For time, specify the time, in seconds, between scan operations on a file system. By default, time is interpreted in seconds. By default, interval=600, which is 10 minutes. You can specify a unit of time, such as minutes, hours, and so on. For information on specifying a unit of time, see The archiver.cmd File Directive Units.
If the archiver receives the samu(1M) utility's :arrun command, it begins scanning all file systems immediately. If the examine=scan directive is also specified in the archiver.cmd file, a scan is performed after :arrun or :arscan is issued.
If the hwm_archive mount option is set for the file system, the archive interval can be shortened automatically. This mount option specifies that the archiver commence its scan when the file system is filling up and the high water mark is crossed. The high=percent mount option sets the high water mark for the file system.
For more information on specifying the archive interval, see the archiver.cmd(4) man page. For more information on setting mount options, see the mount_samfs(1M) man page.
The archiver can produce a log file that contains information about each file that is archived, rearchived, or automatically unarchived. The log file is a continuous record of archival action. To specify a log file, use the logfile directive. This directive has the following format:
For pathname, specify the absolute path and name of the log file. By default, this file is not produced.
Example. Assume that you want to back up the archiver log file every day by copying the previous day's log file to an alternate location. This can be accomplished if you make sure that the copy is performed when the archiver log file is closed. In other words, you must not perform the copy operation while the archiver log file is open for a write operation.
The steps you need to take are as follows:
1. Use the mv(1) command to move the archiver log file within UFS.
This gives any sam-arfind(1M) or sam-arcopy(1M) operations time to finish writing to the archiver log file.
2. Use the mv(1) command to move the previous day's archiver log file to the Sun StorEdge SAM-FS file system.
The logfile directive can also be set for an individual file system.
The notify directive sets the name of the archiver's event notification script file to filename. This directive has the following format:
For filename, specify the name of the file containing the archiver event notification script or the full path to this file.
The default file name is as follows:
/etc/opt/SUNWsamfs/scripts/archiver.sh
The archiver executes this script to process various events in a site-specific manner. The script is called with a keyword for the first argument. The keywords are as follows: emerg, alert, crit, err, warning, notice, info, and debug.
Additional arguments are described in the default script. For more information, see the archiver.sh(1M) man page.
Volume overflow is the process of allowing archived files to span multiple volumes. Volume overflow is enabled when you use the ovflmin directive in the archiver.cmd file. When a file size exceeds the ovflmin directive's minimum_file_size argument, the archiver writes another portion of this file to another available volume of the same type, if necessary. The portion of the file written to each volume is called a section.
The archiver controls volume overflow through the ovflmin directive. The ovflmin directive specifies the minimum size file that is allowed to overflow a volume. By default, volume overflow is disabled.
This directive has the following format:
|
The media type. For a list of valid media types, see the mcf(4) man page. |
|
Example 1. Assume that many files exist with a length that is a significant fraction (say 25 percent) of an mo media cartridge. These files partially fill the volumes and leave unused space on each volume. To get better packing of the volumes, set ovflmin for mo media to a size slightly smaller than the size of the smallest file. The following directive sets it to 150 megabytes:
Note that enabling volume overflow in this example also causes two volumes to be loaded for archiving and staging the file.
The ovflmin directive can also be set for an individual archive set.
Example 2. The sls(1) command lists the archive copy showing each section of the file on each VSN. CODE EXAMPLE 4-4 shows the archiver log file and the sls -D command output for a large file named file50 that spans multiple volumes.
CODE EXAMPLE 4-4 shows that file50 spans three volumes with VSNs of DLT000, DLT001, and DLT005. The position on the volume and the size of each section is indicated in the seventh and tenth fields respectively, and matches the sls -D output also shown. For a complete description of the archiver log entry, see the archiver(1M) man page.
CODE EXAMPLE 4-5 shows the sls -D command and output.
Volume overflow files do not generate checksums. For more information on using checksums, see the ssum(1) man page.
The wait directive causes the archiver to wait for a start signal from samu(1M) or SAM-QFS Manager. When this signal is received, typical archiver operations are begun. By default, the archiver begins archiving when started by sam-fsd(1M). To delay archiving, use the wait directive. This directive has the following format:
The wait directive can also be set for an individual file system.
You can use the fs= directive to include directives specific to a particular file system in the archiver.cmd file after the general directives. After an fs= directive is encountered, the archiver assumes that all subsequent directives specify actions to be taken only for individual file systems.
By default, archiving controls apply to all file systems. However, you can confine some controls to an individual file system. To specify an individual file system, use the fs directive. This directive has the following format:
For fsname, specify the file system name as defined in the mcf file.
The general directives and archive set association directives that occur after these directives apply only to the specified file system until another fs= directive is encountered. For instance, you can use this directive to specify a different log file for each file system.
Several directives can be specified both as global directives for all file systems and as directives that are specific to one file system. Their effects are the same regardless of where they are specified. These directives are as follows:
By default, files are archived as part of the archive set named for the file system. In SAM-QFS Manager, an archive policy defines archive sets. However, you can specify archive sets to include files that share similar characteristics. If a file does not match one of the specified archive sets, it is archived as part of the default archive set named for the file system.
The archive set membership directives assign files with similar characteristics to archive sets. The syntax of these directives is patterned after the find(1) command. Each archive set assignment directive has the following format:
Example 1. CODE EXAMPLE 4-6 shows typical archive set membership directives.
hmk_files net/home/hmk -user hmk datafiles xray_group/data -size 1M system . |
Example 2. You can suppress the archiver by including files in an archive set named no_archive. CODE EXAMPLE 4-7 shows lines that prevent archiving of files in a tmp directory, at any level, and regardless of the directory in which the tmp directory resides within the file system.
fs = samfs1 no_archive tmp no_archive . -name .*/tmp/ |
The following sections describe the search_criteria that you can specify.
You can use the -access age characteristic to specify that the age of a file be used to determine archive set membership. When you use this search_criteria, files with access times older than age are rearchived to different media. For age, specify an integer number followed by one of the suffixes shown in TABLE 4-9.
For example, you can use this directive to specify that files that have not been accessed in a long time be rearchived to cheaper media.
The size of a file can be used to determine archive set membership using the -minsize size and -maxsize size characteristics. For size, specify an integer followed by one of the letters shown in TABLE 4-10.
Example. The lines in this example specify that all files of at least 500 kilobytes, but less than 100 megabytes, belong to the archive set big_files. Files bigger than 100 megabytes belong to the archive set huge_files. CODE EXAMPLE 4-8 shows the lines.
big_files . -minsize 500k -maxsize 100M huge_files . -minsize 100M |
The ownership and group affiliation can be used to determine archive set membership using the -user name and -group name characteristics. CODE EXAMPLE 4-9 shows examples of these directives.
adm_set . -user sysadmin mktng_set . -group marketing |
All files belonging to user sysadmin belong to archive set adm_set, and all files with the group name of marketing are in the archive set mktng_set.
The names of files that are to be included in an archive set can be specified by using regular expressions. The -name regex specification as a search_criteria specifies that any complete path matching the regular expression regex is a member of the archive set.
The regex argument follows the conventions as outlined in the regexp(5) man page. Note that regular expressions do not follow the same conventions as UNIX wildcards.
Internally, all files beneath the selected directory are listed (with their specified paths relative to the mount point of the file system) and passed along for pattern matching. This allows you to create patterns in the -name regex field to match both file names and path names.
1. The following directive restricts files in the archive set images to those files ending with .gif:
2. The following directive selects files that start with the characters GEO:
3. You can use regular expressions with the no_archive archive set. The following specification prevents any file ending with .o from being archived:
4. Assume that your archiver.cmd file contains the lines shown in CODE EXAMPLE 4-10.
# File selections. fs = samfs1 1 1s 2 1s no_archive share/marketing -name fred\. |
With this archiver.cmd file, the archiver does not archive fred.* in the user directories or subdirectories. Archiving occurs for files as follows:
/sam1/share/marketing/fred.anything /sam1/share/marketing/first_user/fred.anything /sam1/share/marketing/first_user/first_user_sub/fred.anything |
/sam1/fred.anything /sam1/share/fred.anything /sam1/testdir/fred.anything /sam1/testdir/share/fred.anything /sam1/testdir/share/marketing/fred.anything /sam1/testdir/share/marketing/second_user/fred.anything |
5. Assume that your archiver.cmd file contains the lines shown in CODE EXAMPLE 4-13.
# File selections. fs = samfs1 1 1s 2 1s no_archive share/marketing -name ^share/marketing/[^/]*/fred\. |
The archiver.cmd file in CODE EXAMPLE 4-13 does not archive fred.* in the user home directories. This archives fred.* in the user subdirectories and in the directory share/marketing. In this case, the user home directories happen to be first_user. This example takes anything as a user's home directory from share/marketing/ until the next slash character (/). Archiving occurs for files as follows:
/sam1/share/fred.anything /sam1/share/marketing/fred.anything /sam1/share/marketing/first_user/first_user_sub/fred.anything /sam1/fred.anything /sam1/testdir/fred.anything /sam1/testdir/share/fred.anything /sam1/testdir/share/marketing/fred.anything /sam1/testdir/share/marketing/second_user/fred.anything /sam1/testdir/share/marketing/second_user/sec_user_sub/fred.any |
You can set the release and stage attributes associated with files within an archive set by using the -release and -stage options, respectively. Both of these settings override stage or release attributes that a user might have set previously.
The -release option has the following format:
The attributes for the -release directive follow the same conventions as the release(1) command and are as shown in TABLE 4-11.
|
Release the file following the completion of the first archive copy. |
|
The -stage option has the following format:
The attributes for the -stage directive follow the same conventions as the stage(1) command and are as shown in TABLE 4-12.
The following example shows how you can use file name specifications and file attributes in order to partially release Macintosh resource directories:
Sometimes the choice of path and other file characteristics for inclusion of a file in an archive set results in ambiguous archive set membership. These situations are resolved in the following manner:
1. The membership definition occurring first in the archive set is chosen.
2. Membership definitions local to a file system are chosen before any globally defined definitions.
3. A membership definition that exactly duplicates a previous definition is noted as an error.
As a consequence of these rules, more restrictive membership definitions should be placed earlier in the directive file.
When controlling archiving for a specific file system (using the fs=fsname directive), the archiver evaluates the file system specific directives before evaluating the global directives. Thus, files can be assigned to a local archive set (including the no_archive archive set) instead of being assigned to a global archive. This has implications when setting global archive set assignments such as no_archive.
CODE EXAMPLE 4-15 shows an archiver.cmd file.
no_archive . -name .*\.o$ fs = samfs1 allfiles . fs = samfs2 allfiles . |
In reading CODE EXAMPLE 4-15, it appears that the administrator did not intend to archive any of the .o files across both file systems. However, because the local archive set assignment allfiles is evaluated before the global archive set assignment no_archive, the .o files in the samfs1 and samfs2 file systems are archived.
CODE EXAMPLE 4-16 shows the directives to use to ensure that no .o files are archived in both file systems.
fs = samfs1 no_archive . -name .*\.o$ allfiles . fs = samfs2 no_archive . -name .*\.o$ allfiles . |
If you do not specify archive copies, the archiver writes a single archive copy for files in the archive set. By default, this copy is made when the archive age of the file is four minutes. If you require more than one archive copy, all copies, including the first, must be specified using archive copy directives.
The archive copy directives begin with a copy_number that is an integer digit. This digit (1, 2, 3, or 4) is the copy number. The digit is followed by one or more arguments that specify archive characteristics for that copy.
The archive copy directives must appear immediately after the archive set assignment directive to which they pertain. Each archive copy directive has the following format:
The following sections describe the archive copy directive arguments.
You can specify that the disk space for files be automatically released after an archive copy is made by using the -release directive after the copy number. This option has the following format:
In CODE EXAMPLE 4-17, files with the group images are archived when their archive age reaches 10 minutes. After archive copy 1 is made, the disk cache space is released.
ex_set . -group images 1 -release 10m |
You might not want to release disk space until multiple archive copies are completed. The -norelease option prevents the automatic release of disk cache until all copies marked with -norelease are made. This option has the following format:
CODE EXAMPLE 4-18 specifies an archive set named vault_tapes. Two copies are created, but the disk cache associated with this archive set is not released until both copies are made. This scenario can be used at a site that requires online access to files before creating volumes for offsite storage.
vault_tapes 1 -norelease 10m 2 -norelease 30d |
Note that the -norelease specification on a single copy has no effect on automatic releasing because the file cannot be released until it has at least one archive copy. Also, the -norelease and -release specifications are mutually exclusive.
You can set the archive age for files by specifying the archive age as the next field on the directive. The archive age can be specified with a suffix character such as h for hours or m for minutes. The archiver.cmd File Directive Units, shows the complete list of suffix characters and their meanings.
In CODE EXAMPLE 4-19, the files in directory data are archived when their archive age reaches one hour.
If you specify more than one archive copy of a file, it is possible to unarchive all but one of the copies automatically. This might occur when the files are archived to various media using various archive ages.
CODE EXAMPLE 4-20 shows directives that specify the unarchive age.
ex_set home/users 1 6m 10w 2 10w 3 10w |
The first copy of the files in the path home/users is archived six minutes after modification. When the files are 10 weeks old, second and third archive copies are made. The first copy is then unarchived.
For more ways to control unarchiving, see Controlling Unarchiving.
If more than one copy of metadata is required, you can place copy definitions in the directive file immediately after an fs= directive.
CODE EXAMPLE 4-21 shows an archiver.cmd file that specifies multiple metadata copies.
fs = samfs7 1 4h 2 12h |
In this example, copy 1 of the metadata for the samfs7 file system is made after four hours and a second copy is made after 12 hours.
File system metadata includes changes to path names in the file system. For this reason, if you have frequent changes to directories, new archive copies are created. This results in frequent loads of the volumes specified for metadata.
The archive set parameters section of the archiver.cmd file begins with the params directive and ends with the endparams directive. CODE EXAMPLE 4-22 shows the format for directives for an archive set.
params archive_set_name.copy_number[R] [ -param1 -param2 ...] . . . endparams |
The pseudo archive set allsets provides a way to set default archive set directives for all archive sets. All allsets directives must precede those for actual archive set copies. Parameters set for individual archive set copies override parameters set by the allsets directive. For more information on the allsets archive set, see the archiver.cmd(4) man page.
All archive set processing parameters are described in this section, with the exception of the -disk_archive parameter. For information on the -disk_archive parameter, see Disk Archiving.
The -archmax directive sets the maximum file size for an archive set. The format is as follows:
This directive is very similar to the archmax global directive. For information on this directive, and the values to enter for target_size, see The archmax Directive: Controlling the Size of Archive Files.
By default, a file being archived is stored in memory in a buffer prior to writing the file to archive media. You can use the -bufsize parameter to specify a nondefault buffer size. These actions can improve performance, and you can experiment with various buffer_size values.
This parameter has the following format:
For buffer_size, specify a number from 2 through 32. The default is 4. This value is multiplied by the dev_blksize value for the media type, and the resulting buffer size is used. The dev_blksize is specified in the defaults.conf file. For more information on this file, see the defaults.conf(4) man page.
For example, this parameter can be specified in the archiver.cmd file in a line such as the following:
The equivalent of this directive can also be specified on a global basis by specifying the bufsize=media buffer_size directive. For more information on this topic, see The bufsize Directive: Setting the Archiver Buffer Size.
By default, the archiver uses only one media drive to archive one archive set's files. When an archive set has a many files or large files, it can be advantageous to use more than one drive. In addition, if the drives in your automated library operate at different speeds, using these directives can contribute to archiving efficiency.
CODE EXAMPLE 4-23 shows the parameters you can use to split archive requests across drives and to balance variations in tape drive transfer speeds.
-drivemax max_size -drivemin min_size -drives number |
An archive request is evaluated against the parameters that are specified, as follows:
When you use the -drives parameter, multiple drives are used only if data that is more than the min_size is to be archived at once. The number of drives to be used in parallel is the lesser of arch_req_total_size/min_size and the number of drives specified by the -drives parameter.
To set these parameters, users need to consider file creation rates, the number of drives, the time it takes to load and unload drives, and drive transfer rates.
Example 1. You can use the -drivemin and -drives parameters if you want to divide an archive request among drives, but you want to avoid tying up all the drives with small archive requests. This might apply to operations that use very large files.
Example 2. Assume that you are splitting an archive set named big_files over five drives. Depending on its size, this archive set could be split as shown in TABLE 4-15.
CODE EXAMPLE 4-24 shows the lines to use in the archiver.cmd file to split the archive request over multiple drives.
params bigfiles.1 -drives 5 -drivemin 10G endparams |
Example 3. The following line is specified in the archiver.cmd file:
When the total size of the files in archive set huge_files.2 is equal to or greater than two times drivemin for the media, two drives are used to archive the files.
Example 4. An archive request is 300 gigabytes. The following line is specified in the archiver.cmd file to archive 10 gigabytes at a time on each of 5 drives:
By default, the archiver uses all volumes assigned to an archive set when it writes archive copies. When writing the archive copies, the archiver selects a volume with enough space for all the files. This action can result in volumes not being filled to capacity. If -fillvsns is specified, the archiver separates the archive request into smaller groups.
By default, a file being archived is stored in memory in a buffer prior to writing the file to archive media. If direct I/O is enabled, you can use the -lock parameter to lock this buffer. This action can improve performance, and you can experiment with this parameter.
This parameter has the following format:
The -lock parameter indicates whether or not the archiver should use locked buffers when making archive copies. If -lock is specified, the archiver sets file locks on the archive buffer in memory for the duration of the sam-arcopy(1M) operation. This avoids paging the buffer, and it can improve performance.
The -lock parameter should be specified only on large systems with large amounts of memory. Insufficient memory can cause an out-of-memory condition.
The -lock parameter is effective only if direct I/O is enabled for the file being archived. By default, -lock is not specified and the file system sets the locks on all direct I/O buffers, including those for archiving. For more information on enabling direct I/O, see the setfa(1) man page, the sam_setfa(3) library routine man page, or the -O forcedirectio option on the mount_samfs(1M) man page.
For example, this parameter can be specified in the archiver.cmd file in a line such as the following:
You can also specify the equivalent of this parameter on a global basis by specifying the lock argument to the bufsize=media buffer_size [lock] directive. For more information on this topic, see The bufsize Directive: Setting the Archiver Buffer Size.
A file is a candidate for being released after one archive copy is made. If the file releases and goes offline before all the archive copies are made, the archiver uses this parameter to determine the method to be used when making the other archive copies. In choosing the method to be used, consider the number of drives available to the Sun SAM-FS system and the amount of disk cache available. This parameter has the following format:
Specify one of the keywords shown in for method.
The recycling process allows you to reclaim space on archive volumes that is taken up by expired archive images. By default, no recycling occurs.
If you want to recycle, you can specify directives in both the archiver.cmd file and in the recycler.cmd file. For more information on the recycling directives supported in the archiver.cmd file, see Recycling.
The archiver employs associative archiving if you specify the -join path parameter. Associative archiving is useful if you want an entire directory to be archived to one volume and you know that the archive file can physically reside on only one volume. Otherwise, if you want to keep directories together, use either the -sort path or -rsort path parameters to keep the files contiguous. The -rsort performs a reverse sort.
When the archiver writes an archive file to a volume, it efficiently packs the volume with user files. Subsequently, when accessing files from the same directory, you can experience delays as the stage process repositions through a volume to read the next file. To alleviate delays, you can archive files from the same directory paths contiguously within an archive file. The process of associative archiving overrides the space efficiency algorithm to archive files from the same directory together. The -join path parameter allows these files to be archived contiguously within an archive set copy.
Associative archiving is useful when the file content does not change but you want to access the group of files together at the same time all the time. For example, you might use associative archiving at a hospital for accessing medical images. Images associated with the same patient can be kept in a directory and the doctor might want to access those images together at one time. These static images can be accessed more efficiently if you archive them contiguously based on their directory location. For example:
It is also possible to sort files within an archive set copy by age, size, or path. The age and size arguments are mutually exclusive. CODE EXAMPLE 4-25 shows how to sort an archive set using the -sort parameter with the argument age or size.
cardiac.1 -sort path cardiac.2 -sort age catscans.3 -sort size |
The first line forces the archiver to sort an archive request by path name. The second example line forces the archiver to sort an archive set copy called cardiac.2 by the age of the file, oldest to youngest. The third line forces the archive set copy called catscans to be sorted by the size of the file, smallest to largest. If you had wanted a reverse sort, you could have specified -rsort in place of -sort.
Unarchiving is the process by which archive entries for files or directories are deleted. By default, files are never unarchived. Files are unarchived based on the time since last access. All frequently accessed data can be stored on a fast media, such as disk, and all older, infrequently accessed data can be stored on tape.
Example 1. CODE EXAMPLE 4-26 shows an archiver.cmd file.
arset1 dir1 1 10m 60d 2 10m 3 10m vsns arset1.1 mo OPT00[0-9] arset1.2 lt DLTA0[0-9] arset1.3 lt DLTB0[0-9] |
If archiver.cmd file shown in CODE EXAMPLE 4-26 controls a file that is accessed frequently, it remains on disk all the time, even if it is older than 60 days. The copy 1 information is removed only if the file is not accessed for 60 days.
If the copy 1 information is removed (because the file was not accessed for 60 days) and someone stages the file from copy 2, it is read from tape. After the file is back online, the archiver makes a new copy 1 on disk and the 60-day access cycle starts all over again. The Sun StorEdge SAM-FS archiver regenerates a new copy 1 if the file is accessed again.
Assume that a patient is in the hospital for four weeks. During this time, all of this patient's files are on fast media (copy 1=mo). After four weeks, the patient is released from the hospital. If no data has been accessed for this patient for up to 60 days after the patient is released, the copy 1 entry in the inode is unarchived, and only copy 2 and copy 3 entries are available. The volume can now be recycled in order to make room for more current patients without having to increase the disk library. If the patient comes back to the hospital after six months for a checkup, the first access of the data is from tape (copy 2). Now the archiver automatically creates a new copy 1 on disk to ensure that the data is back on the fast media during the checkup, which could take several days or weeks.
By default, the archiver writes a tape mark, an EOF label, and two more tape marks between archive files. When the next archive file is started, the driver backs up to the position after the first tape mark, causing a loss of performance. The -tapenonstop parameter directs the archiver to write only the initial tape mark. In addition, if the -tapenonstop parameter is specified, the archiver enters the archive information at the end of the copy operation.
For more information on the -tapenonstop parameter, see the archiver.cmd(4) man page.
By default, the archiver writes archive set copies to any volume specified by a regular expression as described in the volume associations section of the archiver.cmd file. However, it is sometimes desirable for archive set volumes to contain files from only one archive set. The process of reserving volumes can be used to satisfy this data storage requirement.
|
Note - The -reserve parameter reserves a volume for exclusive use by one archive set. A site that uses reserved volumes is likely to incur more cartridge loads and unloads. |
The -reserve parameter reserves volumes for an archive set. When the -reserve parameter is set and a volume has been assigned to an archive set copy, the volume identifier is not assigned to any other archive set copy, even if a regular expression matches it.
As volumes are selected for use by an archive set, a reserved name is assigned to the volume. The reserved name is a unique identifier that ties the archive set to the volume.
The format for the -reserve parameter is as follows:
The keyword specified depends on the form you are using. The possible forms are archive set form, owner form, and file system form, as follows:
The three owner forms shown in CODE EXAMPLE 4-27 are mutually exclusive. That is, only one of the three owner forms can be used on an archive set and copy.
In the archiver.cmd file, you can specify a -reserve parameter for one, two, or all three possible forms. The three forms can be combined and used together in an archive set parameter definition.
For example, CODE EXAMPLE 4-28 shows an archiver.cmd file fragment. The line that begins with arset.1 creates a reserved name based upon an archive set, a group, and the file system.
params arset.1 -reserve set -reserve group -reserve fs endparams |
The information regarding reserved volumes is stored in the library catalog. The lines within the library catalog contain the media type, the VSN, the reserve information, and the reservation date and time. The reserve information includes the archive set component, path name component, and file system component, separated by slashes (//).
These slashes are not indicative of a path name; they are merely separators for displaying the three components of a reserved name. As CODE EXAMPLE 4-29 shows, the lines that describe reserved volumes begin with #R characters in the library catalog.
Note that some lines in CODE EXAMPLE 4-29 have been truncated to fit on the page.
One or more of the reserve information fields can be empty, depending on the options defined in the archiver.cmd file. The date and time indicate when the reservation was made. A reservation line is appended to the file for each volume that is reserved to an archive set during archiving.
You can display the reserve information by using the samu(1M) utility's v display or by using the archiver(1M) or dump_cat(1M) commands in one of the formats shown in CODE EXAMPLE 4-30.
archiver -lv dump_cat -V catalog_name |
The following formats illustrate each form showing the parameter, keywords, and examples of reserved names assigned to volumes.
For example, in CODE EXAMPLE 4-31, the archiver.cmd file fragment shows that the line that begins with the allsets archive set name sets reserve by archive set for all archive sets.
Example 4 shows a complete archive example using reserved volumes.
The archiver records volume reservations in the library catalog files. A volume is automatically unreserved when it is relabeled because the archive data has been effectively erased.
You can also use the reserve(1M) and unreserve(1M) commands to reserve and unreserve volumes. For more information on these commands, see the reserve(1M) and unreserve(1M) man pages.
The Sun StorEdge SAM-FS file systems offer a configurable priority system for archiving files. Each file is assigned a priority computed from properties of the file and priority multipliers that can be set for each archive set in the archiver.cmd file. Properties include online/offline, age, number of copies made, and size.
By default, the files in an archive request are not sorted and all property multipliers are zero. This results in files being archived in first found, first archived order. For more information on priorities, see the archiver(1M) and archiver.cmd(4) man pages.
You can control the order in which files are archived by setting priorities and sort methods. The following are examples of priorities that you can set:
TABLE 4-20 lists the archive priorities.
For value, specify a floating-point number in the following range:
As the archiver scans a file system, it identifies files to be archived. Files that are recognized as candidates for archiving are placed in a list known as an archive request. At the end of the file system scan, the system schedules the archive request for archiving. The -startage, -startcount, and -startsize archive set parameters control the archiving workload and assure timely archival of files. TABLE 4-21 shows the formats for these parameters.
|
Specifies the amount of time that can elapse between the first file in a scan being marked for inclusion in an archive request and the start of archiving. For time, specify a time in the format used in Setting the Archive Age. |
|
|
Specifies the number of files to be included in an archive request. When the number of files in the archive request reaches count, archiving begins. Specify an integer number for count. By default, count is not set. |
|
|
Specifies the minimum total size, in bytes, of all files to be archived in an archive request. Archiving work is accumulated and archiving begins when the total size of the files reaches size. By default, size is not set. |
The examine=method directive and the interval=time directives are global directives that interact with the -startage, -startcount, and -startsize directives. The -startage, -startcount, and -startsize directives optimize archive timeliness versus archive work done. These values override the examine=method specification, if any. For more information on the examine directive, see The examine Directive: Controlling Archive Scans. For more information on the interval directive, see The interval Directive: Specifying an Archive Interval.
Only one of the -startage, -startcount, and -startsize directives can be specified in an archiver.cmd file. If more than one of these directives is specified, the first condition encountered starts the archival operation. If neither -startage, -startcount, nor -startsize are specified, the archive request is scheduled based on the examine=method directive, as follows:
The archiver.cmd(4) man page has examples that show how to use these directives.
The VSN associations section of the archiver.cmd file assigns volumes to archive sets. This section starts with a vsns directive and ends with an endvsns directive.
Collections of volumes are assigned to archive sets by directives of the following form:
An association requires at least three fields: the archive_set_name and copy_number, the media_type, and at least one volume. The archive_set_name and copy_number are connected by a period (.).
The following examples specify the same VSNs in different ways.
Example 1. CODE EXAMPLE 4-32 shows two lines of VSN specifications.
vsns set.1 lt VSN001 VSN002 VSN003 VSN004 VSN005 set.1 lt VSN006 VSN007 VSN008 VSN009 VSN010 endvsns |
Example 2. CODE EXAMPLE 4-33 shows a VSN specification that uses a backslash character (\) to continue a line onto a subsequent line.
vsns set.1 lt VSN001 VSN002 VSN003 VSN004 VSN005 \ VSN006 VSN007 VSN008 VSN009 VSN010 endvsns |
Example 3. CODE EXAMPLE 4-34 specifies VSNs using a regular expression in a shorthand notation.
The volumes are noted by one or more vsn_expression keywords, which are regular expressions as described in the regexp(5) man page. Note that these regular expressions do not follow the same conventions as wildcards. In addition to a regular expression, you can also specify VSN pools from which volumes are to be selected. Pools are expressed with the -pool vsn_pool_name directive with a VSN association.
When the archiver needs volumes for the archive set, it examines each volume of the selected media type in all automated libraries and manually mounted drives to determine if it would satisfy any VSN expression. It selects the first volume that fits an expression that contains enough space for the archive copy operation. For example:
If your Sun StorEdge SAM-FS environment is configured to recycle by archive set, do not assign a VSN to more than one archive set.
The VSN pools section of the archiver.cmd file starts with a vsnpools directive and ends with either an endvsnpools directive or with the end of the archiver.cmd file. This section names a collection of volumes.
A VSN pool is a named collection of volumes. VSN pools are useful for defining volumes that can be available to an archive set. As such, VSN pools provide a useful buffer for assigning volumes and reserving volumes to archive sets.
You can use VSN pools to define separate groups of volumes for use by departments within an organization, users within a group, data types, and other convenient groupings. The pool is assigned a name, media type, and a set of volumes. A scratch pool is a set of volumes used when specific volumes in a VSN association are exhausted or when another VSN pool is exhausted. For more information on VSN associations, see VSN Association Directives.
If a volume is reserved, it is no longer available to the pool in which it originated. Therefore, the number of volumes within a named pool changes as volumes are used. You can view the VSN pools by entering the archiver(1M) command in the following format:
A VSN pool definition requires at least three fields separated by white space: the pool name, the media type, and at least one VSN. The syntax is as follows:
The following example uses four VSN pools: users_pool, data_pool, proj_pool, and scratch_pool. If one of the three specific pools is out of volumes, the archiver selects the scratch pool VSNs. CODE EXAMPLE 4-35 shows an archiver.cmd file that uses four VSN pools.
Archiving is the process of copying a file from online disk to archive media. Often, archive copies are written to volumes on magneto optical or tape cartridges in an automated library, but with disk archiving, online disk in a file system is used as archive media.
Disk archiving can be implemented so that the files are archived from one Sun StorEdge SAM-FS file system to another file system on the same host computer system. Disk archiving can also be implemented so the source files are archived to another file system on a different Sun Solaris system. When disk archiving is implemented using two host systems, the systems involved act as a client and a server. The client system is the system that hosts the source files. The server system is the destination system that hosts the archive copies.
The file system to which the archive files are written can be any UNIX file system. It does not have to be a Sun StorEdge SAM-FS file system. If disk archive copies are written to a different host, the host must have at least one Sun StorEdge SAM-FS file system installed upon it.
The archiver treats files archived to disk volumes the same as it treats files archived to volumes in a library. You can still make one, two, three, or four archive copies. If you are making multiple archive copies, one of the archive copies could be written to disk volumes while the others are written to removable media volumes. In addition, if you typically archive to disk volumes in a Sun StorEdge SAM-FS file system, the archive file copies are themselves archived according to the archiver.cmd file rules in that file system.
The following list summarizes some of the similarities and differences between archiving to online disk and archiving to removable media:
A diskvols.conf file must be created on the system upon which the source files reside. Depending on where the archive copies are written, this file also contains the following information:
While there are no restrictions on where disk archive volumes can reside, it is recommended that the disk volumes reside on a disk other than the one upon which the original files reside. Preferably, the archive copies from a client system would be written to disk volumes on a server system. It is recommended that you make more than one archive copy and write to more than one type of archive media. For example, copy 1 could be archived to disk volumes, copy 2 to tape, and copy 3 to magneto-optical disk.
If you are archiving files to a file system on a server system, the archive files themselves can be archived to removable media cartridges in a library attached to the destination server.
When archiving to online disk, the archiver recognizes most of the archiver.cmd directives. The directives it recognizes define the archive set and configure recycling. The directives that are silently ignored are those that are meaningless in a disk archiving enviroment because they specifically pertain to working with removable media cartridges. Specifically, the system recognizes the following directives for disk archive sets:
For VSN_Name, specify a VSN that is defined in the diskvols.conf file.
For client_system, specify the hostname of the client system that contains the source files.
For more information on directives for disk archiving, see the archiver.cmd(4) man page.
|
You can enable disk archiving at any time. The procedure in this section assumes that you have archiving in place and you are adding disk archiving to your environment. If you are enabling disk archiving as part of an initial installation, see the Sun StorEdge QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide for information. Do not use this procedure because it contains steps that are not needed if you add disk archiving at installation time.
1. Make certain that the host to which you want to write your disk archive copies has at least one Sun StorEdge SAM-FS file system installed on it.
2. Become superuser on the host system that contains the files to be archived.
3. Follow the procedures in the Sun StorEdge QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide for enabling disk archiving.
The Sun StorEdge SAM-FS initial installation procedure contains a step called Enabling Disk Archiving. That step is broken down into two procedures.
4. Become superuser on the host that contains the files to be archived.
5. On the host that contains the files to be archived, use the samd(1M) config command to propagate the configuration file changes and restart the system.
6. Become superuser on the host system to which the archive copies are written. (Optional)
Perform this step only if you are archiving to disk on a different host.
7. On the host to which the archive copies will be written, use the samd(1M) config command to propagate the configuration file changes and restart the destination system. (Optional)
Perform this step only if you are archiving to disk on a different host.
CODE EXAMPLE 4-38 shows the diskvols.conf file that resides on client system pluto.
In the preceding diskvols.conf file, VSNs identified as disk01 and disk02 are written to the host system upon which the original source files reside. VSN disk03 is written to a VSN on server system mars.
CODE EXAMPLE 4-39 shows the diskvols.conf file on server system mars.
# This is file /etc/opt/SUNWsamfs/diskvols.conf on mars # clients pluto endclients |
CODE EXAMPLE 4-40 shows a fragment of the archiver.cmd file on pluto.
params arset1.2 -disk_archive disk01 arset2.2 -disk_archive disk02 arset3.2 -disk_archive disk03 endparams |
In this example, file /sam1/testdir0/filea is in the archive set for arset0.1, and the archiver copies the content of /sam1/testdir0/filea to the destination path named /sam_arch1. CODE EXAMPLE 4-41 shows the diskvols.conf file.
# This is file /etc/opt/SUNWsamfs/diskvols.conf # # VSN Name [Host Name:]Path # disk01 /sam_arch1 disk02 /sam_arch12/proj_1 |
CODE EXAMPLE 4-42 shows the archiver.cmd file lines that pertain to disk archiving:
. . . params arset0.1 -disk_archive disk01 endparams . . . |
The following shows output from the sls(1) command for file filea, which was archived to disk. In CODE EXAMPLE 4-43, note the following:
In this example, file /sam2/my_proj/fileb is on client host snickers in archive set arset0.1, and the archiver copies the content of this file to the destination path /sam_arch1 on server host mars.
CODE EXAMPLE 4-44 shows the diskvols.conf file on snickers.
# This is file /etc/opt/SUNWsamfs/diskvols.conf on snickers # # VSN Name [Host Name:]Path # disk01 mars:/sam_arch1 |
CODE EXAMPLE 4-45 shows the diskvols.conf file on mars.
# This is file /etc/opt/SUNWsamfs/diskvols.conf on mars # clients snickers endclients |
CODE EXAMPLE 4-46 shows the directives in the archiver.cmd file that relate to this example.
. . . params arset0.1 -disk_archive disk01 endparams . . . |
TABLE 4-24 shows the directory structure that all examples in this section use.
This example illustrates the action of the archiver when no archiver.cmd file is used. In this example, a Sun StorEdge SAM-FS environment includes one file system, an optical automated library with two drives, and six cartridges.
CODE EXAMPLE 4-47 shows the output produced by the archiver(1M) -lv command. It shows that the default media selected by the archiver is type mo. Only the mo media are available.
CODE EXAMPLE 4-48 shows output that indicates that the archiver uses two drives. It lists the 12 volumes, storage capacity, and available space.
CODE EXAMPLE 4-49 shows that both the metadata and data files are included in the archive set samfs. The archiver makes one copy of the files when their archive age reaches the default four minutes (240 seconds).
Archive file selections: Filesystem samfs Logfile: samfs Metadata copy:1 arch_age:240 samfs1 path:. copy:1 arch_age:240 |
CODE EXAMPLE 4-50 shows the files in the archive sets archived to the volumes in the indicated order.
This example shows how to separate data files into two archive sets separate from the metadata. There is a manually mounted DLT tape drive in addition to the optical automated library from Example 2. The big files are archived to tape, and the small files are archived to optical cartridges.
CODE EXAMPLE 4-51 shows the content of the archiver.cmd file.
CODE EXAMPLE 4-52 shows the media and drives to be used, not the addition of the DLT and its defaults.
CODE EXAMPLE 4-53 shows the organization of the file system. Files bigger than 512000 bytes (500 kilobytes) are archived after four minutes; all other files are archived after 30 seconds.
CODE EXAMPLE 4-54 shows the division of the archive sets among the removable media in the following output.
In this example, user files and project data files are archived to various media. Files from the directory data are segregated by size to optical and tape media. Files assigned to the group ID pict are assigned to another set of volumes. Files in the directories tmp and users/bob are not archived. Archiving is performed on a 15-minute interval, and an archiving record is kept.
CODE EXAMPLE 4-55 shows this example.
In this example, user files and project data files are archived to optical media. Note that CODE EXAMPLE 4-56 does not use the directory structure presented in TABLE 4-24.
Four VSN pools are defined; three pools are used for user, data, and project, and one is a scratch pool. When the proj_pool runs out of media, it relies on the scratch_pool to reserve volumes. This example shows how to reserve volumes for each archive set based on the set component, owner component, and file system component. Archiving is performed on a 10-minute interval, and an archiving log is kept.
CODE EXAMPLE 4-56 shows the archiver.cmd file and archiver output.
The archiver automates storage management operations using the archiver.cmd file. Before writing this file, it is useful to review some general guidelines that can improve the performance of your Sun StorEdge SAM-FS file system and the archiver. This ensures that your data is stored in the safest way possible.
Each site is unique in its application of computing, data storage hardware, and software. The following recommendations are based upon the experiences of Sun Microsystems. When writing the archiver.cmd file for your site, be sure that you reflect the data storage requirements at your site by considering the following aspects.
1. Save your archive logs. The archive logs provide information that is essential to recovering data, even when the Sun StorEdge SAM-FS software is unavailable. It is recommended that you keep these logs in a safe place in the event of a catastrophic disaster during which the Sun StorEdge SAM-FS software is unavailable.
2. Use regular expressions for volumes. Let the system work for you by allowing it to put files on many different volumes. Volume ranges (specified using regular expressions) allow the system to run continuously. Using specific volume names for archive set copies can rapidly fill a volume, causing undue workflow problems as you remove a piece of media and replace it with another.
3. Base your archive interval on how often files are created and modified, and whether you want to save all modification copies. Remember, that the archive interval is the time between file system scans. A very short archive interval keeps the archiver scanning almost continuously.
4. Consider the number of file systems you are using. Multiple Sun StorEdge SAM-FS file systems generally increase the performance of the archiver as compared to a single Sun StorEdge SAM-FS file system. The archiver uses a separate process for each file system. Multiple file systems can be scanned in considerably less time than a single file system.
5. Use directory structures to organize your files within the Sun StorEdge SAM-FS file system like UNIX file systems. For performance considerations, Sun Microsystems recommends that you do not place more than 10,000 files in a directory.
6. Always make a minimum of two file copies on two separate volumes. Putting data on a single media type puts your data at risk if physical problems with the media occur. Do not rely on a single archive copy if at all possible.
7. Make sure you are dumping your metadata using samfsdump(1M) on a regular basis. The metadata (directory structure, file names, and so on) is stored in an archive set that has the same name as the file system. You can use this information to recover a file system in the event of a disaster. If you do not want to do this, you can prevent this data from being archived by assigning this archive set to a nonexistent VSN. For more information on preserving metadata, see the Sun QFS, Sun SAM-FS, and Sun SAM-QFS Disaster Recovery Guide or the Sun StorEdge QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide.
Upon initial setup, the archiver might not perform the tasks as intended. Make sure that you are using the following tools to monitor the archiving activity of the system:
The samu(1M) utility's a display includes messages for each file system. It indicates when the archiver will scan the .inodes file again and the files currently being archived.
Occasionally, you might see messages to indicate that the archiver either has run out of space on cartridges or has no cartridges. These messages are as follows:
The following checklist includes reasons why your Sun StorEdge SAM-FS environment might not be archiving files.
1. The archiver.cmd file has a syntax error. Run the archiver -lv command to identify the error, then correct the flagged lines.
2. The archiver.cmd file has a wait directive in it. Either remove the wait directive or override it by using the samu(1M) utility's :arrun command.
3. No volumes are available. You can view this from archiver(1M) -lv command output. Add more volumes as needed. You might have to export existing cartridges to free up slots in the automated library.
4. The volumes for an archive set are full. You can export cartridges and replace them with new cartridges (make sure that the new cartridges are labeled), or you can recycle the cartridges. For more information on recycling, see Recycling.
5. The VSN section of the archiver.cmd file fails to list correct media. Check your regular expressions and VSN pools to ensure that they are correctly defined.
6. There is not enough space to archive any file on the available volumes. If you have larger files and it appears that the volumes are nearly full, the cartridges might be as full as the Sun StorEdge SAM-FS environment allows. If this is the case, add cartridges or recycle.
If you have specified the -join path parameter, and there is not enough space to archive all the files in the directory to any volume, no archiving occurs. You should add cartridges, recycle, or use one of the following parameters: -sort path or -rsort path. For more information on these parameters, see Associative Archiving: -join.
7. The archiver.cmd file has the no_archive directive set for directories or file systems that contain large files.
8. The archive(1) -n (archive never) command has been used to set too many directories, and files are never archived.
9. Large files are busy. Thus, they never reach their archive age and are not archived.
10. Hardware or configuration problems exist with the automated library.
11. Network connection problems exist between client and server. Ensure that the client and the server have established communications.
In addition to examining the items on the previous list, you should check the following when troubleshooting the archiver.
1. The syslog file (by default, /var/adm/sam-log). This file can contain archiver messages that can indicate the source of a problem.
2. Volume capacity. Ensure that all required volumes are available and have sufficient space on them for archiving.
3. If the archiver appears to cause excessive, unexplainable cartridge activity or appears to be doing nothing, turn on the trace facility and examine the trace file. For information on trace files, see the defaults.conf(4) man page.
4. You can use the truss(1) -p pid command on the archiver process (sam-archiverd) to determine the system call that is not responding. For more information on the truss(1) command, see the truss(1) man page.
5. The showqueue(1M) command displays the content of the archiver queue files. You can use this command to observe the state of archiver requests that are being scheduled or archived. Any archive request that cannot be scheduled generates a message that indicates the reason. This command also displays the progress of archiving.
The archiver and the releaser work together to balance the amount of data available on the disk cache. The main reason that files are not released automatically from disk cache is that they have not yet been archived.
For more information on why files are not being released, see Troubleshooting the Releaser.
| Sun StorEdge SAM-FS Storage and Archive Management Guide | 817-4093-10 |
|
Copyright © 2004, Sun Microsystems, Inc. All rights reserved.
To Create or Modify an archiver.cmd File and Propagate Your Changes
value