NAME

savelogs - Save log files

SYNOPSIS

savelogs saves your log files in a nice way (by default).

    savelogs --postmovehook='/usr/local/bin/restart_apache' \
             --apacheconf=/www/conf/httpd.conf /var/log/messages

    savelogs `cat list_of_logs_to_process.txt`

    savelogs --loglevel=2 /var/log/maillog /var/log/cronlog \
             /var/log/messages

    savelogs --config=/etc/savelogs.conf

    savelogs --period=15 /var/log/messages

DESCRIPTION

savelogs is a flexible and robust log file archival system. Its logic is simple: move (rename) the log file, filter data from the log file, store the log file in an archive (via tar or gtar), and compress the archive (via gzip or compress). After successful compression, the original log file is deleted.

All of the above phases are optional. This means that you may simply delete files if you wish. Or you may simply compress existing log files. Or you may move files and add them to a tar file but leave the tar file uncompressed, etc. You pick ;o)

(If you just want to cut to the chase and don't care how savelogs works, see the EXAMPLES section near the bottom of this document.)

Savelogs Phases

The processing order may be abbreviated into these five phases:

    move -> filter -> archive -> compress -> delete

any of which may be subtracted from the process order. In addition to these phases are some intermediate 'hooks' where you may supply an external program or shell command to invoke. This is useful if you're rotating web server log files and you need to HUP (restart) your web server after moving the logs (for example).

Subtracting phases is done in one of two possible ways. The first way is to specify it in the configuration file:

    Process          move,archive,delete

which will move log files and archive them (but not filter or compress them). After successful archival, the original log files will be deleted.

The second way is to specify it on the command-line:

    --process=compress,delete

which will simply compress log files (but not move, filter, or archive them).

In addition to the five phase processing options above, you may also employ the following abbreviations:

(no option specified): If you specify no process option, the default is move,compress.
none: Do none of the phases. This isn't a very useful option.
all: Do all of the phases.

An Overview

A typical savelogs session might begin by typing this command:

    savelogs --process=all /var/log/messages

After which the following phases will execute:

move: /var/log/messages --> /var/log/messages.010523
filter: (no filter supplied, so skip this phase)
archive: add messages.010523 to /var/log/messages.tar
compress: /var/log/messages.tar --> /var/log/messages.tar.gz
delete: (/var/log/messages.010523 is deleted)

A Word About Paths

All paths you specify to savelogs should be relative to your home directory. You do not need to use a tilde (~). You may assume that savelogs runs from your home directory and knows how to handle absolute paths.

If my real home directory were located in /usr/home/joe and I wanted to rotate the log file located in /usr/home/joe/var/log/messages, I would do something like this:

    savelogs /var/log/messages

and savelogs would Do What I Mean.

The only exception to this are external commands given to postmovehook and other such options. Paths you specify here really are full paths.

CONFIGURATION

Configuration file option format

savelogs will read its configuration options from a configuration file (which you must supply) or from the command-line. Creating a configuration file is easy: it is a simple Apache-style plaintext file that has options specified in this format:

    Option          value

where Option is one of the options below and value is either a true/false; yes/no; on/off combination or some string value, such as a pathname and file (depending on the nature of the option).

Your distribution of savelogs may have included a sample configuration file for you to edit as you wish in ~/etc/savelogs.conf.sample.

Option processing order

Configuration options are first read from savelogs internal defaults, which are sprinkled throughout this document. Next savelogs reads its configuration file, if any is specified. Lastly, savelogs uses options from the command-line.

For example, the default value for the period directive is 10. If you ran savelogs like this:

    savelogs --period /var/log/messages

every day for 10 days, you would have 10 archived log files.

If you have in your configuration file:

    Period            5

and ran the same command above every day for 10 days, you'd only have 5 archived log files because the configuration file overrides savelogs internal defaults. Finally, if you had a configuration file with the previously mentioned value for period and ran this command:

    savelogs --period=7 /var/log/messages

every day for 10 days, you would have 7 archived log files because command-line options override configuration file options (which override internal default values).

Available options

All options you may specify on the command-line you may also sepcify in a configuration file (except the configuration file directive itself). For example, if you had a cronjob that did this:

    savelogs --process=delete --postmovehook=/usr/local/bin/restart_apache \
    --apacheconf=/www/conf/httpd.conf

you could make a nice configuration file (call it ~/etc/savelogs1.conf) that would do the same thing:

    Process         delete
    PostMoveHook    /usr/local/bin/restart_apache
    ApacheConf      /www/conf/httpd.conf

and then invoke your cron like this:

    savelogs --config=/etc/savelogs1.conf

Case-sensitivity

A sample configuration file may be located in ~/etc/savelogs.conf.sample which you may copy and edit as you wish. Configuration file directives are case-insensitive:

    Process    move,compress
    PROCESS    move,compress
    process    move,compress
    pROceSs    move,compress

are all the same directive to savelogs. Configuration file values ARE case-sensitive.

    ApacheConf    /www/conf/httpd.conf
    ApacheConf    /WWW/conf/httpd.conf
    ApacheConf    /www/CONF/httpd.conf
    ApacheConf    /www/conf/Httpd.conf

are four distinct files.

Testing configurations

When you are testing new configuration directives, use the dry-run option and watch the log output (see the loglevel and logfile directives). This will help you avoid losing data unnecessarily.

You may also specify the settings option which will show you all the current settings after defaults, configuration file, and command-line options have been processed.

OPTIONS

Options given below are configuration directives that may appear in a configuration file or on the command-line. Options are case-insensitive, i.e., ApacheLog is the same as apachelog, though the values associated with the options are often case-sensitive (e.g., paths, filenames, etc.)

Options specified on the command-line should be prefixed with two hyphens. Some options do not make sense in a configuration file or need to occur before the configuration file is parsed such as config, help, or home.

Running Savelogs

help

Shows a brief usage statement and exits.

Example:

    savelogs --help

version

Shows the current version of savelogs and exits.

Example:

    savelogs --version

settings

Shows the current settings of savelogs and exits.

Example:

    savelogs --settings --apacheconf=/www/conf/httpd.conf \
             /var/log/messages

dry-run

Shows you what will happen if you were to run savelogs with the current settings without actually doing it. This is a useful option to specify if you want to see what effect some changes might have, or to see which files are going to get archived with the current settings.

Note that savelogs running under the dry-run directive will sometimes produce errors that wouldn't occur during normal use. This can happen for a variety of reasons, mostly related to savelogs looking for files that don't yet exist, or archives that don't yet exist because they weren't actually created. In this respect, dry-run doesn't give you precisely what will happen, but it does give you a good idea.

Example:

    savelogs --dry-run /var/log/foo

home

Changes the default base location of where savelogs runs from. This is mostly a debugging utility. Consider this an advanced feature which should probably be ignored. Defaults to the process owner's home directory (which is almost always what you want).

Example:

    Home                /usr/home/joe/usr/home/bob

config

Changes the default configuration file savelogs reads at startup. This should be done from the command-line or it won't have any effect.

Example:

    savelogs --config=/etc/my_other_savelogs.conf

process=[move],[filter],[archive],[compress],[delete]

Tells savelogs which phases to execute. If you just want to move (rename) logs, do this:

    savelogs --process=move /var/log/messages

and ~/var/log/messages will become ~/var/log/messages.yymmdd (where yymmdd are today's date).

For just removing logs, specify the delete option. You can get fancy:

    savelogs --process=move,compress /var/log/messages

which renames the log file ~/var/log/messages to ~/var/log/messages.yymmdd and then compresses it, not filtering, archiving (i.e. putting into a separate tar file), or deleting it (the compress option also renames the file so that delete becomes useless since the file as it was no longer exists).

move,compress is the default value for the process option, so the above directive could have also been specified:

    savelogs /var/log/messages

You may also specify all (the default) or none as shortcuts.

Savelogs Logging

savelogs has an internal logging facility that helps you diagnose problems, or just see what's going on. By default, savelogs writes to stdout (i.e., your screen if you're running this from a tty).

loglevel=#

Determines how verbose savelogs is when it's writing its own internal messages. Valid values are between 0 and 5 inclusive.

Example:

    LogLevel            3

The general rule of thumb for savelogs logging is this:

    Level  What will be logged
    =====  ===================
    0      no output except fatal errors
    1      Level 0 + start/finish stats, errors
    2      Level 1 + warnings, logfiles to process
    3      Level 2 + chdir, filter, phase completion
    4      Level 3 + phase core actions, phase beginning
    5      Level 4 + everything else

The first few times you run savelogs, try a higher loglevel value to see what's happening with your log files. Once you're comfortable with how savelogs works, you may turn it down a few notches (level 1 is usually fine) so at least you can check to see if your cronjob actually ran ;o)

logfile=[stdout|stderr|/path/to/log]

savelogs, depending on the loglevel you've specified, writes what it's doing, such as moving, archiving, or deleting, etc. logfile tells savelogs where to write all these messages.

The default value for logfile is stdout which means that your output will go to the screen unless you've redirected STDOUT. You may also specify stderr or the path to a file where you'd like messages to be appended.

Example:

    LogFile             /var/log/savelogs.log

How Savelogs Finds Logs to Process

savelogs normally just processes the logs you specify on the command-line (items on the command-line that are not recognized as options are assumed to be log files to process).

You may also save wear on your finger tips and phosphor in your monitor by using the following directives.

Log=/path/to/log

This works just like adding a file on the command-line, but is included so that you can put log files you want processed in a configuration file. It will also work on the command-line:

Example:

    savelogs --log=/var/log/messages

is the same as:

    savelogs /var/log/messages

which is also equivalent to a config file named '~/etc/savelogs.conf' with this single line:

    Log                  /var/log/messages

and invoked like this:

    savelogs --config=/etc/savelogs.conf

apacheconf=/path/to/httpd.conf

If you specify this option, giving it a valid httpd.conf file, savelogs will parse your Apache configuration file looking for standard log file directives. Any files found will be processed.

Example:

    savelogs --apacheconf=/www/conf/httpd.conf

or in your configuration file:

    ApacheConf          /usr/local/etc/httpd/conf/httpd.conf

apachelog

This option allows you to tell savelogs which logs to process in the httpd.conf file specified by apacheconf. The default value for the apachelog directive is:

    TransferLog|ErrorLog|AgentLog|RefererLog|CustomLog

You may do something clever like this:

    savelogs --apacheconf=/www/conf/httpd.conf --apachelog=TransferLog

which would archive all of your access_log files. Then after running this command, you could do this:

    savelogs --process=delete --apacheconf=/www/conf/httpd.conf \
             --apachelog=ErrorLog

which would delete your error_log files.

In general, the fewer values you specify in the apachelog directive the faster savelogs will find your log files (though the speedup really is negligible, it may also save you from rotating logs you didn't want to).

    ApacheLog           TransferLog|ErrorLog

would be sufficient for most people using combined Apache logs.

apachelogexclude

For those who want somewhat finer control of which logs get processed in their Apache configuration file, the apachelogexclude directive allows you to specify a Perl regular expression (which may simply be a string like 'error') of log files to exclude when processing logs. This way you could do something like this:

    savelogs --apacheconf=/www/conf/httpd.conf --apachelogexclude=logs/bob

which would process all logs found in your httpd.conf file except log files whose names contain the string 'logs/bob'. Maybe Bob likes to rotate his logs using another program or system (conceivable, though unlikely).

Multiple occurances of apachelogexclude are allowed:

    ApacheLogExclude          /dev/null
    ApacheLogExclude          \|
    ApacheLogExclude          logs/bob

which would exclude log files whose names contained '/dev/null' or 'logs/bob' from being processed. Any valid Perl regular expression will work, so:

    ApacheLogExclude          ^/dev/null$

is not the same as the previous example. This example will only match log files whose name is exactly /dev/null, no more, no less.

By default, savelogs uses the following patterns to determine logs to exclude:

    ^/dev/null$
    \| (this is a literal pipe character)

This means that by default savelogs will not attempt to archive a log whose name is '/dev/null' or whose name contains a pipe (|). If for some bizarre reason you wish to remove these defaults when you run savelogs, you can give an empty apachelogexclude option on the command-line:

    % savelogs --apachelogexclude= --config=/etc/savelogs.conf

Moving (Renaming) Files

touch

Touches the original file, creating it if necessary. This is useful for programs that log in "append-only" mode and do not create the log file if it is missing. Once savelogs has renamed a log file, touch will create the file if it does not exist, or reset the timestamp if it does.

size=kbytes

Logs smaller than kbytes will not be included in any processing. To override a default setting, specify --size with no arguments on savelogs command-line.

    Size                      5000

will skip all log files smaller than 5 megs, regardless of other settings.

ext=string

Set the filename extension to 'string'. When a file is moved, it is renamed to the original filename plus the extension you specify. If no extension is specified, today's date is used in 'yymmdd' format. Other options include today and yesterday.

You may not use ext in a configuration file with a value in backticks (e.g., the line:

    ## this directive will not work: don't use it!
    Ext                  `/usr/local/bin/smalldate -u`

in a configuration file will not work). Any other (static) value for ext in a configuration file will work.

See also hourly below for information on how to modify ext even further.

sep=char

The separator character to use when moving files. By default this character is a dot ('.'). Other favorites are underscore ('_') and hyphen/minus sign ('-').

Example:

Use an underscore character:

    savelogs --sep='_' --process=move /var/log/foo

will rename ~/var/log/foo to ~/var/log/foo_yymmdd (where yymmdd is today's date).

Use no separator (just concatenate the extension with the filename):

    savelogs --sep= --process=move /var/log/foo

will rename ~/var/log/foo to ~/var/log/fooyymmdd.

hourly

Adds a letter of the alphabet to the back of the filename extension. This is useful if you are rotating logs several times a day. For example, if you specified your extension (via ext) as 'foo', any log files rotated in the 10am hour will be named 'log.fook'. At 11am, all logs will be called 'log.fool' and so forth.

Example:

    savelogs /path/to/log_file

This will rename the log file to log_file.yymmdd where yymmdd are today's year, month, and day of month.

    savelogs --hourly /path/to/log_file

If you specify the hourly option (or in your configuration file, the Hourly directive), the log will be renamed to log_file.yymmddz where the z represents the current hour as a letter of the alphabet (0 = a, 1 = b, 2 = c, etc.).

The hourly option is available for log file rotation that occurs several times per day. If you need to rotate more often than several times per day, you'll need to either:

use period to rotate logs with a unique number as the extension. period, discussed below, renames log files with a simple integer: log becomes log.0, the former log.0 becomes log.1 and so forth.
specify an extension with --ext that suits your needs. For example, you might write a small program in Perl that outputs a longer date extension and include it like this:
```
    savelogs --ext=`/path/to/prog`
```
A sample program called smalldate has been included with savelogs to do this. Try:
```
    savelogs --ext=`~/usr/local/bin/smalldate`
```
Type smalldate -u for usage.
Download your log files often and delete them from the server (you can use savelogs to delete them for you)

period[=count]

Renames the file based on a period, which is how frequently you run savelogs. If you specify period you may also optionally specify a 'count', which is how many log files to save using the period option:

    savelogs --period=8 /var/log/messages

You may also use the count option, which is deprecated for backward compatibility and some possible future enhancements:

    savelogs --period --count=8 /var/log/messages

If you do not specify a count value (either in period or count), a count of 10 is assumed.

The period option will rename the current log to logfile.0, the log that was previously named logfile.0 to logfile.1 and so on, much like newsyslog(8).

When you specify the period option, the process phases move and compress are assumed. If you also specify a process phase of filter, that will be honored also.

The period option will override any other sep and ext options specified, using the default dot ('.') for the separator and an integer for the extension.

The author also recommends you don't try to mix period named log files with other log files in the same directory, since savelogs may not be able to tell which logs are oldest based on the filename extension and destroy the wrong files. You can safely use any default savelogs extensions (e.g., the default 'today' or 'yesterday' extensions) or your own extension if your own extension contain at least one non-digit (0-9) or your own extension has 5 or more digits. If you meet either of these criteria in your own extension, you may feel confident about mixing logs.

For those familiar with Unix system adminstration, period works like newsyslog(8) with the B option specified in the newsyslog configuration file (the B option tells newsyslog to treat logs as binary files and not append the status message to the log).

An example use of the period option:

    savelogs --touch --period=15 /var/log/maillog /var/log/messages

will move any existing ~/var/log/maillog and ~/var/log/messages to ~/var/log/maillog.0 and ~/var/log/messages.0 and compress them. By specifying the touch option, the original ~/var/log/maillog and ~/var/log/messages will be 'touched', recreating the files. When this command is run again, ~/var/log/maillog.0 is moved to ~/var/log/maillog.1 and ~/var/log/maillog is moved to ~/var/log/maillog.0.

count

Limits the number of logs saved using the period option. The internal default value for count is 10.

If you are using the period option, as of version 1.21 you may now simply specify the count as part of the period option:

    savelogs --period=5 /var/log/messages

postmovehook=command

Runs an arbitrary system command you specify after moving files. If you are rotating Apache log files, you should use a command that will tell your web server to close its log file descriptors and re-open them (e.g., 'restart_apache' would do that on a Virtual Server).

Example:

    savelogs --apacheconf=/www/conf/httpd.conf \
             --postmovehook='/usr/local/bin/restart_apache'

or even nicer in your configuration file:

    PostMoveHook        /usr/local/bin/restart_apache

Paths in postmovehook are NOT relative to your home directory, as most other paths are. You should specify the full path to the file if the file is not in your environment's $PATH.

Variables

Some internal savelogs variables are available during the postmovehook phase. These variables are automatically interpolcated by savelogs during execution and are guaranteed to contain some useful value.

$APACHE_CONF

Contains the full path to the Apache configuration file as specified with the apacheconf option.

    savelogs --apacheconf=/www/conf/httpd.conf \
             --postmovehook='touch $APACHE_CONF'

$HOME

Contains the path to your home directory.

    savelogs --postmovehook='$HOME/bin/myprogram'

force-pmh

Executes the postmovehook command even if there are no logs to process. By default, savelogs will not execute the postmovehook command if there are no logs.

stem

Like ext except the stem is used in addition to ext. After the move and postmovehook phases have completed, savelogs checks to see if you have defined a stemhook. If a stemhook has been defined, a symbolic link to the log is made using stem.

As an example, say you were processing ~/var/log/messages. During the stem phase, savelogs would do this:

    messages.today -> messages.<ext>

    (where <ext> is the extension you specified or today's date by
    default)

Once the stemhook command has executed, the symbolic link (or hard link or copy, as specified by stemlink) is removed.

stemhook=command

A command to execute, much like postmovehook, except that this phase is suited for log file analysis tools that require a predictably named, dead (i.e., no logging is currently being done to it) log file. urchin and analog are good examples of programs that require such logs.

An example use for stemhook would be:

    % savelogs --stemhook="$HOME/usr/local/urchin/urchin" \
      /www/logs/access_log /www/logs/error_log

urchin should be instructed to operate on /www/logs/access_log.today and /www/logs/error_log.today. urchin should also be instructed to do nothing to the log files since we're allowing savelogs to manage them for us. Any changes resulting from the stemhook command to the log file will occur in the original log file.

The same variables for postmovehook are available for stemhook.

stemlink=linktype

Specify the type of link that should be made during the stem phase. The default is symlink. Other options are hard which creates hard links and copy which creates a copy of the original file. hard links are useful for stemhook commands that cannot process symbolic links. If your stemhook command modifies the log file, you may wish to choose the copy option which will be discarded after the stemhook command is executed.

Filtering

A filter is simply a program that generates something on STDOUT. They may be pipelines or other programs.

filter=filter_command

If the filter process option is specified (via the process directive), you should supply a program to filter your log (via the filter command), such as egrep or perl (or a pipeline or a shell script containing your commands, etc.).

If no filter command is given, the filter phase will be skipped (even if you specify --process=filter as the process directive). Consider following filter command:

    --filter='/usr/bin/egrep -v "/images/" \$LOG'

When savelogs gets to its filter phase, it will open a pipe to the above command. Output from this command will be saved to a temporary file, then the temporary file will be renamed to replace the original log file.

Notice the strange $LOG variable. This is an internal savelogs variable that refers to the location of the log file savelogs is currently working on. It is automatically replaced with the right file during execution.

If you are supplying a filter command on the command-line, the backslash (\) in front of $LOG is necessary to tell the shell to not interpolate <$LOG> as a shell variable, but instead pass it along untouched to savelogs. The backslash is not necessary if you are specifying a filter directive in the configuration file:

    Filter          /usr/bin/egrep -v "/images/" $LOG

Archiving and Compressing

gtar

tar

Specifies the location of the tar binary to use. This defaults to whatever it can find on your system. You usually don't need to modify this option.

Example:

    Gtar                 /usr/local/bin/gtar

If both gtar and tar are specified, gtar will be used.

archive

Specifies the name of the archive to which files will be appended.

This directive is somewhat tricky to understand. Under normal use, savelogs uses the name of the file being archived as the archive name. For example, if you were archiving a file named ~/var/log/messages, the name of the archive would be ~/var/log/messages.tar.

If you are archiving multiple files (which is common), each file will be stored in its own archive by name in the directory where the file is located. For example, if you had several files located in ~/var/log, each file would be stored in its own archive named filename.tar in the ~/var/log directory.

If you want to lump together all files in a particular directory into one archive, use the archive directive without any path information:

    savelogs --archive=system.tar \
             /var/log/messages /var/log/proftpd /var/log/foo \
             /www/logs/access_log /www/logs/error_log

This will archive ~/var/log/messages, ~/var/log/proftpd, and ~/var/log/foo in a single file named ~/var/log/system.tar (which may later be compressed if you've so specified). ~/www/logs/access_log and ~/www/logs/error_log will also be lumped together in a file called ~/www/logs/system.tar.

If you want to lump together all files for this savelogs session into one archive, use the archive directive and specify the full path to the archive:

    savelogs --archive=/var/tmp/logs.tar /var/log/messages \
             --apacheconf=/www/conf/httpd.conf

This will archive ~/var/log/messages and all log files found in the Apache configuration file into a single archive named ~/var/tmp/logs.tar (which may later be compressed).

If you wish to place the archive in your home directory, you may be tempted to just do this:

    savelogs --archive=/logs.tar

This won't work. Because of the way savelogs tries to simplify things relative to your home directory, the leading slash is dropped and savelogs doesn't find any path information (and therefore places the archive in where the files are).

To put files in your home directory, preceed the archive command with a dot-slash:

    savelogs --archive=./logs.tar

This won't put the archive in your current working directory, as some are wont to assume, but in your home directory. savelogs has no notion of a current working directory because it is always changing directories from your home to the directory where the log files are and back.

full-path

Specifies whether files stored in archives are full paths relative to your home directory or relative paths relative to the directory in which the file is found.

If full-path is not specified, paths are stored in the tar file relative to their parent directory.

Example:

    savelogs /var/log/messages

will create an archive with the following file in it:

    messages

while

    savelogs --full-path /var/log/messages

will create an archive with the following file in it:

    var/log/messages

When you extract this file later on, the paths will be created for you if they don't exist, which may not be what you want (but, of course, it may be what you want which is why we have this directive).

gzip

compress

uncompress

Specifies the location of the gzip/compress/uncompress binaries for decompressing files (files ending with '.gz' or '.Z'). Use of compress and uncompress is deprecated. If your system has a gzip binary in a non-standard location, specify its location with this directive. If gzip and compress are specified, gzip will be used.

By default (if none of the above options are specified), savelogs will search for a gzip binary in your path and use it.

clobber

If a compressed archive already exists along side a non-compressed archive (e.g., archive.tar and archive.tar.gz), and you've instructed savelogs to compress archive.tar, some compression programs (like gzip) will ask you for confirmation before overwriting existing files.

To get around this, savelogs by default enables the 'force' option on compression programs (usually -f). This way, if you're running savelogs from a cron job or another method where there is no controlling terminal, savelogs keeps running.

If you're running savelogs interactively (i.e., from a tty) and want savelogs to prompt you to overwrite existing compressed files, specify the noclobber option:

Example:

    savelogs --noclobber

or in your configuration file:

    Clobber             no

EXAMPLES

The author recommends liberal use of the dry-run option when testing these examples or when making big changes to your configuration file or command-line options. Doubly-so when you have the delete process option enabled. There's no 'undelete' for UNIX.

Archiving a single log file

    savelogs /path/to/log_file

By default, savelogs will move, and compress a log file. This is its simplest use. If this command is run daily, the result will be a file name like the old file with a yymmdd extension. This file will be compressed.

Nuking logs

You can use savelogs (contrary to its name) to just wipe out log files and reclaim the disk space. If you've got a couple of files that from time to time just get too big and there's really no valuable information in them, do something like this:

    savelogs --process=delete /path/to/log_file1 /path/to/log_file2

If you want to nuke all Apache log files, do something like this:

    savelogs --process=delete --postmovehook=/usr/local/bin/restart_apache \
    --apacheconf=/www/conf/httpd.conf

When you specify only the delete as a process option, no logs are moved, archived, or compressed. They're just deleted.

Compressing logs daily

Compressing logs daily is easy:

    savelogs /var/log/messages

will make:

    -rw-r--r--  1 test  vuser  751327 Jul  6 12:48 messages

become:

    -rw-r--r--  1 test  vuser   84625 Jul  6 12:48 messages.010706.gz

Compressing logs daily if needed

    savelogs --size=5000 /var/log/messages

will only compress the log if the size of the file is 5000 kilobytes (5 megabytes) or larger.

Using periodic log rotation

You want to save 3 days worth of Apache log files and 5 days worth of system log files. You might try the following lines in your crontab:

    1 0 * * * $HOME/usr/local/bin/savelogs --logfile=/var/log/savelogs \
    --postmovehook=/usr/local/bin/restart_apache --period=3 \
    --apacheconf=/www/conf/httpd.conf

    5 0 * * * $HOME/usr/local/bin/savelogs --logfile=/var/log/savelogs \
    --period=5 /var/log/messages /var/log/ftp.log

Most crontab files require that no lines wrap, so you'd need to make sure to keep everything on one line.

Your ~/www/logs directory may look something like this after a week:

    access_log
    access_log.0.gz
    access_log.1.gz
    access_log.2.gz
    error_log
    error_log.0.gz
    error_log.1.gz
    error_log.2.gz

and your system logs directory:

    messages
    messages.0.gz
    messages.1.gz
    messages.2.gz
    messages.3.gz
    messages.4.gz
    ftp.log
    ftp.log.0.gz
    ftp.log.1.gz
    ftp.log.2.gz
    ftp.log.3.gz
    ftp.log.4.gz

Archiving all logs

Most people want to group their log files in an archive. This makes storing them and retrieving them later for post-processing simple and efficient. Your directory tree might look like this:

    usr/local/etc/httpd/logs
    usr/local/etc/httpd/logs/bar
    usr/local/etc/httpd/logs/bar/access_log
    usr/local/etc/httpd/logs/bar/error_log
    usr/local/etc/httpd/logs/baz
    usr/local/etc/httpd/logs/baz/access_log
    usr/local/etc/httpd/logs/baz/error_log
    usr/local/etc/httpd/logs/biz
    usr/local/etc/httpd/logs/biz/access_log
    usr/local/etc/httpd/logs/biz/error_log
    usr/local/etc/httpd/logs/buz
    usr/local/etc/httpd/logs/buz/access_log
    usr/local/etc/httpd/logs/buz/error_log
    usr/local/etc/httpd/logs/foo
    usr/local/etc/httpd/logs/foo/access_log
    usr/local/etc/httpd/logs/foo/error_log

Issue this command:

    savelogs --process=all --apacheconf=/www/conf/httpd.conf

and your directory tree now looks like this:

    usr/local/etc/httpd/logs
    usr/local/etc/httpd/logs/bar
    usr/local/etc/httpd/logs/bar/access_log.tar.gz
    usr/local/etc/httpd/logs/bar/error_log.tar.gz
    usr/local/etc/httpd/logs/baz
    usr/local/etc/httpd/logs/baz/access_log.tar.gz
    usr/local/etc/httpd/logs/baz/error_log.tar.gz
    usr/local/etc/httpd/logs/biz
    usr/local/etc/httpd/logs/biz/access_log.tar.gz
    usr/local/etc/httpd/logs/biz/error_log.tar.gz
    usr/local/etc/httpd/logs/buz
    usr/local/etc/httpd/logs/buz/access_log.tar.gz
    usr/local/etc/httpd/logs/buz/error_log.tar.gz
    usr/local/etc/httpd/logs/foo
    usr/local/etc/httpd/logs/foo/access_log.tar.gz
    usr/local/etc/httpd/logs/foo/error_log.tar.gz

Inside of each compressed archive is a single file:

    access_log.010523

which is the old log renamed with today's date. When you run this command again (e.g., from a cron job) tomorrow, you'll see the same list of files above, except that inside each compressed archive is an additional file:

    access_log.010523
    access_log.010524

Archiving all logs per directory

Say you want to group together all logs in a single directory in one archive. Your directory tree might look like this:

    usr/local/etc/httpd/logs
    usr/local/etc/httpd/logs/bar
    usr/local/etc/httpd/logs/bar/access_log
    usr/local/etc/httpd/logs/bar/error_log
    usr/local/etc/httpd/logs/baz
    usr/local/etc/httpd/logs/baz/access_log
    usr/local/etc/httpd/logs/baz/error_log
    usr/local/etc/httpd/logs/biz
    usr/local/etc/httpd/logs/biz/access_log
    usr/local/etc/httpd/logs/biz/error_log
    usr/local/etc/httpd/logs/buz
    usr/local/etc/httpd/logs/buz/access_log
    usr/local/etc/httpd/logs/buz/error_log
    usr/local/etc/httpd/logs/foo
    usr/local/etc/httpd/logs/foo/access_log
    usr/local/etc/httpd/logs/foo/error_log

Try this:

    savelogs --process=all --archive=logs.tar --apacheconf=/www/conf/httpd.conf

and your directory tree now looks like this:

    usr/local/etc/httpd/logs
    usr/local/etc/httpd/logs/bar
    usr/local/etc/httpd/logs/bar/logs.tar.gz
    usr/local/etc/httpd/logs/baz
    usr/local/etc/httpd/logs/baz/logs.tar.gz
    usr/local/etc/httpd/logs/biz
    usr/local/etc/httpd/logs/biz/logs.tar.gz
    usr/local/etc/httpd/logs/buz
    usr/local/etc/httpd/logs/buz/logs.tar.gz
    usr/local/etc/httpd/logs/foo
    usr/local/etc/httpd/logs/foo/logs.tar.gz

Inside each 'logs.tar.gz' are two files:

    access_log.010523
    error_log.010523

Archiving all logs per session

Say you want to lump all logs in a savelogs session into a single archive. Specify the archive option with a full path, like this:

    savelogs --process=all --archive=/tmp/all_logs.tar \
             --apacheconf=/www/conf/httpd.conf /var/log/messages

This will create a single file /tmp/all_logs.tar.gz that contains all logs found in httpd.conf as well as /var/log/messages.

BUGS

savelogs will not properly interpolate backticks inside the configuration file. Because backticks are shell operators, they must be visible to the shell (meaning they have to be done on the command-line). An example is the ext command where the command-line option might look like this:
```
    savelogs --ext=`smalldate -m`
```
there is no corresponding configuration file entry.
You must obey shell escaping rules when you're doing filter commands. Sometimes these can be tricky. For example, double quotes will do different things than single quotes (I recommend single quotes because you can then use the $LOG variable without big headaches).

AUTHOR

Scott Wiersdorf <scott@perlcode.org>

COPYRIGHT

1 POD Error

The following errors were encountered while parsing the POD:

Around line 2063:: =back doesn't take any parameters, but you said =back 4

To install savelogs, copy and paste the appropriate command in to your terminal.

cpanm

cpanm savelogs

CPAN shell

perl -MCPAN -e shell
install savelogs

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

Savelogs Phases

An Overview

A Word About Paths

CONFIGURATION

Configuration file option format

Option processing order

Available options

Case-sensitivity

Testing configurations

OPTIONS

Running Savelogs

Savelogs Logging

How Savelogs Finds Logs to Process

Moving (Renaming) Files

Filtering

Archiving and Compressing

EXAMPLES

Archiving a single log file

Nuking logs

Compressing logs daily

Compressing logs daily if needed

Using periodic log rotation

Archiving all logs

Archiving all logs per directory

Archiving all logs per session

BUGS

AUTHOR

COPYRIGHT

Module Install Instructions