Lorenzo Taviani

# NAME

Win32::Backup::Robocopy - a simple backup solution using robocopy

# SYNOPSIS

    use Win32::Backup::Robocopy;

# RUN mode
my $bkp = Win32::Backup::Robocopy->new( name => 'my_perl_archive', # mandatory source => 'c:\scripts', # mandatory destination => 'x:\backup', # '.' if not specified history => 1, ); my($stdout, $stderr,$exit, $exitstr,$createdfolder ) = $bkp->run(); # JOB mode my$bkp = Win32::Backup::Robocopy->new( configuration => './backup_conf.json' );
$bkp->job( name => 'my_backup_name', # mandatory src =>'./a_folder', # mandatory dst => 'y:/', # '.' if not specified history => 1, cron => '0 0 25 12 *', # mandatory first_time_run => 1, );$bkp->runjobs;     

# DESCRIPTION

This module is a wrapper around robocopy.exe and try to make it's behaviour as simple as possible using a serie of sane defaults while letting you the possibility to leverage the robocopy.exe invocation in your own way.

The module offers two modes of being used: the RUN mode and the JOB mode. In the RUN mode a backup object created via new is a_folder single backup intended to be run using the run method. In the JOB mode the object is a container of scheduled jobs filled reading a JSON configuration file and/or using the job method. runjobs is then used to cycle the job list and see if some job has to be run.

In the RUN mode, if not history is specified as true, the backup object (using the run method) will copy all files to one folder, named as the name of the backup (the mandatory name parameter used while creating the object). All successive invocation of the backup will write into the same destination folder.

    # RUN mode with all files to the same folder
use Win32::Backup::Robocopy;

my $bkp = Win32::Backup::Robocopy->new( name => 'my_perl_archive', # mandatory source => 'x:\scripts' # mandatory ); my($stdout, $stderr,$exit, $exitstr ) =$bkp->run();


If you instead specify the history parameter as true during construction, then inside the main destination folder ( always named using the name ) there will be one folder for each run of the backup named using a timestamp like 2022-04-12T09-02-36

    # RUN mode with history folders in destination
my $bkp = Win32::Backup::Robocopy->new( name => 'my_perl_archive', # mandatory source => 'x:\scripts', # mandatory history => 1 # optional ); my($stdout, $stderr,$exit, $exitstr,$createdfolder ) = $bkp->run(); The second mode is the JOB one. In this mode you must only specify a config parameter during the object instantiation. You can add different jobs to the queue or load them from a configuration file. Configuration file is read and written in JSON. Then you just call runjobs method to process them all. The JOB mode adds the possibility of scheduling jobs using crontab like strings (using Algorithm::Cron under the hoods).  # JOB mode - loading jobs from configuration file my$bkp = Win32::Backup::Robocopy->new( configuration => './my_conf.json' ); # mandatory configuration file

$bkp->runjobs;  You can add jobs to the queue using the job method. This method will accepts all parameters and assumes all defaults of the new method in the RUN mode and of the run method of the RUN mode. In addition the job method wants a crontab like entry to have the job run only when needed. You can also specify first_time_run to 1 to have the job run a first time without checking the cron scheduling, ie at the first invocation of runjobs  # JOB mode - adding jobs my$bkp = Win32::Backup::Robocopy->new( configuration => './my_conf.json' ); # mandatory configuration file

$bkp->job( name=>'my_backup_name', # mandatory as per new src=>'./a_folder', # mandatory as per new history=>1, # optional as per new cron=>'0 0 25 12 *', # job specific, mandatory first_time_run=>1 # job specific, optional ); # add more jobs..$bkp->runjobs;



## IMPORTANT: used executable and robocopy.exe used defaults

This module needs a valid copy of robocopy.exe to be present in the system and to be available in the PATH

Alternatively a full path of an alternate copy of the robocopy.exe executable can be specified using the ENV variable PERL_ROBOCOPY_EXE and in this case it will be given precedence over the copy present in the system.

Unfortunately robocopy.exe was distributed over the years in many different versions and with doubious version numbers. Notably version 5.1.2600.26 named XP026 is bugged: it returns a success errorlevel even when it fails.

Because of the above the current module will try to spot the position and the version of robocopy.exe and the build of the module will fail if no version are found or a bugged version is the only available.

The robocopy.exe program is full of options. This module is aimed to facilitate the backup task and so it assumes some defaults. Every call to robocopy.exe made by run and runjobs if nothing is specified will result in:

    robocopy.exe SOURCE DESTINATION *.* /S /E /M /R:0 /W:0 /NP /256



Apart from source and destination, first six parameters can be modified during the run call (see below the method description for details). Last two switches will be present anyway: /NP eliminates the progress bar that can show the copied percentage and that it is not useful as the module will collect all the output from the command.

More important is the /256 switch that disable the discutible feature permitting robocopy to create folders with more than 256 characters in the path (the OS has a treshold of 260). Without this switch, an eventual erroneous invocation can lead to a folder structure very difficult to remove because the explorer subsystem is not even able to remove nor rename it.

Even specialized tools can fail ( booting Linux live distro and good old rm -rf can help though ;). Even if other checks in the module are to prevent these bad results the switch will be always present.

By other hand, if nothing is specified, every call of the restore method will result in:

    robocopy.exe SOURCE DESTINATION *.* /S /E /R:0 /W:0 /NP /256


with the only but important difference in respect to archive bit that are not looked for nor reset ( no /M switch passed ).

Please note that robocopy.exe will use by default /COPY:DAT ie will copy data, attributes and timestamp.

Verbosity of the module can vary from 0 (default value, no outptut at all) to 2 giving lot of informations and dumping jobs and configuration. The verbose parameter can be set in the main backup object during the construction made by new and in this case will be inherited by all other methods. But run job runjobs and restore methods can be feed too with a verbose parameter that will be in use only during the call.

# METHODS (RUN mode)

## new

As already stated new only needs two mandatory parameters: name ( the name of the backup governing the destination folder name too) and source ( you can use also the abbreviated src form ) that specify what you intend to backup. The new method will emit a warning if the source for the backup does not exists but do not exit the program: this can be useful to spot a typo leaving to you if that is the right thing (maybe you want to backup a remote folder not available at the moment).

If you do not specify a destination ( or the abbreviated form dst ) you'll have backup folders created inside the current directory, ie the module assumes destination to be '.' unless specified. During the object construction destination will be crafted using the provided path and the name you used for the backup.

If your current running program is in the c:/scripts directory the following invocation

    my $bkp = Win32::Backup::Robocopy->new( name => 'my_perl_archive', source => 'x:\perl_stuff', ); will produces a destination equal to c:/scripts/my_perl_archive and here will be backed up your files. By other hand:  my$bkp = Win32::Backup::Robocopy->new(
name        => 'my_perl_archive',
source      => 'x:\scripts',
destination => 'Z:\backups'
);

will produces a destination equal to Z:/backups/my_perl_archive

All paths and filenames passed in during costruction will be checked to be absolute and if needed made absolute using File::Spec so you can be quite sure the rigth thing will be done with relative paths.

The new method does not do any check against destination folders existence; it merely prepare folder names to be used by run

The module provides a mechanism to spot unavailable destination drive and ask the user to connect it. If you specify waitdrive => 1 during the object construction then the program will not die when the drive specified as destination is not present. Instead it opens a prompt asking the user to connect the appropriate drive to continue. The deafult value of waitdrive is 0 ie. the program will die if the drive is unavailable and creation of the destination folder impossible.

To wait for the drive is useful in case of backups with destination, let's say, an USB drive: see the "backup to external drive" example.

Overview of parameters accepted by new and their defaults:

• name mandatory. Will be used to create the destination folder appended to dest

• source or src mandatory.

• destination or dst defaults to './'

• history defaults to 0 meaning all invocation of the backup will write to the same folder or folder with timestamp if 1

• waitdrive defaults to 0 stopping the program if destination drive does not exists, asking the user if 1

• verbose defaults to 0 governs the amount of output emitted by the program

## run

This method will effectively run the backup. It checks needed folder for existence and try to create them using File::Path and will croak if error are encountered. If run is invoked without any optional parameter run will assume some default options to pass to the robocopy system call:

• files defaults to *.* robocopy will assume all file unless specified: the module passes it explicitly (see below)

• archive defaults to 0 and will set the /A if 1 ( copy only files with the archive attribute set ) robocopy switch

• archiveremove defaults to 1 and will set the /M ( like /A, but remove archive attribute from source files ) robocopy switch

• subfolders defaults to 1 and will set the /S if 1 ( copy subfolders ) robocopy switch

• emptysubfolders defaults to 1 and will set the /E ( copy subfolders, including empty subfolders ) robocopy switch

• retries defaults to 0 and will set the /R:0 or N if specified (number of retries on error on file) robocopy switch

• wait defaults to 0 and will set the /W:0 or N if specified (seconds between retries) robocopy switch

• extraparam defaults to undef and can be used to pass any valid option to robocopy (see below)

So if you dont want empty subfolders to be backed up you can run:

        $bkp->run( emptysufolders => 0 )  Pay attention modifying archive and archiveremove parameters: infact this is the basic machanism of the backup: on MSWin32 OSs whenever a file is created or modified the archive bit is set. This module with it's defualts values of archive and archiveremove will backup only new or modified files and will unset the archive bit in the original file. The run method effectively executes the robocopy.exe system call using Capture::Tiny capture method. The run method returns four elements: 1) the output emitted by the system call, 2) the error stream eventually produced, 3) the exit code of the call ( first three elements provided by Capture::Tiny ) and 4) the text relative to the exit code. A fifth returned value will be present if the backup has history => 1 and it's value will be the name of the folder with timestamp just created.  my($stdout, $stderr,$exit, $exitstr ) =$bkp->run();

# or in case of history backup:
# my( $stdout,$stderr, $exit,$exitstr, $createdfolder ) =$bkp->run();

# an exit code of 7 or less is a success
if ( $exit < 8 ){ print "backup successful:$exitstr\n";
}
else{ print "some problem occurred\n",
"OUTPUT: $stdout\n", "ERROR:$stderr\n",
"EXIT: $exit\n", "EXITSTRING:$existr\n";
}


Read about robocopy.exe exit codes here

robocopy.exe accepts, after source and destination, a third parameter in the form of a list of files or wildcard. robocopy.exe assumes this to be *.* unless specified but the present module passes it always explicitly to let you to modify it at your will. To backup just *.pl files invoke run as follow:

    $bkp->run( files => '*.pl');  You can read more about Windows wildcards here robocopy.exe accepts a lot of parameters and the present module just plays around a handfull of them, but you can pass any desired parameter using extraparam so if you need to have all destination files to be readonly you can profit the /A+:[RASHCNET] robocopy option: $bkp->run( extraparam => '/A+:R');

extraparam accepts both a string or an array reference.

Read about all parameters accepted by robocopy.exe here

# METHODS (JOB mode)

## new

The only mandatory parameter needed by new is conf (or config or configuration) while in JOB mode. The value passed will be transformed into an absolute path and if the file exists and is readable and it contains a valid JSON datastructure, the configuration is loaded and the job queue filled accordingly.

If, by other hand, the file does not exists, new does not complain, assuming the queue of jobs to be filled soon using the job method described below.

    $bkp->job( name => 'documents', src => 'e:\me\docs', dst => 'x:\my_backups' cron => '0 0 25 12 *', ); This method will push job in the queue. It accepts all parameters of the new and the run methods described in RUN mode above. Infact a job, when run, will instantiate a new backup object and will run it via the run method. In addition it must be feed with a valid crontab like string via the cron parameter with a value something like, for example, '15 14 1 * *' to setup the schedule for this job to the first day of the month at 14:15 You can specify the optional parameter first_time_run => 1 to have the job scheduled as soon as possible. Then, after the first time the job will run following the schedule given by the cron parameter. Everytime a job is added, the configuration file will be updated accordingly. If the verbose option is passed in during the job call (or if it is inherited by the main backup object) informations are displayed. With verbose set to 2 each job added is dumped and the resulting configuration will be also printed. ## runjobs This is the method used to cycle the job queue to see if something has to be run. If so the job is run and the configuration file is immediately updated with the correct time for the next execution. The runjobs method without any parameter will check all jobs in order to see if is time to run them. Optionally you can pass to it a range or a string representing a range to just process selected jobs: $bkp->runjobs(0..1,5);
# or the same in the string form
$bkp->runjobs('0..1,5'); ## listjobs With listjobs you can list all jobs currently present in the configuration. In scalar context it just returns the number of jobs while in list context it returns the list of jobs. In the list form you have the possibility to define the format used to represent the job with the format parameter: if it is short (and is the default value) each job will be represented on his own line. By other hand with format => 'long' a more fancy multiline string will be printed for each job. You can also specify a list of fields you want to show instead to have them all present, passing an array reference as value of the fields parameter.  # sclar context my$jobcount = $bkp->listjobs; print "there are$jobcont jobs configured";

# list context: all fields returned in compact mode
print "$_\n" for$bkp->listjobs;

# output:
name = job1 src = x:\path1 dst = F:\bkp\job1 files =  ...(all other fields and values following)
name = job2 src = y:\path2 dst = F:\bkp\job2 files =  ...

# list context: some field returned in compact mode
print "$_\n" for$bkp->listjobs(fields => [qw(name src next_time_descr)]);

# output:
name = job1 src = x:\path1 next_time_descr = Tue Jan  1 00:05:00 2019
name = job2 src = y:\path2 next_time_descr = Mon Apr  1 00:03:00 2019

# list context, long format just some field
print "$_\n" for$bkp->listjobs( format=>'long', fields => [qw(name src next_time_descr)]);

# output:
JOB 0:
name = job1
src = x:\path1
next_time_descr = Tue Jan  1 00:05:00 2019

JOB 1:
name = job2
src = x:\path2
next_time_descr = Mon Apr  1 00:03:00 2019



# RESTORE

## restore

The module provides a method to restore from a backup to a given destination. It is just a copy of all files and directories found in a given source directory, to a given destination (that will be created if it does not already exists).

This method just needs two parameters: from and to like in:

    $bkp->restore( from => 'X:/external/scripts_bkp' , to => 'D:/local/scripts' ); The restore method will accept all parameter concerning robocopy options as the run method does, with the only important difference about archive bit: the default is to ignore it. • files defaults to *.* robocopy will assume all file unless specified: the module passes it explicitly (see below) • archive defaults to 0 and will set the /A if 1 ( copy only files with the archive attribute set ) robocopy switch • archiveremove defaults to 0 (the only difference in respect of the run method) and will set the /M ( like /A, but remove archive attribute ) robocopy switch • subfolders defaults to 1 and will set the /S if 1 ( copy subfolders ) robocopy switch • emptysubfolders defaults to 1 and will set the /E ( copy subfolders, including empty subfolders ) robocopy switch • retries defaults to 0 and will set the /R:0 or N if specified (number of retries on error on file) robocopy switch • wait defaults to 0 and will set the /W:0 or N if specified (seconds between retries) robocopy switch • extraparam defaults to undef and can be used to pass any valid option to robocopy (see run method) ## history restore When each folder contained in the given source to restore has a name as given by a history backup, eg. like 2022-04-12T09-02-36 and the folder used as source to restore contains only folders and no other object at all, then, if these conditions are met, each folder will be used as source starting from the older one to the newer one. This behaviour permits a restore to a point in time using the upto parameter in the restore call. Let's say you have backed up some folder with an history backup and now you have the following folders:  2019-01-04T20-29-10 2019-01-05T20-29-10 2019-01-06T20-29-10 2019-01-07T20-29-10  all contained in X:\external\photos and you discover that the day 7 of January at 14:00 all your pictures got corrupted ( so the last backup contains a lot of invalid files ) you can restore only pictures up to the January 6 using: $bkp->restore(
from => 'X:\external\photos',
to => 'C:\PICS\restore_up_to_january_6',
upto => '2019-01-06T20-29-10',
);


and you'll have restored only the photos backed up in the firsts three folder and not in the fourth one.

The upto parameter can be: 1) a string as used to create folders by history backups, like in the above example 2019-01-06T20-29-10 or 2) a string as created by DateTime::Tiny as_string method (ISO 8601) ie 2019-01-06T20:29:10 or 3) seconds since epoch like 1546806550 or 4) a DateTime::Tiny object or 5) a DateTime object.

Pay attention to what is said in the DateTime::Tiny documentation about time zones and locale: in other words the conversion will be using gmtime and not localtime see the following example to demonstrate it:

    use DateTime::Tiny;
my $epoch = DateTime::Tiny->from_string('2019-01-06T20:29:10')->DateTime->epoch; say 'epoch: ',$epoch;
say 'localtime: ',scalar localtime($epoch); say 'gmtime: ',scalar gmtime($epoch);

# output

epoch:     1546806550
localtime: Sun Jan  6 21:29:10 2019
gmtime:    Sun Jan  6 20:29:10 2019

The restore method will execute a robocopy.exe call with defaut arguments '*.*', '/S', '/E', '/NP' '/256' but you can pass other ones using the extraparam parameter being it a string or an array reference with a list of valid robocopy.exe parameters.

Both history and normal restore can output more informations if verbose is set in the main backup object or if it is passed in directly during the restore method call.

## returned value

The return value of a restore call will be an anonymous array with an element for each operation done by the method. If it was a simple restore the array will hold just one element but if it was a history restore each operation (using a different folder as source) will push an element in the array. These array elements are anonymoous hashes with four keys: stdout, stderr, exit and exitstring of each operation respectively.

# CONFIGURATION FILE

While in JOB mode if the configutaion file passed during object contruction contains valid data, such data will be imported into the main ojbect. Each new job added using the job method will be added too and the configuration will be wrote accordingly. This will speed up the backup setup but can also lead in duplicate jobs: see "on demand backup in job mode" example to see how deal with this.

The configuration file holds JSON data into an array each element of the array being a job, contained in a hash. Writing to the configuration file done by the present module will maintain the job hash ordered using JSON::PP

    my $bkp = Win32::Backup::Robocopy->new(conf=>'bkpconfig.json');$bkp->job( src => '.', dst => 'x:/dest', name => 'first', cron => '* 4 * * *' );



Will produce the following configuration:

  [
{
"name" : "first",
"src" : "D:\\path\\stuff_to_backup",
"dst" : "X:\\dest\\first",
"files" : "*.*",
"history" : 0,
"cron" : "* 4 * * *",
"next_time" : 1543546800,
"next_time_descr" : "Fri Nov 30 04:00:00 2018",
"first_time_run" : 0,
"archive" : 0,
"archiveremove" : 1,
"subfolders" : 1,
"emptysubfolders" : 1,
"retries" : 0,
"wait" : 0,
"waitdrive" : 0,
"verbose" : 0
}
]

you can freely add and modify by hand the configuration file, paying attention to the next_time and next_time_descr entries that are respectively seconds since epoch for the next scheduled run and the human readable form of the previous entry. Note that next_time_descr is just a label and does not affect the effective running time.

# EXAMPLES

## a simple case

You can use an on the fly backup, for example, if you load a configurtion file and you modify it but you are not sure the whole process will be successful:

    use strict;
use warnings;
use Win32::Backup::Robocopy;

# the following will backup into x:\conf_bkp
# ie the destination plus the name of the backup
my $bkp = Win32::Backup::Robocopy->new( name => 'conf_bkp', source => 'c:\path\to\conf', destination => 'x:\', ); my($stdout, $stderr,$exit, $exitstr ) =$bkp->run( archiveremove => 0 );

if ( $exit < 8 ){ print "backup of configuration OK:$exitstr\n";
}

# something goes wrong: need to restore the original configuration

print "starting restore:\n";

$bkp->restore( from => 'x:\conf_bkp', # the name of backup appended to => 'c:\path\to\conf', # to the backup destination verbose => 2, ); In the above example we pass to restore verbose with the value of 2 to have printed out many details of the restore operation. Pay attention to the run call: we used archiveremove => 0 and it also use the default archive => 0 and this will means that we are not looking at all to the archive bit of the file, nor we remove it: this setting will always copy the file. Defaults are set to backup only modified and new files. ## maintain more copies If you instead have a program running monthly, which modify a configuration file you can use the history backup type to have more copies of the same file, one for each run of your monthly task. Now we use verbose with value of 1 inside the run method call:  my$bkp = Win32::Backup::Robocopy->new(
name        => 'conf_bkp',
source      => 'c:\path\to\conf',
destination => 'x:/',
history     => 1,
);

my( $stdout,$stderr, $exit,$exitstr ) = $bkp->run( verbose => 1); if ($exit < 8 ){
print "\nbackup of configuration OK\n";
}

    backup SRC: [c:\path\to\conf]
backup DST: [x:\conf_bkp\2019-01-11T23-11-09]
mkdir x:\conf_bkp\2019-01-11T23-11-09
executing [robocopy.exe c:\path\to\conf x:\conf_bkp\2019-01-11T23-11-09 *.* /S /E /M /R:0 /W:0 /256 /NP]
robocopy.exe exit description:  One or more files were copied successfully (that is, new files have arrived).

backup of configuration OK

If your program run monthly you'll have under conf_bkp the following folders:

    2019-01-11T23-11-09
2019-02-11T23-11-09
2019-03-11T23-11-09
2019-04-11T23-11-09
..



## backup to external drive

The waitdrive option is useful when dealing with network shares or external drives. Infact the module will check if the drive is not present and will ask to connect before proceding:

    my $bkp=Win32::Backup::Robocopy->new( name => 'test', src => '.', dst => 'H:/bkp', # drive H: is unplagged waitdrive => 1 # force asking the user );$bkp->run();

# output:

Backup of:     D:\my_current\dir
To:            H:\bkp\test
Waiting for drive H: to be available..
(press ENTER when H: is connected or CTRL-C to terminate the program)

# I press enter before plugging the drive..

Backup of:     D:\my_current\dir
To:            H:\bkp\test
Waiting for drive H: to be available..
(press ENTER when H: is connected or CTRL-C to terminate the program)

# I plug the external hard disk that receive the H: letter, then  I press ENTER
# the backup runs OK

With waitdrive set to 0 instead the above program dies complaining about directory creation errors and destination drive H: is not accessible!

## on demand backup in job mode

The JOB mode is mainly intended to implement scheduled backups using the cron like mechanism. But, if it is the need, you can have backups run only on demand using a cron string of five asterisks * * * * * and using listjobs and runjobs to ask the user if they want to run the backup:

    use Win32::Backup::Robocopy;
my $bkp = Win32::Backup::Robocopy->new( config => './my_bkp.json' ); # configuration made in previous runs is already populated # so check if we need to specify jobs if ( 0 ==$bkp->listjobs ){

# a first job for 'documents'
$bkp->job( name=>'documents', src=> 'c:\DOCS', dst => "x:\\", cron =>'* * * * *', first_time_run => 1, # or during the current minute will be skipped ); # a second job for 'scripts'$bkp->job(
name=>'SCRIPTS',
src=> 'c:\perl\scripts',
dst  => 'x:\\',
cron=>'* * * * *',
first_time_run => 1, # see above: cron works on minutes!
);
}
else{ print scalar $bkp->listjobs, " jobs retrieved from configuration file..\n"} # iterate over jobs my$job_num = 0;
foreach my $descr($bkp->listjobs( format=>'long', fields => [qw(name src dst)]) ){

print $descr; print "do you want to execute JOB$job_num? [y|n]\n\n";
my $input = <STDIN>; if ($input =~/^y/i ){
$bkp->runjobs($job_num );
}
\$job_num++;
}

# AUTHOR

LorenzoTa, <lorenzo at cpan.org>

# BUGS

Please report any bugs or feature requests to bug-win32-backup-robocopy at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Win32-Backup-Robocopy. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

# SUPPORT

Main support site for this module is perlmonks.org You can find documentation for this module with the perldoc command.

    perldoc Win32::Backup::Robocopy

You can also look for information at:

# ACKNOWLEDGEMENTS

This software, as all my works, would be impossible without the continuous support and incitement of the perlmonks.org community

This program is free software; you can redistribute it and/or modify it under the terms of the the Artistic License (2.0). You may obtain a copy of the full license at: