NAME

Net::Appliance::Session::Cookbook::Recipe06 - Storing Device Configurations in Files

NOTE

This Cookbook was contributed to the Net::Appliance::Session project by Nigel Bowden. Source code from the Cookbook is shipped in the examples folder of this module's distribution.

PROBLEM

You want to telnet or SSH to all of your Cisco IOS devices and pull off their configurations so that you can build up a historic record of configuration files across your network.

SOLUTION

Well, if this is a cookbook, then this particular section is a 6 course banquet. This is a huge script compared to the previous examples we have looked at.

It brings together all of the areas we have looked at previously, but I have pulled in the services of a number of other Perl modules to try to provide a usable utility that could perhaps be used on your network to gather your Cisco device configurations.

All of the credentials that are required to access your devices are now stored in a CSV file, so devices can be added or removed as required. As different devices may have differing access requirements (e.g. transports, username/passwords required etc.) each device is configured with its own set of access credentials. CSV files (comma seperated values) are easy to update using many spreadsheet applications, making it easy to maintain the device credential data for this script.

Also, the script uses an external intialization file (.ini) to configure the operation of the script (e.g. where configuration files will be dumped), rather than having to hard-code values in the body of the script itself.

The script comprises two files:

Recipe_06.pl: The file below, in this Cookbook.
setup.ini: The initialization file that must be in the same directory as Recipe_06.pl.
device_credentials.csv: The CSV file whose location is configured in setup.ini.

The main body of the script should be relatively easy to follow, as it is very similar to previous examples. Where new functions are introduced (e.g. reading and parsing a CSV file), then those functions have (in the main) been separated off in to additional subroutines.

Here is the full solution:

 use strict;
 use warnings;
 use Carp;
 
 use Net::Appliance::Session;
 use Config::INI::Simple;
 use Text::CSV_XS;
 use File::Basename;
 
 # create Config::INI::Simple object and read in our ini file data
 my $ini_filename = dirname($0) . "/setup.ini"; # where we can find ini file
 my %ini_data = parse_ini_file($ini_filename);
 
 # get the device credentials for our devices
 my @device_data = parse_data_file($ini_data{device_csv_file});
 
 # step through each device and try to get and store our configs 
 DEVICE:
 for my $device_ref (@device_data) {
     
     my $device_name = $device_ref->{device_name} || $device_ref->{device_ip};
     
     # set up some logging
     my $debug_log = "$ini_data{debug_dir}/$device_name.debug.log";
     my $error_log = "$ini_data{error_dir}/$device_name.error.log";
     
     # create our config file name
     my $file_timestamp = file_timestamp();
     my $running_config_file = "$ini_data{repository_dir}/$device_name.$file_timestamp.conf";
     
     # create our Net::Appliance::Session with the transport for this device
     my $session_obj = Net::Appliance::Session->new(
         Host      => $device_ref->{device_ip},
         Transport => $device_ref->{transport},
     );
     
     # send the debug for this session to a device-specific file
     $session_obj->input_log($debug_log);
     my @running_config;
     
     # generate the required fields for the priv_array subroutine
     my @priv_array = priv_array($device_ref);
     
     # tell our session object we don't need enable password if none supplied
     unless ($priv_array[0]) {
         $session_obj->do_privileged_mode(0);
     }
     
     # do our interactive (Telnet/SSH) stuff...
     eval {
     
         # try to login to the ios device, ignoring host check
         $session_obj->connect( connect_hash($device_ref), SHKC => 0 );
           
         if ( $priv_array[0] ) {
     
             # if we need to use some enable credentials, supply them
             $session_obj->begin_privileged( @priv_array );
         }
         
         # get our running config
         @running_config =  $session_obj->cmd('show running');
     };
     
     # did we get an error ?
     if ($@) {
         
         # log error to file and move on to next device
         log_error( error_report($@, $device_name), $error_log );
         next DEVICE;
     }
     
     # chop out the extra info top and bottom of the config
     @running_config = @running_config[ 2 .. (@running_config -1)];
     
     # dump the config to a file
     open(CONFIG , " > $running_config_file")
        or warn("Unable to open config file for : $device_name : $!");
     print CONFIG @running_config;
     close CONFIG;
     
     # close down our session
     $session_obj->close;
 }   
 
 #####################################
 # Subroutines
 #####################################
 sub parse_ini_file {
 
     # parse our ini file to get the parameters we need in to
     # some convenient variables
     
     my $ini_filename = shift or croak("No ini file name passed");
     
     my $config_obj = Config::INI::Simple->new();
 
     $config_obj->read($ini_filename) or die("Cannot open ini file : $ini_filename (reason: $!)");
     
     my %ini_data; # variable to use as data hash to hold all ini file data
     
     # set up some variables for later use
     $ini_data{error_dir}        = $config_obj->{Logs}->{error_dir};
     $ini_data{debug_dir}        = $config_obj->{Logs}->{debug_dir};
     $ini_data{device_csv_file}  = $config_obj->{CSV}->{device_csv_file};
     $ini_data{repository_dir}   = $config_obj->{Repository}->{repository_dir};
     $ini_data{timestamp_format} = $config_obj->{Repository}->{timestamp_format};
 
     return %ini_data;
 }
 
 sub parse_data_file {
     
     # parse the CSV data file we are using to hold our  device
     # credential data
     
     my $device_csv_file = shift or croak("No csv file named passed");
 
     # create our csv object ready to parse in the data from our csv file
     my $csv_obj = Text::CSV_XS->new();
     
     #read in our csv file
     open my $csv_fh, "< $device_csv_file"
        or croak("Cannot open device csv file : $device_csv_file (reason: $!)");
     
     # take off the top row that has the field names
     my $top_row = $csv_obj->getline($csv_fh); 
     
     my @device_data;
     
     # take each entry in the CSV file and massage it into a complex
     # data structure
     while (my $data_row = $csv_obj->getline($csv_fh)) {
         my $hash_ref;
         map { ($hash_ref->{$_} = shift @$data_row) } @$top_row;
         
         push(@device_data, $hash_ref);
     }
     
     close $csv_fh;
     return @device_data;
 }
 
 sub connect_hash {
     
     # depending on the combination of credentials supplied, determine
     # the combination of login username/password to use
     
     my $device_ref = shift or croak("No device credentials ref passed !");
 
     # decide which set of credentials we have
     if ( exists($device_ref->{username}) && exists($device_ref->{password}) ) {
         
         # username & password supplied
         return ( Name => $device_ref->{username}, Password => $device_ref->{password} );
     }
     elsif ( exists($device_ref->{password}) ) {
         
         # password only supplied
         return ( Password => $device_ref->{password} );
     }
     else {
         croak("Invalid or missing credentials to log in to this device : "
                    . $device_ref->{device_name} );
     }
 }
 
 sub priv_array {
     
     # depending on the combination of credentials supplied, determine
     # the combination of enable username/password to use
     
     my $device_ref = shift or croak("No device credentials ref passed !");
 
     # decide which set of priv credentials we have
     if ( $device_ref->{enable_username} && $device_ref->{enable_password} ) {
         
         # username & password supplied
         return( $device_ref->{enable_username}, $device_ref->{enable_password} );
     }
     elsif ( $device_ref->{enable_password} ) {
         
         # password only supplied
         return( $device_ref->{enable_password} );
     }    
     elsif ( $device_ref->{enable_username} ) {
         
         # username only supplied - error !
         croak( "Invalid enable login credentials provided (only username provided !" );
     }    
     else {
         
         # no enble pwd required (assume drop straight in to enable mode)
         return 0;
     }
 }
 
 sub error_report {
     
     # standard subroutine used to extract failure info when
     # interactive session fails
     
     my $err         = shift or croak("No err !");
     my $device_name = shift or croak("No device name !");
     
     my $report; # holder for report message to return to caller
     
     if ( UNIVERSAL::isa($err, 'Net::Appliance::Session::Exception') ) {
             
             # fault description from Net::Appliance::Session
             $report  =  "We had an error during our Telnet/SSH session to device  : $device_name \n"; 
             $report .= $err->message . " \n";
                 
             # message from Net::Telnet
             $report .= "Net::Telnet message : " . $err->errmsg . "\n"; 
                 
             # last line of output from your appliance  
             $report .=  "Last line of output from device : " . $err->lastline . "\n\n";
 
         }
         elsif (UNIVERSAL::isa($err, 'Net::Appliance::Session::Error') ) {
             
             # fault description from Net::Appliance::Session
             $report  = "We had an issue during program execution to device : $device_name \n";
             $report .=  $err->message . " \n";
     
         }
         else {
             
             # we had some other error that wasn't a deliberately created exception
             $report  = "We had an issue when accessing the device : $device_name \n";
             $report .= "The reported error was : $err \n";
         }
         
         return $report;
 }
 
 sub log_error {
     
     # log an error message to a file
     
     my $error_message = shift;
     my $file_name     = shift;
     
     open(ERR , " > $file_name") or carp("Unable to error file : $file_name : $!");
     print ERR $error_message;
     close ERR;
 }
 
 sub file_timestamp {
 
     # create a timestamp to add to the conf files created
 
     my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
     
     if ($ini_data{timestamp_format} eq 'uk') {
     
         # UK format
         return sprintf( "%02d-%02d-%4d-%02d%02d", $mday, ($mon + 1), ($year + 1900), $hour, $min );
     }
     else {
         
         # US format
         return sprintf( "%02d-%02d-%4d-%02d%02d", ($mon + 1), $mday, ($year + 1900), $hour, $min );   
     }
 }

DISCUSSION

As discussed above, the main body of this script pulls together most of the principles that are discussed in previous examples. So, I'm not going to discuss the whole script in detail (my fingers can't take the typing!). Instead I'll hightlight the major differences to previous examples and focus more on the features provided by the various subroutines that have been added.

One thing to mention before we go on are the couple of additional modules that we've pulled in to the script to provide us with some valuable features for our solution:

Config::INI::Simple: Allows us to read in a standard Windows-format ini file in to a Perl data structure.
Text::CSV_XS: Allows us to easily read in the contents of a CSV file.

Looking through the main body of the code, after the intial declarations to use the new modules listed above, we call a couple of new subroutines to read in the device credential data in our CSV file and the .ini file that configures the operation of the script:

 use strict;
 use warnings;
 use Carp;
 
 use Net::Appliance::Session;
 use Config::INI::Simple;
 use Text::CSV_XS;
 use File::Basename;
 
 # create Config::INI::Simple object and read in our ini file data
 my $ini_filename = dirname($0) . "/setup.ini"; # where we can find ini file
 my %ini_data = parse_ini_file($ini_filename);
 
 # get the device credentials for our devices
 my @device_data = parse_data_file($ini_data{device_csv_file});

The call to the parse_ini_file subroutine will read in the .ini file that is passed as a parameter and make its data available as a hash.

The call to the parse_data_file subroutine will read in the credentials CSV file that is passed as a parameter and make its data available as an array.

Next, we step through each of the devices whose credentials we retrieved from the CSV file, and create some file names for the various files we may create for this particular device:

 # step through each device and try to get and store our configs 
 DEVICE:
 for my $device_ref (@device_data) {
     
     my $device_name = $device_ref->{device_name} || $device_ref->{device_ip};
     
     # set up some logging
     my $debug_log = "$ini_data{debug_dir}/$device_name.debug.log";
     my $error_log = "$ini_data{error_dir}/$device_name.error.log";
     
     # create our config file name
     my $file_timestamp = file_timestamp();
     my $running_config_file = "$ini_data{repository_dir}/$device_name.$file_timestamp.conf";
     
     # create our Net::Appliance::Session with the transport for this device
     my $session_obj = Net::Appliance::Session->new(
         Host      => $device_ref->{device_ip},
         Transport => $device_ref->{transport},
     );
     
     # send the debug for this session to a device-specific file
     $session_obj->input_log($debug_log);

We will create a debug file for each device, that will be placed in the debug_dir that is defined in our .ini file.

Also, if we experience an error accessing a device, we will also dump an error message in to a device-specific error file in the error_dir that is defined in our .ini file.

One other new item is the file_timestamp subroutine that is called to give us a timestamp to add to our device configuration file. Each time a device configuration is succesfully retrieved, a file based on the device name and the date/time will be created. This allows us to keep a historical view of configurations retrieved from our network devices.

Once we get into our actual interactive session, we have to make a few decisions about how we are going to access each device based on its credentials. For instance, some devices will require a username/login combination for access, others may only require a password.

To cater for these varying access requirements, a couple of new subroutines have been added to provide the correct combination of credentials, dependant upon the credentials provided in our CSV file:

     # generate the required fields for the priv_array subroutine
     my @priv_array = priv_array($device_ref);
     
     # tell our session object we don't need enable password if none supplied
     unless ($priv_array[0]) {
         $session_obj->do_privileged_mode(0);
     }
     
     # do our interactive (Telnet/SSH) stuff...
     eval {
     
         # try to login to the ios device, ignoring host check
         $session_obj->connect( connect_hash($device_ref), SHKC => 0 );
           
         if ( $priv_array[0] ) {
     
             # if we need to use some enable credentials, supply them
             $session_obj->begin_privileged( @priv_array );
         }
         
         # get our running config
         @running_config =  $session_obj->cmd('show running');
     };

The connect_hash subroutine will look at the username and password fields of the credentials CSV file and supply the correct format array to pass to the connect method of our session object.

To get in to enable mode with the begin_privileged method, we need to ensure that we supply the correct credentials for each device. These are provided by the enable_username and enable_password fields of the CSV file. The priv_array subroutine will format an array to ensure that we can access enable mode (if required). If no enable mode credentials are provided, it is assumed that we are already in enable mode, and the session object is advised using the do_privileged_mode method.

Well, that's pretty much it in terms of discussing this script. As I say, much of it has been covered in previous examples.

One footnote I would like to add is that although this is a fully functioning script (apart from the bugs in it I haven't found yet!), it could still use quite a lot of improvements for a production environment. For instance, there is no checking of the type or presence of device credential data in the CSV file. If there is no IP address for a device, the whole thing falls in a heap. But, I had to draw the line somewhere when developing this script to demonstrate how it might be used in the real world, so please bear these limitations in mind.

INSTALLATION

As stated previously, the script comprises three files.

In addition to this script, there is a CSV file that contains the device credential data, and an initialization file that contains various parameters to control the operation of the script.

In case you don't have the two accompanying files, below are examples of the files that you require. If both of the files are placed in to the same directory as this script, then everything should work just fine.

To run the script (once you have all three files configured and together), just run it from the command line:

 $ perl Recipe_06.pl

If all runs well, you can maybe run it from your system scheduler to automate the process. However, don't forget to check for error logs periodically to make sure things are still running OK!

Configuration files that are retrieved should be available in the directory configured by the repository_dir parameter of the setup.ini file.

`device_credentials.csv`

 device_name,device_ip,username,password,enable_username,enable_password,transport
 rtr_1,10.250.249.215,cisco,cisco,,,Telnet
 rtr_2,10.250.249.216,cisco,cisco,,,Telnet

`setup.ini`

 [Logs]
 # directory for error logs
 error_dir=/home/nigel/perl/logs
 
 # directory for session debugging logs 
 debug_dir=/home/nigel/perl/logs
 
 [CSV]
 # CSV file with containing device credential info
 device_csv_file=./device_credentials.csv
 
 [Repository]
 # directory where configs are dumped
 repository_dir=/home/nigel/perl/configs
 
 # format of config file timestamp: 'uk' or 'us'
 timestamp_format=uk

AUTHOR

Nigel Bowden, with POD formatting by Oliver Gorwits.

COPYRIGHT & LICENSE

You may distribute and/or modify this documentation under the same terms as Perl itself.

To install Net::Appliance::Session, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Net::Appliance::Session

CPAN shell

perl -MCPAN -e shell
install Net::Appliance::Session

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)