The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

Name

IPC::Run3::Shell - Perl extension for easy execution of system commands

Synopsis

 use strict;
 use warnings;             # any problems with system commands are warnings
 use IPC::Run3::Shell;                        # nothing exported by default
 use warnings FATAL=>"IPC::Run3::Shell";      # fatal warnings (optional)
 
 # ### Basic Usage (Functional Interface)
 use IPC::Run3::Shell qw/ echo cat /;    # these functions call system cmds
 echo "Hello, World!";                   # void context: display output
 my $groups = cat "/etc/group";          # scalar context: get all output
 my @passwd = cat "/etc/passwd";         # list context: get output lines
 my $foo = echo ' "" \\ ; $HOME 1>&2 ';  # shell metachars are unaffected
 
 # ### OO Interface - methods become system commands
 my $sh = IPC::Run3::Shell->new( show_cmd=>1 );    # independent options
 $sh->cp(-v, '/etc/passwd', '/tmp/passwd.copy');   # Perl note: -v eq "-v"
 
 # ### Options
   # set default options in "use", or per command as first/last arguments
 use IPC::Run3::Shell { chomp=>1 }, qw/ ls /;      # chomp the return value
 my @files = ls -lA, '/etc', {chomp=>0};           # override an option
 my $motd = cat "/etc/motd", {allow_exit=>[0,1]};  # allowed exit codes
 my @words = echo {irs=>"-"}, -n, "A-B-C-D";       # input record separator
 echo "Foo", "Bar", {fail_on_stderr=>1};           # fail on any STDERR
 my $bla = echo {show_cmd=>1}, "blah", "blah";     # show cmd (debugging)
   # IPC::Run3 options binmode_* and append_* are also supported
 
 # ### I/O Redirection with Options stdin, stdout, and stderr
 use IPC::Run3::Shell qw/ :run /;        # and a demo of the run() function
 my ($i,$o,$e) = ("Hello,");
 run '/usr/bin/perl', '-pe', 'warn "World!\n"',    # a tricky example
     {stdin=>\$i, stdout=>\$o, stderr=>\$e};
 print "$o $e";                                    # prints "Hello, World!"
   # Note use of "stdout" means return value will be the exit code
 # ## A *simplified* reference for stdin/stdout/stderr options:
   # undef = inherit filehandle        # \undef = /dev/null
   # $filename = use that filename     # \*FILEHANDLE = use that filehandle
   # \$scalarref = use that string     # \@arrayref = lines* to/from array
   # \&coderef = called for each line*     ( * no newlines added to stdin )
 
 # ### Aliasing
 use IPC::Run3::Shell [ psw => {chomp=>1}, '/bin/ps', -ww ];
 my @ps = psw;        # function psw() is now an alias for "/bin/ps -ww"

More examples in the documentation!

Description

This module's goal is to provide a relatively transparent interface to system commands from regular-looking Perl functions. It is built on the module IPC::Run3 and is heavily inspired by the module Shell, but with several important differences and added features:

  • functions are only exported at the request of the user (no autoloading in the user's namespace)

  • system commands and their arguments are never sent through the shell (so you don't have to worry about escaping shell metacharacters)

  • failures of the system commands are (optional) warnings, which can be made fatal (in the spirit of autodie or bash's set -e)

  • advanced capturing of STDIN, STDOUT and STDERR

Read on for more information and examples!

Version

This document describes version 0.51 of IPC::Run3::Shell.

Security

Calling system commands can be dangerous. This module does not provide any security checks. All of the usual warnings apply, see perlsec!

General

This module provides Perl functions with the following basic pattern: The name of the function (or method, if you're using the OO interface) becomes the name of the system command that is executed when you call the function, and any arguments given to the Perl function become the arguments of the system command. Options may be given to the Perl function as a hashref, see "Options" for details. The return value of the Perl function depends on a few things, this is described in "Return Values", but normally it'll be either the output (STDOUT) of the system command or its exit code. If something goes wrong, an optional and optionally fatal warning is raised, see "Warnings". Additional features are described in the rest of this documentation, such as "Additional Functions".

The command and its argument list should be plain scalars, and not references, objects, or undef, otherwise this will trigger a (optional/optionally fatal) warning. This module changes undef to "" and passes references through to IPC::Run3, which may cause unintended behavior! If you want to use objects that stringify to a useful value, you should explicitly stringify them yourself (e.g. "$object") to avoid the warnings.

A note on terminology: This documentation uses the term "system command" to refer to external executables / programs / scripts; you may prefer to use a different term and that's okay ;-)

Warnings

This module ties into Perl's warnings system and provides a warnings category with the same name as the module. Several possible error conditions generate optional warnings, which can also be made fatal.

Any "modern" Perl script should start with the following, which enables all warnings:

 use warnings;
 use strict;    # or "use v5.12;" or a newer version

In addition, you can turn the warnings into fatal errors like this (see perllexwarn):

 # use the module, then make the module's warnings fatal
 use IPC::Run3::Shell;
 use warnings FATAL=>"IPC::Run3::Shell";

Note there are a few error conditions that are currently always fatal: incorrect arguments to use, a completely empty command, and a failure of IPC::Run3.

Functional Interface

 use IPC::Run3::Shell {show_cmd=>1},  # default options for the following
     qw/ echo cat /,       # these Perl functions will call system commands
     [ lsl => 'ls', '-l' ];     # alias function "lsl" to command "ls -l"
 echo "Hello,", "World!";       # similar to system() (due to void context)
 my @passwds = cat {chomp=>1}, "/etc/passwd";  # get output lines as list
 my $filelist = lsl;            # get the full output of the command

Arguments to use

This section describes the arguments after the module's name and optional module version, i.e. the "LIST" part of Perl's use function.

The functions you name in use will be made available in your script, these Perl functions will be translated into the system commands of the same name following the usual pattern described in "General".

Default options may be specified as one or more hashrefs at the beginning of the list, see "Options". These default options apply only to the functions you specify in use, including "run", and with the exception of "make_cmd".

Aliasing

Passing an arrayref to use allows you to alias functions to commands, with a full path name of the executable, any number of arguments, as well as default options that only affect that command. The first item in the array must be the function name, followed by any default options, then the command name and any arguments.

 use IPC::Run3::Shell [ 'e', {show_cmd=>1}, 'echo', '-n' ],
     [ d => '/bin/date' ];
 e d '+%Z';      # calls "/bin/date +%Z" and passes its output to "echo -n"

OO Interface

 use IPC::Run3::Shell;                        # nothing exported by default
 # the following default options apply to all commands called on the object
 my $sh = IPC::Run3::Shell->new( show_cmd => 1 );
 $sh->echo("Hello, World!");
 my $x = $sh->echo({show_cmd=>0}, "Test!");   # overriding a default option

The OO interface has the advantages that no functions need to be imported to your namespace at all, and that each object has its own independent set of options.

When you make an object with new, almost all of the methods you call on that object are translated into system commands following the usual pattern described in "General". Default options, which apply to all commands called via the object, may be given to new as a plain hash (not a hashref).

Methods are provided via Perl's autoloading feature. So, the only methods that will not be available as system commands are the built-ins of the UNIVERSAL class, as well as "AUTOLOAD" and "DESTROY". So if you want to run a command named, for example, "can", you can use "run" to do that (note that "run" is not available via the OO interface).

(Note on internals: The objects will actually not be of type IPC::Run3::Shell, they will be of type IPC::Run3::Shell::Autoload, so they have a clean namespace.)

Additional Functions

run

 use IPC::Run3::Shell qw/ :run /;               # NOTE the colon
 my $root = run "grep", "root", "/etc/passwd";  # run system command "grep"
 # Perl's built-in "grep" function remains unaffected

run mostly works like the other functions provided by this module (see "General"), with the difference that the name of the system command is the first argument to the function. This is useful, for example, if you don't want to override Perl built-in functions, or if you want to call a command with the full path.

It can be accessed either by specifying :run (note the colon!) to use (in this case it is affected by default options, if you specified them), or by calling IPC::Run3::Shell::run() directly (in this case it is not affected by default options, only by those that you give to IPC::Run3::Shell::run() directly).

make_cmd

 use IPC::Run3::Shell qw/ :make_cmd /;  # NOTE the colon
 my $pl = make_cmd 'perl', '-le';    # $pl->(ARGS) will run "perl -le ARGS"
 $pl->('print "Hello, World!"');

make_cmd is an advanced function which gives you full customizability over the generated Perl function and the executed system command. (It is actually the central function of this module.)

make_cmd takes optional options (hashref(s)), a command name (optionally with full pathname), optional command-line arguments, and returns a code ref which behaves as described in "General" and "Return Values" and, when called, executes the specified system command. Any additional arguments to the code ref become additional command-line arguments. The code ref may also be given additional options.

It can be accessed either by specifying :make_cmd (note the colon!) to use or by calling IPC::Run3::Shell::make_cmd() directly. Note that make_cmd and the functions it generates are only affected by the options passed directly to them, and never by default options given elsewhere (such as to use).

Options

Options can be set in several different places:

  • In the functional interface, default options can be given to use, when making aliases, or to "make_cmd". They are always specified as one or more hashrefs as the first argument(s). In the case of use, default options apply to all functions named in the use statement (with the exception of "make_cmd").

  • In the OO interface, the default options are passed to the constructor, and they then apply to all commands called on that object. They are not affected by use. Each object has its own independent set of default options. Note that the options to new are not specified as a hashref but as a plain hash.

  • In any case, options can always be specified or overridden by passing one or more hashrefs as the first and/or last argument(s) to any individual command.

Whenever multiple option sets are present, they are cumulative, with options specified later overriding the earlier ones. Note that there are a few options, such as "stdout", that cannot be un-set. Specifying unknown options triggers warnings and is strongly discouraged for forwards compatibility.

allow_exit

Either a single value or an arrayref of allowed exit values. Defaults to only zero (0). May also be set to the string "ANY" to mean any exit code is allowed. If set to undef, resets the option to its default value. Any other non-numeric values or an empty arrayref cause a warning. This option does not make much sense unless you have warnings enabled.

irs

Sets a local $/, see Perl's input record separator. Note that, like "stdout", irs => undef has a meaning, so the mere presence of this option in the options hash means the option is set.

chomp

Setting this boolean option causes the return value of the function to be chomped. This option has no effect if you're using the option "stdout".

stdin

stdout

stderr

 use IPC::Run3::Shell {chomp=>1}, qw/ cat find /;
 cat { stdin=>\"Hello, World!\n" };
 # there will probably be some "permission denied" errors in /etc,
 # so we're allowing for an exit value of 1 and are capturing stderr
 my $rv = find { allow_exit=>[0,1] }, '/etc', '-name', 'passwd',
     '-print0', { stdout=>\my @files, stderr=>\my $err, irs=>"\0" };
 print "find exited with $rv\n";
 print "Found files: ".join(', ',@files)."\n";
 print "## Warnings ##\n$err##\n" if $err=~/\S/;

These options give you powerful redirection of STDIN, STDOUT and STDERR of the executed command. The possible values are the same as in IPC::Run3; the following is a brief summary of the possible values:

undef

inherit the filehandle from parent

\undef

\dev\null (or equivalent)

scalar

a filename

filehandle

use that filehandle

scalar reference

read/write from/into that scalar (string)

array reference

read/write from/into that array, splitting output (not input) "lines" on $/

code reference

For stdin, the sub is called repeatedly and its return values are fed to the process until the sub returns undef. For stdout and stderr, the sub is called with each "line" of output (split on $/ as usual).

Note that if you specify the option stdout, the function will always return the exit code of the command, and not its output - see "Return Values". stdin and stderr do not have this effect.

Since setting an option like stdout => undef has a meaning, the mere presence of any of the hash keys stdout or stderr in the options hash will be interpreted as the respective option being set. This also means that these options cannot be un-set by overriding them. So normally, you'll want to set these in the per-command options only, and not in the default options.

Note this module does not make any guarantees about the capabilities of IPC::Run3 to redirect streams! For example, at this time of writing, IPC::Run3 does not support swapping of STDOUT and STDERR. Also, you will get unexpected results if you attempt to redirect STDOUT to STDERR and capture STDERR at the same time. (It is at the moment possible to redirect STDOUT to STDERR or vice versa if you don't touch the other.) If IPC::Run3's behavior changes in the future, that behavior will simply be passed through by this module.

fail_on_stderr

If this boolean option is set, any output to STDERR (except for a single $/) is considered a fatal error. This option may not be specified together with "stderr" (this is currently a fatal error). Note that turning this option on means that STDERR is always captured and not displayed.

show_cmd

Setting this option to a true value causes the command which is about to be executed to be printed to STDERR. You may also specify a filehandle / glob reference here to have the command printed there (e.g. show_cmd => \*STDOUT).

This is meant for debugging and/or user information only, as the output may not be safely escaped, so don't try to parse this output!

Options from IPC::Run3

The following options may be specified and they will be passed through to IPC::Run3.

  • binmode_stdin

  • binmode_stdout

  • binmode_stderr

  • append_stdout

  • append_stderr

  • return_if_system_error

    (Note this module turns this option on by default. It's recommended not to turn it off or you may get unexpected results in case of errors.)

Return Values

If executing the command fails in any way, such as the command returning an unexpected exit code, this generates an optional and optionally fatal warning, see "Warnings". In the case that the underlying system call (or equivalent) fails, the function will return undef or the empty list depending on context. Otherwise, either the exit code or the (possibly unfinished) command's STDOUT is returned, as described here. If you're expecting a nonzero return code from a system command, see the option "allow_exit". See also the option "fail_on_stderr". (Hint: If you want to catch fatal errors, you can look into Try::Tiny or a similar module.)

If you're using the option "stdout", the return value of the function will be the exit code of the system command and never the command's STDOUT.

If you call the function in list context, the return value of the function will be a list consisting of the command's STDOUT output split on the input record separator $/. In other words, you'll normally get a list of the lines of the output. See also the option "irs".

If you call a function in scalar context, the function will return the entire STDOUT output of the command as one string.

If you call a function in void context, and you're not using the option "stdout", the output of the command will be sent to your STDOUT.

You should always be able to inspect the exit code of the system command in Perl's $? variable. See the documentation of Perl's system() for an example of how to interpret $?.

Portability

This module strives to be as portable as IPC::Run3. Before reporting bugs concerning portability, please first test if maybe it's a bug in IPC::Run3.

The tests for this module require that you have the perl executable in your PATH.

If you notice any other portability issues, please let me know!

(Of course, scripts using this module to call system commands will often not be portable, but that's not really this module's concern ;-) )

Caveats

A few things to be aware of:

As described in "Return Values", the functions provided by this module act differently depending on the context they are called in (void, scalar and list) and some of the options (e.g. "stdout"). While this is an attempt to provide a DWIM (Do What I Mean) interface, the difference in behavior might be subtle and lead to a bit of confusion.

Some functions are package methods, while others are not, i.e. don't mix up :: and ->. Examples: IPC::Run3::Shell->new(), IPC::Run3::Shell->import(), IPC::Run3::Shell::run(), IPC::Run3::Shell::make_cmd(). (For the latter two, it's probably easier to use use IPC::Run3::Shell qw/:run :make_cmd/;.)

If you plan on subclassing this module, note that the OO interface doesn't follow traditional Perl OO design patterns (e.g. new isn't actually part of the package of the object).

Advanced Hints

Prototypes

Even though they're usually discouraged and often rightly so, there are still a few rare cases when prototypes are useful. One way to give the Perl functions prototypes is with Scalar::Util's set_prototype:

 # a silly example for demonstration purposes only :-)
 use IPC::Run3::Shell [ xuc => 'perl', '-e', 'print uc "@ARGV"', '--' ];
 use Scalar::Util qw/set_prototype/;
 BEGIN { set_prototype \&xuc, '$'; }     # make xuc() a unary operator
 # normally xuc() would take all the arguments, now it only takes one
 my $foo = join ",", xuc "a", "b", "c";
 print $foo;                             # prints "A,b,c"

Importing at Run-Time

If you know the internals of Perl's use, you know that you can import functions at run-time too, as follows. For a description of the arguments of import please see "Arguments to use".

 use IPC::Run3::Shell;
 IPC::Run3::Shell->import('echo');
 echo("foo");                       # NOTE parentheses are required here

If you like playing with globs, you could install the code refs created by "make_cmd" into your namespace manually. For example:

 BEGIN {              # not required, could do this at run-time
     require IPC::Run3::Shell;
     *foo = IPC::Run3::Shell::make_cmd('echo','foo');
 }
 foo "bar";           # calls "echo foo bar"

Calling the Shell

Warning: Make sure you understand the safety implications of this hint before you use it!

If you really want to invoke the shell and have your commands subject to its various interpolations and such, with all the possible safety implications (such as attackers injecting arbitrary commands into your shell!), you could do something like this:

 use IPC::Run3::Shell [sh => 'sh', '-c'];
 sh 'echo $HOME >&2';     # prints _the_shell's_ $HOME to its STDERR

Autoloading

Warning: Make sure you understand the safety implications of this hint before you use it!

If you really, really wanted to have Shell's autoloading behavior back, with all the possible safety implications (such as any typos leading to the execution of system commands!), you could do this:

 BEGIN {
     require IPC::Run3::Shell;
     *AUTOLOAD = \&IPC::Run3::Shell::Autoload::AUTOLOAD;
 }
 echo("foo","bar");   # calls "echo foo bar" via autoloaded "echo"

See Also

Larry Wall and others wrote the original Shell in 1994, with various contributions throughout the years. Shell is Copyright (c) 2005 by the Perl 5 Porters.

IPC::Run3

Capture::Tiny - yet another capturing module. Its documentation includes a list of lots more such modules.

Shell Scripts

If you'd like to write "shell script-like" Perl scripts, here are a few modules that I have found very useful in doing so. Many of these are in the Perl core.

Author, Copyright, and License

Copyright (c) 2014 Hauke Daempfling (haukex@zero-g.net).

This library is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself.

For more information see the Perl Artistic License, which should have been distributed with your copy of Perl. Try the command "perldoc perlartistic" or see http://perldoc.perl.org/perlartistic.html.