Name
IPC::Run3::Shell - Perl extension for calling system commands as if they were Perl functions
Synopsis
#!/usr/bin/env perl
use warnings;
use strict;
use IPC::Run3::Shell qw(echo cat ps cp);
my $foo = echo("howdy", "<funny>", "world");
print $foo; # prints "howdy <funny> world\n"
my @passwd = cat("/etc/passwd", {chomp=>1});
print "$_\n" for @passwd; # prints lines of /etc/passwd
print ps -ww; # prints output of "ps -ww"
cp("/etc/passwd", "/tmp/passwd.orig");
The above is adapted from Larry Wall's 1994 email about Shell.
Reference / Cheat Sheet
use strict;
use warnings; # any problems with system commands are warnings
use IPC::Run3::Shell; # nothing exported by default
use warnings FATAL=>"IPC::Run3::Shell"; # fatal warnings (optional)
# ### Basic Usage (Functional Interface)
use IPC::Run3::Shell qw/ echo cat /; # these functions call system cmds
echo "Hello, World!"; # void context: display output
my $groups = cat "/etc/group"; # scalar context: get all output
my @passwd = cat "/etc/passwd"; # list context: get output lines
# Note: option "both" changes return values, see below
my $foo = echo ' "" \\ ; $HOME 1>&2 '; # shell metachars are unaffected
# ### OO Interface - methods become system commands
my $sh = IPC::Run3::Shell->new( show_cmd=>1 ); # independent options
$sh->cp(-v, '/etc/passwd', '/tmp/passwd.copy'); # Perl note: -v eq "-v"
# ### Options
# set default options in "use", or per command as first/last arguments
use IPC::Run3::Shell { chomp=>1 }, qw/ ls /; # chomp the return value
my @files = ls -lA, '/etc', {chomp=>0}; # override an option
my $motd = cat "/etc/motd", {allow_exit=>[0,1]}; # allowed exit codes
my @words = echo {irs=>"-"}, -n, "A-B-C-D"; # input record separator
my ($out,$err,$exitcode) = echo "Foo", {both=>1}; # return both STDOUT/ERR
my $out_and_err = echo "FooBar", {both=>1}; # STDOUT+STDERR merged
echo "Foo", "Bar", {fail_on_stderr=>1}; # fail on any STDERR
my $bla = echo {show_cmd=>1}, "blah", "blah"; # show cmd (debugging)
# IPC::Run3 options binmode_* and append_* are also supported
# ### I/O Redirection with Options stdin, stdout, and stderr
use IPC::Run3::Shell qw/ :run /; # and a demo of the run() function
my ($i,$o,$e) = ("Hello,");
run '/usr/bin/perl', '-pe', 'warn "World!\n"', # a tricky example
{stdin=>\$i, stdout=>\$o, stderr=>\$e};
# Note use of "stdout" option means return value will be the exit code
print "$o $e"; # prints "Hello, World!"
# ## A *simplified* reference for stdin/stdout/stderr options:
# undef = inherit filehandle # \undef = /dev/null
# $filename = use that filename # \*FILEHANDLE = use that filehandle
# \$scalarref = use that string # \@arrayref = lines* to/from array
# \&coderef = called for each line* ( * no newlines added to stdin )
# ### Aliasing
use IPC::Run3::Shell [ psw => {chomp=>1}, '/bin/ps', -ww ];
my @ps = psw; # function psw() is now an alias for "/bin/ps -ww"
Description
This module's goal is to provide a relatively transparent interface to system commands from regular-looking Perl functions. It is built on the module IPC::Run3 and is heavily inspired by the module Shell, but with several important differences and added features:
functions are only exported at the request of the user (no autoloading in the user's namespace)
system commands and their arguments are never* sent through the shell (so you don't have to worry about escaping shell metacharacters)
failures of the system commands are (optional) warnings, which can be made fatal (in the spirit of autodie or bash's
set -e
)advanced capturing of
STDIN
,STDOUT
andSTDERR
* this should hold true for all *NIX systems, and on Windows, IPC::Run3 versions 0.047 and up use Win32::ShellQuote so that you shouldn't have to worry about shell metacharacters there either.
Read on for more information and examples!
Version
This document describes version 0.52 of IPC::Run3::Shell.
Security
Calling system commands can be dangerous. This module does not provide any security checks. All of the usual warnings apply, see perlsec!
General
This module provides Perl functions with the following basic pattern: The name of the function (or method, if you're using the OO interface) becomes the name of the system command that is executed when you call the function, and any arguments given to the Perl function become the arguments of the system command. Options may be given to the Perl function as a hashref, see "Options" for details. The return value of the Perl function depends on a few things, this is described in "Return Values", but normally it'll be either the output of the system command or its exit code. If something goes wrong, an optional and optionally fatal warning is raised, see "Warnings". Additional features are described in the rest of this documentation, such as "Additional Functions".
The command and its argument list should be plain scalars, and not references, objects, or undef
, otherwise this will trigger a (optional/optionally fatal) warning. This module changes undef
to ""
and passes references through to IPC::Run3, which may cause unintended behavior! If you want to use objects that stringify to a useful value, you should explicitly stringify them yourself (e.g. "$object"
) to avoid the warnings.
A note on terminology: This documentation uses the term "system command" to refer to external executables / programs / scripts; you may prefer to use a different term and that's okay ;-)
Warnings
This module ties into Perl's warnings system and provides a warnings category with the same name as the module. Several possible error conditions generate optional warnings, which can also be made fatal.
Any "modern" Perl script should start with the following, which enables all warnings:
use warnings;
use strict; # or "use v5.12;" or a newer version
In addition, you can turn the warnings into fatal errors like this (see perllexwarn):
# use the module, then make the module's warnings fatal
use IPC::Run3::Shell;
use warnings FATAL=>"IPC::Run3::Shell";
Note there are a few error conditions that are currently always fatal: incorrect arguments to use
, incompatible options, a completely empty command, and a failure of IPC::Run3.
(Hint: If you want to catch fatal errors, you can look into Try::Tiny or a similar module.)
Functional Interface
use IPC::Run3::Shell {show_cmd=>1}, # default options for the following
qw/ echo cat /, # these Perl functions will call system commands
[ lsl => 'ls', '-l' ]; # alias function "lsl" to command "ls -l"
echo "Hello,", "World!"; # similar to system() (due to void context)
my @passwds = cat {chomp=>1}, "/etc/passwd"; # get output lines as list
my $filelist = lsl; # get the full output of the command
Arguments to use
This section describes the arguments after the module's name and optional module version, i.e. the "LIST
" part of Perl's use
function.
The functions you name in use
will be made available in your script, these Perl functions will be translated into the system commands of the same name following the usual pattern described in "General".
Default options may be specified as one or more hashrefs at the beginning of the list, see "Options". These default options apply only to the functions you specify in use
, including "run", and with the exception of "make_cmd".
Aliasing
Passing an arrayref to use
allows you to alias functions to commands, with a full path name of the executable, any number of arguments, as well as default options that only affect that command. The first item in the array must be the function name, followed by any default options, then the command name and any arguments.
use IPC::Run3::Shell [ 'e', {show_cmd=>1}, 'echo', '-n' ],
[ d => '/bin/date' ];
e d '+%Z'; # calls "/bin/date +%Z" and passes its output to "echo -n"
OO Interface
use IPC::Run3::Shell; # nothing exported by default
# the following default options apply to all commands called on the object
my $sh = IPC::Run3::Shell->new( show_cmd => 1 );
$sh->echo("Hello, World!");
my $x = $sh->echo({show_cmd=>0}, "Test!"); # overriding a default option
The OO interface has the advantages that no functions need to be imported to your namespace at all, and that each object has its own independent set of options.
When you make an object with new
, almost all of the methods you call on that object are translated into system commands following the usual pattern described in "General". Default options, which apply to all commands called via the object, may be given to new
as a plain hash (not a hashref).
Methods are provided via Perl's autoloading feature. So, the only methods that will not be available as system commands are the built-ins of the UNIVERSAL class, as well as "AUTOLOAD
" and "DESTROY
". So if you want to run a command named, for example, "can
", you can use "run" to do that (note that "run" is not available via the OO interface).
(Note on internals: The objects will actually not be of type IPC::Run3::Shell
, they will be of type IPC::Run3::Shell::Autoload
, so they have a clean namespace.)
Additional Functions
run
use IPC::Run3::Shell qw/ :run /; # NOTE the colon
my $root = run "grep", "root", "/etc/passwd"; # run system command "grep"
# Perl's built-in "grep" function remains unaffected
run
mostly works like the other functions provided by this module (see "General"), with the difference that the name of the system command is the first argument to the function. This is useful, for example, if you don't want to override Perl built-in functions, or if you want to call a command with the full path.
It can be accessed either by specifying :run
(note the colon!) to use
(in this case it is affected by default options, if you specified them), or by calling IPC::Run3::Shell::run()
directly (in this case it is not affected by default options, only by those that you give to IPC::Run3::Shell::run()
directly).
make_cmd
use IPC::Run3::Shell qw/ :make_cmd /; # NOTE the colon
my $pl = make_cmd 'perl', '-le'; # $pl->(ARGS) will run "perl -le ARGS"
$pl->('print "Hello, World!"');
make_cmd
is an advanced function which gives you full customizability over the generated Perl function and the executed system command. (It is actually the central function of this module.)
make_cmd
takes optional options (hashref(s)), a command name (optionally with full pathname), optional command-line arguments, and returns a code ref which behaves as described in "General" and "Return Values" and, when called, executes the specified system command. Any additional arguments to the code ref become additional command-line arguments. The code ref may also be given additional options.
It can be accessed either by specifying :make_cmd
(note the colon!) to use
or by calling IPC::Run3::Shell::make_cmd()
directly. Note that make_cmd
and the functions it generates are only affected by the options passed directly to them, and never by default options given elsewhere (such as to use
).
import_into
Lets you export functions from this module into any package of your choosing. The first argument is the package name, the rest are the usual arguments to import.
use IPC::Run3::Shell;
IPC::Run3::Shell->import_into('Foo::Bar', 'echo');
{
package Foo::Bar;
echo("Hello, World");
}
Options
Options can be set in several different places:
In the functional interface, default options can be given to
use
, when making aliases, or to "make_cmd". They are always specified as one or more hashrefs as the first argument(s). In the case ofuse
, default options apply to all functions named in theuse
statement (with the exception of "make_cmd").In the OO interface, the default options are passed to the constructor, and they then apply to all commands called on that object. They are not affected by
use
. Each object has its own independent set of default options. Note that the options tonew
are not specified as a hashref but as a plain hash.In any case, options can always be specified or overridden by passing one or more hashrefs as the first and/or last argument(s) to any individual command.
Whenever multiple option sets are present, they are cumulative, with options specified later overriding the earlier ones. Note that there are a few options, such as "stdout", that cannot be un-set. Specifying unknown options triggers warnings and is strongly discouraged for forwards compatibility.
allow_exit
Either a single value or an arrayref of allowed exit values. Defaults to only zero (0). May also be set to the string "ANY
" to mean any exit code is allowed. If set to undef
, resets the option to its default value. Any other non-numeric values or an empty arrayref cause a warning. This option does not make much sense unless you have warnings enabled.
irs
Sets a local $/
, see Perl's input record separator. Note that, like "stdout", irs => undef
has a meaning, so the mere presence of this option in the options hash means the option is set.
chomp
Setting this boolean option causes the return value(s) of the function to be chomped. This option has no effect if you're using the option "stdout".
both
Setting this boolean option has a different effect depending on whether the function is called in list context or in scalar context - see also "Return Values". You cannot use this option together with "stdout", "stderr" or "fail_on_stderr".
In scalar context, the function returns the entire STDOUT
plus STDERR
of the executed command merged as one string. Note that the order of output on these two streams is not always well-defined; they may be mixed together. This is still useful if, for example, you know the command outputs on STDERR
only and you want an easier way to capture it than the "stderr" option.
In list context, the function returns three values: the entire STDOUT
as one string, the entire STDERR
as one string, and the exit code of the command. This is somewhat similar to Capture::Tiny's capture
used together with system
(except that returns $?
and not the exit code).
# call an external perl process for this example
use IPC::Run3::Shell {both=>1}, "perl";
my ($stdout, $stderr, $exitcode) = perl {allow_exit=>123}, '-e',
' print "Foo\n"; warn "Bar\n"; exit 123 ';
# $stdout gets "Foo\n", $stderr gets "Bar\n" and $exitcode==123
stdin
stdout
stderr
use IPC::Run3::Shell {chomp=>1}, qw/ cat find /;
cat { stdin=>\"Hello, World!\n" };
# there will probably be some "permission denied" errors in /etc,
# so we're allowing for an exit value of 1 and are capturing stderr
my $rv = find { allow_exit=>[0,1] }, '/etc', '-name', 'passwd',
'-print0', { stdout=>\my @files, stderr=>\my $err, irs=>"\0" };
print "find exited with $rv\n";
print "Found files: ".join(', ',@files)."\n";
print "## Warnings ##\n$err##\n" if $err=~/\S/;
These options give you powerful redirection of STDIN
, STDOUT
and STDERR
of the executed command. The possible values are the same as in IPC::Run3; the following is a brief summary of the possible values:
undef
-
inherit the filehandle from parent
\undef
-
\dev\null (or equivalent)
- scalar
-
a filename
- filehandle
-
use that filehandle
- scalar reference
-
read/write from/into that scalar (string)
- array reference
-
read/write from/into that array, splitting output (not input) "lines" on
$/
- code reference
-
For
stdin
, the sub is called repeatedly and its return values are fed to the process until the sub returnsundef
. Forstdout
andstderr
, the sub is called with each "line" of output (split on$/
as usual).
Note that if you specify the option stdout
, the function will always return the exit code of the command, and not its output - see "Return Values". stdin
and stderr
do not have this effect.
Since setting an option like stdout => undef
has a meaning, the mere presence of any of the hash keys stdout
or stderr
in the options hash will be interpreted as the respective option being set. This also means that these options cannot be un-set by overriding them. So normally, you'll want to set these in the per-command options only, and not in the default options.
Note this module does not make any guarantees about the capabilities of IPC::Run3 to redirect streams! For example, at this time of writing, IPC::Run3 does not support swapping of STDOUT
and STDERR
. Also, you will get unexpected results if you attempt to redirect STDOUT
to STDERR
and capture STDERR
at the same time. (It is at the moment possible to redirect STDOUT
to STDERR
or vice versa if you don't touch the other.) If IPC::Run3's behavior changes in the future, that behavior will simply be passed through by this module.
You can't use the stderr
option together with "fail_on_stderr". You can't use stdout
or stderr
together with "both".
fail_on_stderr
If this boolean option is set, any output to STDERR
(except for a single $/
) is considered a fatal error. This option may not be specified together with "stderr" or "both" (this is currently a fatal error). Note that turning this option on means that STDERR
is always captured and not displayed.
show_cmd
Setting this option to a true value causes the command which is about to be executed to be printed to STDERR
. You may also specify a filehandle / glob reference here to have the command printed there (e.g. show_cmd => \*STDOUT
).
This is meant for debugging and/or user information only, as the output may not be safely escaped, so don't try to parse this output!
Options from IPC::Run3
The following options may be specified and they will be passed through to IPC::Run3.
binmode_stdin
binmode_stdout
binmode_stderr
append_stdout
append_stderr
return_if_system_error
(Note this module turns this option on by default. It's recommended not to turn it off or you may get unexpected results in case of errors.)
Return Values
In the following table, "OUT
" and "ERR
" refer to the STDOUT
and STDERR
of the executed command. The options "both", "stdout" and "stderr" are described in their respective sections.
| Void Context | Scalar Context | List Context |
==========+====================+====================+====================+
system() | | return undef | return empty list |
fails | also generates optional warning that can be made fatal[1] |
----------+--------------------+--------------------+--------------------+
Bad exit | | output as below, but may be incomplete |
code[2] | also generates optional warning that can be made fatal[1] |
==========+====================+====================+====================+
Default | OUT and ERR are | OUT is returned | OUT is returned as |
| not redirected, | as one big string | a list of lines[4] |
| they are passed | ERR not redirected by default[3] |
==========+ thru to script's +====================+====================+
Option | STDOUT and STDERR | return merged OUT | return a list of |
"both" | | and ERR | (OUT,ERR,exitcode) |
==========+====================+====================+====================+
Option | OUT is redirected as requested (ERR handled separately) |
"stdout" | Function always returns the exit code |
==========+==============================================================+
Option | ERR is redirected as requested (OUT handled separately) |
"stderr" | |
==========+==============================================================+
You should always be able to inspect the exit code of the system command in Perl's $?
variable. See the documentation of Perl's system()
for an example of how to interpret $?
.
[1] This is described in "Warnings".
[2] If you're expecting a nonzero return code from a system command, see the option "allow_exit". Currently, a command being terminated due to a signal will generate a (optional/optionally fatal) warning and there is no equivalent of "allow_exit" for signals.
[3] STDERR
can be handled via the options "stderr" or "fail_on_stderr".
[4] More exactly, the command's STDOUT
output is split on the input record separator $/
. Since that's the line separator by default, you'll normally get a list of the lines of the output, unless you change $/
, which you can also do via the option "irs".
Portability
This module strives to be as portable as IPC::Run3. Before reporting bugs concerning portability, please first test if maybe it's a bug in IPC::Run3.
The tests for this module require that you have the perl
executable in your PATH
.
If you notice any other portability issues, please let me know!
Of course, scripts using this module to call system commands will often not be portable, but that's simply the nature of scripts that call system commands.
Caveats
A few things to be aware of:
As described in "Return Values", the functions provided by this module act differently depending on the context they are called in (void, scalar and list) and some of the options (e.g. "stdout" and "both"). While this is an attempt to provide a DWIM (Do What I Mean) interface, the difference in behavior might be subtle and lead to a bit of confusion.
Some functions are package methods, while others are not, i.e. don't mix up ::
and ->
. Examples: IPC::Run3::Shell->new()
, IPC::Run3::Shell->import()
, IPC::Run3::Shell->import_into()
, IPC::Run3::Shell::run()
, IPC::Run3::Shell::make_cmd()
. (For the latter two, it's probably easier to use use IPC::Run3::Shell qw/:run :make_cmd/;
.)
If you plan on subclassing this module, note that the OO interface doesn't follow traditional Perl OO design patterns (e.g. new
isn't actually part of the package of the object).
Advanced Hints
Prototypes
Even though they're usually discouraged and often rightly so, there are still a few rare cases when prototypes are useful. One way to give the Perl functions prototypes is with Scalar::Util's set_prototype
:
# a silly example for demonstration purposes only :-)
use IPC::Run3::Shell [ xuc => 'perl', '-e', 'print uc "@ARGV"', '--' ];
use Scalar::Util qw/set_prototype/;
BEGIN { set_prototype \&xuc, '$'; } # make xuc() a unary operator
# normally xuc() would take all the arguments, now it only takes one
my $foo = join ",", xuc "a", "b", "c";
print $foo; # prints "A,b,c"
Importing at Run-Time
If you know the internals of Perl's use
, you know that you can import functions at run-time too, as follows. For a description of the arguments of import
please see "Arguments to use".
use IPC::Run3::Shell;
IPC::Run3::Shell->import('echo');
echo("foo"); # NOTE parentheses are required here
If you like playing with globs, you could install the code refs created by "make_cmd" into your namespace manually. For example:
BEGIN { # not required, could do this at run-time
require IPC::Run3::Shell;
*foo = IPC::Run3::Shell::make_cmd('echo','foo');
}
foo "bar"; # calls "echo foo bar"
Also, note the "import_into" function for more powerful exporting.
Calling the Shell
Warning: Make sure you understand the safety implications of this hint before you use it!
If you really want to invoke the shell and have your commands subject to its various interpolations and such, with all the possible safety implications (such as attackers injecting arbitrary commands into your shell!), you could do something like this:
use IPC::Run3::Shell [sh => 'sh', '-c'];
sh 'echo $HOME >&2'; # prints _the_shell's_ $HOME to _its_ STDERR
Autoloading
Warning: Make sure you understand the safety implications of this hint before you use it!
If you really, really wanted to have Shell's autoloading behavior back, with all the possible safety implications (such as any typos leading to the execution of system commands!), you could do this:
BEGIN {
require IPC::Run3::Shell;
*AUTOLOAD = \&IPC::Run3::Shell::Autoload::AUTOLOAD;
}
echo("foo","bar"); # calls "echo foo bar" via autoloaded "echo"
See Also
Larry Wall and others wrote the original Shell in 1994, with various contributions throughout the years. Shell is Copyright (c) 2005 by the Perl 5 Porters.
IPC::Run3 - I chose this module from the plethora of similar modules for several reasons: it handles all three STDOUT
, STDERR
and STDIN
; it runs on Win32; it allows avoiding the shell; no non-core dependencies (except on Windows); and its test statistics are currently nearly perfect. While it doesn't support a few features that some of the more advanced modules do, like timeouts or interactively communicating with the subprocess, I hope it'll be enough for the majority of cases.
Shell::Tools - reduce boilerplate in Perl shell scripts, also includes easy access to this module!
Capture::Tiny - yet another capturing module. Its documentation includes a list of lots more such modules.
Env - tie environment variables to Perl variables, similar in spirit to this module
Author, Copyright, and License
Copyright (c) 2014 Hauke Daempfling (haukex@zero-g.net).
This library is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself.
For more information see the Perl Artistic License, which should have been distributed with your copy of Perl. Try the command "perldoc perlartistic
" or see http://perldoc.perl.org/perlartistic.html.