IPC::Run3::Shell - Perl extension for calling system commands as if they were Perl functions
#!/usr/bin/env perl use warnings; use strict; use IPC::Run3::Shell qw(echo cat ps cp); my $foo = echo("howdy", "<funny>", "world"); print $foo; # prints "howdy <funny> world\n" my @passwd = cat("/etc/passwd", {chomp=>1}); print "$_\n" for @passwd; # prints lines of /etc/passwd print ps -ww; # prints output of "ps -ww" cp("/etc/passwd", "/tmp/passwd.orig");
The above is adapted from Larry Wall's 1994 email about Shell.
use strict; use warnings; # any problems with system commands are warnings use IPC::Run3::Shell # nothing exported by default qw/ :FATAL /; # fatal warnings (optional) # ### Basic Usage (Functional Interface) use IPC::Run3::Shell qw/ echo cat /; # these functions call system cmds echo "Hello, World!"; # void context: display output my $groups = cat "/etc/group"; # scalar context: get all output my @passwd = cat "/etc/passwd"; # list context: get output lines # Note: option "both" changes return values, see below my $foo = echo ' "" \\ ; $HOME 1>&2 '; # shell metachars are unaffected # ### OO Interface - methods become system commands my $sh = IPC::Run3::Shell->new( show_cmd=>1 ); # independent options $sh->cp(-v, '/etc/passwd', '/tmp/passwd.copy'); # Perl note: -v eq "-v" # ### Options # set default options in "use", or per command as first/last arguments use IPC::Run3::Shell { chomp=>1 }, qw/ ls /; # chomp the return value my @files = ls -lA, '/etc', {chomp=>0}; # override an option my $motd = cat "/etc/motd", {allow_exit=>[0,1]}; # allowed exit codes my @words = echo {irs=>"-"}, -n, "A-B-C-D"; # input record separator my ($out,$err,$exitcode) = echo "Foo", {both=>1}; # return both STDOUT/ERR my $out_and_err = echo "FooBar", {both=>1}; # STDOUT+STDERR merged echo "Foo", "Bar", {fail_on_stderr=>1}; # fail on any STDERR my $outfilt = echo "Hello", { stdout_filter => sub { s/[eo]/X/g } }; # provide a callback that modifies $_ to filter stdout return value my $bla = echo {show_cmd=>1}, "blah", "blah"; # show cmd (debugging) # IPC::Run3 options binmode_* and append_* are also supported # ### I/O Redirection with Options stdin, stdout, and stderr use IPC::Run3::Shell qw/ :run /; # and a demo of the run() function my ($i,$o,$e) = ("Hello,"); run '/usr/bin/perl', '-pe', 'warn "World!\n"', # a tricky example {stdin=>\$i, stdout=>\$o, stderr=>\$e}; # Note use of "stdout" option means return value will be the exit code print "$o $e"; # prints "Hello, World!" # ## A *simplified* reference for stdin/stdout/stderr options: # undef = inherit filehandle # \undef = /dev/null # $filename = use that filename # \*FILEHANDLE = use that filehandle # \$scalarref = use that string # \@arrayref = lines* to/from array # \&coderef = called for each line* ( * no newlines added to stdin ) # ### Aliasing use IPC::Run3::Shell [ psw => {chomp=>1}, '/bin/ps', -ww ]; my @ps = psw; # function psw() is now an alias for "/bin/ps -ww"
This module's goal is to provide a relatively transparent interface to system commands from regular-looking Perl functions. It is built on the module IPC::Run3 and is heavily inspired by the module Shell, but with several important differences and added features:
functions are only exported at the request of the user (no autoloading in the user's namespace - unless you explicitly ask for it)
system commands and their arguments are never* sent through the shell (so you don't have to worry about escaping shell metacharacters)
failures of the system commands are (optional) warnings, which can be made fatal (in the spirit of autodie or bash's set -e)
set -e
advanced capturing of STDIN, STDOUT and STDERR
STDIN
STDOUT
STDERR
* this should hold true for all *NIX systems, and on Windows, IPC::Run3 versions 0.047 and up use Win32::ShellQuote so that you shouldn't have to worry about shell metacharacters there either.
Read on for more information and examples!
This document describes version 0.58 of IPC::Run3::Shell.
Calling system commands can be dangerous. This module does not provide any security checks. All of the usual warnings apply, see perlsec!
This module provides Perl functions with the following basic pattern: The name of the function (or method, if you're using the OO interface) becomes the name of the system command that is executed when you call the function, and any arguments given to the Perl function become the arguments of the system command. Options may be given to the Perl function as a hashref, see "Options" for details. The return value of the Perl function depends on a few things, this is described in "Return Values", but normally it'll be either the output of the system command or its exit code. If something goes wrong, an optional and optionally fatal warning is raised, see "Warnings". Additional features are described in the rest of this documentation, such as "Additional Functions".
The command and its argument list are stringified and should only consist of plain scalars or objects that stringify, as well as option hashrefs. Other references, objects, or undefs will trigger a (optional/optionally fatal) warning. Objects that use overloading but don't support stringification will cause a fatal error as they normally would. (Note that before v0.53, objects that stringified also triggered a warning, and references were passed through unchanged to IPC::Run3 instead of being stringified. undefs have been stringified to "" since v0.51.)
undef
""
A note on terminology: This documentation uses the term "system command" to refer to external executables / programs / scripts; you may prefer to use a different term and that's okay ;-)
;-)
This module ties into Perl's warnings system and provides a warnings category with the same name as the module. Several possible error conditions generate optional warnings, which can also be made fatal.
Any "modern" Perl script should start with the following, which enables all warnings:
use warnings; use strict; # or "use v5.12;" or a newer version
In addition, you can turn the warnings into fatal errors like this (added in v0.56):
use IPC::Run3::Shell ':FATAL';
Which is shorthand for this (see perllexwarn):
use IPC::Run3::Shell; use warnings FATAL=>"IPC::Run3::Shell";
Note there are a few error conditions that are currently always fatal: incorrect arguments to use, incompatible options, a completely empty command, and a failure of IPC::Run3. (In the future, some of these may be reduced into warnings, if a useful default behavior exists.)
use
(Hint: If you want to catch fatal errors, you can look into Try::Tiny or a similar module.)
use IPC::Run3::Shell {show_cmd=>1}, # default options for the following qw/ echo cat /, # these Perl functions will call system commands [ lsl => 'ls', '-l' ]; # alias function "lsl" to command "ls -l" echo "Hello,", "World!"; # similar to system() (due to void context) my @passwds = cat {chomp=>1}, "/etc/passwd"; # get output lines as list my $filelist = lsl; # get the full output of the command
This section describes the arguments after the module's name and optional module version, i.e. the "LIST" part of Perl's use function.
LIST
The functions you name in use will be made available in your script, these Perl functions will be translated into the system commands of the same name following the usual pattern described in "General".
Default options may be specified as one or more hashrefs at the beginning of the list, see "Options". These default options apply only to the functions you specify in use, including "run", and with the exception of "make_cmd".
Passing an arrayref to use allows you to alias functions to commands, with a full path name of the executable, any number of arguments, as well as default options that only affect that command. The first item in the array must be the function name, followed by any default options, then the command name and any arguments.
use IPC::Run3::Shell [ 'e', {show_cmd=>1}, 'echo', '-n' ], [ d => '/bin/date' ]; e d '+%Z'; # calls "/bin/date +%Z" and passes its output to "echo -n"
use IPC::Run3::Shell; # nothing exported by default # the following default options apply to all commands called on the object my $sh = IPC::Run3::Shell->new( show_cmd => 1 ); $sh->echo("Hello, World!"); my $x = $sh->echo({show_cmd=>0}, "Test!"); # overriding a default option
The OO interface has the advantages that no functions need to be imported to your namespace at all, and that each object has its own independent set of options.
When you make an object with new, almost all of the methods you call on that object are translated into system commands following the usual pattern described in "General". Default options, which apply to all commands called via the object, may be given to new as a plain hash (not a hashref).
new
Methods are provided via Perl's autoloading feature. So, the only methods that will not be available as system commands are the built-ins of the UNIVERSAL class, as well as "AUTOLOAD" and "DESTROY". So if you want to run a command named, for example, "can", you can use "run" to do that (note that "run" is not available via the OO interface).
AUTOLOAD
DESTROY
can
(Note on internals: The objects will actually not be of type IPC::Run3::Shell, they will be of type IPC::Run3::Shell::Autoload, so they have a clean namespace.)
IPC::Run3::Shell
IPC::Run3::Shell::Autoload
run
use IPC::Run3::Shell qw/ :run /; # NOTE the colon my $root = run "grep", "root", "/etc/passwd"; # run system command "grep" # Perl's built-in "grep" function remains unaffected
run mostly works like the other functions provided by this module (see "General"), with the difference that the name of the system command is the first argument to the function. This is useful, for example, if you don't want to override Perl built-in functions, or if you want to call a command with the full path.
It can be accessed either by specifying :run (note the colon!) to use (in this case it is affected by default options, if you specified them), or by calling IPC::Run3::Shell::run() directly (in this case it is not affected by default options, only by those that you give to IPC::Run3::Shell::run() directly).
:run
IPC::Run3::Shell::run()
make_cmd
use IPC::Run3::Shell qw/ :make_cmd /; # NOTE the colon my $pl = make_cmd 'perl', '-le'; # $pl->(ARGS) will run "perl -le ARGS" $pl->('print "Hello, World!"');
make_cmd is an advanced function which gives you full customizability over the generated Perl function and the executed system command. (It is actually the central function of this module.)
make_cmd takes optional options (hashref(s)), a command name (optionally with full pathname), optional command-line arguments, and returns a code ref which behaves as described in "General" and "Return Values" and, when called, executes the specified system command. Any additional arguments to the code ref become additional command-line arguments. The code ref may also be given additional options.
It can be accessed either by specifying :make_cmd (note the colon!) to use or by calling IPC::Run3::Shell::make_cmd() directly. Note that make_cmd and the functions it generates are only affected by the options passed directly to them, and never by default options given elsewhere (such as to use).
:make_cmd
IPC::Run3::Shell::make_cmd()
import_into
Lets you export functions from this module into any package of your choosing. The first argument is the package name, the rest are the usual arguments to import.
use IPC::Run3::Shell; IPC::Run3::Shell->import_into('Foo::Bar', 'echo'); { package Foo::Bar; echo("Hello, World"); }
The import_into method was added in v0.52.
Options can be set in several different places:
In the functional interface, default options can be given to use, when making aliases, or to "make_cmd". They are always specified as one or more hashrefs as the first argument(s). In the case of use, default options apply to all functions named in the use statement (with the exception of "make_cmd").
In the OO interface, the default options are passed to the constructor, and they then apply to all commands called on that object. They are not affected by use. Each object has its own independent set of default options. Note that the options to new are not specified as a hashref but as a plain hash.
In any case, options can always be specified or overridden by passing one or more hashrefs as the first and/or last argument(s) to any individual command.
Whenever multiple option sets are present, they are cumulative, with options specified later overriding the earlier ones. Note that there are a few options, such as "stdout", that cannot be un-set. Specifying unknown options triggers warnings and is strongly discouraged for forwards compatibility.
allow_exit
Either a single value or an arrayref of allowed exit values. Defaults to only zero (0). May also be set to the string "ANY" to mean any exit code is allowed. If set to undef, resets the option to its default value. Any other non-numeric values or an empty arrayref cause a warning. This option does not make much sense unless you have warnings enabled.
ANY
irs
Sets a local $/, see Perl's input record separator. Note that, like "stdout", irs => undef has a meaning, so the mere presence of this option in the options hash means the option is set.
local $/
irs => undef
chomp
Setting this boolean option causes the return value(s) of the function to be chomped. This option has no effect if you're using the option "stdout".
stdout_filter
Setting this option to a coderef causes this coderef to be called to filter the command's STDOUT. In list context, the filter is applied per line, while in scalar context, the filter is applied over the entire output. The coderef should modify $_ to modify the output string(s); its return value is ignored.
$_
For example, say you've got a command that outputs JSON, you can turn the output from that command into a Perl data structure like this (this is just a demonstration using echo that passes its input through):
echo
use JSON::PP qw/encode_json decode_json/; use IPC::Run3::Shell [ foo => { fail_on_stderr => 1, stdout_filter => sub { $_=decode_json($_) } }, qw/ echo / ]; my $foo = foo( encode_json( { test=>'123' } ) ); print $foo->{test}, "\n"; # prints "123"
This option has no effect if you're using the option "stdout". It is applied after the chomp operation, if that option is set.
The stdout_filter option was added in v0.58.
both
Setting this boolean option has a different effect depending on whether the function is called in list context or in scalar context - see also "Return Values". You cannot use this option together with "stdout", "stderr" or "fail_on_stderr".
In scalar context, the function returns the entire STDOUT plus STDERR of the executed command merged as one string. Note that the order of output on these two streams is not always well-defined; they may be mixed together. This is still useful if, for example, you know the command outputs on STDERR only and you want an easier way to capture it than the "stderr" option.
In list context, the function returns three values: the entire STDOUT as one string, the entire STDERR as one string, and the exit code of the command. This is somewhat similar to Capture::Tiny's capture used together with system (except that returns $? and not the exit code).
capture
system
$?
# call an external perl process for this example use IPC::Run3::Shell {both=>1}, "perl"; my ($stdout, $stderr, $exitcode) = perl {allow_exit=>123}, '-e', ' print "Foo\n"; warn "Bar\n"; exit 123 '; # $stdout gets "Foo\n", $stderr gets "Bar\n" and $exitcode==123
The both option was added in v0.52.
stdin
stdout
stderr
use IPC::Run3::Shell {chomp=>1}, qw/ cat find /; cat { stdin=>\"Hello, World!\n" }; # there will probably be some "permission denied" errors in /etc, # so we're allowing for an exit value of 1 and are capturing stderr my $rv = find { allow_exit=>[0,1] }, '/etc', '-name', 'passwd', '-print0', { stdout=>\my @files, stderr=>\my $err, irs=>"\0" }; print "find exited with $rv\n"; s/\0\z// for @files; # the "chomp" option doesn't work with "stdout" print "Found files: ".join(', ',@files)."\n"; print "## Warnings ##\n$err##\n" if $err=~/\S/;
These options give you powerful redirection of STDIN, STDOUT and STDERR of the executed command. The possible values are the same as in IPC::Run3; the following is a brief summary of the possible values:
inherit the filehandle from parent
\undef
\dev\null (or equivalent)
a filename
use that filehandle
read/write from/into that scalar (string)
read/write from/into that array, splitting output (not input) "lines" on $/
$/
For stdin, the sub is called repeatedly and its return values are fed to the process until the sub returns undef. For stdout and stderr, the sub is called with each "line" of output (split on $/ as usual).
Note that if you specify the option stdout, the function will always return the exit code of the command, and not its output - see "Return Values". stdin and stderr do not have this effect.
Since setting an option like stdout => undef has a meaning, the mere presence of any of the hash keys stdout or stderr in the options hash will be interpreted as the respective option being set. This also means that these options cannot be un-set by overriding them. So normally, you'll want to set these in the per-command options only, and not in the default options.
stdout => undef
Note this module does not make any guarantees about the capabilities of IPC::Run3 to redirect streams! For example, at this time of writing, IPC::Run3 does not support swapping of STDOUT and STDERR. Also, you will get unexpected results if you attempt to redirect STDOUT to STDERR and capture STDERR at the same time. (It is at the moment possible to redirect STDOUT to STDERR or vice versa if you don't touch the other.) If IPC::Run3's behavior changes in the future, that behavior will simply be passed through by this module.
You can't use the stderr option together with "fail_on_stderr". You can't use stdout or stderr together with "both".
fail_on_stderr
If this boolean option is set, any output to STDERR (except for a single $/) is considered a fatal error. This option may not be specified together with "stderr" or "both" (this is currently a fatal error). Note that turning this option on means that STDERR is always captured and not displayed.
show_cmd
Setting this option to a true value causes the command which is about to be executed to be printed to STDERR. You may also specify a filehandle / glob reference here to have the command printed there (e.g. show_cmd => \*STDOUT).
show_cmd => \*STDOUT
This is meant for debugging and/or user information only, as the output may not be safely escaped, so don't try to parse this output!
The following options may be specified and they will be passed through to IPC::Run3.
binmode_stdin
binmode_stdout
binmode_stderr
append_stdout
append_stderr
return_if_system_error
(Note this module turns this option on by default. It's recommended not to turn it off or you may get unexpected results in case of errors.)
In the following table, "OUT" and "ERR" refer to the STDOUT and STDERR of the executed command. The options "both", "stdout" and "stderr" are described in their respective sections.
OUT
ERR
| Void Context | Scalar Context | List Context | ==========+====================+====================+====================+ system() | | return undef | return empty list | fails | also generates optional warning that can be made fatal[1] | ----------+--------------------+--------------------+--------------------+ Bad exit | | output as below, but may be incomplete | code[2] | also generates optional warning that can be made fatal[1] | ==========+====================+====================+====================+ Default | OUT and ERR are | OUT is returned | OUT is returned as | | not redirected, | as one big string | a list of lines[4] | | they are passed | ERR not redirected by default[3] | ==========+ thru to script's +====================+====================+ Option | STDOUT and STDERR | return merged OUT | return a list of | "both" | | and ERR | (OUT,ERR,exitcode) | ==========+====================+====================+====================+ Option | OUT is redirected as requested (ERR handled separately) | "stdout" | Function always returns the exit code | ==========+==============================================================+ Option | ERR is redirected as requested (OUT handled separately) | "stderr" | | ==========+==============================================================+
You should always be able to inspect the exit code of the system command in Perl's $? variable. See the documentation of Perl's system() for an example of how to interpret $?.
system()
[1] This is described in "Warnings".
[2] If you're expecting a nonzero return code from a system command, see the option "allow_exit". Currently, a command being terminated due to a signal will generate a (optional/optionally fatal) warning and there is no equivalent of "allow_exit" for signals.
[3] STDERR can be handled via the options "stderr" or "fail_on_stderr".
[4] More exactly, the command's STDOUT output is split on the input record separator $/. Since that's the line separator by default, you'll normally get a list of the lines of the output, unless you change $/, which you can also do via the option "irs".
This module strives to be as portable as IPC::Run3. Before reporting bugs concerning portability, please first test if maybe it's a bug in IPC::Run3.
The tests for this module require that you have the perl executable in your PATH.
perl
PATH
If you notice any other portability issues, please let me know!
Of course, scripts using this module to call system commands will often not be portable, but that's simply the nature of scripts that call system commands.
A few things to be aware of:
As described in "Return Values", the functions provided by this module act differently depending on the context they are called in (void, scalar and list) and some of the options (e.g. "stdout" and "both"). While this is an attempt to provide a DWIM (Do What I Mean) interface, the difference in behavior might be subtle and lead to a bit of confusion.
Some functions are package methods, while others are not, i.e. don't mix up :: and ->. Examples: IPC::Run3::Shell->new(), IPC::Run3::Shell->import(), IPC::Run3::Shell->import_into(), IPC::Run3::Shell::run(), IPC::Run3::Shell::make_cmd(). (For the latter two, it's probably easier to use use IPC::Run3::Shell qw/:run :make_cmd/;.)
::
->
IPC::Run3::Shell->new()
IPC::Run3::Shell->import()
IPC::Run3::Shell->import_into()
use IPC::Run3::Shell qw/:run :make_cmd/;
If you plan on subclassing this module, note that the OO interface doesn't follow traditional Perl OO design patterns (e.g. new isn't actually part of the package of the object).
Even though they're usually discouraged and often rightly so, there are still a few rare cases when prototypes are useful. One way to give the Perl functions prototypes is with Scalar::Util's set_prototype:
set_prototype
# a silly example for demonstration purposes only :-) use IPC::Run3::Shell [ xuc => 'perl', '-e', 'print uc "@ARGV"', '--' ]; use Scalar::Util qw/set_prototype/; BEGIN { set_prototype \&xuc, '$'; } # make xuc() a unary operator # normally xuc() would take all the arguments, now it only takes one my $foo = join ",", xuc "a", "b", "c"; print $foo; # prints "A,b,c"
If you know the internals of Perl's use, you know that you can import functions at run-time too, as follows. For a description of the arguments of import please see "Arguments to use".
import
use IPC::Run3::Shell; IPC::Run3::Shell->import('echo'); echo("foo"); # NOTE parentheses are required here
If you like playing with globs, you could install the code refs created by "make_cmd" into your namespace manually. For example:
BEGIN { # not required, could do this at run-time require IPC::Run3::Shell; *foo = IPC::Run3::Shell::make_cmd('echo','foo'); } foo "bar"; # calls "echo foo bar"
Also, note the "import_into" function for more powerful exporting.
Warning: Make sure you understand the safety implications of this hint before you use it!
If you really want to invoke the shell and have your commands subject to its various interpolations and such, with all the possible safety implications (such as attackers injecting arbitrary commands into your shell!), you could do something like this:
use IPC::Run3::Shell [sh => 'sh', '-c']; sh 'echo $HOME >&2'; # prints _the_shell's_ $HOME to _its_ STDERR
If you really, really wanted to have Shell's autoloading behavior back, with all the possible safety implications (such as any typos leading to the execution of system commands!), you can do this (added in v0.56):
use IPC::Run3::Shell ':AUTOLOAD'; echo("foo","bar"); # calls "echo foo bar" via autoloaded "echo"
Thank you to LanX for suggesting :AUTOLOAD and dolmen for a documentation suggestion.
:AUTOLOAD
IPC::Run3::Shell::CLIWrapper - makes it easier to build wrappers for command-line interfaces. This module was added in v0.58.
Larry Wall and others wrote the original Shell in 1994, with various contributions throughout the years. Shell is Copyright (c) 2005 by the Perl 5 Porters.
IPC::Run3 - I chose this module from the plethora of similar modules for several reasons: it handles all three STDOUT, STDERR and STDIN; it runs on Win32; it allows avoiding the shell; no non-core dependencies (except on Windows); and its test statistics are currently nearly perfect. While it doesn't support a few features that some of the more advanced modules do, like timeouts or interactively communicating with the subprocess, I hope it'll be enough for the majority of cases.
Shell::Tools - reduce boilerplate in Perl shell scripts, also includes easy access to this module!
Capture::Tiny - yet another capturing module. Its documentation includes a list of lots more such modules. There is also System::Sub, which does something quite similar to this module.
Env - tie environment variables to Perl variables, similar in spirit to this module
Copyright (c) 2014-2020 Hauke Daempfling (haukex@zero-g.net).
This library is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself.
For more information see the Perl Artistic License, which should have been distributed with your copy of Perl. Try the command "perldoc perlartistic" or see http://perldoc.perl.org/perlartistic.html.
perldoc perlartistic
To install IPC::Run3::Shell, copy and paste the appropriate command in to your terminal.
cpanm
cpanm IPC::Run3::Shell
CPAN shell
perl -MCPAN -e shell install IPC::Run3::Shell
For more information on module installation, please visit the detailed CPAN module installation guide.