=head1 NAME
PDL::FAQ - Frequently asked questions about PDL
=head1 VERSION
Current FAQ version: 1.008
=head1 DESCRIPTION
This is version 1.008 of the PDL FAQ, a collection of frequently
asked questions about PDL - the Perl Data Language.
=head1 ABOUT THIS DOCUMENT
=head2 Q: 1.1 Where to find this document
You can find the latest version of this document at
=head2 Q: 1.2 How to contribute to this document
This is a considerably reworked version of the PDL FAQ. As
such many errors might have crept in and many updates might
not have made it in. You are explicitly encouraged to let us
know about questions which you think should be answered in
this document but currently aren't.
Similarly,
if
you think parts of this document are
unclear, please
tell
the FAQ maintainer about it. Where
a specific answer is taken in full from someones posting
the authorship should be indicated, let the FAQ maintainer
know
if
it isn't. For more general information explicit
acknowledgment is not made in the text, but rather there
is an incomplete list of contributors at the end of this
document. Please contact the FAQ maintainer
if
you feel
hard done by.
Send your comments, additions, suggestions or corrections
to the PDL mailing list at pdl-general
@lists
.sourceforge.net.
See Q: 3.2 below
for
instructions on how to
join
the mailing
lists.
=head1 GENERAL QUESTIONS
=head2 Q: 2.1 What is PDL ?
PDL stands
for
I<Perl Data Language> . To
say
it
with
the words of Karl Glazebrook,
initiator of the PDL project:
The PDL concept is to give standard perl5 the ability
to COMPACTLY store and SPEEDILY manipulate the large
N-dimensional data sets which are the bread and butter
of scientific computing. e.g.
$x
=
$y
+
$z
can add two
2048x2048 images in only a fraction of a second.
It provides tons of useful functionality
for
scientific and numeric analysis.
For readers familiar
with
other scientific data evaluation packages it
may be helpful to add that PDL is in many respects similar to IDL,
MATLAB and similar packages. However, it tries to improve on a number
of issues which were perceived (by the authors of PDL) as shortcomings
of those existing packages.
=head2 Q: 2.2 Who supports PDL? Who develops it?
PDL is supported by its users. General informal support
for
PDL
is provided through the PDL mailing list (pdl-general
@lists
.sourceforge.net ,
see below).
As a Perl extension (see Q: 2.5 below) it is devoted to the idea of free and
open
development put forth by the Perl community. PDL was and is being
actively developed by a loosely knit group of people
around
the world who
coordinate their activities through the PDL development mailing list
(pdl-devel
@lists
.sourceforge.net , see Q: 3.2 below). If you would like
to
join
in the ongoing efforts to improve PDL please
join
this list.
=head2 Q: 2.3 Why yet another Data Language ?
There are actually several reasons and everyone should decide
for
themselves which are the most important ones:
=over 4
=item *
PDL is
"free software"
. The authors of PDL think
that this concept
has
several advantages: everyone
has
access to the sources -> better debugging, easily
adaptable to your own needs, extensible
for
your purposes,
etc... In comparison
with
commercial packages such as MATLAB
and IDL this is of considerable importance
for
workers who
want to
do
some work at home and cannot afford the
considerable cost to buy commercial packages
for
personal
use
.
=item *
PDL is based on a powerful and well designed scripting
language: Perl. In contrast to other scientific/numeric data
analysis languages it
has
been designed using the features of
a proven language instead of having grown into existence from
scratch. Defining the control structures
while
features were
added during development leads to languages that often appear
clumsy and badly planned
for
most existing packages
with
similar scope as PDL.
=item *
Using Perl as the basis a PDL programmer
has
all the
powerful features of Perl at their hand, right from the
start. This includes regular expressions, associative arrays
(hashes), well designed interfaces to the operating
system
,
network, etc. Experience
has
shown that even in mainly
numerically oriented programming it is often extremely handy
if
you have easy access to powerful semi-numerical or
completely non-numerical functionality as well. For example,
you might want to offer the results of a complicated
computation as a server process to other processes on the
network, perhaps directly accepting input from other
processes on the network. Using Perl and existing Perl
extension packages things like this are
no
problem at all
(and it all will fit into your
"PDL script"
).
=item *
Extremely easy extensibility and interoperability as PDL is
a Perl extension; development support
for
Perl extensions is
an integral part of Perl and there are already numerous
extensions to standard Perl freely available on the network.
=item *
Integral language features of Perl (regular expressions,
hashes, object modules) immensely facilitated development
and implementation of key concepts of PDL. One of the most
striking examples
for
this point is probably L<PDL::PP>
(see Q: 6.16 below), a code generator/parser/pre-processor that
generates PDL functions from concise descriptions.
=item *
None of the existing data languages follow the Perl language
rules, which the authors firmly believe in:
=over 4
=item *
TIMTOWTDI: There is more than one way to
do
it.
Minimalist languages are interesting
for
computer
scientists, but
for
users, a little bit of redundancy
makes things wildly easier to cope
with
and allows
individual programming styles - just as people speak in
different ways. For many people this will undoubtedly be
a reason to avoid PDL ;)
=item *
Simple things are simple, complicated things possible:
Things that are often done should be easy to
do
in the language,
whereas seldom done things shouldn't be too cumbersome.
=back
All existing languages violate at least one of these rules.
=item *
As a project
for
the future PDL should be able to
use
super
computer features, e.g. vector capabilities/parallel processing,
GPGPU acceleration. This will probably be achieved by having
L<PDL::PP> (see Q: 6.16 below) generate appropriate code
on such architectures to exploit these features.
=item *
[ fill in your personal 111 favourite reasons here...]
=back
=head2 Q: 2.4 What is PDL good
for
?
Just in case you
do
not yet know what the main features of PDL are and
what one could
do
with
them, here is a (necessarily selective) list of
key features:
PDL is well suited
for
matrix computations, general handling
of multidimensional data, image processing, general scientific
computation, numerical applications. It supports I/O
for
many
popular image and data formats, 1D (line plots), 2D (images)
and 3D (volume visualization, surface plots via OpenGL -
for
instance implemented using Mesa or video card OpenGL drivers),
graphics display capabilities and implements many numerical and
semi-numerical algorithms.
Through the powerful pre-processor it is also easy to interface Perl
to your favorite C routines, more of that further below.
=head2 Q: 2.5 What is the connection between PDL and Perl ?
PDL is a Perl5 extension
package
. As such it needs an existing Perl5
installation (see below) to run. Furthermore, much of PDL is written in
Perl (+ some core functionality that is written in C). PDL programs
are (syntactically) just Perl scripts that happen to
use
some of the
functionality implemented by the
package
"PDL"
.
=head2 Q: 2.6 What
do
I need to run PDL on
my
machine ?
Since PDL is just a Perl5
package
you need first of
all an installation of Perl5 on your machine. As of this
writing PDL requires version 5.10.x of perl, or higher. More
information on where and how to get a Perl installation
and at many CPAN sites (
if
you
do
not know what I<CPAN>
is, check the answer to the
next
question).
To build PDL you also need a working C compiler, support
don't have a compiler there might be a binary distribution
available, see
"Binary distributions"
below.
If you can (or cannot) get PDL working on a new (previously
unsupported) platform we would like to hear about it. Please,
report your success/failure to the PDL mailing list at
pdl-general
@lists
.sourceforge.net . We will
do
our
best to
assist you in porting PDL to a new
system
.
=head2 Q: 2.7 Where
do
I get it?
PDL is available as source distribution in the
I<Comprehensive Perl Archive Network> (or CPAN) and from the
The CPAN archives contains not only the PDL distribution but
also just about everything
else
that is Perl-related. CPAN is
mirrored by dozens of sites all over the world. The main site
latest version can also be downloaded from there.
=head2 Q: 2.8 What
do
I have to pay to get PDL?
We are delighted to be able to give you the nicest possible
answer on a question like this: PDL is
*free
software* and all
sources are publicly available. But still, there are some
copyrights to comply
with
. So please,
try
to be as nice as we
(the PDL authors) are and
try
to comply
with
them.
Oh,
before
you think it is
*completely
* free: you
have to invest some
time
to pull the distribution from the net,
compile and install it and (maybe)
read
the manuals.
=head1 GETTING HELP/MORE INFORMATION
=head2 Q: 3.1 Where can I get information on PDL?
The complete PDL documentation is available
with
the PDL distribution.
Use the command C<perldoc PDL> to start learning about PDL.
The easiest way by far, however, to get familiar
with
PDL is to
use
the PDL on-line help facility from within the PDL
shell, C<perldl> Just type C<perldl> at your
system
prompt. Once you are inside the
C<perldl> shell type C<help> . Using the C<help> and C<apropos> commands
inside the shell you should be able to find the way round the
documentation.
Even better, you can immediately
try
your newly acquired
knowledge about PDL by issuing PDL/Perl commands directly at the command
line. To illustrate this process, here is the record of a typical
C<perldl> session of a PDL beginner (lengthy output is only symbolically
reproduced in braces ( <... ...> ) ):
unix> perldl
pdl> help
< ... help output ... >
pdl> help PDL::QuickStart
< ... perldoc page ... >
pdl>
$x
= pdl (1,5,7.3,1.0)
pdl>
$y
= sequence float, 4, 4
pdl> help inner
< ... help on the
'inner'
function ... >
pdl>
$c
= inner
$x
,
$y
pdl> p
$c
[22.6 79.8 137 194.2]
For further sources of information that are accessible through the
Internet see
next
question.
=head2 Q: 3.2 Are there other PDL information sources on the Internet?
First of all,
for
all purely Perl-related questions there are
tons of sources on the net. Good points to start are
The PDL home site can be accessed by pointing your web browser to
=over 4
=item *
PDL distributions
=item *
On-line documentation
=item *
Pointers to an HTML archive of the PDL mailing lists
=item *
A list of platforms on which PDL
has
been successfully tested.
=item *
News about recently added features, ported libraries, etc.
=item *
Name of the current pumpkin holders
for
the different PDL modules (
if
you want to know what that means you better had a look at the web
pages).
=back
If you are interested in PDL in general you can
join
the pdl-general mailing
list. This is a forum to discuss programming issues in PDL, report bugs,
seek
assistance
with
PDL related problems, etc.
If you are interested in all the technical details of the ongoing PDL
development you can
join
the pdl-devel mailing list.
Subscription and current archive links to both mailing lists can be
Cross-posting between these lists should be avoided
unless
there is a
I<very> good reason
for
doing that.
The PDL project, begun in the late 1990s,
has
undergone considerable evolution
since that
time
, and the support
for
it
has
as well. Thus mailing-list
archives are in several places. Originally pdl-general was called
'perldl'
,
and pdl-devel was called
'pdl-porters'
.
|Time Period | URL |
|------------|-------------------------------------------------------|
|--------------------------------------------------------------------|
=head2 Q: 3.3 What is the current version of PDL ?
As of this writing (FAQ version 1.008 of 21 May 2017) the latest stable version
is 2.018. The latest stable version should always be available from a CPAN
mirror site near you (see L<Question 2.7|
"Q: 2.7 Where do I get it?"
>
for
info on where to get PDL).
The most current (possibly unstable) version of PDL can be obtained
from the Git repository, see L<Question 4.10|
"Q: 4.9 How do I get PDL via Git?"
>
and periodic CPAN developers releases of the Git code will be made
for
testing
purposes and more general availability.
=head2 Q: 3.4 How can PDL-2.2 be older than PDL-2.007?
Over its development, PDL
has
used both a single floating point version
number (from the versions 1.x through 2.005) at which point it switched
to a dotted triple version
for
2.1.1 onward---EXCEPT
for
version 2.2
which came out which should have been 2.2.0. To simplify and unify
things, PDL
has
reverted to a single float version representation
with
PDL-2.006. This can cause dependency problems
for
modules that set a
minimum PDL version of 2.2. The work
around
it, note that all extant
PDL releases have version numbers greater than 2.2.1 so that using
0 as the minimum version will work.
=head2 Q: 3.5 I want to contribute to the further development of PDL. How can I help?
Two ways that you could help almost immediately are (1) participate
in CPAN Testers
for
PDL and related modules, and (2) proofreading and
clarifying the PDL documentation so that it is most useable
for
PDL
users, especially new users.
To participate in CPAN Testers and contribute test reports, the page
starting
for
either C<CPAN> or C<CPANPLUS> users.
If you have a certain project in mind you should check
if
somebody
else
is already working on it or
if
you could benefit from existing
modules. Do so by posting your planned project to the PDL developers
mailing list at pdl-devel
@lists
.sourceforge.net . See the subscription
instructions in L<Question 3.2|"Q: 3.2 Are there other PDL information
sources on the Internet?">. We are always looking
for
people to
write
code and/or documentation. The L<Developer Guide|PDL::DeveloperGuide>
should get you started.
=head2 Q: 3.6 I think I have found a bug in the current version of PDL. What shall I
do
?
First, make sure that the bug/problem you came across
has
not already been
dealt
with
somewhere
else
in this FAQ. Secondly, you can check the
searchable archive of the PDL mailing lists to find whether
this bug
has
already been discussed. If you still haven't found
any explanations you can post a bug report to pdl-general
@lists
.sourceforge.net ,
file in the PDL distribution
for
what information to include. If
you are unsure, discussions via the perldl mailing list can be
most helpful.
=head1 INSTALLATION
=head2 Q: 4.1 I have problems installing PDL. What shall I
do
?
First make sure you have
read
the file F<INSTALL> in the distribution.
This contains a list of common problems which are unnecessary to repeat
here.
Next, check the file F<perldl.conf> to see
if
by editing the
configuration options in that file you will be able to successfully
build PDL. Some of the modules need additional software installed,
please refer to the file F<DEPENDENCIES>
for
further details. Make sure
to edit the location of these packages in perldl.conf
if
you have them
in non-standard locations.
N.B. Unix shell specific: If you would like to save an edited perldl.conf
for
future builds just copy it as F<~/.perldl.conf> into your home directory
where it will be picked up automatically during the PDL build process.
Also, check
for
another, pre-existing version of PDL on the build
system
. Multiple PDL installs in the same PATH or
@INC
can cause
puzzling test or build failures.
If you still can't make it work properly please submit a bug
report including detailed information on the problems you
encountered to the perldl mailing list ( pdl-general
@lists
.sourceforge.net ,
see also above). Response is often rapid.
=head2 Q: 4.2 Are there configuration files
for
PDL I have to edit?
Most users should not have to edit any configuration files manually.
However, in some cases you might have to supply some information
about awkwardly placed include files/libraries or you might want
to explicitly disable building some of the optional PDL modules.
Check the files F<INSTALL> and F<perldl.conf>
for
details.
If you had to manually edit F<perldl.conf> and are happy
with
the
results you can keep the file handy
for
future
reference. Place it in F<~/.perldl.conf> where it will be picked
up automatically or
use
C<perl Makefile.PL PDLCONF=your_file_name>
next
time
you build PDL.
=head2 Q: 4.3 Do I need other software
for
successful operation?
For the basic PDL functionality you don't need any
additional software. However, some of the optional PDL
modules included in the distribution (notably most graphics
and some I/O modules)
require
certain other
libraries/programs to be installed. Check the file
F<DEPENDENCIES> in the distribution
for
details and directions
on how to get these.
=head2 Q: 4.4 How can I install PDL in a non-standard location?
To install PDL in a non-standard location,
use
the INSTALL_BASE
option in the C<perl Makefile.PL> configure step. For example,
C<perl Makefile.PL INSTALL_BASE=/mydir/perl5> will configure PDL
to install into the tree rooted at C</mydir/perl5>. For more
details see L<perlfaq8/
"How do I keep my own module/library directory?"
>
and subsequent sections. Another alternative is to
use
L<
local
::lib>
to
do
the heavy lifting
for
the needed configuration.
=head2 Q: 4.5 How can I force a completely clean installation?
To guarantee a completely clean installation of PDL, you will need
to first
delete
the current installation files and folders. These
will be all directories named C<PDL> in the Perl C<
@INC
> path,
files named C<
*Pdlpp
*> in any C<Inline> directories, and the
programs C<pdl, pdldoc, perldl, and pptemplate>. Then just build
and install as usual. This is much easier to keep track of
if
you
always install C<PDL> into a non-standard location. See Q: 4.4 above.
=head1 BINARY DISTRIBUTIONS
=head2 Q: 4.5 What binary distributions are available?
Information about binary distributions of PDL can be found on
of PDL
for
Linux (RedHat and Debian), FreeBSD, Mac OS X and Windows,
though they might not be the most recent version.
If someone is interested in providing binary distributions
for
other
architectures, that would be very welcome. Let us know on the
pdl-devel
@lists
.sourceforge.net mailing list. Also check your Linux
distribution's
package
manager as many now include PDL. PPMs
for
win32 versions (both 32bit and 64bit) are also available.
=head2 Q: 4.6 Does PDL run on Linux? (And what about packages?)
Yes, PDL does run on Linux and indeed much of the development
links to packages
for
some of the major distributions. Also
check your distribution's
package
manager (yum, apt, urpmi, ...)
as PDL is now found by many of these.
=head2 Q: 4.7 Does PDL run under Windows?
PDL builds fine on Win32 using MinGW or Microsoft compilers. See
the F<win32/INSTALL> file in the PDL source distribution
for
details.
Other compilers have not been tested--input is welcome. There is
also a distribution of PDL through ActiveState's ppm, though it
might not always be the latest version. PDL-2.018 builds out of
the box on Strawberry Perl and ActiveState Perl and there are
distributions of Strawberry Perl
with
bundled PDL
=head1 CVS, GIT, AND ON-GOING DEVELOPMENT
=head2 Q: 4.8 Can I get PDL via CVS?
No. PDL development was conducted
with
a CVS repository from December
1999 to April 2009. In April 2009 the project switched to the
=head2 Q: 4.9 How
do
I get PDL via Git?
Assume you have Git installed on your
system
and want to download the
project source code into the directory C<PDL>. To get
read
-only access
to the repository, you type at the command line
If you wish to submit changes to PDL, you should
"fork"
the repository
normal fashion.
To become an official PDL developer, you will need to be added to the
GitHub
"PDLPorters"
organisation.
For official PDL developers, to get
read
/
write
access to the repository
type at the command line
They can still
use
their own
fork
; at least one active developer uses
that model rather than branches on the main repository.
=head2 Q: 4.10 I had a problem
with
the Git version, how
do
I check
if
someone
has
submitted a patch?
somebody
has
submitted a pull request related to your problem.
In addition,
if
you are not subscribing to the mailing list,
check the archive of the C<pdl-devel> and C<pdl-general> mailing lists.
See L<Question 3.2|
"Q: 3.2 Are there other PDL information sources on the Internet?"
>
for
details.
=head2 Q: 4.11 I have gotten developer access to Git, how
do
I upload
my
changes?
The first thing you should
do
is to
read
the Git documentation and
learn the basics about Git. There are many sources available online.
It is very important that you
use
Git
"best practice"
,
with
branches,
but fortunately this is very easy! Here are the basics.
Make sure your copy is up to date
with
the main repo:
git checkout master
git pull --rebase
Make a branch:
git checkout -b mybranch-name
Commit your changes locally:
git add <file1> <file2> ...
git commit
or combine these two
with
:
git commit -a
Test the PDL
before
you
push
it to the main repository. If the
code is broken
for
you, then it is most likely broken
for
others.
Luckily, the rest of this process will test that automatically to help
you
catch
such errors.
Then update the shared repository
with
your changes:
git
push
-u origin mybranch-name
This will still leave your changes on a branch, but this is good. Now
ask you whether you want to make a
"pull request"
- you
do
. Follow the
prompts. This will then initiate the automated
"continuous integration"
tests, on Linux and Windows,
with
various versions of Perl,
with
various
compilers. You will also want to get at least one other developer to
review your changes.
Once this review process is successfully completed, you can merge your
changes to the master branch!
=head1 PDL JARGON
=head2 Q: 5.1 What is broadcasting (is PDL a newsreader)?
Until 2.075,
"threading"
was used to refer to two ideas,
but that ambiguity
has
now been resolved by using the now (as of
2022) industry-standard term
"broadcasting"
for
the vectorisation /
array-programming concept.
=over 4
=item *
When mentioned in the F<INSTALL> directions and possibly
during the build process we have the usual computer science
meaning of multi-threading in mind (useful mainly on
multiprocessor machines or clusters), currently (as of 2.074)
POSIX threads (see L<PDL::ParallelCPU>).
=item *
PDL broadcasting of operations on ndarrays (as mentioned in the
indexing docs) is the iteration of a basic operation over
appropriate
sub
-slices of ndarrays, e.g. the inner product
C<inner
$x
,
$y
> of a (3) pdl C<
$x
> and a (3,5,4) pdl
C<
$y
> results in a (5,4) ndarray where
each
value is the result of an inner product of the (3) pdl
with
a
(3)
sub
-slice of the (3,5,4) ndarray. For details check
L<PDL::Indexing>
=back
The connection is that broadcasting divides up independent operations
that can be done in parallel.
=head2 Q: 5.2 What is an ndarray?
Well, PDL
scalar
variables (which are instances of a particular class
of Perl objects, i.e. blessed thingies (see C<perldoc perlobj> )) are
in common PDL parlance often called I<ndarrays> (
for
example, check the
mailing list archives). Err, clear? If not, simply
use
the term
I<ndarray>
when
you refer to a PDL variable (an instance of a PDL
object as you might remember) regardless of what actual data the PDL
variable contains.
=head1 TECHNICAL QUESTIONS
=head2 Q: 6.1 What is perldl?
Sometimes C<perldl> is used as a synonym
for
PDL. Strictly
speaking, however, the name C<perldl> is reserved
for
the
little shell that comes
with
the PDL distribution and is
supposed to be used
for
the interactive prototyping of PDL
scripts. For details check L<perldl>.
=head2 Q: 6.2 How
do
I get on-line help
for
PDL?
Just type C<help> (shortcut =
"?"
) at the C<perldl> shell
prompt and proceed from there. Another useful command
is the C<apropos> (shortcut =
"??"
) command.
Also
try
the C<demo> command in the C<perldl>
shell
if
you are new to PDL.
=head1 MANIPULATION OF NDARRAYS
=head2 Q: 6.3 I want to access the third element of a pdl but
$x
[2] doesn't work ?!
See answer to the
next
question why the normal Perl array syntax doesn't
work
for
ndarrays.
=head2 Q: 6.4 The docs
say
ndarrays are some kind of array. But why doesn't the Perl array syntax work
with
ndarrays then ?
OK, you are right in a way. The docs
say
that ndarrays can be
thought of arrays. More specifically, it says (
L<PDL::QuickStart> ):
I find
when
using the Perl Data Language it is most useful
to think of standard Perl
@x
variables as
"lists"
of generic
"things"
and PDL variables like
$x
as
"arrays"
which can be
contained in lists or hashes.
So,
while
ndarrays can be thought of as some kind of
multi-dimensional array they are
B< not> arrays in the Perl sense. Rather,
from the point of view of Perl they are some special class
(which is currently implemented as an opaque pointer to some
stuff in memory) and therefore need special functions (or
'methods'
if
you are using the OO version) to access
individual elements or a range of elements. The
functions/methods to check are
C<at> / C<set> (see
L<the section
'Sections'
in PDL::QuickStart|PDL::QuickStart/
"Sections"
> ) or the powerful
C<slice> function and friends (see L<PDL::Slices> and
L<PDL::Indexing> and especially L<PDL::NiceSlice> ).
Finally, to confuse you completely, you can have Perl arrays
of ndarrays, e.g. C<
$spec
[3]> can refer to a pdl representing ,e.g,
a spectrum, where C<
$spec
[3]> is the fourth element of the Perl
list (or array ;) C<
@spec
> . This may be confusing but is
very useful !
=head2 Q: 6.5 How
do
I concatenate ndarrays?
Most people will
try
to form new ndarrays from old ndarrays
using some variation over the theme:
C<
$x
= pdl([
$y
, 0, 2])>. This does work, but may not work in the way
that a novice user would expect. (If C<
$y
>
has
N dimensions then C<
$x
>
will have N+1 dimensions.) Other ways to concatenate ndarrays are
to
use
the functions C<cat>, C<append>, and C<glue>. Similarly you can
split
ndarrays using the command C<dog>.
=head2 Q: 6.6 Sometimes I am getting these strange results
when
using inplace operations?
This question is related to the C<inplace> function. From the
documentation (see L<PDL::QuickStart>):
Most functions, e.g.
log
(),
return
a result which is a
transformation of their argument. This makes
for
good
programming practice. However many operations can be done
"in-place"
and this may be required
when
large arrays are in
use
and memory is at a premium. For these circumstances the
operator inplace() is provided which prevents the extra copy
and allows the argument to be modified. e.g.:
$x
=
log
(
$array
);
log
( inplace(
$bigarray
) );
And also from the doc !!:
Obviously
when
used
with
some functions which can not be
applied in situ (e.g. convolve()) unexpected effects may
occur!
=
for
comment Check the list of PDL functions at the end of PDL.pod which points out
C<inplace>-safe functions. No longer in PDL.pod, need to fix!!
=head2 Q: 6.7 What is this strange usage of the string concatenation operator C<.=> in PDL scripts?
See
next
question on assignment in PDL.
=head2 Q: 6.8 Why are there two different kinds of assignment in PDL ?
This is caused by the fact that currently the assignment
operator C<=> allows only restricted
overloading. For some purposes of PDL it turned out to be
necessary to have more control over the overloading of an
assignment operator. Therefore, PDL peruses the operator
C<.=>
for
certain types of assignments.
=head2 Q: 6.9 How
do
I set a set of
values
in an ndarray?
In Perl 5.6.7 and higher this assignment can be made
using lvalue subroutines:
pdl>
$x
= sequence(5); p
$x
[0 1 2 3 4]
pdl>
$x
->slice(
'1:2'
) .= pdl([5,6])
pdl> p
$x
[0 5 6 3 4]
PDL also supports a more
matrix-like slice syntax via the L<PDL::NiceSlice> module:
pdl>
$x
(1:2) .= pdl([5,6])
pdl> p
$x
[0 5 6 3 4]
With versions of Perl prior to 5.6.7 B<or
when
running under
the perl debugger> this
has
to be done using a temporary variable:
pdl>
$x
= sequence(5); p
$x
[0 1 2 3 4]
pdl>
$tmp
=
$x
->slice(
'1:2'
); p
$tmp
;
[1 2]
pdl>
$tmp
.= pdl([5, 6]);
pdl> p
$x
[0 5 6 3 4]
This can also be made into one expression, which is often
seen in PDL code:
pdl> (
$tmp
=
$x
->slice(
'1:2'
)) .= pdl([5,6])
pdl> p
$x
[0 5 6 3 4]
=head2 Q: 6.10 Can I
use
an ndarray in a conditional expression?
Yes you can, but not in the way you probably tried first. It
is not possible to
use
an ndarray directly in a conditional
expression since this is usually poorly
defined
. Instead PDL
has
two very useful functions:
C<any> and C<all> . Use these to test
if
any or
all elements in an ndarray fulfills some criterion:
pdl>
$x
=pdl ( 1, -2, 3);
pdl>
print
'$x has at least one element < 0'
if
(any
$x
< 0);
$x
has
at least one element < 0
pdl>
print
'$x is not positive definite'
unless
(all
$x
> 0);
$x
is not positive definite
=head2 Q: 6.11 Logical operators and ndarrays -
'||'
and
'&&'
don't work!
It is a common problem that you
try
to make a mask array or something
similar using a construct such as
$mask
= which(
$ndarray
> 1 &&
$ndarray
< 2);
This B< does not> work! What you are looking
for
is the B< bitwise>
logical operators
'|'
and
'&'
which work on an element-by-element
basis. So it is really very simple: Do not
use
logical operators on
multi-element ndarrays since that really doesn't make sense, instead
write
the example as:
$mask
= which(
$ndarray
> 1 &
$ndarray
< 2);
which works correctly.
=head1 ADVANCED TOPICS
=head2 Q: 6.12 What is a null pdl ?
=
for
comment Is Q: 6.12 up-to-date
with
null and empty pdls?
C<null> is a special token
for
'empty ndarray'
. A null pdl
can be used to flag to a PDL function that it should create
an appropriately sized and typed ndarray. I<Null> ndarrays
can be used in places where a PDL function expects an
I<output> or I<temporary> argument. I<Output> and
I<temporary> arguments are flagged in the
I<signature> of a PDL function
with
the C<[o]> and
C<[t]> qualifiers (see
next
question
if
you don't know what
the I<signature> of a PDL function is). For example, you
can invoke the C<sumover> function as follows:
sumover
$x
,
$y
=null;
which is equivalent to
$y
= sumover
$x
;
If this seems still a bit murky check
L<PDL::Indexing> and
L<PDL::PP>
for
details about calling
conventions, the I<signature> and
I<broadcasting> (see also below).
=head2 Q: 6.13 What is the signature of a PDL function ?
The I<signature> of a function is an important concept in PDL.
Many (but not all) PDL function have a I<signature> which
specifies the arguments and their (minimal) dimensionality. As
an example, look at the signature of the C<maximum> function:
'a(n); [o] b;'
this says that C<maximum> takes two arguments, the first of which is
(at least) one-dimensional
while
the second one is zero-dimensional and
an I<output> argument (flagged by the C<[o]> qualifier). If the function
is called
with
ndarrays of higher dimension the function will be repeatedly
called
with
slices of these ndarrays of appropriate dimension(this is called
I<broadcasting> in PDL).
For details and further explanations consult
L<PDL::Indexing> and L<PDL::PP> .
=head2 Q: 6.14 How can I subclass (inherit from) ndarrays?
The short answer is:
read
L<PDL::Objects> (e.g. type
C<help PDL::Objects> in the I<perldl> shell).
The longer answer (extracted from L<PDL::Objects> ):
Since a PDL object is an opaque reference to a C struct, it is not
possible to extend the PDL class by e.g. extra data via
sub
-classing
(as you could
do
with
a hash based Perl object). To circumvent this
problem PDL
has
built-in support to extend the PDL class via the
I<
has
-a> relation
for
blessed hashes. You can get the I<HAS-A> to
behave like I<IS-A> simply in that you assign the PDL
object to the attribute named C<PDL> and
redefine the method initialize(). For example:
@FOO::ISA
=
qw(PDL)
;
sub
initialize {
my
$class
=
shift
;
my
$self
= {
creation_time
=>
time
(),
PDL
=> PDL->null,
};
bless
$self
,
$class
;
}
For another example check the script
F<t/subclass.t> in the PDL distribution.
=head2 Q: 6.15 What on earth is this dataflow stuff ?
Dataflow is an experimental project that you don't need to concern
yourself
with
(it should not interfere
with
your usual programming).
However,
if
you want to know, have a look at
L<PDL::Dataflow> . There
are applications which will benefit from this feature (and it is already
at work behind the scenes).
=head2 Q: 6.16 What is PDL::PP?
Simple answer: PDL::PP is both a glue between external
libraries and PDL and a concise language
for
writing PDL
functions.
Slightly longer answer: PDL::PP is used to compile very
concise definitions into XSUB routines implemented in C that
can easily be called from PDL and which automatically support
broadcasting, dataflow and other things without you having to
worry about it.
For further details check L<PDL::PP> and the
section below on L<Extensions of PDL|
"EXTENSIONS OF PDL"
>.
=head2 Q: 6.17 What happens
when
I have several references to the same PDL object in different variables (cloning, etc?) ?
ndarrays behave like Perl references in many respects. So
when
you
say
$x
= pdl [0,1,2,3];
$y
=
$x
;
then both
$y
and
$x
point to the same object, e.g. then saying
$y
++;
will
*not
* create a copy of the original ndarray but just increment in
place, of which you can convince yourself by saying
print
$x
;
[1 2 3 4]
This should not be mistaken
for
dataflow which connects several
*different
* objects so that data changes are propagated between
the so linked ndarrays (though, under certain circumstances, dataflowed
ndarrays can share physically the same data).
It is important to keep the
"reference nature"
of ndarrays in mind
when
passing ndarrays into subroutines. If you modify the input
ndarrays you modify the original argument, I<not> a copy of it. This
is different from some other array processing languages but makes
for
very efficient passing of ndarrays between subroutines. If you
do
not want to modify the original argument but rather a copy
of it just create a copy explicitly (this example also demonstrates
how to properly check
for
an I<explicit> request to process
inplace, assuming your routine can work inplace):
sub
myfunc {
my
$pdl
=
shift
;
if
(
$pdl
->is_inplace) {
$pdl
->set_inplace(0)
}
else
{
$pdl
=
$pdl
->copy
}
$pdl
->set(0,0);
return
$pdl
;
}
=head1 MISCELLANEOUS
=head2 Q: 6.18 What I/O formats are supported by PDL ?
The current versions of PDL already support quite a number of different
I/O formats. However, it is not always obvious which module implements
which formats. To help you find the right module
for
the
format
you
require
, here is a short list of the current list of I/O formats and
a hint in which module to find the implementation:
=over 4
=item *
A home brew fast raw (binary) I/O
format
for
PDL is implemented by the
L<FastRaw|PDL::IO::FastRaw> module
=item *
The L<FlexRaw|PDL::IO::FlexRaw> module implements generic methods
for
the input and output of `raw' data arrays. In particular, it is
designed to
read
output from FORTRAN 77 UNFORMATTED files and the
low-level C C<
write
> function, even
if
the files are compressed or gzipped.
It is possible that the FastRaw functionality will be included in the
FlexRaw module at some
time
in the future.
=item *
FITS I/O is implemented by the C<wfits>/C<rfits> functions in L<PDL::IO::FITS> .
=item *
ASCII file I/O in various formats can be achieved by using the
C<rcols> and C<rgrep> functions, also in L<PDL::IO::Misc> .
=item *
L<PDL::IO::Pic> implements an interface to the
NetPBM/PBM+ filters to
read
/
write
several popular image formats; also
supported is output of image sequences as MPEG movies, animated GIFs
and a wide variety of other video formats.
=item *
On CPAN you can find the L<PDL::NetCDF> module that works
with
PDL 2.007.
=back
For further details consult the more detailed list in the L<PDL::IO>
documentation or the documentation
for
the individual modules.
=head2 Q: 6.19 How can I stack a set of 2D arrays (images) into a 3D ndarray?
Assuming all arrays are of the same size and in some
format
recognized by
C<rpic> (see L<PDL::IO::Pic> ) you could
say
:
@names
=
qw/name1.tif .... nameN.tif/
;
$dummy
= PDL->rpic(
$names
[0]);
$cube
= PDL->zeroes(
$dummy
->type,
$dummy
->dims,
$#names
+1); # make 3D ndarray
for
(0..
$#names
) {
(
$tmp
=
$cube
->slice(
":,:,($_)"
)) .= PDL->rpic(
$names
[
$_
]);
}
or
$cube
(:,:,(
$_
)) .= PDL->rpic(
$names
[
$_
]);
for
the slice assignment using the new L<PDL::NiceSlice> syntax and lvalue
assignments.
The
for
loop reads the actual images into a temporary 2D ndarray whose
values
are then assigned (using the overloaded C<.=> operator) to the
appropriate slices of the 3D ndarray C<
$cube
> .
=head2 Q: 6.20 Where are test files
for
the graphics modules?
The demo programs can be run most easily from the
C<perldl> interactive shell:
perl -Mblib perldl
followed by C<demo 3d> or C<demo 3d2> at the prompt.
C<demo> by itself will give you a list of the available PDL demos.
=head2 Q: 6.21 What is TriD or PDL::TriD or PDL::Graphics::TriD?
Questions like this should be a thing of the past
with
the PDL
on-line help
system
in place. Just
try
(
after
installation):
un
*x
> perldl
pdl> apropos trid
Check the output
for
promising hits and then
try
to look up
some of them, e.g.
pdl> help PDL::Graphics::TriD
Note that case matters
with
C<help> but not
with
C<apropos> .
=head2 Q: 6.22 PGPLOT does not
write
out PNG files.
As of 2.086, PDL's PGPLOT-using modules are now all supplied
with
the L<PGPLOT> module itself.
=head1 EXTENSIONS OF PDL
=head2 Q: 7.1 I am looking
for
a
package
to
do
XXX in PDL. Where shall I look
for
it?
The first stop is again C<perldl> and the on-line help
or the PDL documentation. There is already a lot of functionality in
PDL which you might not be aware of. The easiest way to look
for
functionality is to
use
the C<apropos> command:
pdl> apropos
'integral'
ceil Round to integral
values
in floating-point
format
floor Round to integral
values
in floating-point
format
intover Project via integral to N-1 dimensions
rint Round to integral
values
in floating-point
format
Since the apropos command is
no
sophisticated search engine make
sure that you search on a couple of related topics and
use
short
phrases.
However there is a good chance that what you need is not part
of the PDL distribution. You are then well advised to check
using PDL. If that does not solve your problem, ask on the
mailing-list,
if
nothing
else
you might get assistance which
will let you interface your
package
with
PDL yourself, see
also the
next
question.
=head2 Q: 7.2 Can I access
my
C/FORTRAN library routines in PDL?
Yes, you can, in fact it is very simple
for
many simple
applications. What you want is the PDL pre-processor PP
(L<PDL::PP> ). This will allow you to make a
simple interface to your C routine.
The two functions you need to learn (at least first) are
C<pp_def> which defines the calling interface to the function,
specifying input and output parameters, and contains the code
that links to the external library. The other command is
C<pp_end> which finishes the PP definitions. For details see
the L<PDL::PP> man-page, but we also have a worked
example here.
double eight_sum(
int
n)
{
int
i;
double sum, x;
sum = 0.0; x=0.0;
for
(i=1; i<=n; i++) {
x++;
sum += x/((4.0
*x
*x
-1.0)*(4.0
*x
*x
-1.0));
}
return
1.0/sum;
}
We will here show you an example of how you interface C
code
with
PDL. This is the first example and will show
you how to approximate the number 8...
The C code is shown above and is a simple function
returning a double, and expecting an integer - the number
of terms in the sum - as input. This function could be
defined
in a library or, as we
do
here, as an inline
function.
We will postpone the writing of the Makefile till
later. First we will construct the
C<.pd> file. This is the file
containing PDL::PP code. We call this
C<eight.pd> .
pp_addhdr (
'
double eight_sum(
int
n)
{
int
i;
double sum, x;
sum = 0.0; x=0.0;
for
(i=1; i<=n; i++) {
x++;
sum += x/((4.0
*x
*x
-1.0)*(4.0
*x
*x
-1.0));
}
return
1.0/sum;
}
');
pp_def (
'eight'
,
Pars
=>
'int a(); double [o]b();'
,
Code
=>
'$b()=eight_sum($a());'
);
pp_done();
A peculiarity
with
our
example is that we have included
the entire code
with
C<pp_addhdr> instead of linking it in. This is only
for
the purposes of
example, in a typical application you will
use
C<pp_addhdr> to include header
files. Note that the argument to
C<pp_addhdr> is enclosed in quotes.
What is most important in this example is however the
C<pp_def> command. The first
argument to this is the name of the new function
I<eight > , then comes a hash which
the real meat:
=over 4
=item *
This gives the input parameters (here C<a>) and the output parameters
(here C<b>). The latter are indicated by the C<[o]> specifier. Both
arguments can have a type specification as shown here.
Many variations and further flexibility in the interface can be
specified. See C<perldoc PDL::PP>
for
details.
=item *
This switch contains the code that should be
executed. As you can see this is a rather peculiar
mix of C and Perl, but essentially it is just as
you would
write
it in C, but the variables that
are passed from PDL are treated differently and
have to be referred to
with
a preceding
'$'
.
There are also simple macros to pass pointers to
data and to obtain the
values
of other Perl
quantities, see the manual page
for
further
details.
=back
Finally note the call to
C<pp_done()> at the end of the
file. This is necessary in all PP files.
OK. So now we have a file
with
code that we dearly would
like to
use
in Perl via PDL. To
do
this we need to
compile the function, and to
do
that we need a
Makefile.
PDL::Core::Dev->
import
();
$package
= [
"eight.pd"
,Eight,PDL::Eight,
''
,1];
%hash
= pdlpp_stdargs(
$package
);
WriteMakefile(
%hash
);
sub
MY::postamble {pdlpp_postamble(
$package
)}
The code above should go in a file called Makefile.PL,
which should subsequently be called in the standard
Perl way:
C<perl Makefile.PL> .
This should give you a Makefile and running
C<make> should compile the module
for
you and
C<make install> will
install it
for
you.
The fifth element in the C<
$package
> array-
ref
is true. This tells PDL
to generate one C file per PP function, which
with
the right C<make>
options can be compiled in parallel,
for
a useful speedup of development
/ installation.
=head2 Q: 7.3 How can I interface
package
XXX in PDL?
This question is closely related to the previous one, and as
we said there, the
L<PDL::PP> pre-processor is the standard
way of interfacing external packages
with
PDL. The most usual
way to
use
PDL::PP is to
write
a short interface routine, see
the L<PDL::PP> perldoc page and
the answer to the previous question
for
examples.
However it is also possible to interface a
package
to PDL by
re-writing your function in PDL::PP directly. This can be
convenient in certain situations, in particular
if
you have a
routine that expects a function as input and you would like to
pass the function a Perl function
for
convenience.
The L<PDL::PP> perldoc page is the main
source of information
for
writing PDL::PP extensions, but it
is very useful to look
for
files in the distribution of PDL as
many of the core functions are written in PDL::PP. Look
for
files that end in C<.pd> which is the generally accepted
suffix
for
PDL::PP files. But we also have a simple example here.
The following example will show you how to
write
a simple
function that automatically allows broadcasting. To make this
concise the example is of an almost trivial function, but
the intention is to show the basics of writing a PDL::PP
interface.
We will
write
a simple function that calculates the minimum,
maximum and average of an ndarray. On
my
machine the resulting
function is 8
times
faster than the built-in function
C<stats> (of course the latter also
calculates the median).
Let's jump straight in. Here is the code (from a file called
C<quickstats.pd> )
pp_def(
'quickstats'
,
Pars
=>
'a(n); [o]avg(); [o]max(); [o]min()'
,
Code
=> '
$GENERIC
(a) curmax, curmin;
$GENERIC
(a) tmp=0;
loop(n) %{
tmp +=
$a
();
if
(!n ||
$a
() > curmax) { curmax =
$a
();}
if
(!n ||
$a
() < curmin) { curmin =
$a
();}
%}
$avg
() = tmp/
$SIZE
(n);
$max
() = curmax;
$min
() = curmin;
'
);
pp_done();
The above might look like a confusing mixture of C and
Perl, but behind the peculiar syntax lies a very
powerful language. Let us take it line by line.
The first line declares that we are starting the
definition of a PDL:PP function called
C<quickstats> .
The second line is very important as it specifies the
input and output parameters of the function.
C<a(n)> tells us that there is one input
parameter that we will refer to as
C<a> which is expected to be a vector of
length
n (likewise matrices, both square and rectangular
would be written as
C<a(n,n)> and
C<a(n,m)> respectively). To
indicate that something is an output parameter we put
C<[o]> in front of their names, so
referring back to the code we see that avg, max and min
are three output parameters, all of which are
scalar
(since they have
no
dimensional size indicated.
The third line starts the code definition which is
essentially pure C but
with
a couple of convenient
functions.
C<
$GENERIC
> is a
function that returns the C type of its argument - here
the input parameter a. Thus the first two lines of the
code section are variable declarations.
The
C<loop(n)> construct is a
convenience function that loops over the dimension
called n in the parameter section. Inside this loop we
calculate the cumulative sum of the input vector and
keep track of the maximum and minimum
values
. Finally
we assign the resulting
values
to the output
parameters.
Finally we finish
our
function declaration
with
C<pp_done()> .
To compile
our
new function we need to create a Makefile,
which we will just list since its creation is discussed in
an earlier question.
PDL::Core::Dev->
import
();
$package
= [
"quickstats.pd"
,Quickstats,PDL::Quickstats,
''
,1];
%hash
= pdlpp_stdargs(
$package
);
WriteMakefile(
%hash
);
sub
MY::postamble {pdlpp_postamble(
$package
)}
An example F<Makefile.PL>
Our new statistic function should now compile using the
tried and tested Perl way:
C<perl Makefile.PL; make> .
You should experiment
with
this function, changing the
calculations and input and output parameters. In conjunction
with
the L<PDL::PP> perldoc page this should allow you to quickly
write
more advanced routines directly in PDL::PP.
=head1 BUGS
If you find any inaccuracies in this document (or dis-functional
URLs) please report to the perldl mailing list pdl-general
@lists
.sourceforge.net.
=head1 ACKNOWLEDGMENTS
Achim Bohnet (ach
@mpe
.mpg.de )
for
suggesting CoolHTML as a
prettypodder (although we have switched to XML now) and various
other improvements. Suggestions
for
some questions were taken
from Perl FAQ and adapted
for
PDL.
=head1 CONTRIBUTORS
Many people have contributed or
given
feedback on the current
version of the FAQ, here is an incomplete list of individuals
whose contributions or posts to the mailing-list have improved
this FAQ at some point in
time
alphabetically listed by first
name: Christian Soeller, Chris Marshall, Doug Burke, Doug Hunt,
Frank Schmauder, Jarle Brinchmann, John Cerney, Karl Glazebrook,
Kurt Starsinic, Thomas Yengst, Tuomas J. Lukka.
=head1 AUTHOR AND COPYRIGHT
This document emerged from a joint effort of several PDL
developers (Karl Glazebrook, Tuomas J. Lukka, Christian
Soeller) to compile a list of the most frequently asked questions
about PDL
with
answers. Permission is granted
for
verbatim
copying (and formatting) of this material as part of PDL.
Permission is explicitly not granted
for
distribution in book
or any corresponding form. Ask on the PDL mailing list
pdl-general
@lists
.sourceforge.net
if
some of the issues covered
in here are unclear.