Bioperl - Design Documentation
Not appropiate. Read on...
Bioperl is a coordinated project which has a number of design features to allow bioperl to be well used, extended and collaborate with other packages. This design can be focused in a number of areas.
bioperl ettiquette and learning about it bioperl root object - exception throwing, exceptions etc. bioperl interface design bioperl sequence object design notes
This was written by Ewan Birney in a variety of airports across the US.
The biggest problem often in reusing a code base like bioperl is that it requires both the people using it and the people contributing to it to change their attitude towards code. Generally people in bioinformatics are more likely to be self-taught, single programmers, who put together most of their scripts/programs as individuals. Bioperl is a truely collaborative project (the core code is the product of about 15 individuals) and anyone will be only contributing some part of it in the future.
Here are some notes about how my coding style has changed to work in collaborative projects.
Reading documentation is sometimes as tough as writing the documentation. Try to read documentation before you ask a question - not only might it answer your question, but more importantly it will give you idea why the person who wrote the module wrote it - and this will be the frame work in which you can understand his or her answer.
If the code does what you want, the fact that it is not written the way you would write should not be a big issue. Of course, if there is some glaring error then that is worth pointing out to someone. Dismissing a module on the basis of its coding style is a tremendously wrong thing to do.
This ranges from giving very accurate bug reports (this script --> makes this error, giving all data), through to pointing out design issues in a constructive manner (not - this *sucks*). If you find a problem, then providing a patch using diff or work around is a great thing to do - the author/maintainer of the module will love you for it.
Providing "I used XXX and it did just what I wanted it to do" feedback is also really great. Developers generally only hear about their mistakes. To hear about successes gives everyone a warm glow.
One trick I have learnt is that when I download a new project/code or use a new module I open up a fresh buffer in emacs and keep a mini diary of everything that I did or think when I started to use the package. After I used it I could go back, edit the buffer and then send it to the author either with "it was great - it did just what I wanted, but I found that the documentation here was misleading" to "to get it to install I had to incant the following things..."
When you want to get involved, hopefully it will be because you want to extend something or provide better facillities to something. The important thing here is not to work in a vacuum. By providing the main list with a good proposal before you start about what you are going to do (and listen to the responses) is a must. I have been pulled up so many times by other people looking at my design that I can't imagine coding stuff now without feedback.
Sadly, you might think that you have written good code, but you don't know that until you manage to test it! The CPAN style perl modules have a wonderful test suite system (delve around into the t/ directories) and I have extended the makefile system so that the test script which you write to test the module can be part of the t/ system from the start. Once a test is in the t/ system it will be run millions of times worldwide when bioperl is downloaded, providing incredible and continual regression testing of your module (for free!).
The coding process should be enjoyable, and I get very proud of people who tell me that they picked up bioperl and it worked for them, even if they don't use a single module that I wrote. There is a brilliant sense of community in bioperl about providing useful, stable code and it should be a pleasure to contribute to it.
So - I am always looking forward to people posting on the guts list with their feedback/questions/proposals. As well as the long standing fun we have making new releases.
All objects in bioperl (but for interfaces - see the next section) inheriet from the Root Object. The bioperl root object allows a number of very useful concepts to be provided. In particular.
Bioperl root object allow exceptions to be throw on the object with very nice debugging output
Bioperl root object have a context which allows, in particular, exceptions that are thrown to say which object as throwing the exception.
Bioperl root object have some helper methods, in particular rearrange to help functions which take hash inputs.
To use the root object, the object has to inheriet from it. This means the @ISA array should have (Bio::Roo::Object) in it and that the module goes "use Bio::Root::Object". The root object provides the ->new function. This new function builds a hash, sets some root object management issues and then calls the _initialize function. It is this function which your object needs to implement. The full code is given below.
# convention is that if you are using the Bio::Root object you should put it # inside the Bio namespace package Bio::MyNewObject; use vars qw(@ISA); use strict; use Bio::Root::Object; @ISA = qw(Bio::Root::Object); # new() is inherited from Bio::Root::Object # _initialize is where the heavy stuff will happen when new is called sub _initialize { my($self,@args) = @_; # call superclasses initialize my $make = $self->SUPER::_initialize(@args); # do your own argument processing here # set default attributes etc... return $make; # success - we hope! }
Exceptions are die functions, in which the $@ variable (a scalar) is used to indicate how it died. The exceptions can be caught using the eval {} system. The bioperl root object has a method called "->throw" which calls die but also provides a full stack trace of where this throw happened on (and also which object the exception was thrown - see the context section). So an exception like $obj->throw("I am throwing an exception");
Provides the following output on STDERR if is not caught.
-------------------- EXCEPTION -------------------- MSG: I am throwing an exception CONTEXT: Error in object Bio::Root::Object "anonymous Bio::Root::Object" SCRIPT: myscript.pl STACK: main::my_subroutine(7) main::(3) ---------------------------------------------------
indicating that this exception was thrown at line 7 of subroutine my_subroutine, in myscript.pl
Exceptions can be caught using an eval block, such as
my $obj = Bio::SomeObject->new(); my $obj2 eval { $obj2 = $obj->method1(); $obj2->method2(10); } if( $@ ) { # exception was thrown &tell_user("Exception was thrown, preventing whatever I wanted to do. Actual exception $@"); exit(0); } # else - use $obj2
notice that the eval block can have multiple statements in it, and also that if you want to use variables outside of the eval block, they must be declared with my outside of the eval block (you are planning to use strict in your scripts, aren't you!).
Each bioperl object has a context, which is given by the name attribute (name is a method defined in the Bio::Root::Object package). This context is displayed when the exception is made, so that the following script:
use Bio::Root::Object; $obj = Bio::Root::Object->new; $obj->name("Context-A"); &my_subroutine($obj); sub my_subroutine { $self = shift; $self->throw("I am throwing an exception"); }
Produces the following exception
-------------------- EXCEPTION -------------------- MSG: I am throwing an exception CONTEXT: Error in object Bio::Root::Object "Context-A" SCRIPT: test2.pl STACK: main::my_subroutine(10) main::test2.pl(6) ---------------------------------------------------
Notice that the Object nows says that it is Context-A.
This context is particularly useful when objects are produced from a database. This is because some exceptions are really due to problems with the data in an object rather than the code. These sort of exceptions are better tracked down when you know where the object came from, not where in the code the exception is thrown.
One of the drawbacks to this scheme is that the attribute ->name is "special" from bioperl's perspective. I believe it is best to stay away from using $obj->name() to mean anything from the object's perspective (for example ->id() ), leaving it free to be used as a context for debugging purposes. You might prefer to overload the name attribute to be "useful" for the object.
Bioperl has been moving to a split between interface and implementation definitions. An interface is solely the definition of what methods one can call on an object, without any knowledge of how it is implemented. An implementation is an actual, working implementation of an object. In languages like Java, interface definition is part of the language. In Perl, like many aspects of Perl you have to roll your own.
In bioperl, the interface names are called Bio::MyObjectI, with the trailing I indicating it is an interface definition of an object. The interface files (sometimes nicknamed the 'I files') provide mainly documentation on what the interface is, and how to use (and implement it). All the functions which the implementation is expected to provide are defined as subroutines, and then die with an informative warning. The exception to this rule are the implementation independent functions (see later).
Objects which want to implement this interface should inheriet the Bio::MyObjectI file in their @ISA array. This means that if the implementation does not provide a method which the interface defines, rather than the user getting a "method not found error" it gets a "mymethod was not defined in MyObjectI, but should have been" which makes it clearer that whoever provided the implementation was to blame, and not the caller/script writer.
When people want to check they have valid objects being passed to their functions they should test the presence of the interface, not the implementation. for example
sub my_sequence_routine { my($seq,$other_argument) = @_; $seq->isa('Bio::SeqI') || die "[$seq] is not a sequence. Cannot process"; # do stuff }
This is in contrast to
sub my_incorrect_sequence_routine { my($seq,$other_argument) = @_; # this line is INCORRECT $seq->isa('Bio::Seq') || die "[$seq] is not a sequence. Cannot process"; # do stuff }
Some people might justifiably argue "why do this?". The main reason is to support external objects from bioperl, and allow them to masquarade as real bioperl objects. For example you might have your own quite intricate sequence object which you want to use in bioperl functions, but don't want to lose your own neat coding. One option would be to have a function which built a bioperl sequence object from your object, but then you would be endlessly building temporary objects and destroying them, in particular if the script yo-yoed between your code and bioperl code.
A better solution would be to implement the Bio::SeqI interface. You would read the Bio::SeqI documentation, and then provide the methods which it required, and put Bio::SeqI in your @ISA array. Then you could pass in your object into bioperl routines and eh voila - you are a bioperl sequence object.
(A problem might arise if your object has the same methods as the Bio::SeqI methods but use them differently - your $obj->id() might mean provide the raw memory location of the object, whereas the documentation for Bio::SeqI $obj->id() says it should return the human-readable name. If so you need to look into providing an 'Adaptor' class, as suggested in the Gang-of-four).
Interface classes really come into their own when we start leaving Perl and enter extensions wrapped over C or over databases, or through systems like CORBA to other languages, like Java/Python etc. Here the "object" is often a very thin wrapper over the a DBI interface, or an XS interface, and how it stores the object is really different. By providing a very clear, implementation free interface with good documentation there is a very clear target to hit.
Some people might complain that we are doing something very "un-perl-like" by providing these separate interface files. They are 90% documentation, and could be provided anywhere, in many ways they could be merged with the actual implementation classes and just made clear that if someone wants to mimic a class they should override the following methods. However, we (and in particular myself - Ewan) prefers a clear separation of the interface. It gives us a much clearer way of defining what is going on. It is in many ways just "design sugar" (as opposed to syntactic sugar) to help us, but it really helps, so thats good enough justification to me.
One of the issues we discovered early on in using Interface files was that there were methods that we would like to provide for classes which were independent of their implementation. A good example is a "Range" interface, which might define the following methods
$obj->start() $obj->end()
Now a client to the object might want to use a $obj->length() method. because it is much easier than retrieving the two attributes and substracting them. However, the ->length() method is just a pain for someone providing the implementation to provide - once start() and end() is defined, length is. There seems to be a catch-22 here: to make an object definition good for a client one needs to have additional, helper methods "on top of" the interface, however to make life easier for the object implementation one wants to have the bare minimum of functions defined which the implementer has to provide.
In the Range interface this became more than annoyance, as alot of the "smarts" of the Range system was that we wanted to have the ability to say
if( $range->intersection($someother_range) )
We wanted a generic RangeI interface that we could apply to many objects, with definitions required only for ->start, ->end and ->strand. However we wanted the ->intersection, and ->union methods to be on all ranges, without us having to reimplement this every time.
Our (Matt Pocock and Ewan Birney's) solution was to allow implementation into the RangeI interface file, but only when these implementations sat "on top" of the interface definition and therefore provided helper client operations. In a language like Java, we would clearly have two classes, with a composition/delegation method:
MyPublicSomethingClass has-a MyInternalSomethingInterface, with ADifferentImplemtation implements MyInternalSomethingInterface
However this is really heavy handed in Perl (and people were complaining about having different implementation/interface classes). We were quite happy about merging the implementation independent functions with the interface definition, and I (Ewan) used this in other interfaces since then. The documentation has to be clear about what is going on, but I think in general it is.
To take advantage of the new bioperl interface/implementation split, you have three options for wrapping external objects as bioperl objects
Assume that bioperl will always be installed, and inheriet off the appropiate "I" file. You will then need to read the I file to see which functions you need to implement.
One of the problems about the previous system is that one needs to have bioperl wherever your package goes. This might be too much of a constraint on you. One solution is to use (or abuse?) the Perl compile time system by putting a conditional use statement in a BEGIN block in your module, such as
BEGIN { eval { require Bio::PrimarySeqI; }; if ( $@ ) { # nothing happens } else { push(@ISA,"Bio::PrimarySeqI"); } }
If what you want to wrap is an out-of-Perl experience (eg, a SQL database or a C struct or a CORBA client) you will need to write an Adaptor class. This will be a Perl object which specifically sits between your object and bioperl. For complex Perl objects which don't map easily to a bioperl object, you will probably be forced to write adaptor objects
With adaptor objects one can generally make their own CPAN module installation, with instructions to install only when bioperl is around.
To install Bio::Seq, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Bio::Seq
CPAN shell
perl -MCPAN -e shell install Bio::Seq
For more information on module installation, please visit the detailed CPAN module installation guide.