Net::Z3950::AsyncZ adds an additional layer of asynchronous support to the Z3950 module through the use of multiple forked processes. Users may also find that it is a useful front end to Z3950. Detailed descriptions of the mechanics, objects, and methods of AsyncZ can be found in the accompanying documentation:
What follows are annotated versions of the example scripts. I start with the basic.pl, which uses the the basics needed to run AsyncZ, and move up through the scripts, each of which adds features to the one previous in this order: basic.pl, basic_pretty.pl, more_pretty.pl, options.pl
basic.pl
AsyncZ
basic.pl, basic_pretty.pl, more_pretty.pl, options.pl
Since each script builds upon the one previous, the only script which is quoted in full is basic.pl. For subsequent scripts, I quote the code added to the predecessor.
use Net::Z3950::AsyncZ qw(isZ_Error); # [1] my @servers = ( # [2] [ 'amicus.nlc-bnc.ca', 210, 'NL'], ['bison.umanitoba.ca', 210, 'MARION'], [ 'library.anu.edu.au', 210, 'INNOPAC' ], ['130.17.3.75', 210, 'MAIN*BIBMAST'], [ 'library.usc.edu', 2200,'unicorn'], [ 'z3950.loc.gov', 7090, 'Voyager' ], [ 'fc1n01e.fcla.edu', 210, 'FI' ], [ 'axp.aacpl.lib.md.us', 210, 'MARION'], [ 'jasper.acadiau.ca', 2200, 'UNICORN'] ); my $query = ' @attr 1=1003 "Henry James" '; # [3] # [4] my $asyncZ = Net::Z3950::AsyncZ->new( servers=>\@servers,query=>$query, cb=>\&output ); showErrors($asyncZ); # [5] exit; #------END MAIN------#
Net::Z3950::AsyncZ
showErrors()
$host, $port, and $database_name
Net::Z3950
servers
query
showErrors
Load Net::Z3950::AsyncZ and import isZ_Error, which is a class method that we will use in the error handling subroutine showErrors().
isZ_Error
Create an array of servers to which we want to send our query. This consists of an array of references to a three-element anonymous array: $host, $port, and $database_name. This is the same structure which is used in Net::Z3950.
Create a query. This query uses the YAZ toolkit's PQN syntax: 'prefix'
Create a Net::Z3950::AsyncZ object, using named-parameters; in addition to passing servers and query into the contrcutor, we also pass in a reference to a callback function which will be called by Net::Z3950::AsyncZ whenever new records become available--it will be up to the callback function to output the records to terminal or browser.
Call showErrors, a subroutine which will output error messages, in the event that some of the servers fail to respond or to return records. We pass in the reference to the Net::Z3950::AsyncZ object, which showErrors() will need to access the errors.
sub output { my($index, $array) = @_; # [1] foreach my $line(@$array) { # [2] print "$line\n" if $line; # [3] } print "\n--------\n\n"; }
AsyncZ.html
Get the two parameters which AsyncZ passes to the output function, namely the index of the server which is reporting back and a reference to an array of record data for the server. This array will contain one or more records. See Output Callbackin AsyncZ.html or AsyncZ.pod.
Output Callback
AsyncZ.pod
Each array element represents a separate line of record output: retrieve each line from the array.
If the line is not null, print it. (The check for a null line is a carry over from an earlier state of AsyncZ: I don't think it's needed anymore but I keep it just in case.)
sub showErrors { my $asyncZ = shift; # [1] print "The following servers have not responded to your query: \n"; for(my $i=0; $i< $asyncZ->getMaxErrors();$i++) { # [2] my $err = $asyncZ->getErrors($i); # [3] next if !isZ_Error($err); # [4] print "$servers[$i]->[0]\n"; # [5] print " $err->[0]->{msg}\n" if $err->[0]->{msg}; # [6] print " $err->[1]->{msg}\n" if $err->[1]->{msg}; # [7] } }
Get the reference to the AsyncZ object.
Get the total number of errors reported and loop through them.
For each server there are two possible errors, since for some servers we make a second attempt to get results if the first attempt fails. (See "Basic Mechanisms of Net::Z3950::AsyncZ" in AsyncZ.pod.) $err is a reference to an anonymous array which may hold 1 or 2 references to Net:AsyncZ::ErrMsg objects, which store all the necessary info about these errors. (See Net::Z3950::AsyncZ::ErrMsg in AsyncZ.pod)
$err
Net:AsyncZ::ErrMsg
Net::Z3950::AsyncZ::ErrMsg
So, get the errors array for the current index.
Check to see whether in fact an error occurred--we may not have gotten any records back on the first attempt; consequently, we may have an error for attempt 1. But we may have been successful on attempt 2 and so the first error is nullified. Or we may have had an error on the first attempt which was fatal and so no second attempt was made. isZ_Error will tell us what happend.
If we've got this far, then there's been some kind of error. So, let's tell our user the name of the server that failed to return results: we pick that up from the @servers array--[$server, $port, $database].
@servers
6 - 7
Now we can make our own use of the ErrMsg objects. The array reference $err holds two ErrMsg objects. $err-[0]> is from attempt 1 and $err->[1] from attempt 2. We check to see if error messages have been saved in these object and if so, we print them.
ErrMsg
$err-
$err->[0]
$err->[1]
basic_pretty.pl is an upgrade to basic_pl. When you run basic_pl, you get a set of headers, which your user doesn't have to see, the records are run together, and the interspersed with the records are various debugging messages. basic_pretty.pl rectifies these problems.
basic_pretty.pl
basic_pl
Instead of reprinting the entire basic.pl, let's look only at the changes.
use Net::Z3950::AsyncZ qw(:header :errors); # [1] use Net::Z3950::AsyncZ::Errors qw(suppressErrors); # [2] . . . . my $asyncZ = Net::Z3950::AsyncZ->new(servers=>\@servers,query=>$query, cb=>\&output, log=>suppressErrors(), # [3] );
1. We import a number of subroutines from Net::Z3950::AsyncZ that will enable us to do something with both the headers and the errors directed to the user.
2. We import a subroutine from Net::Z3950::AsyncZ::Errors that will enable us to get rid of the interspersed debugging messages.
Net::Z3950::AsyncZ::Errors
3. We set log to suppressErrors() so that errors get suppressed.
log
sub output { my($index, $array) = @_; foreach my $line(@$array) { return if noZ_Response($line); # [1] next if isZ_Info($line); # [2] next if isZ_Header($line); # [3] (print "\nServer: ", Z_serverName($line), "\n"), next # [4] if isZ_ServerName($line); # [5] print "$line\n" if $line; } print "\n--------\n\n"; }
1. isZ_Info removes headers.
isZ_Info
2. So, too, does isZ_Header
isZ_Header
3. Z_serverName checks to see if this is the header with the server's name in it
Z_serverName
4. If it is, then extract the server's name with isZ_ServerName and print it for the user's information
isZ_ServerName
sub showErrors { my $asyncZ = shift; # substitute some general statement for a # system level error instead of something # puzzling to the user like: 'illegal seek' my $systemerr = "A system error occurred on the server\n"; # [1] print "The following servers have not responded to your query: \n"; for(my $i=0; $i< $asyncZ->getMaxErrors();$i++) { my $err = $asyncZ->getErrors($i); next if !isZ_Error($err); print "$servers[$i]->[0]\n"; if($err->[0]->isSystem()) { # [2] print $systemerr; } else { # [3] print " $err->[0]->{msg}\n" if $err->[0]->{msg}; } if($err->[1] && $err->[1]->isSystem()) { # [4] print $systemerr; } else { print " $err->[1]->{msg}\n" # [5] if $err->[1]->{msg} && $err->[1]->{msg} != $err->[0]->{msg}; } } }
1. We create a general system-level error message because this time we are going to test for system level errors and print the general statement to screen instead of system level error messages which risk frustrating the user.
2. We use the Net::ErrMsg object, naemly $err->[0]->isSystem(), to test for system-level errors and print the general message if it is system-level.
Net::ErrMsg
$err->[0]->isSystem()
3. If it isn't we ouput the error message for this error.
4. We check first to make sure that $err->[1] exists: remember, $err->[1] is an error that occurs during the second attempt to query the server, and if the first time around we got a fatal (non-retryable) error, then we will not have and $err->[1]. If there is an $err->[1] and it's a system-level error, the print the general system message.
5. Otherwise, print the $err->[1] message. But only if it is not the same error and therefore the same message as the first time around. Since there's no point in repeating it.
The script more_pretty illustrates the use of the format option.
more_pretty
format
my $asyncZ = Net::Z3950::AsyncZ->new(servers=>\@servers,query=>$query,cb=>\&output, format=>\&thisRecordRow, # [1] log=>suppressErrors() );
1. We set the format option to thisRecordRow, a subroutine which we will define later.
thisRecordRow
2. num_to_fetch specifies how many records we want returned; the default is 5.
num_to_fetch
use Text::Wrap qw($columns &wrap); # [1] sub thisRecordRow { my ($row) = @_; # [2] $columns = 56; # [3] my $field = $row->[1]; my $indent = ' ' x 25; $field = wrap("",$indent, $field) if length($field) > 56; # [4] return sprintf("%20s: %s\n", $Net::Z3950::AsyncZ::Report::MARC_FIELDS{$row->[0]}, $field); # [5] }
1. Load the Text::Wrap module and import $columns and &wrap.
Text::Wrap
$columns
&wrap
2. Retrieve $row from @_. This is a two element aanoymous array. $row[0] is a reference to the MARC tag for the current row, while $row[1] is a reference to the record data for this row.
$row
@_
$row[0]
$row[1]
3. Set the maximum number of columns for the record data to 56 (we are going to allow 20 for the field identifier). %Net::Z3950::AsyncZ::Report::MARC_FIELDS is a hash in which the MARC tags are the keys and the dentifier strings are the values:
%Net::Z3950::AsyncZ::Report::MARC_FIELDS
'050' => LC call number 245 => title
and so forth. This topic is taken up in the Net::Z3950::AsyncZ::Report documentation.
Net::Z3950::AsyncZ::Report
4. If the record data is longer than 56 characters, wrap it.
5. Finish off our formatting of the line using sprintf and return the line to Net::Report.
sprintf
Net::Report
So instead of this:
050 LC call number: LB1027.S3662 1969b 245 title: The Schools and the challenge of innovation[by] H. Thomas James and others] With introd. by Sterling M. McMurrin. 260 publication: New York,McGraw-Hill[1969]
We get this:
LC call number: LB1027.S3662 1969b56 title: The Schools and the challenge of innovation[by] H. Thomas James [and others] With introd. by Sterling M. McMurrin. publication: New York,McGraw-Hill[1969]
use Net::Z3950::AsyncZ qw(:header :errors asyncZOptions); # [1] use Net::Z3950::AsyncZ::Errors qw(suppressErrors); my @servers = ( [ 'amicus.nlc-bnc.ca', 210, 'NL'], ['bison.umanitoba.ca', 210, 'MARION'], [ 'library.anu.edu.au', 210, 'INNOPAC' ], ['130.17.3.75', 210, 'MAIN*BIBMAST'], [ 'library.usc.edu', 2200,'unicorn'], [ 'z3950.loc.gov', 7090, 'Voyager' ], [ 'fc1n01e.fcla.edu', 210, 'FI' ], [ 'axp.aacpl.lib.md.us', 210, 'MARION'], [ 'jasper.acadiau.ca', 2200, 'UNICORN'] ); my @options = (); # [2] for(my $i = 0; $i < @servers; $i++) { $options[$i] = asyncZOptions(num_to_fetch=>1, # [3] format=>\&thisRecordRow); $options[$i]->set_query(' @attr 1=1003 "James Joyce" ') if $i % 2 == 0; # [4] } $options[0]->set_GRS1(); # amicus # [5] $options[0]->set_raw_on(); # [6] $options[0]->set_log('amicus.log'); # [7] $options[1]->set_raw_on(); # [8] $options[5] = undef; # z3950.loc.gov # [9] my $query = ' @attr 1=1003 "Henry James" '; # [10] my $asyncZ = Net::Z3950::AsyncZ->new(servers=>\@servers,query=>$query,cb=>\&output, log=>suppressErrors(), # [11] options=>\@options, # [12] num_to_fetch=>2 # [13] ); showErrors($asyncZ); exit; #------END MAIN------#
Import asyncZOptions, the class method which returns Net::Z3950::AsyncZ::Option::_params objects-- where we can set options for each server separately.
asyncZOptions
Net::Z3950::AsyncZ::Option::_params
Create an array to hold the references to _params objects.
_params
Loop through the servers, creating a _params object for each. Set num_to_fetch=>1 and format=>\&thisRecordRow for each server.
num_to_fetch=>1
format=>\&thisRecordRow
Note: When you create a _params object for a server, if the num_to_fetch and format options are not set, they will revert to the default values, which are 5 and plain text output, even if you later set these options in the AsyncZ constructor. AsyncZ constructor settings do not apply to num_to_fetch and format if you have previously created a _params object for the server in question.
For every 2nd server we'll ask for books about James Joyce. The odd number servers will use the query about Henry James at #10. Unlike the num_to_fetch and format options, a query set in the AsyncZ constructor will apply to any server which does not have a query set for it in a _params object. The rationale behind this is that you usually will be asking one question across all servers.
We request GRS-1 records from amicus, The National Library of Canada, because this is their default preferredRecordSyntax.
preferredRecordSyntax
We ask to have the amicus records returned to us raw, because we might presumably branch off from our output function to a special handler for raw GRS-1 records. (Although in the case of the National Library of Canada GRS-1 records, our GRS-1 handler in Net::Z3950::AsyncZ:Records works fine.)
Net::Z3950::AsyncZ:Records
Because of our special treatment of amicus records, we set a log to catch any error messages. In the case of logs, the log setting in the AsyncZ constructor will apply to all servers unless a log is specifically set for it in it a server's _params object. The rationale for this is that you probably would want one log file to cover all servers, except in special circumstances.
In the present case, only amicus will get a log; all the other servers will be governed by log=>suppressErrors() in the AsyncZ constructor.
log=>suppressErrors()
Since amicus doesn't always respond, let's get some raw output from another server, just for demonstration purposes: $server[1] is bison.
I undef z3950.loc.gov, Library of Congress. This means that the Library of Congress record output will be govenred by the AsyncZ constructor and a default _params object which will be created for it.
undef
Set the query for any servers which don't have a query set in its <_params>.
Suppress error logs for all servers which don't ask for error logs in their _params objects.
Set options=>\@options
options=>\@options
Fetch 2 records for any server which does not have a _params object--in this case z3950.loc.gov, Library of Congress.
raw.pl illustrates how to access raw records which have not been filtered through Net::Z3950::Record::render().
raw.pl
Net::Z3950::Record::render()
use Net::Z3950::AsyncZ qw(:header :errors asyncZOptions prep_Raw get_ZRawRec); # [1] use Net::Z3950::AsyncZ::Errors qw(suppressErrors); my @servers = ( ['bison.umanitoba.ca', 210, 'MARION'], [ 'z3950.loc.gov', 7090, 'Voyager' ], [ 'jasper.acadiau.ca', 2200, 'UNICORN'] ); my @options = ( asyncZOptions (raw=>1,num_to_fetch=>3, render=>0), #[2] asyncZOptions (raw=>1,num_to_fetch=>3, render=>0), asyncZOptions (raw=>1,num_to_fetch=>3, render=>0), ); my $query = ' @attr 1=1003 "James Joyce" '; my $asyncZ = Net::Z3950::AsyncZ->new(servers=>\@servers,query=>$query,cb=>\&output, monitor=>45, maxpipes=>2, log=>suppressErrors(), options => \@options, ); exit; #------END MAIN------# sub output { my($index, $array) = @_; my $count=0; return if noZ_Response($array=>[0]); #[3] my $recs = prep_Raw($array); #[4] while(($rec = get_ZRawRec($recs))) { #[5] my $outfile = "> raw_${index}_$count"; #[6] open OUTFILE, $outfile; print OUTFILE $rec; close OUTFILE; $count++; } }
Import functions from AsyncZ which are needed for error handling, reading headers, and handling unfiltered raw records
Create _params objects for each of the servers: set raw to true and render to false.
raw
true
render
false
Check to make sure there has been a response: no reponse headers will always consist of an array with one element.
Prepare the unfiltered records by passing them to prep_Raw(). This subroutine strips the headers from all the records in the current group, creates a single string from the array, and sets markers between each record.
Fetch one record at a time--get_ZRawRec()is a "get_next" type function, starting with the first record.
For this example, we'll write each record to a file: so we create a file name for each record as it is fetched and write the output to the file.
This script demonstrates a number of things having to do with handling of HTML and MARC records. In addition, it gives an example of the use of the option Z3950_options of the _params object.
Z3950_options
I reprint the script here, which is fully annotated, and give a few fuller explanations below.
#!/usr/bin/perl ## This script demonstrates a number of things ## 1. how to create you own MARC fields hash by adding fields to %$Net::Z3950::Report:all ## 2. use of the Z3950_options _params option ## 3. formatting HTML by starting with the default HTML row format ## 4. use of utf8 for unicode output TO browser ## use Net::Z3950::AsyncZ qw(:header :errors asyncZOptions); use Net::Z3950::AsyncZ::Errors qw(suppressErrors); use Net::Z3950::AsyncZ::Report; use strict; my @servers = ( ['128.118.88.200',210,'catalog'], ['bison.umanitoba.ca', 210, 'MARION'] ); # [1] create hash of additional MARC fields my %my_MARC_fields = ( 651 => "location", 654 => "terms", 655 => "genre", 656 => "occupation", 760 => "main series", 762 => "subseries", 765 => "original language", 767 => "translation entry", 770 => "supplement/special issue", 772 => "supplement parent", 773 => "host item entry", 774 => "constituent unit", 775 => "other edition", 776 => "add. physical form", 777 => "issued with", 780 => "preceding", 785 => "succeeding", 786 => "data source", 787 => "nonspecific rel.", 800 => "personal name", 810 => "corporate name", 811 => "meeting name", 830 => "uniform title" ); # [2] create a new hash which adds the additional MARC fields to %$Net::Z3950::AsyncZ::Report::all, # ($Net::Z3950::AsyncZ::Report::all is a reference to %Net::Z3950::AsyncZ::Report::MARC_Fields_All my %my_MARC_hash = (%$Net::Z3950::AsyncZ::Report::all, %my_MARC_fields); # [3] set options for both servers # --assign \%my_MARC_hash to marc_userdef # --ask for full records, the default is brief, by setting the Z3950 option elementSetName =>'f'. # The 'f' option is used by the Net::Z3950::ResultSet module. We set this option by # using Z3950_options. (Options set in the Manager are inherited by the other Z3950 modules.) # --set format to &Net::Z3950::AsyncZ::Report::_defaultRecordRowHTML or else set HTML to true. my @options = ( asyncZOptions( num_to_fetch=>8, format=>\&Net::Z3950::AsyncZ::Report::_defaultRecordRowHTML, marc_userdef=>\%my_MARC_hash,Z3950_options=>{elementSetName =>'f'}), asyncZOptions( num_to_fetch=>8, HTML=>1, marc_userdef=>\%my_MARC_hash) ); # [4] set the utf8 option to true--you could also do that above in step 3 $options[0]->set_utf8(1); $options[1]->set_utf8(1); # [5] set the query my $query = ' @attr 1=1016 "Baudelaire" '; # [6] Output headers which notify the browser that this script is outputting utf8 print "Content-type: text/html;charset=utf-8'\n\n"; print '<head><META http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body>', "\n"; # [7] send out the query to the servers my $asyncZ = Net::Z3950::AsyncZ->new(servers=>\@servers,query=>$query,cb=>\&output, options=>\@options, #log=>suppressErrors() ); exit; #------END MAIN------# sub output { my($index, $array) = @_; # [8] stipulate that the output stream is utf8--required! binmode(STDOUT, ":utf8"); # [9] create a table structure for the rows of <TD>'s which are output by the # default format subroutine my $table_started = 0; my $server_found= 0; print "<TABLE><TR><TD>"; foreach my $line(@$array) { return if noZ_Response($line); next if isZ_Info($line); # remove internal data if (isZ_Header($line)) { print '<tr><td> <td> </TABLE>' if $table_started; $table_started = 1; # [10] Add space around table elements and set the alignments for the columns print '<TABLE cellspacing = "6" cellpadding="2" border="0" width = "600">'; print '<colgroup span="2"><COL ALIGN = "RIGHT" WIDTH="150" VALIGN="TOP"><COL ALIGN="LEFT"></COLGROUP>'; next; } my $sn = Z_serverName($line); if($sn && ! $server_found) { print "\n<br><br><br><b>Server: ", $sn, "</b><br>\n"; $server_found = 1; } # [11] substitute a fancier style for the field names $line =~ s/<TD>/<TD NOWRAP style="color:blue" align="right">/i; print "$line\n" if $line; } print "</TABLE>"; }
%my_MARC_fields is drawn from the Library of Congress MARC documentation.
%my_MARC_fields
It is added to %Net::Z3950::AsyncZ::Report::MARC_FIELDS_ALL, which is referenced by $Net::Z3950::AsyncZ::Report::all (and is not itself directly accessible). We create this extended set of fields in order to get as much data as possible, since we are going to be setting elementSetName to 'f', asking for "full" as opposed to "brief" records.
%Net::Z3950::AsyncZ::Report::MARC_FIELDS_ALL
$Net::Z3950::AsyncZ::Report::all
elementSetName
To use utf8 support, you must have MARC::Charset installed; otherwise, this option will be ignored.
utf8
MARC::Charset
This query should get us some French accented characters with which to test out utf8 support
These steps notify the browser that it will be receiving a utf8 stream and notify perl that it should output a utf8 stream. Unless you call binmode(STDOUT,":utf8"), perl will not output the utf8 code.
binmode(STDOUT,":utf8")
We will be using the default HTML format function, which outputs individual rows of data formatted for insertion into a table. Its structure is:
<TD>field name<TD>field data
The output() callback takes advantage of this formatting byt specifying HTML attributes for the table and by reconstructing one of the <TD> tags.
output()
Myron Turner <turnermm@shaw.ca> or <mturner@ms.umanitoba.ca>
Copyright 2003 by Myron Turner
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
4 POD Errors
The following errors were encountered while parsing the POD:
You forgot a '=back' before '=head1'
Expected '=item 6'
Expected text after =item, not a bullet
To install Net::Z3950::AsyncZ::ZSend, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Net::Z3950::AsyncZ::ZSend
CPAN shell
perl -MCPAN -e shell install Net::Z3950::AsyncZ::ZSend
For more information on module installation, please visit the detailed CPAN module installation guide.