NAME
CGI::UploadEngine - File Upload Engine for Multi-App Web Server
VERSION
This document describes CGI::UploadEngine version 0.9.1
DESCRIPTION
The main design goal of CGI::UploadEngine (CGI::UE) has been to enable developers to use file upload select boxes in a regular HTML form without needing to handle multi-part forms (or making any number of rookie mistakes). This is accomplished by injecting the necessary code into the developers template to submit the form twice; once to upload the file, and a second time to submit the rest of the data along with an acknowledgement token. This means that the "Upload Engine" (UE) handles all of the html and javascript necessary to upload, validate (and optionally limit) file type, size, and destination directory.
The UE includes the four main methods of CGI::UE and a mysql table. The two middle methods are meant to be implemented in a "Core" controller/script that handles the actual file upload. This package currently only includes working code for a Catalyst supported Core. If you do not have Catalyst, please get it, wait for an update, or send us your working CGI Core to include in the next update.
CGI::UE can be installed with make, but the UE Core should be installed by a server administrator (currently with Catalyst) for the benefit of developers on their server (either developing with Catalyst or "doing it old school"). The developers will use both the first and last CGI::UE methods and the installed UE Core to painlessly create forms that can handle both file and field data, even in "old school" CGI scripts.
SYNOPSIS
Normally, file select boxes have to be in a form with a special multi-part attribute which can significantly complicate handling other values in the same form. By handling the file upload with AJAX, we can replace the file select box with a simple (hidden) text field (with our "success token" pre-filled) that reduces file uploading to just another dynamic field to be filled in a template.
<form action="[% action_url %]" method="POST">
<table cellspacing="2" cellpadding="0" width="600">
<tr>
<td> Select file </td>
<td> [% file_upload %] </td>
</tr>
<tr>
<td> Your fields here </td>
<td> <input type="text" name="other" id="other" value=""> </td>
</tr>
<tr>
<td> </td>
<td> <input type="submit"> </td>
</tr>
</table>
</form>With the template in hand, we only need to inject the UE "Package". The Package contains all of the HTML and most of the JavaScript necessary to upload the file to your specified directory. (Additional JavaScript is used from the Ext library.)
However, additional work is required to make a controller/script. Below is text of form_eg.cgi, which is included in the CGI::UE package. Notice it uses Template Toolkit (TT). It also requires a script level configuration file, template file, and library file.
use CGI::UploadEngine;
use Template;
require "upload_cfg.pl";
require "upload_html.pl";
# Create an upload object
my $upload = CGI::UploadEngine->new();
# Set the view template
my $tmpl = "upload/form_eg.tt2";
# Set the template variables
my $vars = { action_url => $root_url . "cgi-bin/handler_eg.cgi",
file_upload => $upload->upload_prepare({ file_path => "/tmp" }) };
# Process the template
my $result;
my $template = Template->new({ INCLUDE_PATH => $tmpl_incl });
$template->process( $tmpl, $vars, \$result );
# Print the page
header();
print $result;
footer();
In the above script, the functions header() and footer() are from the library file upload_html.pl (as is parse_form_data() in the script below). (Try reading the short configuration file.)
Note: Page layout could be handled through TT. You could grab the CGI form data another way. Database passwords should be in file X. Bleep blurp bleep. In short, these CGI scripts are working samples of developer software, not final software.
Once the end user selects a file, it is submitted via AJAX. On success, the form is altered. The file upload box is removed and replaced with a regular text box containing the returned success token. As a developer, you will need to retrieve the file info using the token to discover the file-path-name and other pertinant facts.
Below is text of handler_eg.cgi, which is also included in the package.
use CGI::UploadEngine;
use Template;
require "upload_cfg.pl";
require "upload_html.pl";
# Handle CGI variables
parse_form_data();
my $token = $FORM_DATA{token};
my $other = $FORM_DATA{other};
# Create an upload object
my $upload = CGI::UploadEngine->new();
# Retrieve file hash_ref
my $file = $upload->upload_retrieve({ token => $token });
# Set the view template
my $tmpl = "upload/handler_eg.tt2";
# Set the template variables
my $vars = { path_name => $file->{file_path} . "/" . $file->{file_name},
other => $other };
# Process the template
my $result;
my $template = Template->new({ INCLUDE_PATH => $tmpl_incl });
$template->process( $tmpl, $vars, \$result );
# Print the page
header();
print $result;
footer();The sample script prints the values on a page. That is probably not what you will do with it. Notice however that "other" form values are passed along with the token, which is used to retrieve file information in a perl hash reference.
INTERFACE
CGI::UE has four public methods (besides new()). The first and last are "developer methods", and are used by the person who wants to create a form mixing a file select box with other more common elements like text input, textareas, radio buttons, or a normal select box. The middle methods are "Core methods", and are accessed via a running URL pre-installed by the server administrator.
Developer Methods
Besides mastering these calls, developers must include the JavaScript library reference in the page header. CGI::UE uses the Ext library for cross-browser support.
new()
Create a new CGI::UE object.
# Create an upload object my $upload = CGI::UploadEngine->new();The returned object is used to access the next four methods.
upload_prepare()
The destination directory and file limits are passed to the engine using upload_prepare(), which returns a text string containing two HTML form elements and javascript.
my $injection = $upload->upload_prepare({ file_path => '/tmp' }) };If you have a template with your caption and a cell for the file upload box:
<tr> <td> Select file </td> <td> [% file_upload %] </td> </tr>You would fill it by assigning the results of upload_prepare() to the file_upload key:
# Set the template variables my $vars = { action_url => $root_url . "cgi-bin/handler_eg.cgi", file_upload => $upload->upload_prepare({ file_path => "/tmp" }) };Example results of a generated Package are shown below.
upload_retrieve()
However you retrieve the CGI variable, it is named "token".
# Handle CGI variables parse_form_data(); my $token = $FORM_DATA{token};Use the token to retrieve a hash reference with relevant results.
# Retrieve file hash_ref my $file = $upload->upload_retrieve({ token => $token }); my $vars = { path_name => $file->{file_path} . '/' . $file->{file_name} };Use the file information to further your nefarious schemes.
Core Methods
So far unmentioned is the fact that the Package injected into the template includes an "attempt token". When a file is selected by an end user, the JavaScript submits the attempt token along with the file to the server UE Core via AJAX. The Package includes the correct URL for UE Core, and swaps the Core URL with your URL before submitting the form for the first time. On success, your URL is replaced along with new HTML code that includes the success token.
upload_validate()
$file_obj = $upload->upload_validate({ token => $token });A Core implementation first calls upload_validate() with the attempt token. The returned file object has relevant details for the Core to validate the file and save it to the appropriate directory.
upload_success()
$success = $upload->upload_success({ token => $token, file_name => $file_name, file_size => $file_size});If the file is valid and uploads without fail, the Core updates the name and size, and returns a success token.
Token Generation
Tokens are generated simply using alphanumerics and underscore, through rand().
our $token_length = 60;
our @token_chars = ("a".."z","A".."Z","0".."9","_");
# Random string created from global package variables
foreach (1..$token_length) { $token .= $token_chars[rand @token_chars]; }Injected Package Example
The following is included as an accessible reference, and is not meant to be copied or used directly. The code below was generated by the private method _generate_html().
<tr>
<td> Select file </td>
<td>
<script type="text/javascript">
Ext.onReady(function() {
var file_upload_box = Ext.get("auto_file");
var auto_file_form = document.forms[0];
var action = document.forms[0].action;
var encoding = document.forms[0].encoding;
var file_id;
document.forms[0].action = "http://DOMAIN/upload";
document.forms[0].encoding = "multipart/form-data";
file_upload_box.addListener("change", function(el) {
Ext.Ajax.request({
form: auto_file_form,
success: function(response, opts) {
if(!response.responseText.match(/ERROR: /)){
file_id = Ext.util.JSON.decode(response.responseText).success;
Ext.fly(file_upload_box).remove();
Ext.fly(file_control).insertHtml("afterBegin", "<img src="http://DOMAIN/images/checked.jpg"> File Uploaded");
var token = document.getElementById("token");
token.value = file_id;
document.forms[0].action = action;
document.forms[0].encoding = encoding;
}else{
//we have an error.
Ext.fly(file_upload_box).remove();
//if possible decode the JSON
try{
file_id = Ext.util.JSON.decode(response.responseText).success;
Ext.fly(file_control).insertHtml("afterBegin", "<img src="http://DOMAIN/images/alert.jpg">"+file_id);
}catch(e){
//JSON decode failed so use responseText
Ext.fly(file_control).insertHtml("afterBegin", "<img src="http://DOMAIN/images/alert.jpg">"+response.responseText);
}
}
},
failure: function(response, opts) {
alert("server-side failure with status code " + response.status);
}
});
});
});
</script>
<div name="file_control" id="file_control">
<input type="file" size="50" name="auto_file" id="auto_file">
<input type="hidden" name="token" id="token" value="xGgbwIKjcM_SJ75uAvwwg4f3OtpQjcrULSmiUuXnSqYaFeZ8LoVxJUgrVg9a">
</div>
</td>
</tr>The elements are wrapped in a div named "file_control", which is used as the anchor to replace the elements with the success token.
DIAGNOSTICS
There are several, but they will not be described in the initial release.
Error message here, perhaps with %s placeholders-
[Description of error here]
Another error message here-
[Description of error here]
CONFIGURATION AND ENVIRONMENT
CGI::UploadEngine is not CPAN "install" friendly and requires significant configuration, but all files and directions are included in this section.
There is a custom perl install script (with bash execs). It might not work for your system, so the steps required are described below. Please read the directions first so you will understand what the script is asking from you.
Download, unzip, and expand the archve. All the commands below should be run from the root package directory, ls -lh should look (mostly) like this:
drwxr-xr-x 2 root root 4096 Oct 28 04:05 cgi-bin
-rw-r--r-- 1 root root 96 Oct 26 02:11 Changes
drwxr-xr-x 2 root root 4096 Oct 28 04:28 Controller
drwxr-xr-x 2 root root 4096 Oct 26 15:43 images
-rwxr-xr-x 1 root root 5677 Oct 26 15:58 install
drwxr-xr-x 3 root root 4096 Oct 26 02:11 lib
-rw-r--r-- 1 root root 535 Oct 26 02:11 Makefile.PL
-rw-r--r-- 1 root root 72 Oct 26 22:10 MANIFEST
-rw-r--r-- 1 root root 1096 Oct 26 02:55 README
drwxr-xr-x 3 root root 4096 Oct 26 02:47 root
drwxr-xr-x 2 root root 4096 Oct 26 02:53 scripts
drwxr-xr-x 2 root root 4096 Oct 26 02:47 sql
drwxr-xr-x 2 root root 4096 Oct 26 02:47 t
-rw-r--r-- 1 root root 162 Oct 26 02:51 uploadengineManual Installation
The following commands are what the install script attempts to accomplish. After you read through the steps, try the install script before trying to follow the steps manually.
If you are attempting manual installation, note that these commands were developed in bash shell. If you are unsure what shell you are using, go ahead and start bash now:
bashInitialize Variables
It will save a few steps if we get this out of the way first. The variables in question here are all related to target directories and server information. Values below are typical (for a Catalyst project MBC), but yours may be different.
UE_APACHE_ROOT=/var/www; export UE_APACHE_ROOT; UE_CGI_DIR=$APACHE_ROOT/cgi-bin; export UE_CGI_DIR; UE_IMAGE_DIR=$APACHE_ROOT/html/images; export UE_IMAGE_DIR; UE_EXT_PATH=$APACHE_ROOT/html/js/ext-core/ext-core-debug.js; export UE_EXT_PATH; UE_CAT_PROJ=MBC; export UE_DOMAIN; UE_CONTROLLER_DIR=/var/www/catalyst/MBC/lib/MBC/Controller; export UE_CONTROLLER_DIR; UE_TMPL_DIR=/var/www/catalyst/MBC/root/src; export UE_TMPL_DIR; UE_DOMAIN=bioinformatics.ualr.edu; export UE_DOMAIN; UE_ROOT_URL=http://$UE_DOMAIN/catalyst; export UE_ROOT_URL; UE_EXT_URL=http://$UE_DOMAIN/js/ext-core/ext-core-debug.js; export UE_EXT_URL;Install CGI::UE config
This configuration file allows server level settings for failure icon, success message, root URL, allowed file types, and future features. It has to be in a directory that is readable by Apache, and the path is currently hard-coded in CGI::UE.
Copy script to apache root directory
The script name is prepended with a period to "hide" it.
cp uploadengine $UE_APACHE_ROOT/.uploadengineUpdate config path in CGI::UE module
sed 's#APACHE_ROOT#$UE_APACHE_ROOT#g';
Install CGI::UE Module
After the apache root directory is updated, install the module:
perl Makefile.PL make install clean rm -f Makefile.old perl t/00.load.tInstall UE Database
The UE only requires a single table, but a new, separate database is recommended. The installation script will ask for your password in order to create the necessary mysql resource. To install manually, first create the database, here named files:
mysqladmin -p create filesThe create the table:
mysql -p files <sql/upload.sqlThe -p switch makes each program prompt you for the password. You will also need to create a user and password on the database for CGI::UE to use. The database name, user, and pass will need to match the values in cgi-bin/upload_cfg.pl.
Note: Do *NOT* use the example password "tmie"below. *DO* use an new, independent user such as "files".
mysql -p files -e "grant all privileges on files.* to 'files'\@'localhost' identified by 'tmie'";Install CGI::UE Core
As mentioned above, the only Core currently provided is a Catalyst module, and therefore requires a working Catalyst project.
Copy template
This template defines the response from the Core to the AJAX file upload.
cp root/src/upload.tt2 $UE_TMPL_DIR/upload.tt2Rename controller namespace
sed 's#UE_CAT_PROJ#$UE_CAT_PROJ#g';Copy controller
cp Controller/Upload.pm $UE_CONTROLLER_DIR/Upload.pmRestart apache
The installation script does not restart apache at all, but if you are installing manually, we recommend a restart at this point to test that CGI::UE is installed well enough to use.
Install Sample Developer App
Two types of sample apps are included: Catalyst controllers and "old school" CGI scripts. You might install both, but you should install at least one to test the installation.
In each case, there are two actions: form_eg and handler_eg. The first creates a sample form with a file select box and one other text box using the upload_prepare() method. The second uses upload_retrieve() to get the results from the engine. The scripts are shown above in the Synopsis. The controllers perform equivilant actions under Catalyst.
Install CGI scripts
The CGI scripts need several support files: templates for each action, a configuration file with web server details, and a library file that handles HTTP responses and HTML layout.
Install the javascript file
For security reasons, the jasvascript path needs to be from your server.
Update javascript path in config file
The javascript file is
Update URLs in config file
The domain
sed 's#APACHE_ROOT#$UE_APACHE_ROOT#g';
Install Catalyst controllers
cp Controller/Upload.pm $UE_CONTROLLER_DIR/Upload.pmRestart apache
The installation script does not restart apache at all, but if you are installing manually and chose to install the sample controllers, we recommend a restart at this point to test that the Controllers will load.
Installation Script
To install, type:
bash
perl installUEDEPENDENCIES
Class::StdINCOMPATIBILITIES
None reported.
BUGS AND LIMITATIONS
No bugs have been reported.
Please report any bugs or feature requests to bug-cgi-uploadengine@rt.cpan.org, or through the web interface at http://rt.cpan.org.
AUTHORS
Roger A Hall C<< <rogerhall@cpan.org> >>
Michael A Bauer C<< <mbkodos@cpan.org> >>
Kyle D Bolton C<< <kdbolton@ualr.edu> >>
Aleksandra A Markovets C<< <aamarkovets@ualr.edu> >>LICENSE AND COPYRIGHT
Copyleft (c) 2010, Roger A Hall <rogerhall@cpan.org>. All rights pre-served.
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.
DISCLAIMER OF WARRANTY
BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENSE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.