The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
App-MFILE-WWW-0.097
River stage one • 1 direct dependent • 1 total dependent
/ App::MFILE::WWW

NAME

App::MFILE::WWW - Generic web front-end with demo app

VERSION

Version 0.097

LICENSE

This software is distributed under the "BSD 3-Clause" license, the text of which can be found in the file named COPYING in the top-level distro directory. The license text is also reprodued at the top of each source file.

DESCRIPTION

This distro contains a generic framework for developing web front-ends to REST resources. The framework consists of a web server based on Plack and Web::Machine, CSS and HTML for the "app frame" (on-screen area where the application's "screens" are displayed), and "widgets" for defining the application's login dialog, menus, forms, and actions.

For illustration, the distro contains a demo app that authenticates against App::Dochazka::REST and contains a single menu, a simple form, and a sample action.

STACK

The full stack of which App::MFILE::WWW is a part consists of the following components:

  • Database engine

    For storing application data.

  • Perl DBI

    For interfacing between the Perl code and the database engine.

  • REST server

    A REST server, such as App::Dochazka::REST, implements a data model and provides a HTTP interface to that model.

  • optional CLI client/frontend

    An optional Command Line Interface (frontend) can provide a command line interface to the REST server.

  • WWW client/frontend

    The WWW frontend, built based on this distro, is a web server that serves HTML, CSS, and JavaScript code to users to provide them with a menu-driven "browser experience" of the application.

Conceptually, the clients (frontends) act as proxies between the user and the REST server. Taking this one step further, the REST server itself is a proxy between the client and the database engine.

From a technical perspective, the strict separation between the REST server and its clients makes the application as a whole more robust.

DERIVED WWW CLIENTS

The philosophy behind the stack design described above is that you, the user, have the freedom and the flexibility to write your own client, on any platform, in any language -- however you see fit. In other words, you are not forced to use any particular client. Conceivably, you can even communicate with the REST server without any client at all.

However, writing a client is time- and labor-intensive. Although App::MFILE::WWW is capable of standalone operation, it is designed as a "foundation" upon which derived clients can be written.

Standalone operation

App::MFILE::WWW can be run as a standalone HTTP server. Authentication is disabled by default, so no REST server is needed in this scenario.

Before a derived client can be written, the developer must first understand how App::MFILE::WWW is structured. This is easily understood by examining how it works in standalone mode.

Assuming App::MFILE::WWW has been installed properly, it can be started in standalone mode by running mfile-www, as a normal user, with no arguments or options. Here is a basic description of what happens in this scenario -- refer to the script source code in bin/mfile-www for better understanding:

  • by default (in the absence of the --ddist option), $ddist is set to the empty string

  • $ddist_dir is not set

  • the script calls the App::MFILE::WWW::init routine, which loads the configuration parameters stored in config/WWW_Config.pm of the core distro (App::MFILE::WWW) sharedir

  • since no sitedir option was specified on the command line, no other configuration files are loaded

  • the configuration parameters and their core default values can be seen in config/WWW_Config.pm under the core distro (App::MFILE::WWW) sharedir

  • a very important configuration parameter is MFILE_WWW_LOG_FILE, which is the full path to the log file where the Perl side of App::MFILE::WWW will write its log messages -- by default, this is set to '.mfile-www.log' in the user's home directory, and the current setting is always reported on-screen by the startup script so the user knows where to look if something goes wrong

  • the HTTP server is started by calling Plack::Runner, and the script reports to the user the port number at which it is listening (5001 by default)

  • the HTTP server always interprets URL paths it receives relative to its "root" (called HTTP_ROOT for the purposes of this document), which is set to the core distro (App::MFILE::WWW) sharedir in this case

  • JS and CSS files are considered "static content" and will be served from HTTP_ROOT/js and HTTP_ROOT/css, respectively

  • when an HTTP 'GET' request comes in on the port where the HTTP server is listening, and it is not requesting static content, the request is passed on to the Web::Machine application (effectively, App::MFILE::WWW::Resource) for processing

  • POST requests are assumed to be AJAX calls and are handled by the process_post routine of App::MFILE::WWW::Resource

  • GET requests are assumed to have originated from a web browser running on a user's computer -- to handle these, the main_html routine of Resource.pm generates HTML code which is sent back in the HTTP response

  • the HTML so generated contains embedded JavaScript code to start up RequireJS with the required configuration and pass control over to "the JavaScript side" of App::MFILE::WWW

The embedded JavaScript code does the following:

  • sets the baseURL to $site-MFILE_WWW_REQUIREJS_BASEURL>, which is set to /js -- in absolute terms, this means HTTP_ROOT/js

  • sets the "app" path config to $site-MFILE_APPNAME> -- for example, if $site-MFILE_APPNAME> is set to 'foobar', the path config for app will be set to foobar and a RequireJS dependency app/bazblat on the JavaScript side will translate to HTTP_ROOT/js/foobar/bazblat.js

  • in this particular case, of course, MFILE_APPNAME is set to mfile-www

  • persuades RequireJS via magic incantations to "play nice" together with jQuery and QUnit

  • by calling requirejs.config, brings in site configuration parameters needed on the JavaScript side so they can be accessed via the cf JS module

  • passes control to the app/main JS module

What happens on the JavaScript side is described in a different section of this documentation.

Derived client operation

In a derived-client scenario, App::MFILE::WWW is basically used as a library, or framework, upon which the "real" application is built.

The derived-client handling is triggered by providing the --ddist command-line option, i.e.

    $ mfile-www --ddist=App-Dochazka-WWW

Where 'App-Dochazka-WWW' refers to the Perl module App::Dochazka::WWW, which is assumed to contain the derived client source code.

So, in the first place it is necessary to create such a Perl module. The App::MFILE::WWW module can be used as a template. It should have a sharedir configured and present.

Here is a "play-by-play" description of what happens in this scenario when the startup script is run. Again, refer to the script source code in bin/mfile-www for better understanding:

  • $ddist is set to the string given in the --ddist option, e.g. App-Dochazka-WWW (or 'App::Dochazka::WWW' in which case it will be converted to the correct, hyphen-separated format)

  • $ddist_dir is set to File::ShareDir::dist_dir( $ddist ), i.e. the derived distro sharedir (extending the above example, the distro sharedir of App::Dochazka::WWW)

  • the presence of the --ddist option triggers a special routine whose purpose is to ensure that the derived distro exists and that its sharedir is properly set up to work with App::MFILE::WWW:

        =over

    =item * error exit if the distro referred to by the C<--ddist> option
    doesn't exist 

    =item * error exit if the distro lacks a sharedir

    =item * C<css> and C<js/core> need to exist and be symlinks to the same
    directories in the L<App::MFILE::WWW> sharedir. If this is not the
    case, the script displays a message asking the user to re-run the
    script as root

    =item * if already running as root, the symlinks are created and the
    script displays a message asking to be re-run as a normal user

    =item * once the symlinks are in place, the script runs some sanity
    checks (mainly verifying the existence of certain files in their
    expected places)

    =back

  • the script calls the App::MFILE::WWW::init routine, which loads the configuration parameters stored in the following places:

        =over

    =item * the L<App::MFILE::WWW> distro sharedir (under C<config/WWW_Config.pm>)

    =item * the derived distro sharedir (also under C<config/WWW_Config.pm>)

    =item * finally and optionally, if a sitedir was specified on the
    command line -- for example C<--sitedir=/etc/dochazka-www> --,
    configuration parameters are loaded from a file C<WWW_SiteConfig.pm> in
    that directory, overriding the defaults

    =back

  • the derived distro's configuration should override the MFILE_APPNAME parameter -- in our example, it could be set to 'dochazka-www'

  • also refer to the previous section to review the explanation of the MFILE_WWW_LOG_FILE parameter

  • the HTTP server is started by calling Plack::Runner, and the script reports to the user at what port number it is listening (5001 by default)

  • the HTTP server always interprets URL paths it receives relative to its "root" (called HTTP_ROOT for the purposes of this document), which is set to the derived distro's sharedir

  • the rest of the description is the same as for "Standalone operation"

REQUEST-RESPONSE CYCLE

The HTTP request-response cycle is implemented as follows:

  • nginx listens for incoming connections on port 80/443 of the server

  • When a connection comes in, nginx decrypts it and forwards it to a high-numbered port where a PSGI-compatible HTTP server (such as Starman) is listening

  • The HTTP server takes the connection and passes it to the Plack middleware. The key middleware component is Plack::Middleware::Session, which assigns an ID to the session, stores whatever data the server-side code needs to associate with the session, links the session to the user's browser via a cookie, and provides the application a hook (in the Plack environment stored in the HTTP request) to access the session data

  • if the connection is asking for static content (defined as anything in images/, css/, or js/), that content is served immediately and the request doesn't even make it into our Perl code

  • any other path is considered dynamic content and is passed to Web::Machine for processing -- Web::Machine implements the HTTP standard as a state machine

  • the Web::Machine state machine takes the incoming request and runs it through several functions that are overlayed in App::MFILE::WWW::Resource - an appropriate HTTP error code is returned if the request doesn't make it through the state machine. Along the way, log messages are written to the log.

  • as part of the state machine, all incoming requests are subject to "authorization" (in the HTTP sense, which actually means authentication). First, the session data is examined to determine if the request belongs to an existing authorized session. If it doesn't, the request is treated as a login/logout attempt -- the session is cleared and control passes to the JavaScript side, which, lacking a currentUser object, displays the login dialog.

  • once an authorized session is established, there are two types of requests: GET and POST

  • incoming GET requests happen whenever the page is reloaded - in an authorized session, this causes the main menu to be displayed, but all static content (CSS and JavaScript modules) are reloaded for a "clean slate", as if the user had just logged in.

  • Note that App::MFILE::WWW pays no attention to the URI - if the user enters a path (e.g. http://mfile.site/some/bogus/path), this will be treated like any other page (re)load and the path is simply ignored.

  • if the session is expired or invalid, any incoming GET request will cause the login dialog to be displayed.

  • well-formed POST requests are assumed to be AJAX calls and are directed to the process_post routine, which first examines the request body, which must adhere to a simple structure:

        { method: "GET", path: "employee/current", body: { ... } }

    where 'method' is any HTTP method accepted by the REST server, 'path' is a valid path to a REST server resource, and 'body' is the content body to be sent in the HTTP request to the REST server. Provided the request is properly authorized and the body is well-formed, the request is forwarded to the REST server via the App::MFILE package's rest_req routine and the REST server's response is sent back to the user's browser, where it is processed by the JavaScript code.

  • under ordinary operation, the user will spend 99% of her time interacting with the JavaScript code running in her browser, which will communicate asynchronously as needed with the REST server via AJAX calls.

DEVELOPMENT NOTES

The App::MFILE::WWW codebase has two parts, or "sides": the "Perl side" and the "JavaScript side". Control passes from the Perl side to the JavaScript side

  • synchronously whenever the user (re)loads the page

  • asynchronously whenever the user triggers an AJAX call

JavaScript side

Modular (RequireJS)

The JavaScript code is modular. Each code module has its own file and modules are loaded asynchronously by RequireJS.

Unit testing (QUnit)

The JavaScript code included in this package is set up for unit testing using the QUnit http://qunitjs.com/ library.

UTF-8

In conformance with the JSON standard, all data passing to and from the server are assumed to be encoded in UTF-8. Users who need to use non-ASCII characters should check their browser's settings.

Deployment

To minimize latency, App::MFILE::WWW can be deployed on the same server as the back-end (e.g. App::Dochazka::REST), but this is not required.

PACKAGE VARIABLES

For convenience, the following variables are declared with package scope:

FUNCTIONS

init

Initialization routine - run from bin/mfile-www, the server startup script. This routine loads configuration parameters from files in the distro and site configuration directories, and sets up logging.