The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
App-Wallflower-1.015
River stage one • 1 direct dependent • 1 total dependent
8 ++
/ Wallflower::Tutorial

NAME

Wallflower::Tutorial - Turn your Plack app into a static site

VERSION

version 1.015

SYNOPSIS

    # do cool stuff using Plack-aware frameworks,
    # to generate static web sites

INTRODUCTION: THE BENEFITS OF STATIC WEBSITES

Static websites have a lot of advantages, when serving data that changes rarely:

  • speed

  • security

  • deployment by copy

Dynamic websites have more to do with JavaScript than with POST or query strings. JavaScript code run on the client can take advantage of JSON files containing useful data. Updates to the data do not necessarily have to be performed on the public side of the site.

WRITING A PSGI APPLICATION FOR WALLFLOWER

The point of saving URLs from a PSGI application to files is not simply to make a static version of a fully dynamic site.

Unlike many static website generators, wallflower does not enforce any conventions on its users.

wallflower makes it very easy to write a static website using the web framework of your choice.

To make the most of a static website, there are a few simple rules to follow, listed below:

All URLs should have an extension

A PSGI/Plack application can specify the Content-Type returned for any URL content.

On the other hand, static servers use the file extension to decide what Content-Type to send for a given file. When unable to decide what the file content is, servers usually send application/octet-stream as the Content-Type.

Since the goal of a Wallflower application is to generate a static website, all URL pathnames should have an extension.

All responses should have a Content-Type header

Similarly, wallflower decides to check the body of a response for links it may contain based on its Content-Type header, as links only make sense for HTML and CSS files.

If the response has no Content-Type header, wallflower will miss some of the links, and thus not properly crawl the whole site.

Hide the non-static elements in a specific Plack environment

Because the website can be written with any modern web framework, it's also very easy to have URLs that reply to POST requests. Obviously, these pages cannot be saved on the static destination.

It's very easy to use the Plack environment to decide which pages to enable. By default, plackup sets the Plack environment to development, while wallflower sets it to deployment.

It's therefore possible to hide the dynamic parts of the application in the development environment, while only the static elements are reachable from the deployment environment.

Speed up site regeneration by sending 304 responses

Since version 1.002, if the target file exists, wallflower will send the If-Modified-Since header with its modification date. If the application sends a 304 Not Modified response, the target file will not be modified.

As of version 1.008, if the application sends the Last-Modified header, it we be used to set the modification time of the target file.

As of version 1.014, if the 304 response contains a Content-Type header, and wallflower was run with the --follow option, it will looked for more links to follow in the existing file.

Note that since wallflower works without using the network, this feature is only really useful if the response generation takes time and it's possible to decide to send a 304 Not Modified without computing the entire response content.

See Plack::Middleware::ConditionalGET for basic support of 304 responses.

Make all URLs eventually reachable from the root

By default, wallflower starts by loading /, and automatically and repeatedly follows all URLs found in HTML and CSS documents, until all reachable URLs have been processed.

The simplest way to generate the full static site is to make sure that all URLs can be reached by repeatedly following links from the root.

When using the --url option to "mount" your application under some path, don't forget to call wallflower with that path as the root:

    wallflower --application myapp.psgi --url http://localhost/myapp /myapp/

In case all of the application URL are not reachable from the root, it is possible to pass wallflower a list of starting points (one per line), using the --filter option:

    wallflower --application myapp.psgi --filter urls.txt

Note that the --filter option turns wallflower into a "filter" command: it takes a list of files as argument, and if no file is given, it reads the list of URL from the command-line.

URL SEMANTICS COMPARED TO DIRECTORY SEMANTICS

A Plack application deals with URLs. Nothing prevents the application from treating /thunk and /thunk/ differently and returning different content bodies for them.

From the perspective of a filesystem, however, if thunk is a directory, then it is semantically equivalent to thunk/. And it is impossible to have both a file and a directory with the same name inside the same parent directory.

When dealing with a static site, the server maps the URLs to the filesystem. If thunk is a directory, the server usually redirects the client to thunk/ (using a 301 Moved Permanently response). Then the default file is picked for the content, traditionally thunk/index.html.

Because Wallflower does not know the conventions used by the Plack application it calls, it cannot decide if /thunk should be understood as a "file" or as a "directory" when generating the file name that will receive the content for that URL.

So, if your application treats /thunk and /thunk/ as identical, you should:

  • Always request /thunk/ when providing a list of URLs to wallflower. Otherwise you might end up with the thunk file already existing when it tries to create files under the thunk/ directory, or vice-versa.

  • Make sure that links provided by your application are always to /thunk/, so that when wallflower follows links automatically, we're back to the previous case.

  • Try to make the application reply with status 301 Moved Permanently to requests for thunk. This is what a well-behaved web server serving static pages will do when a user-agent requests a "directory" without a final /.

At this point, you know that an application that treats /thunk ("file") and /thunk/ ("directory") differently will not work with wallflower.

wallflower will show a status code of 999 (not a valid HTTP status) in the following two cases:

  • when trying to create thunk/pam after having created a thunk file, (warning: Can't open thunk/pam for writing: Not a directory)

  • when trying to create thunk after having created a thunk/ directory. (warning: Can't open thunk for writing: Is a directory)

See also the section "All URLs should have an extension", for why you should avoid extensionless URLs.

REFERENCES

A few articles about wallflower have been published, and are listed below:

If you're interested in static web site generators, you can check:

AUTHOR

Philippe Bruhat (BooK) <book@cpan.org>

COPYRIGHT AND LICENSE

Copyright 2012-2018 by Philippe Bruhat (BooK).

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.