The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
WebAPI-DBIC-0.002007
River stage one • 1 direct dependent • 1 total dependent
18 ++
/ data_service_api

AUTHENTICATION

Currently we're using HTTP Basic authentication and using the supplied credentials as the credentials for database access. So whatever username and password are provided with the request will be used to perform the request on the database.

Clearly https is needed if auth is required and the interface is public!

Current Basic auth leaves db password in browsers auth cache or keychain.

    Digest auth http://en.wikipedia.org/wiki/Digest_access_authentication
    http://search.cpan.org/perldoc?Plack%3A%3AMiddleware%3A%3AAuth%3A%3ADigest

    Also http://stackoverflow.com/questions/319530/restful-authentication

IMPLEMENTATION NOTES

We use the existing DBIx::Class classes directly as the entity definitions exposed by the data service. This is reasonable given that it's an internal service so there's little value in abstracting it further.

The pattern of set, set/id, set/id/relation, set/id/relation/id can continue to any depth but we're using a flat namespace for now. If we used a deeper path it would represent 'is identified by' (or 'belongs to') relationships. This complicates the caching/invalidation though, so we'll keep it flat for now.

XXX Allow overriding the method via a url param, e.g. for testing: GET /dogs?method=delete TBD

XXX Allow overriding the return status via a url param, e.g. for testing: GET /dogs?suppress_response_codes=true would return 200 status, but the content would still be the error details.

ETag etc [LATER]

For datetime use UTC and ISO8601 - automation via TBD

Review the Web Linking spec (RFC5899) and HAL for use of rel links, eg first/prev/next/last. [LATER]

Formalize the error response contents.

Validation

Validation in the data service requires defining an approach to parameter validation (eg adopting a module like Params::Validate plus Moose attribute validations) and defining an approach to throwing and handling exceptions.

Specifically it should provide sufficient information to the client, via the JSON response, to enable the client to update the form to indicate *which* field(s) are associated with the error.

Similarly exceptions raised due to database constraint errors should also generate client-useful exceptions with field information if possible. Note that this may require some mapping of database field names to json entity type field names.

CACHING

Implementing caching is easy. Implementing efficient caching (where each resource is only cached one - a canonical copy) and cache invalidation (eg a trigger on a table can invalidate the cached copy of affected rows) is, er, non-trivial.

ESI

Edge Side Include is the "secret sauce" that enables caching (and cache invalidation) to work nicely with prefetch and HAL.

Consider a simple request like /foo/42. The response looks like:

    {
        id: 42,
        ...foo fields...
    }

A database trigger on the foo table could be used to invalidate the cache for a particular /foo/:id record when that record is updated. So far so good.

Now consider a request with prefetch: /foo/42&prefetch=bar where the response looks like:

    {
        id: 42,
        ...other foo fields...
        _embedded: {
            bar: { id: 97, ...other bar fields...}
        }
    }

Now there's no simple way to invalidate that cached response when the corresponding record in the bar table is updated.

This is where ESI comes in. The response from the API would look like this:

    {
        id: 42,
        ...other foo fields...
        _embedded: {
            bar: <esi:include src="/bar/97">
        }
    }

the ESI processor (eg varnish) caches that unprocessed response and then processes the ESI requests embedded in it. So it makes a separate request for "/bar/97" (which may well be resolved from its own cache) and builds the response to send to the client.

The same database triger mechanism on the bar table will invalidate the cached /bar/97 response when the corresponding record in the bar table is updated.

With ESI, this invalidation affects all cached responses.

    http://odino.org/some-common-questions-about-edge-side-includes/
    http://stackoverflow.com/questions/11781576/most-secure-javascript-json-inline-technique

Also look into "Surrogate-Capability & Surrogate- Control headers for ESI based block caching"

Varnish

The varnish cache (see http://varnish-cache.org) supports basic ESI and also enables alternative approaches that might be useful:

* X-depends-on - e.g. http://www.smashingmagazine.com/2014/04/23/cache-invalidation-strategies-with-varnish-cache/

* https://www.varnish-cache.org/utilities?field_utility_category_tid=16

* http://www.hward.com/varnish-cache-invalidation-with-fastly-surrogate-keys

RESEARCH

REST Core concepts and specifications:

    http://en.wikipedia.org/wiki/Representational_State_Transfer
    http://www.w3.org/Protocols/rfc2616/rfc2616.html (HTTP Spec)
    https://github.com/basho/webmachine/wiki/Diagram

Best practice (hint: there's isn't one, just lots of suggestions):

    http://www.slideshare.net/Wombert/designing-http-interfaces-and-restful-web-services-sfliveparis2012-20120608
    http://www.infoq.com/articles/webber-rest-workflow
    https://restful-api-design.readthedocs.org/en/latest/
    http://www.stormpath.com/blog/designing-rest-json-apis
    http://www.slideshare.net/guilhermecaelum/rest-in-practice (XML)
    http://www.slideshare.net/apigee/restful-api-design-second-edition (107 slides)
    http://www.foxycart.com/blog/the-hypermedia-debate#.UT8PSKVYXdk

PUT vs POST

    http://jcalcote.wordpress.com/2008/10/16/put-or-post-the-rest-of-the-story/
    http://benramsey.com/blog/2009/11/post-vs-put/
    http://techno-weenie.net/2011/4/28/my-put-requests-bug-me/

Example APIs

    http://developer.github.com/v3/
    http://bitworking.org/projects/atom/rfc5023.html

API Design

    http://www4.in.tum.de/~blanchet/api-design.pdf

Linking

    http://amundsen.com/media-types/linkrelations/
    http://www.iana.org/assignments/link-relations/link-relations.xml
    http://tools.ietf.org/html/rfc5988
    http://www.mnot.net/blog/2011/11/25/linking_in_json (see also the comments)

HAL - Hypertext Application Language

    http://blog.stateless.co/post/13296666138/json-linking-with-hal
    http://stateless.co/hal_specification.html
    http://tools.ietf.org/html/draft-kelly-json-hal
    http://haltalk.herokuapp.com/explorer/browser.html
    http://www.quora.com/REST-software-architectural-style/JSON-+-Hypermedia-Using-HAL-in-Production

URI Template

    http://tools.ietf.org/html/rfc6570
    https://metacpan.org/module/URI::Template

CURIE Syntax - Compact URIs

    http://www.w3.org/TR/curie/

Partial reponses

    http://blog.apigee.com/detail/restful_api_design_can_your_api_give_developers_just_the_information
    https://developers.google.com/+/api/#partial-response

Other references:

    http://www.programmableweb.com
    http://www.programmableweb.com/apis/directory/1?protocol=REST&format=JSON
    http://www.slideshare.net/jmusser/j-musser-apishotnotgluecon2012
    http://nocarrier.co.uk/2012/09/hypermedia-types-and-connection-negotiation/

Restful Objects:

    http://en.wikipedia.org/wiki/Restful_Objects
    http://www.infoq.com/articles/Intro_Restful_Objects
    Demo: http://simple-dusk-6870.herokuapp.com/arow-ronet.html#
    http://skillsmatter.com/podcast/design-architecture/restful-objects (video)

Assorted Proposed Standards

    http://json-ld.org

    https://github.com/kevinswiber/siren

    http://librelist.com/browser//hypermedia/2012/5/2/notes-on-hal-and-collection+json/

Error Formats

    https://github.com/blongden/vnd.error
    http://tools.ietf.org/html/draft-nottingham-http-problem-02