• HTTP Callback

(note: HTTP Callback replaces JSONRequest which had to be renamed due to a conflict with an existing project)

HTTP Callback

This article is in a Draft form. It should not be treated as canonical.

Introduction and Motivation

One common pattern that seems to come up building microapps is abstracting HTTP requests in a way that they can be stored, repeated, etc. Eg, a microapp that is in planning is a "cron" like app that will make requests to other microapps on a recurring basis. A first pass at designing the interface might look like this:

   % curl -X PUT -d "recurrence=hourly&url=http://app2.example.com/some/path/" http://cron.example.com/jobs/my-new-job/

And it would make a GET request to 'http://app2.example.com/some/path/' every hour. Of course, it's immediately obvious that we probably shouldn't limit things to GET requests. So next, we allow a method to be specified:

   % curl -X PUT -d "recurrence=hourly&method=POST&url=http://app2.example.com/some/path/" http://cron.example.com/jobs/my-new-job/

Then the cron app can make a POST (or PUT, or DELETE or whatever) request to the url every hour. But if you're making a POST or PUT request, you probably want some way to specify additional parameters (since they wouldn't go into the url), and maybe a body for the request, and even headers.

That's where this spec comes in. Since it seems to happen fairly often that a "request" needs to specified and stored somewhere or passed around between microapps, it should be useful to have a single, agreed upon format for them so that standard libraries and tools can be built to handle and process them. Essentially, a request object functions like a callback in a more traditional system.

This spec is an attempt to roughly standardize the request object that gets passed around and stored. If there is a single agreed-upon set of attributes, it should be possible to have tools and libraries to further simplify this type of pattern. Eg, converting between the HTTP Callback and different 'formats' like a WSGI environ dictionary, normalizing parameters, or integrating directly with HTTP/REST wrapper libraries like restclient.

Format

HTTPCallback attempts to be as close as possible to the XHR object that MochiKit uses but does not include the mimetype attribute that only makes sense within a javascript context. It also includes a couple extra attributes to support httplib2 type functionality (auth, redirect following) extensibility (kwargs), and forward-compatibility (version).

A HTTPCallback is a JSON Object with the following attributes:

The only required attribute is the url.

Typically, a request will be serialized to JSON for transmission. An XML or other representation should also be possible (and may be specified later).

Variable Substitution

Sometimes it's also useful for the application dealing with the request to be able to dynamically substitute in values for some of the parameters, etc. This spec should cover a standard way of specifying how those substitutions are carried out.

For substitution in the URI, the existing URI Template syntax should be used.

Substitution in the method should not be allowed (and wouldn't make much sense anyway).

Substitution in the params and headers may be possible. Probably using a syntax similar to URI Templates. Will require more thought.

The application executing the request should also be able to insert additional params and headers. Which params/headers are added and their values should be specified and documented on a per-application basis.

Examples

Some samples of HTTCallback objects and the rough HTTP request that would be generated from it. (note: Actual HTTP requests might include additional headers, etc. These examples are trying to just show the relevant ones).

Simplest Possible:

    {"url" : "http://www.example.com/foo"}
    
    GET /foo HTTP/1.1
    Host: www.example.com
    Content-Length: 0

Specifying a method:

     {"url" : "http://www.example.com/foo", "method" : "POST"}
     
     POST /foo HTTP/1.1
     Host: www.example.com
     Content-Length: 0

Specifying params:

     {"url" : "http://www.example.com/foo", "method" : "POST", "params" : {"foo" : "bar"}}
     
     POST /foo HTTP/1.1
     Host: www.example.com
     Content-Length: 7
     Content-Type: application/x-www-form-urlencoded
     
     foo=bar

With a queryString instead:

    {"url" : "http://www.example.com/foo","method" : "POST", "queryString" : "foo=bar"}

    POST /foo HTTP/1.1
    Host: www.example.com
    Content-Length: 7
    Content-Type: application/x-www-form-urlencoded

    foo=bar

Specifying params on a GET:

     {"url" : "http://www.example.com/foo", "method" : "GET", "params" : {"foo" : "bar"}}
     
     GET /foo?foo=bar HTTP/1.1
     Host: www.example.com
     Content-Length: 0

Or a queryString on a GET:

     {"url" : "http://www.example.com/foo", "method" : "GET", "queryString" : "foo=bar"}

     GET /foo?foo=bar HTTP/1.1
     Host: www.example.com
     Content-Length: 0

If both a queryString and params are specified, they get combined:

     {"url" : "http://www.example.com/foo", "method" : "POST", "queryString" : "foo=bar", "params" : [["baz","qux"]]}

     POST /foo HTTP/1.1
     Host: www.example.com
     Content-Length: 15
     Content-Type: application/x-www-form-urlencoded
     
     foo=bar&baz=qux

Specifying headers:

     {"url" : "http://www.example.com/foo", "headers" : [["Accept","text/*"]]}

     GET /foo HTTP/1.1
     Host: www.example.com
     Content-Length: 0
     Accept: text/*

Reference Implementation

There is a reference implementation of a HTTPCallback library for Python. It performs some of the basic normalization, can marshal into and out of python dictionaries, JSON strings, and WSGI environ dictionaries, and does basic URI Templating.

Open Issues

• proxy support?
• 'queryString' is camelCase to mirror mochikit, but 'follow_all_redirects' uses underscores due to its roots in python's httplib2. Is it more important for the parameter names to reflect their origins or to be consistent? If consistency is more important, I'd be inclined to go with camelCase since these are JSON objects.
• support for HTTP versions other than 1.1? (I'm thinking there's no real good reason for support for older versions, but maybe forward compatibility?)
• encoding. allow it to be specified separately, or just default to utf-8 and make them override the entire Content-Type header anytime they want to use something different?
• instead of a 'kwargs' attribute for extensibility would it be cleaner to just specify that any additional parameters can be added to the JSONRequest object but they will be stripped out before the request is made?