Blog

- December 10, 2015

HTTP provides a smart mechanism to manage formats of exchanged payloads. This mechanism is responsible for selecting the appropriate representation when servicing an HTTP request. This feature is called content negotiation or CONNEG. In this post, we will describe how to use it, as well as the different headers involved and the possible status codes for corresponding errors.

As it’s used by browsers under the hood when interacting with a HTTP server, you can just as well leverage this mechanism when interacting with a RESTful service. This allows you to get data formats you expect and you can handle in your application. In addition, when implementing such a service, you can provide support for several formats and allow the client to choose the one he wants to use.

Before actually dealing with content negotiation, let’s define what a content type is and how to use it within HTTP requests and responses.

Defining content format

When you put data in the payload of a HTTP message within a request or a response, you must specify the corresponding format using a Content-Type header. Usable values correspond to media types (initially called MIME types).

A comprehensive list of media types is available on IANA website: http://www.iana.org/assignments/media-types/media-types.xhtml.

It is mandatory to allow the client or the server to know the format and make it possible to handle data. Here is a sample use of the header within a HTTP request:

POST /contacts HTTP/1.1
Content-Type: application/json

{
  "lastName": "Fielding",
  "firstName": "Roy"
}

HTTP/1.1 201 OK
Location: http://myrestservice/contacts/1

Media type structure

More generally media types follow the format top-level type name / [ tree. ] subtype name [ +suffix ] [ ; parameters ]. This is defined in the RFC 4288 (https://tools.ietf.org/html/rfc4288). So the value of this header contains a value with a lop-level type like text or application and a sub type like xml or json. This value can also include some optional parameters like the character encoding with the charset one. Notice that these parameters can differ depending on the header used.

Here is a sample with the Content-Type header:

Content-Type: application/json; charset=UTF-8

This header is used by the server or the client to select the right approach to handle (and eventually parse) the data.

Additional content headers

Some other headers can also be used to give more details about the content:

  • The Content-Encoding header indicates which additional content encodings have been applied to the payload, and thus what decoding mechanisms must be applied in order to obtain the media type referenced by the Content-Type header field. The Content-Encoding header is primarily used to allow a document to be compressed without losing the identity of its underlying media type.
  • The Content-Language header describes the natural language(s) of the intended audience for the enclosed entity.

Here is a use sample of these two additional headers:

GET /contacts HTTP/1.1
(...)

HTTP/1.1 201 OK
Content-Type: application/json
Content-Language: en
Content-Encoding: deflate

To get the content, the client needs to uncompress what was received in the payload.

Unsupported media type

There is no obligation for the server to support the format sent. In this case, it will send back a 415 status code (Unsupported media type) to notify the client that made the request. “The request entity has a media type that the server or resource does not support”, according to the specification.

415

Now you understand what a media type is and how it identifies the payload format, let’s dive into the heart of the topic: how to tell the server to return the expected content.

Telling the server what format you expect

When sending a request, you can specify the data format(s) you expect within the response payload. Defining several values is possible since clients are able to handle several formats.

The Accept header should be used for this purpose. Here is a simple sample to get a list of contacts with the JSON format:

GET /contacts HTTP/1.1
Accept: application/json

HTTP/1.1 200 OK
Content-Type: application/json

[
  {
    "id": 1,
    "lastName": "Fielding",
    "firstName": "Roy"
  }
]

Defining a set of media types

As discussed in the first section, media type values are structured. This is particularly important in the context of the content negotiation since you can use the asterisk * character to group media types in ranges. For example, the */* value indicates all media types and type/* all subtypes of that type.

If you can handle all text formats, simply use the following value in the Accept header:

Accept: text/*

When no Accept header is defined, it corresponds to the */* value i.e. any format can be returned and you let the server choose the format.

If the server cannot support a particular media type, you can define alternatives since the Accept header lets you define a set of media types.

Defining preferences

When defining several values in the Accept header, it’s important to specify your preference order, i.e. which is your first preference according to what the server supports. The q parameter gives you the ability to define this order. Let’s take a sample:

Accept: application/json,application/xml;q=0.9,*/*;q=0.8

Let’s explain what this means. This allows you to ask the server a JSON format. If it can’t, perhaps it could return XML format (the second level). If it’s still not possible, let it return what it can. The preference order is defined through the q parameter with values from 0 to 1. When nothing is specified, the implicit value is 1.

Note that an implicit order is supported as shown in this simple sample:

Accept: text/*, application/json, */*

In this case, the preference order will be application/json, text/* and finally */*. Moreover with two values like application/json and application/xml, the order in the header value will be used for the preference order.

Additional headers

Other headers can also apply within content negotiation to build more advanced expectations using charsets, encodings and languages:

  • Accept-Charset defines the character sets that are acceptable.
  • Accept-Encoding contains the list of acceptable encodings. Possible values are gzip and deflate.
  • Accept-Language contains the list of acceptable human languages for response.

When you specify a set of Accept-like headers with possible preferences, the server implementation needs to take everything into account in order to return the correct content. This can be tricky and the RFC 2296 specification “HTTP Remote Variant Selection Algorithm” describes the algorithm to implement based on scores.

Non supported requested format

If the server doesn’t support the requested format, it will send back a 406 status code (Not Acceptable) to notify the client that made the request. “The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request”, according to the specification.

406

Conclusion

As described throughout this post, HTTP natively supports a mechanism based on headers to tell the server about the content you expect and you’re able to handle. Based on these hints, the server is responsible for returning the corresponding content in the correct format.

Such mechanism is typically used by browsers under the hood when trying to interact with a website. RESTful services can also implement such a mechanism to adapt the exchanged contents according to the formats supported by clients.

The Restlet framework and the Restlet Cloud platforms both natively support such mechanisms.