ploeh blog danish software design

NDC 2013 session recordings are now available, including mine.

The NDC 2013 conference is over, and most (if not all) of the session recordings are now available. Specifically, both of the talks I gave were recorded and are now available for general viewing.

• Faking Homoiconicity in C# with graphs
• Big Object Graphs Up Front

This article describes a rule of thumb for formatting unit tests.

The Arrange Act Assert (AAA) pattern is one of the most fundamental and important patterns for writing maintainable unit tests. It states that you should separate each test into three phases (Arrange, Act, and Assert).

Like most other code, unit tests are read more than they are written, so it's important to make the tests readable. This article presents one way to make it easy for a test reader easily to distinguish the three AAA phases of a test method.

The way of AAA #

The technique is simple:

• As long as there are less than three lines of code in a test, they appear without any special formatting or white space.
• When a test contains more than three lines of code, you separate the three phases with a blank line.
• When a single phase contains so many lines of code that you'll need to divide it into subsections to make it readable, you should explicitly mark the beginning of each phase with a code comment.

Motivating example #

Many programmers use the AAA pattern by explicitly demarking each phase with a code comment:

[Fact]
public void UseBasketPipelineOnExpensiveBasket()
{
    // Arrange
    var basket = new Basket(
        new BasketItem("Chocolate", 50, 3),
        new BasketItem("Gruyère", 45.5m, 1),
        new BasketItem("Barolo", 250, 2));
    CompositePipe<Basket> pipeline = new BasketPipeline();
    // Act
    var actual = pipeline.Pipe(basket);
    // Assert
    var expected = new Basket(
        new BasketItem("Chocolate", 50, 3),
        new BasketItem("Gruyère", 45.5m, 1),
        new BasketItem("Barolo", 250, 2),
        new Discount(34.775m),
        new Vat(165.18125m),
        new BasketTotal(825.90625m));
    Assert.Equal(expected, actual);
}

Notice the use of code comments to indicate the beginning of each of the three phases.

Given an example like the above, this seems like a benign approach, but mandatory use of code comments starts to fall apart when tests are very simple.

Consider this Structural Inspection test:

[Fact]
public void SutIsBasketElement()
{
    // Arrange
    // Act?
    var sut = new Vat();
    // Assert
    Assert.IsAssignableFrom<IBasketElement>(sut);
}

Notice the question mark after the // Act comment. It seems that the writer of the test was unsure if the act of creating an instance of the System Under Test (SUT) constitutes the Act phase.

You could just as well argue that creating the SUT is part of the Arrange phase:

[Fact]
public void SutIsBasketElement()
{
    // Arrange
    var sut = new Vat();
    // Act
    // Assert
    Assert.IsAssignableFrom<IBasketElement>(sut);
}

However, now the Act phase is empty. Clearly, using code comments to split two lines of code into three phases is not helpful to the reader.

Three lines of code and less #

Here's a simpler alternative:

[Fact]
public void SutIsBasketElement()
{
    var sut = new Vat();
    Assert.IsAssignableFrom<IBasketElement>(sut);
}

When there's only two lines of code, the test is so simple that you don't need help from code comments. If you wanted, you could even reduce that test to a single line of code, by inlining the sut variable:

[Fact]
public void SutIsBasketElement()
{
    Assert.IsAssignableFrom<IBasketElement>(new Vat());
}

Such a test is either a degenerate case of AAA where one or more phase is empty, or else it doesn't really fit into the AAA pattern at all. In these cases, code comments are only in the way, so it's better to omit them.

Even if you have a test that you can properly divide into the three distinct AAA phases, you don't need comments or formatting if it's only three lines of code:

[Theory]
[InlineData("", "", 1, 1, 1, 1, true)]
[InlineData("foo", "", 1, 1, 1, 1, false)]
[InlineData("", "bar", 1, 1, 1, 1, false)]
[InlineData("foo", "foo", 1, 1, 1, 1, true)]
[InlineData("foo", "foo", 2, 1, 1, 1, false)]
[InlineData("foo", "foo", 2, 2, 1, 1, true)]
[InlineData("foo", "foo", 2, 2, 2, 1, false)]
[InlineData("foo", "foo", 2, 2, 2, 2, true)]
public void EqualsReturnsCorrectResult(
    string sutName,
    string otherName,
    int sutUnitPrice,
    int otherUnitPrice,
    int sutQuantity,
    int otherQuantity,
    bool expected)
{
    var sut = new BasketItem(sutName, sutUnitPrice, sutQuantity);
    var actual = sut.Equals(
        new BasketItem(otherName, otherUnitPrice, otherQuantity));
    Assert.Equal(expected, actual);
}

Three lines of code, and three phases of AAA; I think it's obvious what goes where - even if this single test method captures eight different test cases.

Simple tests with more than three lines of code #

When you have more than three lines of code, you'll need to help the reader identify what goes where. As long as you can keep it simple, I think that you accomplish this best with simple whitespace:

[Fact]
public void UseBasketPipelineOnExpensiveBasket()
{
    var basket = new Basket(
        new BasketItem("Chocolate", 50, 3),
        new BasketItem("Gruyère", 45.5m, 1),
        new BasketItem("Barolo", 250, 2));
    CompositePipe<Basket> pipeline = new BasketPipeline();
 
    var actual = pipeline.Pipe(basket);
 
    var expected = new Basket(
        new BasketItem("Chocolate", 50, 3),
        new BasketItem("Gruyère", 45.5m, 1),
        new BasketItem("Barolo", 250, 2),
        new Discount(34.775m),
        new Vat(165.18125m),
        new BasketTotal(825.90625m));
    Assert.Equal(expected, actual);
}

This is the same test as in the motivating example, only with the comments removed. The use of whitespace makes it easy for you to identify three phases in the method, so comments are redundant.

As long as you can express each phase without using whitespace within each phase, you can omit the comments. The only whitespace in the test marks the boundaries between each phase.

Complex tests requiring more whitespace #

If your tests grow in complexity, you may need to divide the code into various sub-phases in order to keep it readable. When this happens, you'll have to resort to using code comments to demark the phases, because use of only whitespace would be ambiguous:

[Fact]
public void PipeReturnsCorrectResult()
{
    // Arrange
    var r = new MockRepository(MockBehavior.Default)
    {
        DefaultValue = DefaultValue.Mock
    };
 
    var v1Stub = r.Create<IBasketVisitor>();
    var v2Stub = r.Create<IBasketVisitor>();
    var v3Stub = r.Create<IBasketVisitor>();
 
    var e1Stub = r.Create<IBasketElement>();
    var e2Stub = r.Create<IBasketElement>();
    e1Stub.Setup(e => e.Accept(v1Stub.Object)).Returns(v2Stub.Object);
    e2Stub.Setup(e => e.Accept(v2Stub.Object)).Returns(v3Stub.Object);
 
    var newElements = new[]
    {
        r.Create<IBasketElement>().Object,
        r.Create<IBasketElement>().Object,
        r.Create<IBasketElement>().Object,
    };
    v3Stub
        .Setup(v => v.GetEnumerator())
        .Returns(newElements.AsEnumerable().GetEnumerator());
 
    var sut = new BasketVisitorPipe(v1Stub.Object);
    // Act
    var basket = new Basket(e1Stub.Object, e2Stub.Object);
    Basket actual = sut.Pipe(basket);
    // Assert
    Assert.True(basket.Concat(newElements).SequenceEqual(actual));
}

In this example, the Arrange phase is so complicated that I've had to divide it into various sections in order to make it just a bit more readable. Since I've had to use whitespace to indicate the various sections, I need another mechanism to indicate the three AAA phases. Code comments is an easy way to do this.

As Tim Ottinger described back in 2006, code comments are apologies for not making the code clear enough. A code comment is a code smell, because it means that the code itself isn't sufficiently self-documenting. This is also true in this case.

Whenever I need to add code comments to indicate the three AAA phases, an alarm goes off in my head. Something is wrong; the test is too complex. It would be better if I could refactor either the test or the SUT to become simpler.

When TDD’ing, I tend to accept the occasional complicated unit test method, but if I seem to be writing too many complicated unit tests, it's time to stop and think.

Summary #

In Growing Object-Oriented Software, Guided by Tests, one of the most consistent pieces of advice is that you should listen to your tests. If your tests are too hard to write; if your tests are too complicated, it's time to consider alternatives.

How do you know when a test has become too complicated? If you need to added code comments to it, it probably is.

This article first appeared in the The Developer No 2/2013. It's reprinted here with kind permission.

This article provides arguments for (and against) putting unit tests in libraries different from the production code.

One of my readers ask me "whether unit tests should be done in separate assemblies or if they should be done in [...] the production assemblies?" As he puts it, he "always took for granted that the test should go into separate assemblies," but is this really true, and if it is: what are the arguments?

Despite the fundamental nature of this question, I haven't seen much explicit treatment of it, and to answer it, I had to pause and think. The key to the answer, I think, is to first understand why you write automated tests at all. (This way of thinking about questions and answers is closely related to the (Lean) technique of 5 whys.)

As always, the short answer is that it depends, but that's not very helpful, so here are some common reasons.

Regression testing #

In my experience, most people focus on the quality control aspect of automated testing. If your only purpose of having automated tests is to have a good regression test suite, then it seems that it doesn't really matter where the tests go.

Still, even if that's your only reason for unit testing, I think putting the unit tests in a separate library is the best choice:

• It gives you a clearer separation of concerns. If you put unit tests in your production library, it will blur the distinction between production code and test code. How do you know which code to test? How do you know what not to test? There are different ways to address these concerns (namespaces, TDD), but I think the simplest way to deal with such issues is to put the tests in a separate library. (This argument reminds me a little of Occam's razor.)
• Putting unit tests in your production code will make your compiled libraries bigger. It will take (slightly) more time to load the libraries into memory, and they will take up more memory. This probably doesn't matter at all on a big server, but may be important if your production code runs on a (small) mobile device. Once again, it depends.
• The more code you put in your production software, the larger the potential attack surface becomes. Have you performed a security analysis of your unit tests? It's much easier to put the tests in a separate library that you never deploy to production, because it means that you don't have to waste valuable time and resources doing a security analysis of your test code.

In short, if you put unit tests in the same library as your production code, all that unit test must adhere to the same standards as your production code. Obviously, if you have no standards for your production code, this doesn't apply, but then you probably have bigger problems.

API design feedback #

While I find regression testing useful in itself, I find Test-Driven Development (TDD) particularly valuable because it provides feedback about the API I'm building. In the spirit of GOOS, if the test is difficult to write, it's probably because the API is difficult to use. You'll only get this feedback if you test against the public API of your code; the unit test is the first client of your API.

However, if your unit tests reside in the same library as your production code, you can easily invoke internal classes and members from your unit tests. You simply lose the opportunity to get valuable feedback about the usability of your code. (Note that this argument also applies to the use of the [InternalsVisibleTo] attribute, which means that you should never use it.)

For me, this reason alone is enough that I always prefer putting unit tests in separate libraries.

Shipping production code with tests #

Although I think that there are strong arguments against putting unit tests in production code, there may also be reasons in favor. As one answer on Stack Overflow describes, shipping your production code with a test suite can be valuable as a self-diagnostics tool. This might be particularly valuable if you deploy your production code to heterogeneous environments, and you're unable to predict and test the configuration of all environments before you ship your code.

Summary #

In the end, the answer depends on your context. If you understand why you (want to) have unit tests, you'll be able to answer the initial question: should unit tests go in a separate library, or can they go in the same library as the production code? In mainstream scenarios I will strongly recommend putting tests in a separate library, but you may have special circumstances where it makes sense to put the tests together with the production code. As long as you understand the pros and cons, you can make your own informed decision.

Comments

Add a self-link on RESTful resource.

This suggestion is part of my REST lessons learned series of blog posts. Contrary to previous posts, this advice doesn't originate from a lesson learned the hard way, but is more of a gentle suggestion.

One of the many advantages of a well-designed REST API is that it's 'web-scale'. The reason it's 'web-scale' is because it can leverage HTTP caching.

HTTP caching is based on URLs. If two URLs are different, they represent different resources, and can't be cached as one. However, conceivably, a client could arrive at the same resource via a number of different URLs:

• If a client follows redirect, it may not arrive at the URL it originally requested.
• If a client builds URLs from templates, it may order query string parameters in various ways.

Consider, as courtesy, adding a self-link to all resources. Adding a self link is as easy as this:

<product xmlns="http://fnaah.ploeh.dk/productcatalog/2013/05"
         xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:link href="http://catalog.api.ploeh.dk/products/1337"
             rel="self" />
  <atom:link href="http://catalog.api.ploeh.dk"
             rel="http://catalog.api.ploeh.dk/docs/rels/home" />
  <atom:link href="http://catalog.api.ploeh.dk/categories/chocolate"
             rel="http://catalog.api.ploeh.dk/docs/rels/category" />
  <atom:link href="http://catalog.api.ploeh.dk/categories/gourmet"
             rel="http://catalog.api.ploeh.dk/docs/rels/category" />
  <name>Fine Criollo dark chocolate</name>
  <price>50 DKK</price>
  <weight>100 g</weight>
</product>

Notice the link with the rel value of "self"; from its href value you now know that the canonical URL of that product resource is http://catalog.api.ploeh.dk/products/1337.

Particularly when there are (technically) more than one URL that will serve the same content, do inform the client about the canonical URL. There's no way a client can tell which URL variation is the canonical URL. Only the API knows that.

Redirected clients #

A level 3 RESTful API can evolve over time. As I previously described, you may start with URLs like http://foo.ploeh.dk/orders/1234 only to realize that in a later version of the API, you'd rather want it to be http://foo.ploeh.dk/customers/1234/orders.

As the RESTful Web Services Cookbook explains, you should keep URLs cool. In practice, that means that if you change the URLs of your API, you should at least leave a 301 (Moved Permanently) at the old URL.

If you imagine a mature API, this may have happened more than once, which means that a client arriving at one of the original URLs may be redirected several times.

In your browser, you know that if redirects happen, the address bar will (normally) display the final URL at which you arrived. However, consider that REST clients are applications. It will be implementation-specific whether or not the client realizes that the value of the final address isn't the URL originally requested.

As a courtesy to clients, do consider informing them of the final address at which they arrived.

URL variations #

While the following scenario isn't applicable for level 3 RESTful APIs, level 1 and 2 services require clients to construct URLs from templates. As an example, such services may define a product search resource as /products/search?q={query}&p={page}. To search for chocolate (and see the third (zero-indexed) page) you could construct the URL as /products/search?q=chocolate&p=2. However, most platforms (e.g. the ASP.NET Web API) would handle /products/search?p=2&q=chocolate in exactly the same way. The more query parameters you allow, the more permutations are possible.

This is a well-known problem in search (and computer science); it's called Canonicalization. Only the API knows the canonical value of a given URL: do consider informing the client about this value.

Summary #

Adding a self-link to each resource isn't much of a burden for the service, but could provide advantages for both the service and its clients. In most cases I'd expect the ROI to be high, simply because the investment is low.

Comments

client.FollowRedirects = true;
var response = client.Get("/foo");

Add home links on RESTful resources.

This suggestion is part of my REST lessons learned series of blog posts. Contrary to the previous posts, this advice doesn't originate from a lesson learned the hard way, but is more of a gentle suggestion.

When designing a level 3 RESTful API, I've found that it's often helpful to think about design issues in terms of: how would you design it if it was a web site? On web sites, we don't ask users to construct URLs and enter them into the browser's address bar. On web sites, users follow links (and fill out forms). Many well-designed web sites are actually HATEOAS services, so it makes sense to learn from them when designing RESTful APIs.

One almost universal principle for well-designed web sites is that they always have a 'home' link (usually in the top left corner). It makes sense to transfer this principle to RESTful APIs.

All the RESTful APIs that I've helped design so far have had a single 'home' resource, which acts as a starting point for all clients. This is the only published URL for the entire API. From that starting page, clients must follow (semantic) links in order to accomplish their goals.

As an example, here's a 'home' resource on a hypothetical product catalog API:

<home xmlns="http://fnaah.ploeh.dk/productcatalog/2013/05"
      xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:link href="http://catalog.api.ploeh.dk/products/1234"
             rel="http://catalog.api.ploeh.dk/docs/rels/products/discounted" />
  <atom:link href="http://catalog.api.ploeh.dk/products/5678"
             rel="http://catalog.api.ploeh.dk/docs/rels/products/discounted" />
  <atom:link href="http://catalog.api.ploeh.dk/products/1337"
             rel="http://catalog.api.ploeh.dk/docs/rels/products/discounted" />
  <atom:link href="http://catalog.api.ploeh.dk/categories"
             rel="http://catalog.api.ploeh.dk/docs/rels/product/categories" />
</home>

From this (rather sparse) 'home' resource, you can view each discounted product by following the discounted link, or you can browse the catalog by following the categories link.

If you follow one of the discounted links, you get a new resource:

<product xmlns="http://fnaah.ploeh.dk/productcatalog/2013/05"
         xmlns:atom="http://www.w3.org/2005/Atom">
  <atom:link href="http://catalog.api.ploeh.dk"
             rel="http://catalog.api.ploeh.dk/docs/rels/home" />
  <atom:link href="http://catalog.api.ploeh.dk/categories/chocolate"
             rel="http://catalog.api.ploeh.dk/docs/rels/category" />
  <atom:link href="http://catalog.api.ploeh.dk/categories/gourmet"
             rel="http://catalog.api.ploeh.dk/docs/rels/category" />
  <name>Fine Criollo dark chocolate</name>
  <price>50 DKK</price>
  <weight>100 g</weight>
</product>

Notice that in addition to the two category links, the resource representation also contains a home link, enabling the client to go back to the 'home' resource.

Having a home link on all resources is another courtesy to the client, just like avoiding 204 responses - in fact, if you can't think of anything else to return instead of 204 (No Content), at the very least, you could return a home link.

In the above examples, I didn't obfuscate the URLs, but I would do that for a real REST API. The reason I didn't obfuscate the URLs here is to make the example easier to understand.

Avoid hackable URLs if you are building a REST API.

This is a lesson about REST API design that I learned while building non-trivial REST APIs. If you provide a full-on level 3 REST API, consider avoiding hackable URLs.

Hackable URLs #

A hackable URL is a URL where there's a clear pattern or template for constructing the URL. As an example, if I present to you the URL http://foo.ploeh.dk/products/1234, it's easy to guess that this is a resource representing a product with the SKU of 1234. If you know the SKU of another product, it's easy to 'hack' the URL to produce e.g. http://foo.ploeh.dk/products/5678.

That's a really nice feature of your API if you are doing a level 1 or 2 API, but for a HATEOAS API, this defies the purpose.

The great divide #

Please notice that the shift from level 2 to level 3 RESTful APIs mark a fundamental shift in the way you should approach URL design.

Hackable URLs are great for level 1 and 2 APIs because the way you (as a client) are told to construct URLs is by assembling them from templates. As an example, the Windows Azure REST APIs explicitly instruct you to construct the URL in a particular way: the URL to get BLOB container properties is https://myaccount.blob.core.windows.net/mycontainer?restype=container, where you should replace myaccount with your account name, and mycontainer with your container name. While code aesthetics are subjective, it's not even a particularly clean URL, but it's easy enough to produce. The URL template is part of the contract, which puts the Windows Azure API solidly at level 2 of the Richardson Maturity Model. If I were designing a level 1 or 2 API, I'd make sure to make URLs hackable, too.

However, if you're building a level 3 API, hypermedia is king. Clients are expected to follow links. The addresses of resources are not published as having a particular template; instead, clients must follow semantic links in order to arrive at the desired resource(s). When hypermedia is the engine of application state, it's no good if the client can short-circuit the application flow by 'hacking' URLs. It may leave the application in an inconsistent state if it tries to do that.

Hackable URLs are great for level 1 and 2 APIs, but counter-productive for level 3 APIs.

Evolving URLs #

One of the main attractions of building a level 3 RESTful API is that it's easier to evolve. Exactly because URL templates are not part of the contract, you can decide to change the URL structure when evolving your API.

Imagine that the first version of your API has an (internal) URL template like /orders/{customerId}, so that the example URL http://foo.ploeh.dk/orders/1234 is the address of the order history for customer 1234. However, in version 2 of your API, you realize that this way of thinking is still too RPC-like, and you'd rather prefer /customers/{customerId}/order, e.g. http://foo.ploeh.dk/customers/1234/orders.

With a level 3 RESTful API, you can change your internal URL templates, and as long as you keep providing links, clients following links will not notice the difference. However, if clients are 'hacking' your URLs, their applications may stop working if you change URL templates.

Keep clients safe #

In the end, HATEOAS is about encapsulation: make it easy for the client to do the right thing, and make it hard for the client to do the wrong thing. Following links will make clients more robust, because they will be able to handle changes in the API. Making it easy for clients to follow links is one side of designing a good API, but the other side is important too: make it difficult for clients to not follow links: make it difficult for clients to 'hack' URLs.

The services I've helped design so far are level 3 APIs, but they still used hackable URLs. One reason for that was that this is the default in the implementation platform we used (ASP.NET Web API); another reason was that I thought it would be easier for me and the rest of the development team if the URLs were human-readable. Today, I think this decision was a mistake.

What's the harm of supplying human-readable URLs for a level 3 RESTful API? After all, if a client only follows links, the values of the URLs shouldn't matter.

Indeed, if the client only follows links. However, clients are created by human developers, and humans often take the road of least resistance. While there are long-time benefits (robustness) from following links, it is more work in the short term. The API team and I repeatedly experienced that the developers consuming our APIs had 'hacked' our URLs; when we changed our URL templates, their clients broke and they complained. Even though we had tried to explicitly tell them that they must follow links, they didn't. While we never documented our URL templates, they were simply too easy to guess from pure extrapolation.

Opaque URLs #

In the future, I plan to make URLs opaque when building level 3 APIs. Instead of http://foo.ploeh.dk/customers/1234/orders, I'm going to make it http://foo.ploeh.dk/DC884298C70C41798ABE9052DC69CAEE, and instead of http://foo.ploeh.dk/products/2345, I'm going to make it http://foo.ploeh.dk/598CB0CAC30646E1BB768596BFE91F2C, and so on.

Obviously, that means that my API will have to maintain some sort of two-way lookup table that can map DC884298C70C41798ABE9052DC69CAEE to a request for customer 1234's orders, 598CB0CAC30646E1BB768596BFE91F2C to request for product 2345, but that's trivial to implement.

It puts a small burden on the server(s), but effectively stops client developers from shooting themselves in their feet.

Summary #

Hackable URLs are a good idea if you are building a web site, or a level 1 or 2 REST API, but unless you know that all client developers are enthusiastic RESTafarians, consider producing opaque URLs for level 3 REST APIs.

Comments

<home xmlns="http://fnaah.ploeh.dk/productcatalog/2013/05">
  <link-template href="http://foo.ploeh.dk/0D89C19343DA4ED985A492DA1A8CDC53/{term}"
                 rel="http://catalog.api.ploeh.dk/docs/rels/product/search" />
</home>

POST /108B618F06644379879B9F2FEA1CAE92 HTTP/1.1
Content-Type: application/json
Accept: application/json

{ "term": "chocolate" }

HTTP/1.1 200 OK
Content-Type: application/json

{
  "term": "chocolate",
  "results": [{
    "name": "Valrhona",
    "link": {
      "href": "http://example.com/B69AD9193F6E472AB75D8EBD97331E22",
      "rel": "product"
  }
  }, {
    "name": "Friis-Holm",
    "link": {
      "href": "http://example.com/3C4BEC72BFB54E1B8FADB16CCB3E7A67",
      "rel": "product"
    }
  }]
}

HTTP/1.1 303 See Other
Location: http://example.com/0D25692DE79B4AF9A73F128554A9119E

I'm on Herding Code.

At the Danish Developer Conference 2013 I had the pleasure of meeting Jon Galloway, who interviewed me for a Herding Code podcast. In this interview, we talk about AutoFixture, testing (or not testing) trivial code, as well as lots of the unit testing topics I also cover in my Pluralsight course on Advanced Unit Testing.

Avoid 204 responses if you're building a HATEOAS application.

This is a lesson about REST API design that I learned while building non-trivial REST APIs. In order to be as supportive of the client as possible, a REST API should not return 204 (No Content) responses.

From the service's perspective, a 204 (No Content) response may be a perfectly valid response to a POST, PUT or DELETE request. Particularly, for a DELETE request it seems very appropriate, because what else can you say?

However, from the perspective of a proper HATEOAS-aware client, a 204 response is problematic because there are no links to follow. When hypermedia acts as the engine of application state, when there are no links, there's no state. In other words, a 204 response throws away all application state.

If a client encounters a 204 response, it can either give up, go to the entry point of the API, or go back to the previous resource it visited. Neither option is particularly good.

Giving up is not a good option if there's still work to do. Essentially, this is equivalent to a crashing client.

Going to the entry point of the API may allow the client to move on, doing what it was doing, but state may still be lost.

Going back (the equivalent of using your browser's back button) may be the best option, but has a couple of problems: First, if the client just did a DELETE, the previous resource in the history may now be gone (it was just deleted). The client would have to go back twice to arrive at a proper resource. Second, while your browser has built-in history, a programmatic HTTP client probably hasn't. You could add that feature to your client, but it would require more work. Once more, it would require the client to maintain state, which means that you'd be moving state from hypermedia to the client. It's just not a HATEOAS-compliant approach.

A good REST API should make it easy to be a client. While this is only a variation of Postel's law, I also like to think of this in terms or courtesy. The service has a lot of information available to it, so it might as well be courteous and help the client by sharing appropriate pieces of information.

Instead of a 204 (No Content) response, tell the client what it can do now.

Responding to POST requests #

An HTTP POST request often represents some sort of Command - that is: an intent to produce side effects. If the service handles the request synchronously, it should return the result of invoking the Command.

A common POST action is to create a new resource. At the very least, the REST API should return a 201 (Created) with a proper Location header. That's all described in detail in the RESTful Web Services Cookbook.

Even if the API decides to handle the request asynchronously, it can provide one or more links. Actually, the specification of 202 (Accepted) says that the "entity returned with this response SHOULD include an indication of the request's current status and [...] a pointer to a status monitor". That sounds like a link to me.

Responding to PUT requests #

An HTTP PUT request is often intended to update the state of a particular resource. Instead of returning 204 (No Content), the API should be courteous and return the new state of the resource. You may think that this is redundant because the client just transferred the state of the resource to the service, so why pay transmission costs to get the resource back from the service?

Even if a client PUTs the entire state of a resource to the API, the API may still have information about the resource not available to the client. The most universal example is probably the resource's ETag. If the client wishes to do further processing on the resource, it's likely going to need the ETag value sooner or later.

The representation of the resource may also include optional or denormalized data that the client may not have. If that data is optional, the client can make a PUT without it, but it might want that data to display to a user, so it would have to read the latest state of the resource regardless.

Responding to DELETE requests #

A DELETE request represents the intent to delete a resource. Thus, if the service successfully handles a DELETE request, what else can it do than returning a 204 (No Content)? After all, the resource has just been removed.

A resource is often a member of a collection, or otherwise 'owned' by a container. As an example, http://foo.ploeh.dk/api/tags/rock represents a "rock" tag, but another way of looking at it is that the /rock resource is contained within the tags container (which is itself a resource). This should be familiar to Atom Pub users.

Imagine that you want to delete the http://foo.ploeh.dk/api/tags/rock resource. In order to accomplish that goal, you issue a DELETE request against it. If all your client gets back is a 204 (No Content), it's just lost its context. Where does it go from there? Unless you keep state on the client, you don't know where you came from.

Instead of returning 204 (No Content), the API should be helpful and suggest places to go. In this example I think one obvious link to provide is to http://foo.ploeh.dk/api/tags - the container from which the client just deleted a resource. Perhaps the client wishes to delete more resources, so that would be a helpful link.

Responding to GET requests #

Returning 204 (No Content) as a response to a GET request is a bit weird, but probably not completely unheard of. Such a resource would represent a flag or semaphore, where the existence of the resource (as opposed to a 404 (Not Found)) signifies something. However, consider being helpful to the client and at least provide a link it can use to move on.

Summary #

Technically, responding with 204 (No Content) is perfectly valid, but if a REST API does that, its clients lose their current context. A service should be helpful and inform clients where they can go from the current resource. This will make client development easier, and only puts a small burden on the service.

Comments

Be careful with user-supplied data in URI segments.

This is a lesson about design of REST APIs that I learned the hard way: if you built service URIs from dynamic data, be careful with the data you allow in URI segments.

Some HTTP frameworks (e.g. the ASP.NET Web API) let you define URI templates or routes that handle incoming requests - e.g.

routeTemplate: "api/{controller}/{id}"

which means that a request to http://foo.ploeh.dk/api/fnaah/sgryt would map the value "fnaah" to controller, and the value "sgryt" to id.

If you are building a level 2 or less service, you may even publish your URI templates to consumers. If you are building a level 3 RESTful API, you may not be publishing your URI templates, but you can still use them internally.

Avoid user-supplied data in URI segments #

Be careful with the source of data you allow to populate URI segments of your URI template. At one point, I was involved with designing a REST API that (among other things) included a 'tag cloud' feature. If you wanted to see the contents of a specific tag, you could navigate to it.

Tags were all user-defined strings, and they had no internal ID, so our first attempt was to simply treat the value of the tag as the ID. That seemed reasonable, because we wanted the tag resource to list all the resources with that particular tag value. Thus, we modeled the URI template for tag resources like the above route template.

That worked well for a URL like http://foo.ploeh.dk/api/tags/rock because it would simply match the value "rock" to the id variable, and we could then list all resources tagged with "rock".

However, some user had defined a tag with the value of "sticky & sweet" (notice the ampersand character), which meant that when you wanted to see all resources with this link, you would have to navigate to http://foo.ploeh.dk/api/tags/sticky & sweet. However, that sort of URL is considered dangerous, and IIS will (by default) refuse to handle it.

Can you get around this by URL encoding the value? No, it's part of the request path, not part of any query string, so that's not going to work. The issue isn't that the URL is invalid, but that the server considers it to be dangerous. Even if you URL encode it, the server will decode it before handling it, and that would you leave you at square one.

You can either change the URI template so that the URL instead becomes http://foo.ploeh.dk/api/tags?id=sticky%20%26%20sweet. This URL encodes the query string (the part of the URL that comes after the ?), but gives you an ugly URL.

Another alternative is to be very strict about input validation, and only allow users to create values that are safe when used as URI segments. However, that's putting an unreasonable technical limitation on an application feature. If a user wants to tag a resource with "sticky & sweet", the service should allow it.

In the end, we used a third alternative: we assigned an internal ID to all tags and mapped back and forth so that the URL for the "sticky & sweet" tag became http://foo.ploeh.dk/api/tags/1234. Yes: that makes it impossible to guess the URL, but we were building a level 3 RESTful API, so clients are expected to follow links - not guess the URL.

This post provides an overview of some lessons I learned while bulding non-trivial REST APIs.

Last year I spent a good deal of the year designing and implementing a handful of non-trivial REST APIs for a customer of mine. During that process, I learned some small lessons about the design of RESTful systems that I haven't seen described elsewhere, and I want to share these lessons with you.

In order to learn the concepts and philosphy behind REST, I think that REST in Practice is a great resource (pun intended), but when it comes to practical guidance, I find the RESTful Web Services Cookbook invaluable. It's full of useful and concrete tips and tricks for building RESTful APIs, but I don't remember reading about the following lessons, that I had to learn the hard way.

There's so much hype and misrepresentation about REST that I have to point out that when I'm talking about REST, I mean full-on, level 3 REST, with resources, verbs, hypermedia controls and the works.

Each of these lessons deserves a small article of its own, but here's an overview:

• Avoid user-supplied data in URI segments
• Avoid 204 responses
• Avoid hackable URLs
• Consider a home link on all resources
• Consider a self link on all resources

I hope you find these tips useful.