Keep IDs internal with REST by Mark Seemann

Instead of relying on entity IDs, use hypermedia to identify resources.

Whenever I've helped teams design HTTP APIs, sooner or later one request comes up - typically from client developers: Please add the entity ID to the representation.

In this article I'll show an alternative, but first: the normal state of affairs.

Business as usual #

It's such a common requirement that, despite admonitions not to expose IDs, I did it myself in the code base that accompanies my book Code That Fits in Your Head. This code base is a level 3 REST API, and still, I'd included the ID in the JSON representation of a reservation:

{
  "id": "bf4e84130dac451b9c94049da8ea8c17",
  "at": "2021-12-08T20:30:00.0000000",
  "email": "snomob@example.com",
  "name": "Snow Moe Beal",
  "quantity": 1
}

At least the ID is a GUID, so I'm not exposing internal database IDs.

After having written the book, the id property kept nagging me, and I wondered if it'd be possible to get rid of it. After all, in a true REST API, clients aren't supposed to construct URLs from templates. They're supposed to follow links. So why do you need the ID?

Following links #

Early on in the system's lifetime, I began signing all URLs to prevent clients from retro-engineering URLs. This also meant that most of my self-hosted integration tests were already following links:

[Theory]
[InlineData(867, 19, 10, "adur@example.net", "Adrienne Ursa", 2)]
[InlineData(901, 18, 55, "emol@example.gov", "Emma Olsen", 5)]
public async Task ReadSuccessfulReservation(
    int days,
    int hours,
    int minutes,
    string email,
    string name,
    int quantity)
{
    using var api = new LegacyApi();
    var at = DateTime.Today.AddDays(days).At(hours, minutes)
        .ToIso8601DateTimeString();
    var expected = Create.ReservationDto(at, email, name, quantity);
    var postResp = await api.PostReservation(expected);
    Uri address = FindReservationAddress(postResp);
 
    var getResp = await api.CreateClient().GetAsync(address);
 
    getResp.EnsureSuccessStatusCode();
    var actual = await getResp.ParseJsonContent<ReservationDto>();
    Assert.Equal(expected, actual, new ReservationDtoComparer());
    AssertUrlFormatIsIdiomatic(address);
}

This parametrised test uses xUnit.net 2.4.1 to first post a new reservation to the system, and then following the link provided in the response's Location header to verify that this resource contains a representation compatible with the reservation that was posted.

A corresponding plaintext HTTP session would start like this:

POST /restaurants/90125/reservations?sig=aco7VV%2Bh5sA3RBtrN8zI8Y9kLKGC60Gm3SioZGosXVE%3D HTTP/1.1
Content-Type: application/json
{
  "at": "2021-12-08 20:30",
  "email": "snomob@example.com",
  "name": "Snow Moe Beal",
  "quantity": 1
}

HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Location: example.com/restaurants/90125/reservations/bf4e84130dac451b9c94049da8ea8c17?sig=ZVM%2[...]
{
  "id": "bf4e84130dac451b9c94049da8ea8c17",
  "at": "2021-12-08T20:30:00.0000000",
  "email": "snomob@example.com",
  "name": "Snow Moe Beal",
  "quantity": 1
}

That's the first request and response. Clients can now examine the response's headers to find the Location header. That URL is the actual, external ID of the resource, not the id property in the JSON representation.

The client can save that URL and request it whenever it needs the reservation:

GET /restaurants/90125/reservations/bf4e84130dac451b9c94049da8ea8c17?sig=ZVM%2[...] HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "id": "bf4e84130dac451b9c94049da8ea8c17",
  "at": "2021-12-08T20:30:00.0000000",
  "email": "snomob@example.com",
  "name": "Snow Moe Beal",
  "quantity": 1
}

The actual, intended use of the API doesn't rely on the id property, neither do the tests.

Based on this consistent design principle, I had reason to hope that I'd be able to remove the id property.

Breaking change #

My motivation for making this change was to educate myself. I wanted to see if it would be possible to design a REST API that doesn't expose IDs in their JSON (or XML) representations. Usually I'm having trouble doing this in practice because when I'm consulting, I'm typically present to help the organisation with test-driven development and how to organise their code. It's always hard to learn new ways of doing things, and I don't wish to overwhelm my clients with too many changes all at once.

So I usually let them do level 2 APIs because that's what they're comfortable with. With that style of HTTP API design, it's hard to avoid id fields.

This wasn't a constraint for the book's code, so I'd gone full REST on that API, and I'm happy that I did. By habit, though, I'd exposed the id property in JSON, and I now wanted to perform an experiment: Could I remove the field?

A word of warning: You can't just remove a JSON property from a production API. That would constitute a breaking change, and even though clients aren't supposed to use the id, Hyrum's law says that someone somewhere probably already is.

This is just an experiment that I carried out on a separate Git branch, for my own edification.

Leaning on the compiler #

As outlined, I had relatively strong faith in my test suite, so I decided to modify the Data Transfer Object (DTO) in question. Before the change, it looked like this:

public sealed class ReservationDto
{
    public LinkDto[]? Links { get; set; }
    public string? Id { get; set; }
    public string? At { get; set; }
    public string? Email { get; set; }
    public string? Name { get; set; }
    public int Quantity { get; set; }
}

At first, I simply tried to delete the Id property, but while it turned out to be not too bad in general, it did break one feature: The ability of the LinksFilter to generate links to reservations. Instead, I changed the Id property to be internal:

public sealed class ReservationDto
{
    public LinkDto[]? Links { get; set; }
    internal string? Id { get; set; }
    public string? At { get; set; }
    public string? Email { get; set; }
    public string? Name { get; set; }
    public int Quantity { get; set; }
}

This enables the LinksFilter and other internal code to still access the Id property, while the unit tests no longer could. As expected, this change caused some compiler errors. That was expected, and my plan was to lean on the compiler, as Michael Feathers describes in Working Effectively with Legacy Code.

As I had hoped, relatively few things broke, and they were fixed in 5-10 minutes. Once everything compiled, I ran the tests. Only a single test failed, and this was a unit test that used some Back Door Manipulation, as xUnit Test Patterns terms it. I'll return to that test in a future article.

None of my self-hosted integration tests failed.

ID-free interaction #

Since clients are supposed to follow links, they can still do so. For example, a maître d'hôtel might request the day's schedule:

GET /restaurants/90125/schedule/2021/12/8?sig=82fosBYsE9zSKkA4Biy5t%2BFMxl71XiLlFKaI2E[...] HTTP/1.1
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZXN0YXVyYW50IjpbIjEiLCIyMTEyIiwi[...]

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "name": "The Vatican Cellar",
  "year": 2021,
  "month": 12,
  "day": 8,
  "days": [
    {
      "date": "2021-12-08",
      "entries": [
        {
          "time": "20:30:00",
          "reservations": [
            {
              "links": [
                {
                  "rel": "urn:reservation",
                  "href": "http://example.com/restaurants/90125/reservations/bf4e84130dac4[...]"
                }
              ],
              "at": "2021-12-08T20:30:00.0000000",
              "email": "snomob@example.com",
              "name": "Snow Moe Beal",
              "quantity": 1
            }
          ]
        }
      ]
    }
  ]
}

I've edited the response quite heavily by removing other links, and so on.

Clients that wish to navigate to Snow Moe Beal's reservation must locate its urn:reservation link and use the corresponding href value. This is an opaque URL that clients can use to make requests:

GET /restaurants/90125/reservations/bf4e84130dac451b9c94049da8ea8c17?sig=vxkBT1g1GHRmx[...] HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
{
  "at": "2021-12-08T20:30:00.0000000",
  "email": "snomob@example.com",
  "name": "Snow Moe Beal",
  "quantity": 1
}

In none of these interactions do clients rely on the id property - which is also gone now. It's gone because the Id property on the C# DTO is internal, which means that it's not being rendered.

Mission accomplished.

Conclusion #

It always grates on me when I have to add an id property to a representation in an HTTP API. It's often necessary when working with a level 2 API, but with a proper hypermedia-driven REST API, it may not be necessary.

At least, the experiment I performed with the code base from my book Code That Fits in Your Head indicates that this may be so.

Comments

CREATE TABLE [dbo].[Foo] (
    [Id]           INT              NOT NULL IDENTITY PRIMARY KEY CLUSTERED,
    [Address]      NVARCHAR (850)   NOT NULL UNIQUE NONCLUSTERED,
    -- Other columns go here...
);

• ← Previous
• Archive
• Next →