Keep product knowledge close to the code

I realised this week that documentation next to the code can explain more than what a module or function does.

It can also preserve known limitations that we have deliberately decided not to address yet, and the product processes that future changes to the code must follow.

Turn known limitations into executable documentation

Imagine a hotel booking application that lets receptionists search for a guest by name.

The search ignores differences between uppercase and lowercase letters, but it does not ignore accents. Searching for Jose therefore fails to find a guest named José.

This does cause occasional problems, but receptionists can still find the booking using its reference. The team has decided that fixing more common search problems is currently a higher priority.

Instead of leaving that decision in a ticket that will eventually disappear from view, we can document the limitation on the function itself:

defmodule HotelBookingApp.GuestSearch do
  @doc """
  Checks whether a guest's name matches a search query.

  Matching is case-insensitive:

      iex> HotelBookingApp.GuestSearch.matches_name?("José Silva", "SILVA")
      true

  ## Known limitation: searching without accents

  Receptionists should be able to find guests without typing accents.
  However, searching for "Jose" currently fails to match "José":

      iex> HotelBookingApp.GuestSearch.matches_name?("José Silva", "Jose")
      false

  We have deferred fixing this because reported cases are infrequent,
  and receptionists can find the booking using its reference instead.

  The example above records the current bug. Once accent-insensitive
  matching is implemented, its expected result should become `true`.
  """
  @spec matches_name?(String.t(), String.t()) :: boolean()
  def matches_name?(name, query) do
    String.contains?(
      String.downcase(name),
      String.downcase(query)
    )
  end
end

We also need a small test module to make the examples executable:

defmodule HotelBookingApp.GuestSearchTest do
  use ExUnit.Case, async: true

  doctest HotelBookingApp.GuestSearch
end

This is a simple bug and a simple example, but you can extrapolate it to the complexity you deal with every day in your own application and domain.

The doctest does more than describe the bug. It makes the current behaviour executable. If that behaviour changes, the failing doctest creates a reason to revisit the documentation. It cannot verify that the surrounding explanation remains accurate, but it does connect the illustrated behaviour to the implementation.

This also gives coding agents and automated reviewers the context to recognise an acknowledged limitation, potentially reducing repeated reports of an issue we have already considered.

Document product processes next to the code they govern

Not every piece of product knowledge can or should be executable.

Suppose a new engineer needs to add a permission to the application's permission system. The safe process depends on whether the permission protects a new feature or an existing feature that customers already use.

That knowledge belongs in the module documentation closest to the permission definitions:

defmodule HotelBookingApp.Authorisation.Permission do
  @moduledoc """
  Defines the permissions that can be assigned to a permission role.

  ## Adding a permission

  First, determine whether the permission protects a new feature or an
  existing feature.

  For a new feature, add the permission to the list below.

  For an existing feature used by customers in production:
  - Add the permission to the list below.
  - Create a data migration that grants it to every role whose users could
    previously access the feature, preserving existing access by default.
  - If a customer requested the new restriction, confirm which of their roles
    should retain access and apply those agreed restrictions in the migration.

  The migration prevents existing users from unexpectedly losing access when
  the feature becomes permission-controlled.
  """
end

This is not a doctest. It is still valuable because it puts the product process where an engineer or coding agent is most likely to need it: beside the code they are about to change.

Doctests keep the illustrated behaviour checked against the implementation. The surrounding explanations and product processes still need maintenance, but placing them beside the relevant code makes them easier to discover when someone changes it.