Each test should own its setup

A test should explain itself.

Every time I have to jump to a setup callback to understand why an assertion passes, the test has already failed at its second job: communication.

I understand why they are appealing. A setup callback removes repetition, makes each test shorter, and gives us one place to create the records that several tests need.

But shorter tests are not necessarily simpler tests.

Very often, shared setup makes a test harder to read, harder to review, harder to change, and harder to move. It reduces the number of lines in the test by moving essential information somewhere else.

I much prefer self-contained tests: each test should create and own the state it needs.

A short test with hidden context

Here is a fairly typical Elixir test:

defmodule MyApp.OrdersTest do
  use MyApp.DataCase

  setup do
    customer = insert(:customer)
    product = insert(:product, price: 1_000)

    %{customer: customer, product: product}
  end

  test "uses the product price as the order total", %{
    customer: customer,
    product: product
  } do
    assert {:ok, order} = Orders.create(customer, product)
    assert order.total == 1_000
  end
end

The test itself is short, but it is not self-explanatory.

Why should the total be 1_000? The answer is not in the test. We need to leave it, find the relevant setup, notice the product's price, and remember that detail when we return.

The setup is not incidental. It contains the condition that causes the behaviour we are testing.

I would write the test like this:

test "uses the product price as the order total" do
  customer = insert(:customer)
  product = insert(:product, price: 1_000)

  assert {:ok, order} = Orders.create(customer, product)
  assert order.total == 1_000
end

Now the test tells the whole story:

• the product costs 1_000
• creating the order results in a total of 1_000

There are no important facts hidden elsewhere in the module.

Locality matters more than line count

Shared setup is usually introduced to remove duplication. That is a useful goal in production code, but I do not think it should be applied mechanically to tests.

Production code and test code have different jobs. Production code primarily implements behaviour. A test is also an explanation and an example.

Some repetition helps that explanation.

test "creates an order when the product is in stock" do
  customer = insert(:customer)
  product = insert(:product, stock: 1)

  assert {:ok, order} = Orders.create(customer, product)
  assert order.product_id == product.id
end

test "rejects an order when the product is out of stock" do
  customer = insert(:customer)
  product = insert(:product, stock: 0)

  assert {:error, :out_of_stock} =
           Orders.create(customer, product)
end

Those tests repeat several lines, but the repetition makes the difference between the two cases immediately visible.

If the customer and product were created in a setup callback, we would save a few lines while making the reader reconstruct each scenario from multiple places.

That is not a good trade.

The useful question is not:

How do I remove every repeated line?

It is:

How much context does someone need to understand this test?

For me, a few duplicated fixture calls are usually cheaper than scattered context.

Self-contained tests make better review comments

The cost of shared setup becomes particularly visible during code review.

A reviewer rarely reads a test file from top to bottom with every detail loaded in their head. They often encounter a test in a pull request diff, an inline comment, a suggested patch, or a general PR conversation.

Imagine that an AI agent flags a potential bug while reviewing a pull request. I ask it to draft a failing test that demonstrates the issue. I often find a concrete test case easier to understand than a description in plain English. I run the test locally, confirm that it fails for the expected reason, and paste it into a GitHub comment:

That comment is useful on its own. The author, another reviewer, or an AI agent can understand the scenario without access to anything else.

Now imagine that the same test relies on the module's shared setup:

What kind of product did the setup create? Was it available before the update? Does the setup create anything else that affects the result?

The test may make sense inside the file, after jumping to the setup callback and possibly through one or two helper functions. It does not make sense in isolation.

This creates unnecessary work for every reviewer. They must open the file, find the setup, understand which parts apply, and mentally combine that information with the test in the comment.

It is even worse for an AI agent. We either need to provide the entire file as context, trust the agent to locate every indirect dependency, or accept that it may reason about an incomplete scenario.

Self-contained tests reduce that context requirement. They make the evidence for a review comment portable.

Tests should be portable

I want to be able to copy a test from:

• my editor into a pull request comment
• a pull request comment into my local repository
• one test module into another
• an issue into an AI coding agent conversation
• an AI agent's response into the codebase

The test should carry the facts required to understand it.

Of course, no test is literally independent of the application. It will still use factories, application modules, and common test infrastructure. Portability is not about eliminating every dependency. It is about making the dependencies that define the scenario explicit at the point of use.

A shared setup callback couples a test to its location in the file. Move the test and it may stop compiling, stop passing, or, more dangerously, continue passing under different assumptions.

A self-contained test can usually be moved without bringing along a web of setup callbacks. If it cannot, the missing dependency tends to be obvious.

Shared setup accumulates stale state

Shared setup also has a tendency to grow.

A test needs an account, so we add an account. Another test needs an administrator, so we add an administrator. A third needs a subscription, so we add a subscription and perhaps a helper that constructs it.

Eventually the setup creates a small world:

setup do
  account = insert(:account)
  admin = insert(:user, role: :admin, account: account)
  employee = insert(:user, role: :employee, account: account)
  subscription = insert(:subscription, account: account)
  product = insert(:product)

  %{
    account: account,
    admin: admin,
    employee: employee,
    subscription: subscription,
    product: product
  }
end

Not every test needs that world, but every test pays for it.

Later, tests are removed or rewritten, while setup entries and helper functions remain. It becomes difficult to tell whether a fixture is still necessary because its consumers are distributed across the module. Unused values, stale helpers, and accidental dependencies survive much longer than they would if the setup were local.

Deep setup abstractions also make intermittent failures harder to investigate. When a test fails only occasionally, the first task is to reconstruct its exact starting state. If that state is spread across setup callbacks and helper functions, it becomes difficult to determine what data was created, which defaults mattered, and whether the test accidentally depended on state it did not explicitly request. Self-contained tests do not prevent every flaky test, but they reduce the amount of hidden context we must inspect when one fails.

Local setup makes dead code easier to see. If a test no longer uses subscription, we remove the line directly above it. There is no need to audit every other test before deciding whether it is safe.

It also makes changes easier to reason about. Changing a shared setup value may affect an entire module, including tests that do not visibly reference the changed detail. Changing local setup affects the test in front of us.

This is the same reason global state is difficult to reason about, only at a smaller scale: the distance between cause and effect makes the impact of a change less obvious.

Self-contained does not mean no abstraction

I am not arguing that every test should manually build every struct or repeat a dozen low-level database operations.

Helpers are useful when they express domain intent:

test "a customer cannot order an unavailable product" do
  customer = insert(:customer)
  product = unavailable_product()

  assert {:error, :product_unavailable} =
           Orders.create(customer, product)
end

The test still owns its setup. It explicitly asks for the state it needs, and the helper gives that state a meaningful name.

The important distinction is control:

• a helper runs because the test calls it
• a setup callback runs because the test happens to live in a module

Explicit helpers can make tests clearer. Implicit setup makes their dependencies easier to miss.

I would still be careful with helpers that hide the exact value under test. If product availability is central to the behaviour, spelling it out may be clearer:

product = insert(:product, available: false)

The goal is not to inline everything. The goal is to keep the facts that explain the expected result close to the assertion.

The economics of repetition have changed

Shared setup was more appealing when every repeated fixture had to be written and maintained manually. Even then, I disliked the tradeoff: we saved a few keystrokes by moving important context away from the test.

AI coding agents have weakened that tradeoff even further. Asking an agent to generate a few explicit fixture calls costs almost nothing. The expensive part is no longer producing those lines. It is reading, reviewing, and reasoning about them later.

AI has made code cheaper to write, but it has not made hidden context cheaper to understand. If an abstraction saves typing while making every test harder to comprehend, I no longer see much value in it.

When I still use setup

Elixir's setup is not inherently bad. Uniform infrastructure that is not part of the scenario being described is the strongest case for using it.

For example, a Phoenix controller test might apply the same request header to every connection:

setup %{conn: conn} do
  {:ok, conn: put_req_header(conn, "accept", "application/json")}
end

That configuration is different from silently deciding that every test has a customer, an active subscription, and an available product worth 1_000.

Even in cases like this, I generally prefer to avoid setup. I would rather keep the convention unambiguous: each test owns the state it needs.

Once a setup callback exists, it becomes the obvious place to put the next shared dependency. A human sees the existing pattern and extends it. An AI agent does the same. The callback that started with harmless infrastructure gradually accumulates domain state, and the boundary between the two becomes less clear.

Repeating a small amount of infrastructure is often a reasonable price for making the preferred pattern obvious to every contributor, human or agent.

Each test should own its world

Self-contained tests are sometimes longer. I am happy to accept that.

What bothers me is not abstraction itself. A domain function or module can also hide implementation details, but it is part of the product and carries domain meaning. Calling it from a test still tells the reader something about the system.

A setup callback introduces a different kind of abstraction: an ad hoc API that exists only between the setup and its tests. That API is implicit, specific to the test module, and has no meaning outside it. I do not want readers to learn both the product's API and a separate setup API just to understand one test.

Self-contained tests are easier to read because the relevant state is nearby. They are easier to review because a diff or comment contains the whole example. They are easier to change because their dependencies are explicit. They are easier to move because they carry their setup with them. They also give both humans and AI agents enough context to reason about the behaviour without reconstructing it from the rest of the file.

A test is not better because it has fewer lines. It is better when someone can understand what it proves, why it should pass, and what would make it fail.

Each test should own its world, and that world should be visible wherever the test appears.