How to build software engineering guardrails: An example with Bylaw

Jul 22, 2026

Introduction

In the previous article, I argued that deterministic guardrails are a better place for repeatable engineering policies than an AI agent's context window. It avoids bloating the context window with information that could be encoded elsewhere.

But how do you turn a policy into a guardrail?

Start with something that can be programmatically inspected. That might be source code, an abstract syntax tree, a database schema, a query, generated HTML, or a dependency graph.

Then:

  1. inspect the artifact
  2. recognize the pattern that matters
  3. express your policy as an invariant
  4. find a choke point through which every relevant artifact passes
  5. reject violations with an actionable error

If a policy can be evaluated from data that tooling can inspect, an AI agent should not have to remember it.

A guardrail is only reliable if it runs consistently. A single choke point lets you validate the invariant once instead of relying on every caller to remember to invoke the check.

That is why I built Bylaw. Bylaw is a collection of Elixir packages for encoding engineering policies as deterministic checks:

The following examples show how the same process applies to different kinds of software artifacts.

Inspecting source code

No Repo Transaction

Bylaw.Credo.Check.Ecto.NoRepoTransaction prevents using the deprecated Repo.transaction and suggests using Repo.transact instead.

The artifact is Elixir source code. The pattern is a call to Repo.transaction. The invariant is that application code must use the current Repo.transact API. AI agents often use outdated APIs because older examples are well represented in their training data.

For example, you ask an AI agent to make sure that creating an account and its audit event happen atomically. It writes:

def register_account(attrs) do
  Repo.transaction(fn ->
    account =
      %Account{}
      |> Account.changeset(attrs)
      |> Repo.insert!()

    Repo.insert!(AuditEvent.account_created(account))

    account
  end)
end

The implementation is otherwise correct, but the agent used Repo.transaction/1, an API deprecated in Ecto 3.13. The code looks plausible because years of examples use it.

The Bylaw check rejects the call and points the agent towards the current API:

def register_account(attrs) do
  Repo.transact(fn ->
    account =
      %Account{}
      |> Account.changeset(attrs)
      |> Repo.insert!()

    Repo.insert!(AuditEvent.account_created(account))

    account
  end)
end

I could add “use Repo.transact instead of Repo.transaction” to the agent's instructions, but the agent can still forget or overlook it. Once the policy is a Bylaw check, every violation produces the same result.

Inspecting a database schema

Foreign Key Actions

Suppose your organization's engineering standard is that foreign keys must never cascade deletes. Records should only be removed through explicit application flows, where retention requirements, audit events, and other side effects can be handled deliberately.

The artifact is the PostgreSQL schema. The pattern is any foreign key with an ON DELETE action. The invariant is that every foreign key must restrict deletion while dependent records still exist.

You encode that standard once with Bylaw.Db.Adapters.Postgres.Checks.ForeignKeyActions and apply it to the entire schema:

{Bylaw.Db.Adapters.Postgres.Checks.ForeignKeyActions,
 rules: [on_delete: :restrict]}

The schema-validation test is the choke point. It inspects the current database schema and validates every foreign key in one pass. It does not matter which migration introduced a constraint or which part of the application owns the affected tables: if the schema violates the invariant, the test fails.

Later, an AI agent is asked to add a relationship between orders and accounts. Orders must be retained for accounting and customer support, even after an account is closed, but the agent generates this migration:

alter table(:orders) do
  add :account_id,
      references(:accounts, on_delete: :delete_all),
      null: false
end

The migration looks reasonable, but deleting one account will now silently delete all of its orders. The dangerous behavior is easy to overlook because it is hidden behind the on_delete option. It also violates the organization's engineering standard.

The check inspects every foreign key in the PostgreSQL schema and rejects this cascade.

The agent then fixes the migration:

alter table(:orders) do
  add :account_id,
      references(:accounts, on_delete: :restrict),
      null: false
end

Postgres will now prevent an account from being deleted while orders still reference it. The application has to handle those business records explicitly instead of losing them as a side effect.

The same guardrail applies to every future foreign key. It protects the whole schema against drift, regardless of which tables are involved or whether the migration was written by a person or an AI agent.

Inspecting composed queries

An Ecto query is a data structure

At some point, I realized that an Ecto query is a data structure.

That changed how I thought about query validation. If a query is data, I can inspect it. If I can inspect it, I can understand what it is trying to do and define which structures are valid or invalid for my application.

Take an ordinary query:

from(p in Post, as: :post)
|> join(:inner, [post: p], a in assoc(p, :author), as: :author)
|> where([post: p], p.status == "published")
|> where([author: a], a.active == true)
|> order_by([post: p], desc: p.published_at, desc: p.id)
|> Repo.all()

This reads like a sequence of database operations: start from posts, join their authors, add two filters, order the results, and execute the query.

The from, join, assoc, where, and order_by expressions build the query one step at a time.

Before Repo.all executes it, those pipeline steps have built an %Ecto.Query{} value. Here is an abridged view of the relevant fields:

%Ecto.Query{
  from: %Ecto.Query.FromExpr{
    source: {"posts", Post},
    as: :post
  },
  joins: [
    %Ecto.Query.JoinExpr{
      qual: :inner,
      assoc: {0, :author},
      as: :author
    }
  ],
  wheres: [
    %Ecto.Query.BooleanExpr{
      op: :and,
      expr:
        {:==, [],
         [
           {{:., [], [{:&, [], [0]}, :status]}, [], []},
           %Ecto.Query.Tagged{
             value: "published",
             type: {0, :status}
           }
         ]}
    },
    %Ecto.Query.BooleanExpr{
      op: :and,
      expr:
        {:==, [],
         [
           {{:., [], [{:&, [], [1]}, :active]}, [], []},
           %Ecto.Query.Tagged{
             value: true,
             type: {1, :active}
           }
         ]}
    }
  ],
  order_bys: [
    %Ecto.Query.ByExpr{
      expr: [
        desc: {{:., [], [{:&, [], [0]}, :published_at]}, [], []},
        desc: {{:., [], [{:&, [], [0]}, :id]}, [], []}
      ]
    }
  ],
  aliases: %{
    post: 0,
    author: 1
  }
}

This is Ecto's actual representation, with incidental metadata such as file names, line numbers, parameters, and subquery lists omitted. &0 refers to the root Post binding and &1 refers to the joined author. The important part is that the query exposes its components:

  • from identifies the base table and schema
  • joins describes the joined associations and join types
  • wheres contains the predicates and how they are combined
  • order_bys contains the fields and directions used for sorting
  • aliases maps the named bindings :post and :author to their positions in the query

The repository is the choke point

Bylaw validates the completed query through Ecto.Repo.prepare_query/3. Ecto invokes this callback for query-based repository operations, after the application has composed the query and before the repository executes it.

That location is important. A query can be built dynamically across several functions and files. A source-code linter like Credo can inspect each expression independently, but it may never see the final query shape. prepare_query/3 receives the composed %Ecto.Query{} at the shared repository boundary, giving Bylaw one place to validate the relevant queries before they reach the database.

This means a program can ask questions about the query before it reaches the database:

  • Does it contain contradictory predicates?
  • Can its ordering produce non-deterministic results?
  • Does it join a table that this operation should not access?
  • Is a required tenant or authorization filter present?
  • Does an update or delete have an appropriately narrow filter?

Once I saw the query as data, guardrails around Ecto queries felt like a natural consequence. Bylaw can inspect the same structure that Ecto will execute, recognize unsafe or invalid patterns, and reject the query before it reaches the database.

Conflicting Where Predicates

Suppose your application builds a query incrementally across multiple files.

The artifact is the composed Ecto query. The pattern is multiple equality predicates on the same field. The invariant is that those predicates cannot require mutually exclusive values.

The posts context starts from a scope that only returns published posts:

# lib/my_app/blog.ex
def list_public_posts(params) do
  from(p in Post, as: :post)
  |> where([post: p], p.status == :published)
  |> PostFilters.apply(params)
  |> Repo.all()
end

Later, an AI agent is asked to add a status filter to the API. It works in a separate filter module and writes:

# lib/my_app/blog/post_filters.ex
def apply(query, %{"status" => "draft"}) do
  where(query, [post: p], p.status == :draft)
end

def apply(query, _params), do: query

Each function looks reasonable when read on its own. But a request with ?status=draft produces this query:

WHERE status = 'published' AND status = 'draft'

The query does not crash. It silently returns no records. Because the predicates were added in different files, inspecting either function by itself will not reveal the bug.

Bylaw.Ecto.Query.Checks.ConflictingWherePredicates inspects the composed query and reports the conflicting predicates.

Deterministic Order

Another query invariant is deterministic ordering.

The artifact is again the composed Ecto query. The pattern is a query that returns records in a non-deterministic order. The invariant is that an ordered result set must have a stable order across executions.

An AI agent writes an API endpoint:

def index(conn, _params) do
  posts =
    from(p in Post, as: :post)
    |> order_by([post: p], desc: p.inserted_at)
    |> Repo.all()

  json(conn, %{data: posts})
end

Two posts can have the same inserted_at value. When that happens, Postgres is free to return those posts in a different order between requests.

This is based on a real issue I encountered at work. The bug itself was small: an API endpoint returned a list without deterministic ordering. It passed through tests and code review, was deployed, and eventually affected a customer consuming the API.

Once a bug like this reaches a customer, fixing one missing tie-breaker can consume time across an entire organization:

  1. The customer notices inconsistent results and investigates whether the problem is in their own integration.
  2. The problem has to bother them enough that they spend more time writing a support ticket.
  3. Support investigates whether the issue has already been reported, fixed, or planned.
  4. If it has not, support creates an actionable item for product.
  5. Product evaluates the issue and prioritizes it against other work.
  6. The issue is assigned to a developer, who reproduces it, identifies the missing deterministic order, writes a regression test, and implements the fix.
  7. The change goes through code review and deployment.
  8. Engineering informs support that the fix has been released.
  9. Support informs the customer, who then verifies the behavior in their integration.

Every step is reasonable, but together they consume significant time from the customer, support, product, and engineering. The organization pays that cost because a small invariant was not enforced before release.

Fixing this endpoint does not prevent the same mistake from happening again. A regression test can protect this particular endpoint, but another developer or AI agent can add a different query without deterministic ordering. That query can also pass through tests and code review, reach production, and start the same lifecycle again.

Bylaw.Ecto.Query.Checks.DeterministicOrder requires the root schema's primary key:

def index(conn, _params) do
  posts =
    from(p in Post, as: :post)
    |> order_by([post: p], desc: p.inserted_at, desc: p.id)
    |> Repo.all()

  json(conn, %{data: posts})
end

The primary key acts as a tie-breaker. Even when two posts have the same inserted_at value, their order is now stable between API requests.

Encoding this rule as a guardrail catches the missing primary key while the query is being developed. The agent or developer fixes it immediately, before the bug can begin that lifecycle. Because the check runs at the repository choke point, it protects future queries as well as the endpoint that originally exposed the problem. This is the practical value of guardrails: they preserve attention across the organization by preventing known classes of mistakes at the cheapest point to correct them.

Another field with a unique database constraint could also make the ordering deterministic. However, an Ecto schema does not expose arbitrary unique indexes, so this query check cannot prove that such a field is unique. It only accepts the root schema's primary key, which it can verify through Ecto schema reflection. A database-aware check could inspect unique constraints and safely recognize additional tie-breakers.

A deliberate trade-off

Ecto exposes the %Ecto.Query{} struct, but the internal representation of its expressions is not a stable extension API. A future Ecto release could change that representation and require Bylaw's query checks to be updated.

I decided that trade-off was acceptable for three reasons:

  • Ecto's query API has been fairly stable across releases
  • I run these checks in development and test, not in production
  • if an Ecto upgrade breaks a check, it fails in the development lifecycle instead of changing behavior for customers

The checks may need maintenance after an Ecto upgrade, but that failure should be visible and relatively easy to fix before deployment.

The checks are also intentionally narrow. They do not attempt to prove every semantic property of a query. Each check recognizes a specific set of query shapes and may need to balance false positives against false negatives. Their purpose is to steer the application toward explicit, approved patterns and catch high-value mistakes consistently.

What makes a good guardrail?

The examples above inspect different artifacts, but the guardrails share the same properties:

  • Deterministic: the same artifact always produces the same result
  • Focused: each check enforces one policy
  • Fast: developers and agents receive feedback while the relevant change is still fresh
  • Actionable: the error explains what failed and how to correct it
  • Hard to bypass accidentally: the checks run before a change is merged

Guardrails are also educational tools for humans and AI agents. A developer may not know that a database does not guarantee row order without an ORDER BY, or that ordering by a non-unique field can still produce unstable results. When a guardrail rejects the query, it can identify the violated invariant, explain why the query is unsafe, and show what needs to change.

The timing makes that feedback especially effective. The human or agent receives the explanation while writing the relevant code, with the failing query directly in front of them. The lesson is attached to a concrete decision instead of presented as an abstract rule that they may need someday.

A company-wide CLAUDE.md, style guide, or prompt can document the same policy, but every contributor and agent still has to find, interpret, and remember it. An actionable guardrail delivers the relevant policy at the point of violation and prevents the code from moving forward until the issue is addressed. It enforces the invariant and teaches the reasoning behind it in the same feedback loop.

A noisy or ambiguous check will eventually be ignored. A useful guardrail identifies a real violation, explains why it matters, and gives the developer or agent a clear path forward.

Where to enforce guardrails

A guardrail is most effective when it runs throughout the development lifecycle:

  • in the agent's feedback loop after it changes code
  • before commits or pushes
  • in CI before merge

Fast local feedback helps the agent correct its own output. CI provides the final guarantee that the policy applies to every contribution.

Conclusion

Building a software engineering guardrail starts with an inspectable artifact. Recognize the pattern that carries risk, express the desired policy as an invariant, and enforce it at a choke point through which every relevant artifact passes.

Bylaw applies that process to Elixir source code, PostgreSQL schemas, Ecto queries, and HTML. The result is a set of policies that AI agents and developers do not need to remember because the tooling enforces them every time.

https://ryanzidago.com/posts/feed.xml