evalseal SPECIFICATION

Seal Record Format · version 1 · in use

The format,
in full

A seal is three things: a JSON object, a detached signature over its exact bytes, and the artifacts that object names. This page defines all three. It is written so that a second implementation can produce byte-identical records from the same inputs, and so that anyone can check an existing one without running our software. Where the prose and the issued record disagree, the record is right.

Canonical form

Everything else rests on this. A record is hashed and signed as bytes, so two implementations that disagree about whitespace produce different signatures over identical data. The serialisation is fixed:

UTF-8, no byte-order mark
keys sorted lexicographically, at every level of nesting
two-space indent, one key per line
non-ASCII emitted literally — never \uXXXX escaped
exactly one trailing newline (0x0A)

In the reference implementation that is a single expression — json.dumps(obj, sort_keys=True, indent=2, ensure_ascii=False) + "\n", encoded UTF-8. Reproducing it in another language means reproducing those five properties, not that expression.

Sorting is why the fields on disk are alphabetical and bear no relation to the order they are written in code. Do not reorder a record to make it read better. That is a different document with a different hash and no valid signature.

The record

Every field below is required in v1. Shown here in reading order with hashes cut short; on disk the keys are sorted and every hash is the full 64 hex characters.

{
  "schema":        "evalseal/seal/v1"       ← format identifier
  "id":            "D291-7CAF-C9B1"         ← derived, never chosen
  "grade":         "A"                      ← A | B | C — who executed the run
  "grade_meaning": "independently executed — the subject had no access to the run"
  "caveat":        "…"                      ← plain-language limits, never empty
  "subject":       { file, kind, sha256 }   ← the artifact under test
  "dataset":       { file, name, version, tasks, sha256 }
  "grading":       "A task passes only if every case matches exactly…"
  "conditions":    { deterministic, note }  ← anything that could make a re-run differ
  "result":        { passed: 8, total: 12, rate: "66.7%", failed: [ … ] }
  "traces":        { file, records, bytes, sha256 }
  "environment":   { python, platform, harness: { file, sha256 } }
  "files":         { "<path>": "<sha256>", … }
  "sealed_at":     "2026-07-28T09:14:02Z"   ← RFC 3339, UTC, second precision
  "signature":     { scheme, namespace, identity, file, key }
}
filesMaps every artifact the record names to its own sha256, each trace included individually. A verifier never has to accept a summary hash — any single file can be checked on its own, without the bundle.
environment.harnessThe hash of the program that produced the record. Editing the harness therefore changes every record it regenerates. That is intended: a record states which instrument produced it, and a modified instrument is a different one.
caveatMandatory, and never empty. A record that cannot state its own limits in plain language has not been examined closely enough to issue. This field is the reason the format has no marketing surface.
conditionsEverything that could make a second run differ. For a deterministic subject it is one line. For a sampled model it carries seed, temperature, top-p, stop conditions and the tool set — a score without them is an anecdote, and the format gives it nowhere to hide.
result.rateA string, formatted to one decimal place — not a float. Floats do not round-trip identically across languages, and this value sits inside a document that is hashed byte for byte.

Seal numbers

The number is derived from the content. It is not assigned, not sequential, and cannot be chosen.

digest = sha256( traces_sha256_hex + sealed_at )
id     = first 12 hex of digest, uppercased, grouped 4-4-4

sha256("52b14610…b29684" + "2026-07-28T09:14:02Z")
       → d2917cafc9b1…  →  D291-7CAF-C9B1
What changes itThe traces, or the seal time. A different task set, a different subject or a different run produces different traces, and therefore a new number. The old seal is never overwritten.
What does notThe grade, the caveat, the environment, the harness hash. Editing the harness rewrites a record in place, under the same number — because the run it describes did not change.
CollisionsTwelve hex characters is 48 bits of a sha256 prefix. A number is an address, not an authentication. What authenticates a record is the signature over its bytes, and nothing else.

Signing envelope

SSHSIG — the same detached-signature format used for signed git commits — over the exact record bytes. The verifying side needs ssh-keygen and nothing else.

# issue
ssh-keygen -Y sign -f <private key> -n evalseal record.json   → record.json.sig

# check
ssh-keygen -Y verify -f allowed_signers -I seal@evalseal.com \
                     -n evalseal -s record.sig < record.json
Namespaceevalseal. A signature made under any other namespace will not verify as a seal, so a key used elsewhere — for git, for SSH — cannot have one of its signatures replayed into this format.
DetachedThe signature covers record.json and nothing else. Every other artifact is covered transitively, through its hash in files. Break either link and the check fails at a known point.
Key distribution/keys/allowed_signers, in SSHSIG allowed-signers format. It is served from the domain it vouches for, which is a real limitation and is stated on the seal itself. Pin the fingerprint and watch for it moving.
RotationRotating the key invalidates every seal already issued under the old one. There is no re-signing path in v1. The key therefore does not rotate for convenience.

Verification does not establish that an evaluation was competent, or that its questions were fair. It establishes that these bytes are the bytes that were signed. That is the whole of what a signature is for.

The traces bundle

A hash over an archive is worthless unless the archive is byte-reproducible. Ordinary tar and gzip are not — they record a clock, a username and whatever order the filesystem handed over. Every one of those is pinned:

gzip     mtime = 0                       (no timestamp in the header)
tar      entries sorted by name
         path prefix  traces/
         mtime 0 · mode 0644 · uid 0 · gid 0 · uname "" · gname ""
entries  one per graded task, canonical JSON, same five rules as the record

Each of these lines exists because its default value differs between two machines that ran the same evaluation. Left alone they change the hash without changing the data, which is the one thing a content hash must never do.

Grades

The grade field states who executed the run. It says nothing about the score, and a high score cannot raise it.

A · IndependentEvalSeal provisioned the environment, executed the run and captured the traces. The subject never touched any of it.
B · WitnessedThe run executed in a sealed environment we provisioned, on the subject's own keys and hardware. Inputs and outputs were captured by us, not reported by them.
C · SubmittedTraces were submitted by the subject. The seal attests integrity and time only: this record has not changed since that moment. It does not attest that the run happened as described.

What v1 does not carry

Absences in a format are decisions. These four are deliberate, and the last two are unfinished.

No verdictThere is no pass mark, no ranking and no recommendation field, and none will be added. A record reports what happened. Concluding from it is someone else's job, and keeping those two jobs apart is the point of the format.
No marketing identityA subject is a file and a hash. There is no field for a vendor, a model family or a product name, because none of them are checkable and all of them are what people would read instead of the hash.
No revocationOpen. v1 cannot withdraw a seal. A record found to be wrong is answered by publishing a second record that says so — never by deleting the first. Whether that is sufficient is not yet settled.
No chainOpen. Records do not reference one another. Re-running the same subject produces an unrelated seal, and nothing in the format connects them. A predecessor field is the most likely addition in v2.

Versioning

The schema field carries the version, and a record is always read under the version it names.

BreakingRemoving or renaming a field, changing the canonical form, changing how the id is derived, changing the signing namespace. Any of these becomes evalseal/seal/v2.
Not breakingAdding an optional field. New records carry it and hash differently for it; records already issued are untouched, because nothing regenerates them.
No migrationIssued seals are never upgraded, and there is no path to do so. A v1 record and its v1 signature go on agreeing with each other indefinitely, which is the only property the format actually promises.

A format is only a format if a second implementation can produce the same bytes. Everything above is written to make that possible — including the parts that make it inconvenient.

See a seal in full Why a receipt