What a Real-World .gram Benchmark Measures—and What It Does Not

문서 복사

문서 도구

Markdown 보기MCP 주소 복사문서 오류 신고

자동 복사가 허용되지 않았습니다. 내용을 선택해 직접 복사하세요.

닫기

Unverified 공개 출처와 지원 범위

Status

Unverified

Component

Docs research

Deployment

n/a (research article)

Applies to version

Latest

출처 확인일

미확인

Owner

thegram-one/thegram-blog

Public evidence

None

Basis

This is a research/benchmark article rather than a product release page, and no fetched public evidence supports assigning a product status.

All pages: version and status matrix

출처 확인은 실제 계정 설치·기기 동작·서비스 상태의 검증을 뜻하지 않습니다.

A format comparison is useful only when readers can see what data and representation were measured. This benchmark examines one Codex project-memory document reconstructed from repository history. It is not a measurement of an agent remembering facts during 18 months of live work, nor an evaluation of model accuracy. The source is a final, hash-manifested report with internally self-consistent checks; it does not establish an independent external reproduction.

Corpus and method

The analyzed .gram container is 1,046,361 bytes and contains 18 states, revisions 0 through 17, and 532 records at its final state. The current .gram source projection is 219,837 bytes; the remaining 826,524 bytes are History/6 journal overhead. Its record set includes project, goal, source, history, requirement, assumption, event, evidence, component, decision, task, risk, metric, blocker and artifact types. The reconstruction spans source history from 2025-04-16 through 2026-10-10. It is retrospective repository-history reconstruction, not organically maintained personal memory.

For current-state size comparison, the current .gram source is compared with a chosen reversible Markdown schema and JSON representations of the same normalized persisted semantic state. Current Markdown is 309,305 bytes; minified JSON is 291,140 bytes; pretty JSON is 466,852 bytes. On these selected encodings, the 219,837-byte current .gram source is 28.9% smaller than Markdown and 24.5% smaller than minified JSON. These are conditional results for this corpus and schema, not intrinsic Markdown or JSON overhead and not a universal storage win.

The schema preserves field values, links, order and the raw rich body; semantic type codes are expanded while original field keys remain. It does not preserve source-byte identity. Do not compare the 1,046,361-byte full .gram container with a Markdown or JSON current snapshot as if they had the same scope: the full file includes prior-state journal data.

Separately, exported historical Markdown and JSON totals include per-revision effective/recorded/actor/request metadata and sum to 2,742,707, 4,161,950 and 2,569,734 bytes for Markdown, pretty JSON and minified JSON. Those totals describe 18 state exports under this scheme, not proof that snapshot formats cannot store history.

What was checked

The private benchmark run manifest hashes 108 generated artifacts; it is not a manifest of 108 corpus inputs. Verify-only mode re-checks those artifact hashes, the raw rich body of each revision, the 532 current records and Markdown/JSON semantic round-trips for all 18 revisions. The published report states the round-trip result, but the verification output itself is not published.

Seven deterministic content oracles query selected facts such as project-head counts, an implementation-language decision, architecture links and a latest analyzed change. Historical boundary checks verify that revision 0 contains 12 records, that a later decision appears at revision 2, and that an earlier decision changes status across revisions. These are checks of persisted data, not LLM tests, answer-quality studies or accuracy scores.

The benchmark cites a pinned format revision and a native runtime fingerprint. Independent reproduction would require the corresponding source and runtime evidence; a reader without those exact inputs cannot assume they will produce the same report. The benchmark did not independently fetch an upstream repository checkout. Public repository citations provide provenance for project context, not proof that this reconstructed sample represents all Codex users.

Interpretation and practical value

For product and architecture readers, the example shows how one structured file can combine current prose, linked records and inspectable prior states. A tool can ask what changed, when a decision appeared or what evidence supports a task without relying on separately maintained snapshots. A compact source may also be convenient to store or move. This benchmark does not prove lower model token usage, better retrieval, reduced inference cost, higher factual accuracy or performance over a carefully designed event store.

The useful lesson is methodological: label current and historical scopes separately, disclose encoding choices, retain provenance and test exact historical boundaries. Before applying this size result to another project, measure its content, structure, rich-body size, chosen schema and history retention. Record density, repeated fields, prose, link patterns and export conventions can change the outcome substantially.

Evidence access and further reading

This site includes privacy-redacted report and oracle exports at /evidence/codex/report.json and /evidence/codex/oracle.json. They are publication exports, not byte-identical copies of the original files. The manifest at /evidence/codex/manifest.json lists the two published exports and distinguishes publication hashes from original-source hashes; it is not the private 108-artifact run manifest or a full reproduction package.

The original corpus, source snapshots and verify-only output are not part of these exports. These materials let readers inspect selected report and expectation data; they do not show that this article independently reran the benchmark or that an external reader reproduced it. The internal benchmark source and format specification were reviewed for methodology and semantics but are not publicly available.

Source: Final Codex memory benchmark report

Report and oracle export provenance reviewed 2026-10-12; public copy is privacy-redacted and differs from original bytes.

Source: Benchmark research source package

Internal untracked benchmark research material inspected 2026-10-12; not publicly available and not cited at a commit that lacks it.

Source: Thegram persistent format contract

Internal format source at revision 625edec; not publicly available.

Source: OpenAI Codex public repository

Public repository cited for provenance; benchmark did not independently fetch its checkout.

Reference: https://github.com/openai/codex

Evidence: Seven deterministic content oracles

Seven persisted-data queries and historical boundaries; they do not measure LLM accuracy.

Knowledge: Compare equal current-state scopes

Size comparisons use 219,837 current .gram bytes and selected reversible Markdown/JSON current projections, not full journal against a snapshot.

Knowledge: Avoid universal format claims

One retrospectively reconstructed Codex corpus and selected schemas do not establish universal size, retrieval or model-accuracy performance.