# Storage overview

Kind

overview

For

developer · operator

Verified against

@veritio/storage@0.4.7 · @veritio/core@0.4.7

Storage is part of the evidence model, not an interchangeable persistence detail. Exactly one authoritative store owns each tenant chain. Archives, analytical views, and caches may accelerate access, but they never assign sequence numbers and they are never the definitive input to verification.

That single rule decides most storage questions in a Veritio deployment.

```text
                 host mutation transaction
                            │
                            ▼
              ┌───────────────────────────┐
              │   authoritative store     │  ← owns sequence, idempotency,
              │   (AuditStore contract)   │    previous-hash linkage
              └───────────────────────────┘
                   │            │
      one-way copy │            │ one-way projection
                   ▼            ▼
        object archive     ClickHouse read model
        (R2 / S3 / MinIO)  (episodes, subjects)
                   ╳            ╳
            never a sequence owner, never
            authoritative for verify or DSAR
```

## What `@veritio/storage@0.4.7` ships

[Section titled “What @veritio/storage@0.4.7 ships”](#what-veritiostorage047-ships)

The package exports factories and schema constants; it opens no connections and reads no environment variables. Every driver below is a published export of `@veritio/storage`, except `MemoryAuditStore`, which lives in `@veritio/core`.

Export

Tier

Notes

`MemoryAuditStore` (`@veritio/core`)

Authoritative, ephemeral

Tests, examples, Workbench. Does not survive a restart.

`createPostgresAuditStore`

Authoritative

Postgres dialect over a host-injected `SqlAuditExecutor`.

`createNeonAuditStore`

Authoritative

Same Postgres dialect; Neon preserves the ordering semantics.

`createMysqlAuditStore` / `createMySqlAuditStore`

Authoritative

MySQL dialect. The second name is a spelling alias.

`createMariaDbAuditStore` / `createMariaDBAuditStore`

Authoritative

MariaDB through the MySQL dialect, plus an all-caps alias.

`createMongoAuditStore`

Authoritative

Needs a host-injected collection **and** transaction callback.

`createFileEvidenceStore`

Authoritative, single-writer

Durable JSONL event **and** edge chains under one directory. Not the `AuditStore` interface.

`createRedisAuditTipCache`

Derived cache

Tenant tip only. Records are re-validated on read and write.

`createObjectAuditArchive`, `archiveAuditStoreTenant`

Derived cold tier

Sealed NDJSON segments on any S3-compatible client.

`createClickHouseAuditReadModel`

Derived read model

Episode, session, and subject scans.

`createPostgresOutboxAdapter`, `createNeonOutboxAdapter`, `createMysqlOutboxAdapter`, `createMariaDbOutboxAdapter`, `createMariaDBOutboxAdapter`, `createFileOutboxAdapter`

Transactional outbox

Durable queue between an application commit and evidence append.

`createOutboxDispatcher`, `createHttpOutboxDispatcher`, `createHttpIngestTarget`

Delivery

Leased draining to a local sink or an HTTP ingest endpoint.

Schema constants ship alongside the factories: `POSTGRES_AUDIT_RECORDS_SCHEMA_SQL`, `MYSQL_AUDIT_RECORDS_SCHEMA_SQL`, `MONGO_AUDIT_RECORD_INDEXES`, `POSTGRES_OUTBOX_SCHEMA_SQL`, `MYSQL_OUTBOX_SCHEMA_SQL`, and `clickHouseAuditReadModelSchemaSql()`. You own migrations; the constants only fix the shape the adapters depend on.

Two shapes are load-bearing in the audit table. The primary key is `(tenant_id, sequence)`, which makes a duplicate sequence in one tenant impossible. A separate unique key on `(tenant_id, idempotency_key_hash)` makes a reused key impossible. Both are per tenant, which is why two tenants can each hold sequence `1` without colliding.

## The `AuditStore` contract

[Section titled “The AuditStore contract”](#the-auditstore-contract)

The interface is two methods:

```ts
interface AuditStore {
  append(event: AuditEvent, options?: AuditStoreAppendOptions): Promise<AuditRecord>
  list(scope: EvidenceScope & { tenantId: string }, options?: AuditStoreListOptions): Promise<AuditRecord[]>
}
```

`AuditStoreAppendOptions` carries `idempotencyKey` and `expectedPreviousHash`. `AuditStoreListOptions` carries `afterSequence` and `limit`.

Behind those two methods, an authoritative store must keep four decisions together in one atomic boundary:

1.  assign a gapless sequence inside the tenant boundary;
2.  reject a reused idempotency key when canonical event bytes differ;
3.  reject an unexpected previous hash;
4.  recompute record integrity before returning persisted data.

If a backend cannot make those decisions atomically, it is not an `AuditStore`.

The SQL adapters get this from the engine. Inside the host transaction they first look up the tenant-scoped idempotency hash, then read the tenant tip with `ORDER BY sequence DESC LIMIT 1 FOR UPDATE`, then insert. The row lock is what serializes two concurrent appends for the same tenant into two consecutive sequence numbers instead of one duplicate. The Mongo adapter reaches the same outcome through a host-supplied session and the two unique indexes in `MONGO_AUDIT_RECORD_INDEXES`; without a replica set and real transactions it will not hold.

Reads are equally strict. Every adapter parses the stored `record_json`, re-hashes it with `hashAuditRecord`, and returns a clone. A record edited directly in the database fails closed on the next read instead of being served as evidence, and a caller mutating a returned record cannot reach stored bytes.

Note the exception in the table above: `createFileEvidenceStore` is durable and hash-chained, but it does not implement `AuditStore`. It exposes `recordEvent`, `recordEdge`, `recordBatch`, `listEvents`, `listEdges`, `listCommits`, and `verify` — the recorder-facing sink shape, covering both the event chain and the edge chain, for tools that run as a fresh process per event. One directory holds exactly one tenant’s chains, and it coordinates exactly one writer.

## Host-injected configuration

[Section titled “Host-injected configuration”](#host-injected-configuration)

`@veritio/storage` exports factories, schemas, conformance tests, and transport-neutral executor interfaces. It deliberately does not open database connections or read environment variables. Your server bootstrap owns the vendor client and adapts its transaction callback into the required executor:

```ts
interface SqlAuditSession {
  execute(statement: string, params: readonly unknown[]): Promise<SqlAuditQueryResult>
}
interface SqlAuditExecutor extends SqlAuditSession {
  transaction<T>(run: (session: SqlAuditSession) => Promise<T>): Promise<T>
}
```

This keeps credentials and deployment choices outside protocol code, and makes Neon, self-managed Postgres, or another compatible service a host decision rather than a protocol one.

## Authoritative write path

[Section titled “Authoritative write path”](#authoritative-write-path)

The safe service flow is:

1.  Authenticate the request and resolve tenant scope on the server.
2.  Normalize and deterministically redact the event.
3.  Begin the authoritative database transaction.
4.  Lock or compare the tenant tip.
5.  Apply idempotency and expected-previous-hash checks.
6.  Assign sequence, hash canonical bytes, and append.
7.  Commit before acknowledging success.

When an application mutation and its evidence must share one commit, that flow moves behind a queue: enqueue a minimized governed-change outbox row inside the application transaction, and let a leased dispatcher append retry-safe evidence afterwards using stable event and edge identities. Outbox rows move through `pending`, `leased`, `paused`, `dispatched`, and `dead`, and a paused row re-arms only after an explicit circuit acknowledgement. See [Transactional outbox](/docs/storage/outbox/) for the adapter, dispatcher, and delivery-safety surface.

## Derived tiers, and why the label is a hard rule

[Section titled “Derived tiers, and why the label is a hard rule”](#derived-tiers-and-why-the-label-is-a-hard-rule)

Two derived tiers ship in the package. Both look like storage. Neither is.

**Object archive.** `createObjectAuditArchive` seals already-sequenced records into segments on an injected `ObjectArchiveClient` (`put`, `get`, `list`) backed by R2, S3, or MinIO. Segments hold the exact `canonicalJson` bytes of each record as NDJSON lines, so the chain verifies byte-for-byte from the archive alone. A manifest is written _after_ its segment, so a manifest’s existence means the bytes are durable. Events and edges are the protocol’s two independently sequenced chains, so they archive under separate key namespaces and `verifyTenant` replays both. Sealing validates hashes, tenant scope, and chain continuity before writing, and reads recompute the segment digest and every record hash. `archiveAuditStoreTenant` drains a tenant’s un-archived event tail incrementally, resuming from the archive’s own tip.

It is still derived. Object storage cannot couple gapless per-tenant sequencing to an idempotency-conflict check atomically, and the tip check between two concurrent sealers can race — hosts must run one archiver per tenant.

**ClickHouse read model.** `createClickHouseAuditReadModel` accelerates scans that would otherwise page whole tenant chains into memory: `listEpisodes`, `listEpisodeSteps`, `listBySubject`. Envelope and grouping keys become typed columns for pruning, but the full canonical record travels in a raw `String` column — never ClickHouse’s native JSON type, which parses and re-serializes and would break hash recompute. Projection is at-least-once; duplicates collapse through the ReplacingMergeTree engine and the read helpers query with `FINAL`. Every projected and returned record is hash-revalidated. All value filters bind through ClickHouse query parameters rather than SQL interpolation.

It is still derived, for the same structural reason: no synchronous unique constraints, no transactional tip check.

So the rule stands in both directions. Derived tiers never assign a sequence number, never adjudicate an idempotency conflict, and are never the authority for verification or a data-subject request. They keep canonical bytes as opaque strings and hash-revalidate on read, which is what makes them safe to _hold_ evidence — not what makes them fit to _own_ it. Run verification and definitive exports from the authoritative chain, and use derived tiers for recovery copies, long-range scans, and operational views only once their lag and replay behavior are visible to you.

The Redis tip cache is the smallest case of the same rule: it stores a validated tenant tip to save a lookup, and never the only copy of anything.

## Prove the store before you trust it

[Section titled “Prove the store before you trust it”](#prove-the-store-before-you-trust-it)

`@veritio/storage/conformance` exports the contract as executable checks. `createAuditStoreConformanceTests` returns five named tests; a store is authoritative only when all five pass. Running them against a custom in-process store gives:

verified output

```json
{
  "store": "serialized-in-process-store",
  "checkCount": 5,
  "checks": [
    {
      "name": "appends tenant-scoped chains and lists records deterministically",
      "ok": true
    },
    {
      "name": "returns idempotent records and rejects conflicting idempotency keys",
      "ok": true
    },
    {
      "name": "fails closed for missing tenant scope and expected tip mismatches",
      "ok": true
    },
    {
      "name": "returns cloned records so callers cannot mutate stored evidence",
      "ok": true
    },
    {
      "name": "fails closed when stored record integrity is corrupted",
      "ok": true
    }
  ],
  "conformant": true
}
```

The fifth check is the one that matters most and is easiest to skip. The suite corrupts a stored record behind the store’s back through a `mutateStoredRecord` seam that is deliberately not reachable through the `AuditStore` interface, then requires the store to refuse to hand the record out. The [store conformance guide](/docs/storage/conformance/) walks the full custom-store implementation.

For the repository’s disposable live matrix, run the same five checks against real engines:

Terminal window

```sh
bun run --cwd storage db:up
bun run --cwd storage test:live
bun run --cwd storage db:down
```

The matrix covers Postgres, Neon-compatible Postgres, MySQL, MariaDB, MongoDB replica-set transactions, MinIO-compatible object storage, and ClickHouse. Each backend is skipped when its `VERITIO_*_TEST_*` variable is absent, so a skipped live suite is not production evidence — assert the variables are present in your own CI.

## What a green run does and does not prove

[Section titled “What a green run does and does not prove”](#what-a-green-run-does-and-does-not-prove)

A passing conformance run proves the store honors tenant-scoped ordering, idempotent replay, fail-closed rejection of missing scope and stale tips, cloned reads, and fail-closed integrity on tampered bytes.

It does not prove concurrency safety. The five checks are sequential; nothing in them appends from two callers at once. Concurrent correctness comes from the transaction and locking you wired underneath — `FOR UPDATE` on the tip row, a real replica-set session for Mongo — and from operating the engine that way in production. The repository’s separate `bun run --cwd storage test:stress` suite exercises outbox lease concurrency, not `AuditStore` appends.

It also does not prove durability, backup integrity, retention behavior, or that your host actually resolved tenant scope from an authenticated request rather than from client input. And like everything in Veritio, a conforming store supports compliance evidence; it does not by itself satisfy any legal obligation.

## Next

[Section titled “Next”](#next)

-   [Postgres and Neon](/docs/storage/postgres/) for the concurrent production path.
-   [MySQL and MariaDB](/docs/storage/mysql-mariadb/) for existing transactional SQL estates.
-   [MongoDB](/docs/storage/mongodb/) for replica-set sessions and required indexes.
-   [File store](/docs/storage/file-store/) for the executable single-directory tutorial.
-   [Store conformance suite](/docs/storage/conformance/) before you ship a custom store.
-   [Transactional outbox](/docs/storage/outbox/) when evidence must share a commit with an application mutation.

[Edit page](https://github.com/getveritio/veritio-website/edit/main/src/content/docs/docs/storage/overview.mdx)

Last updated: Aug 23, 2026

[Previous  
Express](/docs/frameworks/express/)[Next  
Postgres & Neon](/docs/storage/postgres/)

Veritio provides evidence support, not legal advice or automatic compliance.

This site uses cookieless, anonymous analytics (Umami) by default. With your consent, we also enable Google Analytics, which sets cookies and sends usage data to Google. [Privacy Policy](/legal/privacy/)