Append-Only Architectures and the Ethics of Preserving History

Abstract

Append-only architecture is commonly justified through auditability, recovery, and simplified concurrency. Its deeper importance is cultural. A mutable table tends to present the latest state as if it were the whole truth, whereas an append-only history records that every state was produced through events, revisions, and institutional decisions. This essay examines event histories, immutable logs, state derivation, bitemporal models, tombstones, corrections, and retention. It formalises state as a fold over ordered events and explores when append-only design is algebraically safe. It also addresses limits: infinite retention is neither technically neutral nor ethically desirable, and immutability can conflict with privacy, legal erasure, and minimisation. The argument is that preserving history is an ethical commitment only when history remains interpretable, governed, and proportionate. A platform should remember enough to explain itself, but it should not convert technical memory into indiscriminate surveillance.

1. State hides its own becoming

A conventional operational table presents a current state. A record says that an order is complete, an account is inactive, a case is closed, or a report is valid. The table rarely reveals, by itself, how that state came to be. Previous values have been overwritten, and the present appears without genealogy.

Append-only design begins from the opposite intuition: change should be represented by adding events rather than replacing the past. Let an event stream for entity $i$ be

ordered by a declared temporal relation. Current state is derived by a fold:

where $\varphi$ applies each event to the prior state. The state is not denied; it is recognised as a projection of history.

This distinction is epistemically important. A state answers “what is believed now?” An event history can answer “what happened, what was believed then, and how the present was constructed?”

2. Immutability and the meaning of correction

Immutability does not mean that mistakes remain authoritative. It means that correction is represented by a new event rather than concealed through destructive replacement. If event $e_k$ was wrong, a correction $e_c$ can supersede or negate it according to explicit semantics.

Let the interpretation function be

A correction changes the interpreted state without deleting the historical fact that the earlier assertion existed. This enables two questions to coexist: what does the platform currently consider valid, and what information was previously published or acted upon?

A tombstone event is a common mechanism:

Consumers interpret subsequent state as absent or inactive while the log retains evidence of the transition. For some domains, a compensating event is more appropriate than deletion. The semantics must be explicit because “delete,” “cancel,” “revoke,” and “correct” are not interchangeable acts.

The ethical value of append-only design lies precisely here: it refuses to make correction indistinguishable from original truth.

3. Ordering and the problem of time

An append-only log is meaningful only if ordering is understood. Distributed systems rarely offer a universal total order corresponding to the world. They provide partition order, source sequence, processing order, or logical clocks.

Let events carry several times:

Event time records occurrence; ingest time records arrival; valid time records when the assertion applies; system time records when the platform stored it; sequence may encode source order. These dimensions can disagree.

State as of event time may require late-arriving events to be inserted conceptually into the past. State as known at a historical system time must exclude information that had not yet arrived. A bitemporal model defines

where $t_v$ is valid time and $t_s$ system time. This allows questions such as: what was believed on Tuesday about the world as of Monday? Without this distinction, retrospective recomputation silently alters the narrative available to historical decision-makers.

Append-only architecture therefore requires temporal literacy, not merely immutable storage.

4. Algebra of event application

Not every event stream is easy to replay. The algebra of state transition determines whether partitioning, parallelism, and reordering are safe.

If events combine through an operation $\oplus$, useful properties include associativity:

commutativity:

and idempotence:

CRDTs exploit such properties to converge under distributed updates. Many business events do not commute. A cancellation before creation is not equivalent to cancellation after fulfilment. Dose administration, inventory movement, and contractual status transitions depend on order and preconditions.

The platform should therefore classify event types by algebraic behaviour. Commutative facts can be aggregated freely; ordered state transitions require sequence constraints; corrections require precedence rules; derived events require lineage to their causes. Treating all append-only data as an unordered set invites subtle corruption.

5. Event history and derived state

Consumers often need current state rather than full history. Event-sourced architecture therefore produces materialised views:

The view may represent active therapies, latest valid reports, open cases, available inventory, or current customer status. It should be reproducible from the event log under a versioned projection ${\psi }_v$.

When projection logic changes, the platform can rebuild the view. This is a major benefit, but it creates a governance question: should history be restated under the new interpretation or should multiple view versions coexist? A derived table should carry both data validity and projection version.

Snapshots can reduce replay cost. If $s_k$ is a trusted snapshot after event $k$, then

Snapshots are performance artefacts, not replacements for history. They require checksums, provenance, and rebuild tests so that convenience does not become a new opaque source of truth.

6. The institutional memory of a platform

Organisations often preserve documents but lose decision paths. A metric changes, a status is reclassified, or a record is corrected, yet the reason survives only in email or personal memory. Append-only architecture can make change legible if events include rationale, actor, authority, and relation to prior assertions.

An institutional event may be represented as

This does not mean every record should become a verbose narrative. It means that significant changes should preserve the metadata needed to distinguish observation, correction, interpretation, and administrative action.

The platform then functions as an institutional memory. Memory, however, is not neutral storage. It is curated capacity for reconstruction. Unindexed logs without semantics are closer to sediment than memory. They preserve traces but not necessarily intelligibility.

7. Privacy, erasure, and the limits of immutability

The moral rhetoric of “never delete” is dangerous. Infinite retention increases exposure, cost, surveillance potential, and legal risk. Privacy law may require erasure or restriction. Security incidents can turn preserved history into accumulated harm.

Append-only should therefore be understood within a retention and cryptographic architecture. Techniques include encryption with destroyable keys, separation of identity from events, tokenisation, redaction events, access-time filtering, and physical compaction after legally defined periods.

Let the retained history be selected by policy $R$:

The objective is not maximal memory but sufficient accountable memory. A retention policy should preserve what is required to explain legitimate actions while removing what no longer has a justified purpose.

This introduces an ethics of selective remembrance. Forgetting can be as responsible as remembering when it limits unnecessary power.

8. Append-only does not eliminate truth disputes

A complete log can record every assertion and still fail to establish which assertion is valid. Events may conflict, sources may disagree, and corrections may be contested. The platform needs resolution rules and confidence models.

Let conflicting assertions about proposition $p$ be $A_p=\{a_1,\dots ,a_m\}$. Resolution may depend on source authority, recency, corroboration, or explicit adjudication:

where $W$ is a governed weighting, not an objective law of nature. The losing assertions should often remain visible for audit even when they do not drive current state.

Append-only architecture preserves disagreement; it does not solve it automatically. This is a virtue if the platform resists manufacturing false certainty. It is a failure if consumers are left without a governed interpretation.

9. Designing memory with proportion

A disciplined append-only platform defines event identity, ordering semantics, correction models, projection versions, retention classes, and reconstruction tests. It separates raw capture from published history, because not every technical event deserves indefinite institutional status.

A useful design objective balances explanatory value against risk and cost:

The weights depend on domain and consequence. Safety-critical processes may require deep history. Low-value telemetry may be aggregated or discarded. Sensitive data may require stronger minimisation despite analytic potential.

The architecture of memory should be explicit about whose future questions it is designed to answer and whose exposure it might increase.

Conclusion

Append-only architecture reveals that state is produced, revised, and governed through time. It enables replay, audit, correction, and alternative projections because it refuses to collapse history into the latest value. Its mathematical foundation lies in ordered event application and versioned state derivation; its operational foundation lies in immutable capture, tombstones, snapshots, and bitemporal semantics.

Yet preservation is not an unconditional good. History without interpretation becomes noise; history without retention limits becomes risk; immutability without correction becomes dogma. The ethical objective is proportionate memory.

A trustworthy platform remembers enough to explain how its present came to be. It also records its corrections, distinguishes past belief from current validity, and forgets when continued retention no longer has legitimate purpose. In this balance, append-only design becomes an architecture not merely of storage, but of accountable historical consciousness.

Conceptual references

Fowler, M. “Event Sourcing” (2005). Snodgrass, R. T. Developing Time-Oriented Database Applications in SQL (1999). Lamport, L. “Time, Clocks, and the Ordering of Events in a Distributed System” (1978). Ricoeur, P. Memory, History, Forgetting (2000). Mayer-Schönberger, V. Delete: The Virtue of Forgetting in the Digital Age (2009).