Abstract

The modern data lake, warehouse, or lakehouse promises consolidation, but physical colocation is not semantic integration. A thousand incompatible datasets stored in one object store remain a thousand incompatible datasets. This essay distinguishes topological centralisation from epistemic coherence. It examines schema heterogeneity, identity, temporal semantics, units, missingness, and provenance, and shows mathematically why co-residence does not imply composability. The essay also critiques the architectural tendency to treat storage technology as a substitute for modelling. A shared platform becomes meaningful only when it introduces contracts, mappings, quality constraints, and explicit relations between local representations and shared concepts. The philosophical argument draws on the difference between collection and order: an archive becomes knowledge infrastructure only when rules of relation are made visible. Centralisation is useful, but it is the beginning of integration, not its completion.

1. The topology of storage

Suppose an organisation copies data from many systems into a shared storage environment. The network topology has changed: data that were distributed are now physically or logically colocated. This reduces certain costs—connectivity, persistence, scalable processing, backup, and access—but it does not establish that records can be joined, compared, or interpreted consistently.

Let datasets $D_1$ and $D_2$ reside in the same platform. Composability requires more than co-location:

where $S$ is schema compatibility, $I$ identity compatibility, $T$ temporal compatibility, $U$ unit and domain compatibility, $P$ provenance adequacy, and $G$ governance permissibility. Physical proximity contributes to none of these conditions automatically.

The statement “the data are in the lake” is therefore analogous to saying that books are in the same library. It tells us where they are, not whether they use the same concepts, refer to the same entities, or can answer the same question.

2. Schema equality is not semantic equality

Two tables may have identical column names and different meanings. Conversely, they may have different schemas and represent the same phenomenon. Structural comparison is necessary but insufficient.

Let ${\Sigma }_i$ be the schema of dataset $D_i$, and $M_i$ its semantic interpretation. It is possible that

or

A field named status may encode workflow position, validity, billing state, or physical availability. A field named date may refer to creation, occurrence, collection, ingestion, or publication. Automated schema ingestion preserves syntax while potentially obscuring meaning.

The platform must therefore manage semantic metadata and mapping rationale. Otherwise, centralisation increases the surface area for accidental joins: datasets look compatible because they are technically accessible.

3. Identity is an architectural service

Most cross-system questions depend on identifying the same entity across sources. Yet identifiers are often local, recycled, context-bound, or created at different stages of a process.

Let $i{d}_i(e)$ be the identifier assigned to real-world entity $e$ by system $i$. Integration requires a relation

possibly with confidence scores and validity intervals. A simple deterministic crosswalk may suffice in one domain; probabilistic entity resolution may be required in another. The important point is that the relation must be explicit and governed.

Co-located tables without identity reconciliation produce misleading counts and joins. Duplicate individuals appear as distinct; distinct entities are collapsed; historical identifiers are mistaken for current ones. A platform that centralises data but delegates identity to every analyst has centralised storage while decentralising risk.

4. Time is not one column

Temporal incompatibility is one of the most common causes of analytical error. A dataset may contain event time, system entry time, processing time, arrival time, correction time, and validity intervals. Different systems may use local time, UTC, unsynchronised clocks, or date-only precision.

A temporally explicit record can be modelled as

where $t_e$ is event time, $t_p$ processing time, $¿$ the interval during which the record is valid, and $z$ timezone or clock context. Many datasets expose only one ambiguous timestamp.

Joining records “by date” may therefore combine events that were not contemporaneous. Current-state queries may accidentally use a value that was later corrected. A central platform needs a temporal model, not merely timestamp columns.

The philosophy of time enters architecture here. Events occur; records are created; claims become valid; knowledge is revised. These are distinct temporal orders. A system that collapses them cannot reconstruct what was known when.

5. Units, codes, and the false comfort of numbers

Numbers appear universal, but measurements are local achievements. A value without unit, method, scale, and reference context is not safely comparable. Even categorical codes may collide across systems.

Suppose two sources provide values $x_1$ and $x_2$. A comparison is legitimate only after transformation into a common measurement space:

where $u_i$ denotes unit and $m_i$ method or coding context. The mapping functions ${\varphi }_i$ may be exact, approximate, or undefined.

A platform should preserve local codes and units while optionally projecting them into governed standards. Immediate replacement is dangerous because mapping may be lossy. Late semantic binding allows the organisation to ingest without pretending that every local distinction has already been reconciled.

6. Missingness has meaning

Centralised platforms often standardise nulls while losing why values are missing. Yet “not measured,” “not applicable,” “unknown,” “withheld,” “failed acquisition,” and “not yet available” are different states.

Let missingness be a categorical variable $m\in M$, not merely the absence symbol $\perp$. Analytical validity depends on the missingness mechanism. In statistical terms, data may be missing completely at random, missing at random conditional on observed variables, or not missing at random. In operational systems, missingness is frequently process-generated and therefore informative.

A central store that replaces all absence with null may increase apparent uniformity while destroying process evidence. Data quality must capture the reason and stage at which information became unavailable.

7. Provenance and the epistemic chain Data become trustworthy not because they are central, but because their provenance is inspectable. Provenance includes source system, extraction time, transformation version, reference data version, quality outcomes, and publication context.

Let an output $o$ be produced by a computation graph $G$. Its lineage can be represented as a subgraph

containing all upstream nodes and edges that contributed to the result. Without this graph, centralisation can make data less explainable: records are copied into a shared zone and detached from the systems and events that gave them meaning.

The archive, in the Foucauldian sense, is not a heap but a system of enunciability: rules that determine how statements can be formed and related. Provenance is one of the technical forms of those rules.

8. Centralisation can amplify disorder A poorly governed data lake lowers the cost of producing new datasets but also lowers the cost of producing redundant, poorly documented, and contradictory datasets. This is the familiar “data swamp,” but the metaphor is often used too casually. The underlying mechanism is uncontrolled derivation.

Let $D_t$ be the number of datasets at time $t$, $p$ the average probability that a consumer creates a derivative copy, and $r$ the retirement rate. Then

where $C_t$ is consumer activity. If $pC_t > rD_t, dataset proliferation accelerates. Without contracts, ownership, and lifecycle management, centralisation increases combinatorial ambiguity.

The solution is not to prohibit derivation. It is to distinguish governed products, temporary work products, and deprecated assets; to track their ancestry; and to make authoritative status visible.

9. What must be added to storage A meaningful platform adds at least:

• canonical or domain-level identities;
• explicit event and validity time;
• semantic contracts and controlled vocabularies;
• unit and code mappings with preserved local values;
• quality rules and quarantine mechanisms;
• lineage and transformation versioning;
• ownership and lifecycle states;
• access policies linked to purpose;
• published data products with service expectations.

We can summarise the platform as

Removing any term changes the nature of the system. Storage and compute alone form a processing environment, not necessarily a trustworthy platform.

Conclusion

Putting all data in one place solves a logistical problem. It does not solve the epistemic problem of determining what the data mean, whether they refer to the same entities, whether their times align, whether their measurements are comparable, or whether their use is legitimate.

Centralisation is valuable because it creates the technical possibility of shared computation. But possibility must be converted into composability through contracts, identity, temporal modelling, semantic mappings, quality controls, lineage, and governance. The mature architectural question is therefore not “Where are the data?” but “Under which explicit relations can these data become evidence for the same claim?”

Conceptual references

Foucault, M. The Archaeology of Knowledge (1969/1972). Kitchin, R. The Data Revolution (2014). Sowa, J. F. Knowledge Representation (2000). Stonebraker, M., and Hellerstein, J. M., writings on database architecture and data integration.