Most Knowledge Is Dark Matter We Cannot Directly Observe

June 30, 2022

The publicly visible knowledge in any society, what is published, taught, and discussed, represents a small fraction of the total stock. The rest is intellectual dark matter: knowledge whose existence we can infer but cannot directly observe.

"Since collapse in material technology is always preceded by collapse in the practice of social technology, Dark Ages are always preceded by Intellectual Dark Ages." Samo Burja

Burja identifies three categories:

Lost knowledge once existed but no living person possesses it; we infer it from artifacts, like the Antikythera mechanism or the Roman concrete that has outlasted every modern equivalent.
Proprietary knowledge is actively concealed because its value depends on exclusivity; Renaissance Technologies' Medallion Fund returns are evidence of knowledge that Jim Simons will never publish.
Tacit knowledge cannot be fully articulated even by those who possess it; a master craftsman knows how to do something that no written manual can capture.

The total stock of intellectual dark matter in any society dwarfs the visible stock. Most analysis of institutions and civilizations ignores this entirely, which is why such analysis is systematically wrong. When we evaluate a civilization's capabilities by examining its published knowledge, we are like astronomers estimating the mass of a galaxy by counting only the visible stars. The estimate will be off by an order of magnitude.

This has unsettling implications for our own era. We cannot assess what we have lost because the loss is by definition invisible to us. The people who would notice that a critical body of knowledge has degraded are precisely the people whose knowledge has degraded. FOGBANK, the classified nuclear material whose manufacturing process was lost by the NNSA, was rediscovered only because the need was urgent. How many comparable losses have occurred in domains where no urgent need has yet forced the question?

What you can see is never the whole picture; the most consequential knowledge is the knowledge you do not know is missing.