The Price of Knowing: Szilard, Brillouin, Jaynes
Details
Last week we found that erasing a bit of information costs heat. This week we ask a deeper question: why should knowing anything cost us at all? Three thinkers answer.
We begin with Leó Szilárd. In his 1929 paper "On the Decrease of Entropy in a Thermodynamic System by the Intervention of Intelligent Beings," he looked at Maxwell's demon, an imaginary creature that watches molecules in a box and lets only the fast ones through a little door, seeming to sort hot from cold for free. Szilárd found the catch: before it can sort a molecule, the demon has to know whether it's fast or slow, and getting that knowledge costs at least as much order as the sorting creates. He had put a price on a single piece of information, long before the word "bit" existed. Contribution: knowing is not free.
We then turn to Léon Brillouin, who asked what it costs just to see. In "Maxwell's Demon Cannot Operate" (1951), he noted that inside a sealed box everything glows the same color and the demon is sitting in the dark. To spot a molecule, it has to shine its own light, and that light throws off more disorder than the sorting could recover. Brillouin named what gets spent: negentropy, the order a system burns to learn. Contribution: to know the world, you have to pay the world.
We arrive finally at Edwin Jaynes, who showed the two ideas were really one. In "Information Theory and Statistical Mechanics" (1957), he argued that the entropy physicists measure in a warm gas isn't a substance in the world. It is just a measure of how much we don't know about the details we can't track. Seen this way, the physics of heat is careful guessing under partial knowledge, and the physicist's entropy and information theory's entropy are the same thing. Contribution: entropy is ignorance, measured.
Links
Leó Szilárd
- "On the Decrease of Entropy…" (1929, Zeitschrift für Physik 53, 840–856) — https://doi.org/10.1007/BF01341281
- English translation (1964, Behavioral Science 9(4), 301–310) — https://doi.org/10.1002/bs.3830090402
- Maxwell's demon — Wikipedia — https://en.wikipedia.org/wiki/Maxwell%27s_demon
- Leó Szilárd — Wikipedia — https://en.wikipedia.org/wiki/Le%C3%B3_Szil%C3%A1rd
Léon Brillouin
- "Maxwell's Demon Cannot Operate: Information and Entropy. I" (1951, Journal of Applied Physics 22(3), 334–337) — https://doi.org/10.1063/1.1699951
- Science and Information Theory (1956, Academic Press; Dover reprint, 2013)
- Negentropy — Wikipedia — https://en.wikipedia.org/wiki/Negentropy
- Léon Brillouin — Wikipedia — https://en.wikipedia.org/wiki/L%C3%A9on_Brillouin
Edwin Jaynes
- "Information Theory and Statistical Mechanics" (1957, Physical Review 106, 620–630) — https://doi.org/10.1103/PhysRev.106.620
- "Information Theory and Statistical Mechanics. II" (1957, Physical Review 108, 171–190) — https://doi.org/10.1103/PhysRev.108.171
- Principle of maximum entropy — Wikipedia — https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
- Edwin Thompson Jaynes — Wikipedia — https://en.wikipedia.org/wiki/Edwin_Thompson_Jaynes
Timezones
6:00 AM, Pacific (USA)
7:00 AM, Mountain (USA)
8:00 AM, Central (USA)
9:00 AM, Eastern (USA)
About Our Group
We welcome open minded, respectful conversation on Stoicism and its relevance to daily life, personal growth, and modern thought. Our discussions connect ancient philosophy with contemporary science, psychology, economics, and culture with the shared aim of cultivating wisdom together. The meeting begins at 9:00 AM Eastern, with dialogue starting promptly at 9:15 AM.