Entropy, Time, and Maxwell's Demon

Credit: Peter MacDonald, Edmonds UK.

While I don’t plan on giving a complete rundown of every sentence in my new Symmetry book every now and again, I figured I’d run some ideas by you and see if you have any followup questions. At the moment, I’m working on Chapter 2, with the working title, “Does Entropy Increase with Time or does it Make Time?” (Rest assured, there’s a lot of material that wasn’t in the original io9 article, but I like the title.)

Apropos of this, I’ve spent the weekend reading fellow Pennsbury High School alum and Dutton author Sean Carroll’s book, “From Eternity to Here,” who addresses this very question is a fair amount of detail. It’s a very well-reasoned book and has a very good tone, but in the end, I remain unconvinced. I still fall in the “time makes entropy” camp (which, to be fair, is the orthodox view, which is presumably one of the reasons why Sean wrote his book).

As you may know, the 2nd law of thermodynamics says, colloquially, that entropy always increases with time. Practically speaking, this means that energy will flow from hot materials to cooler ones in the form of heat.

Is there any loophole that could allow us to get around the Second Law? In the 19th century, James Clerk Maxwell devised a cool thought experiment to cheat entropy. Maxwell was no slouch. He unified all of electromagnetism into a single theory, combining a number of very disparate looking ideas into “Maxwell’s Equations,” which served as the principle inspiration for Einstein’s theory of Special Relativity.
Maxwell imagined a box filled with air molecules, some moving faster, and some moving slower than one another, but thoroughly mixed. In the middle of the box was a partition, separation the left from the right with a little hole and a trap door in front of it.

Maxwell’s idea was that whenever a “cold” molecule (one moving slower than average) approached the trapdoor from the left side of the box, a very clever demon would open the door and let the molecule through to the right side of the box. Likewise, whenever a “hot” molecule approached from the right, the demon would open the door and let the molecule go into the left side of the box. Otherwise, the door would remain closed.

That’s it, but it this has profound implications. Assuming the trapdoor is one of those no friction, the demon is basically making the left side of the box hot and the right side of the box cold. And he’s doing it without breaking a sweat.

Image Credit Science Photo Library

This is exactly the opposite of what’s supposed to happen in thermodynamics. Remember how it’s supposed to work. The takeaway about how the 2nd Law works is that heat should generally flow from hot regions to cooler ones. It’s probably not too much of a stretch to suppose that you didn’t even need a popular physics book to tell you that.
What gives? I admit that I first saw this problem when I was an undergraduate and was profoundly unimpressed with it. Who cares about a few atoms here and there? Besides, if the 2nd law is really only statistical in nature, does it really matter if we can circumvent it?

Yes, younger me. It does.

The 2nd law is supposed to be a hard and fast law of the universe, and a back door would be amazing. Why do we need to continuously burn coal, petroleum, or natural gas? Because most of the energy used by our machinery gets wasted as heat. If we could somehow employ a few million of Maxwell’s demons to recover the heat into useful energy, we’d be pretty much set.

It was until nearly a century after Maxwell came up with his demon that we really understood why the 2nd law remained inviolate. In 1948, Claude Shannon, a research scientist at Bell Labs, founded a branch of research known as “Information Theory.” Just as quantum mechanics made all of modern computing physically possible, information theory revolutionized cryptography, communication, and made innovations like the Internet possible.

One of the major results of information theory is that information and entropy are more or less the same thing. Suppose I send a message that is exactly two characters long. How many different messages can I send? I could in principle send you 26×26=676 different “words,” but most of those letter combinations are completely meaningless. Only a few (the Scrabble dictionary lists 101) are actual words.

To the computer scientists among you, this means that while in principle it would require about 10 bits (the 1’s and 0’s that are used to store data) to differentiate between every possible 2 letter combination, if you know that you’re transmitting a word, you only need about 7 bits. What a savings!

Communications can be significantly compressed by noticing that certain letters are used less frequently than others. E’s, for example, show up way more often than Z’s in the English language. This is why the former is worth only 1 point in Scrabble and the latter is worth 10. It also explains why “E” in Morse code is:

while Z is:

— — ••

Z takes far longer to tap out, but that’s okay, because you’re going to do it far less frequently. Another way of thinking about this is that the more complicated (or unlikely) a message is, the more information it carries, and the more bytes of data you’d need to store it on a computer.

What does all of this have to do with Maxwell’s Demon? Every time the demon has to decide whether or not to let an atom through his trap door, he takes a measurement and makes a recording of the speed of the atom. The very existence of that recording (whether in the brain of the demon, on a pad of paper, or in a computer) adds information to the universe, and information and entropy are the same thing.
The demon doesn’t really decrease entropy by playing his gas games; it’s just that the increased entropy goes in large part to creating his measurements and memories.

Your own memories, then, are in some sense a testament to the fact that the entropy of your brain is increasing with time. The fact that you remember the past and not the future really does just seem to be a testament to the increasing entropy of your brain.


Followup questions are appreciated.


This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

10 Responses to Entropy, Time, and Maxwell's Demon

  1. Kevin Yeh says:

    Is it really accurate to say that the entropy of our brains “Increases” with age? We increase the entropy of the total system by metabolizing food, giving off heat, producing CO2, etc… But I would like to think that my brain becomes more ordered with time rather than less.

    I guess the piece that is confusing for me still is the relationship between information, entropy and uncertainty. Would a restatement of the 2nd law of thermo using the term uncertainty say that the total uncertainty is increasing or decreasing?

    • dave says:

      It’s a good point. The information in our brains certainly increase. Brains are a tricky business (as you know). It may be easier to think of the bits in a disk drive or something similar. Prior to writing anything out, everything is set to zero or some other low entropy pattern (010101…, for instance). It’s possible to write random sequences of 0’s and 1’s (high entropy), but we generally assume that the truly “random” sequences have no connection to the past.

      Good suggestion. I’ll have to think of a nice way to clarify this point.

  2. jin says:

    >One of the major results of information theory is that information and entropy are more or less the same thing.

    it may be beyond the scope of the article but i don’t see how or why this is true… the passage that follows this statement does not seem to bolster the initial assertion.

    re: maxwell’s demon – i must be missing something because i don’t get why it was ever an issue. the little demon is doing work to defeat entropy. we do that all the time. what am i missing?

    finally, is the argument that entropy makes time or is it that it makes time DIRECTIONAL. i’ve never heard that it said that time’s existence is emergent from entropy… only that entropy is the reason why time marches inexorably forward instead of being “random access” like space.

    • dave says:

      Ah, but the demon isn’t doing work. The door could be made as lightweight as you like, and as on a perfectly oiled track. Physically, it is trivial to move the door.

      Your other points are very good. I definitely would like to add more discussion (similar to what Kevin was pointing out above) relating memory/information to entropy. I think the computer analogy is appropriate; I’ll definitely want to flesh that out.

      And you’re also correct that there is a difference (albeit a subtle one) between “creating time” and “defining an arrow of time” or even “defining that such a thing as an arrow of time exists.” But the “standard” interpretation is that entropy really doesn’t cause any of these things. Most physicists believe (mostly because we don’t have anything more fundamental) that having a time-like coordinate is simply part of our universe, and that the increase in entropy is simply a natural consequence of probability theory.

      Interesting points. Thank you for the feedback.

  3. Kevin Yeh says:

    If you inspect Schrodinger’s cat, are entropy and uncertainty each increasing, decreasing or staying the same?

    Similarly, if you (or perhaps Maxwell’s demon) measures the speed (hot or cold) and position (near the wall) of the particles does quantum mechanics say anything about how those measurements affects the entropy of the system?

  4. Seth says:

    Saying “that entropy always increases with time” leaves out the possibility that entropy can remain constant.

    I recall the change in entropy is allowed to equal zero under some special circumstances. But it seems as though if Maxwell’s Demon allows a fast moving particle through, regrets the act, so lets a fast moving particle through from the opposite side in some inconspicuous amount of time, that this would be an adiabatic process with delta_S = 0. You say that because he has made two measurements that delta_S > 0 because of the information he gained- but I can’t see how this wouldn’t qualify to be an adiabatic process.

  5. Boris says:

    Dave, recently I saw a demonstration of the fountain effect in superfluid Helium. I was wondering if this actually is a small form of a Maxwell Demon? As it was explained to me, part of the effect is based on “colder” molecules being allowed through to then become a fountain. Or is this based on the phase change, which then begs the question why a phase change might be allowed an exemption…

    Sorry for asking, as you can see I am not a professional physicist 🙂

  6. jin says:

    “Ah, but the demon isn’t doing work. The door could be made as lightweight as you like, and as on a perfectly oiled track. Physically, it is trivial to move the door.”

    hiya doc,

    hmmm ok… but isn’t the decision making of letting a particle in or out and the sensing of which particles meet the criteria “intellectual work”? so the demon is the “ordering agent”… i guess like a writer… or a mathematician… and is incurring entropy penalties in itself?

    • dave says:

      Precisely, but the relevant part of it is captured in the Shannon information. As I commented to Kevin (above), perhaps this will be made much clearer if we consider a computer or a robot, rather than a demon.

      • Larry Olson says:

        If it were shown that the second law was broken for 10 minutes or 10 seconds, would Shannon’s information have to be re-looked at? For example if someone found a way to make the brownian ratchet work (say by isolating one half of the ratchet in a viscous fluid, causing the frictionous fluid to heat up faster than the regular non viscous fluid)… separating hot from cold.. what would happen to shannon’s information and equations? Would science fall apart and would shannon’s math fall apart? Another example to consider is the brownian ratchet where hydrostatic pressure is different on each side of the system instead of a temperature differential.. Or one could consider a vibrating brownian ratchet that simply gives off sparks of static electricty by rubbing on nano wool and shorting out to metal. In this case the brownian ratchet need not go in one direction but can randomly vibrate erratically and still produce useful electricity. All these ideas could be falsified, however let’s pretend for a moment one or more of these ideas turns out to break the second law for more than 20 seconds. What happens to shannon’s information?

        As Daniel Sheehan says, the second law may be a general principle rather than an actual law, or a general guideline. You’ve said similar statments in your google Talk, Dave.

        I will email you some more info soon on some potential violations and simpler devices than the brownian ratchet which could be falsified or even verified to work… and then the second law and shannon information become questionable science.

Leave a Reply

Your email address will not be published. Required fields are marked *