2.3. Thermodynamics and statistical mechanics; entropy and the direction of time
Thermodynamics is the physics of heat flow and of the interconversion between heat energy and other forms of energy. Statistical mechanics is the theory that describes macroscopic properties such as pressure, volume and temperature of a system in terms of the average properties of its microscopic constituents, the atoms and molecules. Thermodynamics and statistical mechanics are both concerned with predicting the same properties and describing the same processes, thermodynamics from a macroscopic point of view, and statistical mechanics from a microscopic point of view.
In 1850, the German physicist Rudolf Clausius (1822 - 1888) proposed the first law of thermodynamics, which states that energy may be converted from one form to another, such as heat energy into the mechanical rotation of a turbine, but it is always conserved. Since 1905 when German-Swiss-American physicist Albert Einstein (1879 - 1955) invented the special theory of relativity, we know that energy and matter can be converted into each other. Hence, the first law actually applies jointly to both matter and energy. This law is probably the most fundamental one in nature. It applies to all systems, no matter how small or large, simple or complex, whether living or inanimate. We do not think it is ever violated anywhere in the universe. No new physical theory is ever proposed without checking to see whether it upholds this law.
Question: What are some everyday examples of the first law of thermodynamics?
The second law of thermodynamics can be stated in several ways. The first statement of it, made by Rudolf Clausius in 1850, is that heat can flow spontaneously from a hot to a cold object but it cannot spontaneously pass from a cold to a hot object. The second statement of the second law was made later by Scottish physicist William Thomson Kelvin (1824 - 1907) and German physicist Max Planck (1858 - 1947): Heat energy cannot be completely transformed into mechanical energy, but mechanical energy can be completely transformed into heat energy.
Question: What are some everyday examples of the second law of thermodynamics?
The third statement of the second law depends on a new concept, that of entropy. In order to discuss entropy, we first discuss the number of distinguishable arrangements of a system, from which the concept of entropy is derived. Some readers may wish to skip this discussion and go directly to the definition of entropy that is given afterwards.
Example #1: Imagine a box divided into two compartments, each of which can hold only one ball. Put a ball into one of the compartments. Clearly, the number of ways that you can do this is two--the ball can be put into either compartment. (Mathematically, this is the number of combinations of two objects taken one at a time; this is given by the binomial coefficient).
Example #2: If there are three compartments, the number of ways you can put a ball in is three (the number of combinations of three objects taken one at a time).
Example #3: If there are four compartments, the number of ways you can put a ball in is four (the number of combinations of four objects taken one at a time).
Example #4: Now put two identical balls into a box with two compartments. The number of ways you can do this is only one (the number of combinations of two objects taken two at a time) because if the balls are interchanged, there is no distinguishable difference in the arrangements.
Example #5: Now put two identical balls into a box with three compartments. The number of ways you can do this can be counted in the following way:
a) The first ball in compartment #1, the second in either of the other two. This adds up to two.
b) The first ball in compartment #2, the second in either #1 or #3. But the first arrangement is identical to the first arrangement of a), so we don't count it. The second arrangement is new, so we count it. If now the first ball is now put into #3, the second can be put into either #1 or #2, but these are not new, so we don't count them.
Thus, the total number of distinguishable arrangements for two identical balls in three compartments is three (the number of combinations of three objects taken two at a time).
Example #6: Now put two identical balls into a box with four compartments. We count the number of possible ways as follows:
a) The first ball in #1, the second in #2, #3, or #4. This adds up to three.
b) The first ball in #2, the second in #1, #3, or #4. The first is the same as the first arrangement of a), so there are two new distinguishable arrangements.
c) The first ball in #3, the second in #1, #2, or #4. Only the last arrangement is new, so there is one additional distinguishable arrangement.
d) The first ball in #4, the second in #1, #2, or #3. Each of these arrangements is identical to a), b), or c), so these are not new.
Thus, the total number of distinguishable arrangements for two identical balls in four compartments is six (the number of combinations of four objects taken two at a time).
Example #7: Now put two balls into only the first three of four compartments. This is identical to Example #5 except that now there are two balls in four compartments instead of two balls in three compartments. The number of distinguishable arrangements is now three as long as we know that the balls are in the first three compartments. This example shows that the number of distinguishable arrangements depends not only on the number of balls and compartments, but also on how the balls are distributed in the compartments.
The methods of probability allow us to calculate the number of distinguishable arrangements in any number of compartments whether the balls are identical or not, and for any given distribution of balls. For a given number of compartments and for identical balls, the number of distinguishable arrangements is smallest (equal to one) when the number of balls is the same as the number of compartments (example #4). This would correspond to a pure crystalline solid material. For a given number of compartments and identical balls, the number of distinguishable arrangements is maximum when the number of balls is equal to half the number of compartments (example #6). This would correspond to a highly compressed gas. For a rarefied gas, the number of compartments (each equal to the size of a molecule) is vastly larger than the number of molecules, and the number of distinguishable arrangements is much greater than one (example #3) but less than the maximum (example #6).
We are now able to define entropy. Entropy is related to (actually, is proportional to the logarithm of) the total number of distinguishable possible arrangements of the system (in a six-dimensional position-velocity space--three dimensions for position and three for velocity--rather than in the three-dimensional position space of the example above). Entropy quickly increases as we increase the volume of the system, the number of objects in it, and the total energy of the objects. For a macroscopic system, say of 1023 particles, the entropy is enormously larger than for the system of two balls described above. Entropy also is larger when the objects are uniformly distributed (example #6) than when they are clumped together (example #7). It turns out that it is also larger when energy as well as mass is distributed uniformly. Since energy is related to temperature, entropy is larger when the temperature is uniform, and it increases when the temperature increases.
Entropy is related to the amount of disorder and order in the system. Decreasing entropy is equivalent to decreasing disorder or disorganization (increasing order or organization) of an object or system (example #7); while increasing entropy is equivalent to increasing disorder or disorganization (decreasing order or organization) (example #6).
It turns out that the second law of thermodynamics can be stated in the following way: Natural processes of an isolated macroscopic system normally proceed in the direction of maximum probability (maximum disorder), which is the direction of maximum number of distinguishable arrangements of the system. (It is highly improbable, although not totally impossible, for them to proceed in the opposite direction.) The forward direction of time is the direction in which entropy increases. Thus, the second law of thermodynamics can be restated in terms of entropy: Natural processes of an isolated macroscopic system always proceed in the direction of increasing entropy (disorder). In classical physics, this defines the forward direction of time. In Section 6.4, we shall see what determines this direction in quantum physics. (Note that we have emphasized that the second law applies only to a system that is isolated from the rest of the universe, or to the universe as a whole.)
The second law is based on our observations of the direction in time of natural processes. We say that this direction depends on the universe proceeding in time from a state of lower entropy to a state of higher entropy but, in classical physics, the increasing direction of entropy is identical to the forward direction of time. We cannot distinguish one from the other. It is tautological to say that entropy increases with time because the increase of entropy cannot be distinguished from the forward direction of time. However, in quantum physics, the direction of time is determined by the nature of a quantum event (see Chapter 6).
[Note: The fact that the forward direction of time is the direction of increasing entropy implies that the universe began in a much lower entropy state than that of today. What caused the entropy to be so low when the universe began is not known. This is a subject of intensive research in theoretical cosmology.]
The direction of time can also be inferred from the first two statements of the second law of thermodynamics: 1) The unidirectional flow of heat from hot to cold bodies, and 2) the possibility of total conversion of mechanical energy to heat energy, but not the reverse.
A mistake made by some people is to think that the second law applies to individual objects or systems, such as automobiles, plants, or human bodies, even if they are not isolated from the rest of the universe, and that this is the reason that such objects decay and disintegrate with time. This is a fallacy, however, because the second law does not prevent the entropy of an individual object from continuously decreasing with time and thus becoming more ordered and organized as long as it receives energy from something else in the universe whose entropy continues to increase. In our solar system, it is primarily the sun’s entropy that continually increases as its fuel is burned and it becomes more disordered.
The above discussion is based on our understanding of objective time within classical physics. It says nothing about our subjective sense of the passage of time. Regardless of what the clock says, everybody experiences time as passing faster in some circumstances than in others. Our experience of duration depends on our perception of the degree of change and on our accompanying emotional state ("Time flies faster when you're having fun"; "How time flies!").
Question: Do you think that the entropy of your body is decreasing as you get older, or is it increasing?
Economist Jing Chen has written that natural selection in biological evolution can be seen as a force towards low entropy states of the human mind at the cost of increasing entropy of the universe (see The Entropy Theory of Human Mind at http://web.unbc.ca/~chenj/1.pdf):
People generally think that physical laws only have limited utility in understanding human behavior because our mind is free. However, human mind is shaped by natural selection and sexual selection. Living organisms need to extract low entropy from the environment, to defend their low entropy sources and to reduce the diffusion of the low entropy. The struggle to stay in low entropy states is called natural selection. In human societies, agriculture is the main low entropy source. Part of health care systems aim at defending our own low entropy sources to be accessed by viruses and bacteria. The military forces are established to extract low entropy from others and to defend own low entropy sources. Clothing and housing reduces the diffusion of low entropy.
Sexual selection is the struggle between the individuals of one sex, generally the males, to communicate their attractiveness to the other sex in order to form a partnership for reproduction. Human beings, as well as other sexually reproducing species, are the successful descendants of the earliest sexually reproducing species about a billion years ago. For the system of communication to be successful in different kinds of environments over such a long time, the mode of communication has to be simple, stable and universal. Since the entropy law, which states that closed systems tend towards states of higher entropy, is the most universal law of the nature, it is natural that the display of low entropy levels evolves as the universal signal of attractiveness in the process of sexual selection.
As both natural selection and sexual selection favor low entropy state, the pursuit of low entropy becomes the main motive of human mind and animal mind. Indeed the low entropy state is the main way of advertisement for most sexually reproducing species. Large body size, colorful and highly complex feather patterns with large amount of information content and exotic structures are all different representations of low entropy states. Since a low probability event corresponds to a state of low entropy, a novel feature is often attractive in the competition for reproduction. It has been generally recognized that sexual selection is the main drive of diversity.
Besides communication with members of the opposite sex, social animals need to communicate their attractiveness and power in order to influence the behavior of others. For the same reason as in sexual selection, the most general signal is display of low entropy. Among all social species, human beings have developed the most complex social structure. The creation of distinct art works, the demonstration of athletic prowess, the accumulation of wealth, and conspicuous consumption - all of which represent different forms of low entropy - are the major methods of advertising one’s attractiveness.
As the social groups become larger and the division of labor becomes finer, people become less familiar with each other in their daily interactions, which make it more difficult for people to judge the ability of others. The need for people to advertise their attractiveness through external accumulation of low entropy also becomes stronger. People usually signal their capability by buying more expensive houses, cars, clothes, going to more expensive restaurants and attending more exclusive schools. The great efforts human beings put into non-food activities reflect the high cost of communication in a large and complex society. Historical evidences show that the transaction costs have been increasing over time.
The "transaction costs" that Chen speaks about mean that the struggle to achieve and sustain localized low entropy states leads to higher entropy in the universe as a whole.
Question: In your own life, how do you experience the struggle to achieve and sustain low entropy states? What are the transaction costs of this struggle, i.e., in what way does your struggle increase the overall entropy of the world? How successful have you been in your efforts to achieve and maintain low entropy states? How sustainable are such states?
An extremely important property of Newton’s laws is that they are time reversal invariant. What this obscure-sounding term means is that, if the direction of time is reversed, the directions of motion of all particles are also reversed, and this reversed motion is completely allowed by Newton’s laws. In other words, the motion in reversed time is just as valid as the motion in forward time, and nature herself does not distinguish between the two. A simple example of this is the time-reversed motion of a thrown baseball, which follows a parabolic trajectory in either the forward or the reversed direction. Without seeing the act of throwing, and without air resistance, we would not be able to distinguish the forward parabola from the reversed parabola. Another way to state it is that a movie of a thrown baseball seems just as valid to us if it is run in the reverse direction as in the forward direction. Time reversal invariance is also apparent in the seemingly random motion of the molecules in a gas. If we could see their motion in a movie and then reverse it, we could not distinguish between the forward motion and the reversed motion (see http://www.colorado.edu/physics/phet/web-pages/simulations-base.html Physics→Heat and Thermo→Gas Properties).
However, if we consider the motion of an object containing many ordered particles (for example, with a recognizable size, shape, position, velocity, and orientation), we encounter a different phenomenon. It is easy to tell the difference between the reversed and forward motions of a person, a horse, a growing plant, a cup falling from a table and breaking, and most other examples from everyday life. Another example is the free expansion of a gas that initially is confined to one side of a box by a membrane. If the membrane is broken, the gas immediately expands into the other side (initially assumed to be evacuated), and we can easily tell the time reversed motion from the forward motion. In all of these cases, the motion at the individual molecule level is time reversal invariant, but it is clear that the gross motion of the macroscopic object is not.
Our question now is, "Why does nature seem to be time reversal invariant at the individual, or few, particle level, but apparently not at the level of many particles contained in an ordered system such as any common macroscopic object?" In classical physics, irreversibility is always due to the second law of thermodynamics, which determines the forward direction of time. The entropy of a system of moving horse or person, gravitational force, earth, and surroundings increases with time because the motion dissipates energy and increases the disorder in the body, earth, and surroundings. The forward direction of a falling cup is apparent after the cup hits the floor and breaks because the broken cup is more disordered (has higher entropy) than the unbroken cup. However, even before the cup breaks, the entropy of the combined system of cup, gravitational force, and earth increases as the cup falls.
Question: How is it possible that the entropy of a system of falling cup, gravitational force, and earth increases even before the cup breaks? Assume that the earth is a collection of a large number of particles that are in thermal equilibrium at all times, i.e., that the energy of tidal motion is constantly being converted into heat energy.
French physicist Charles Augustin de Coulomb (1736 - 1806) discovered the force law obeyed by stationary, electrically charged objects between 1785 and 1791. In 1820, Danish physicist Hans Christian Oersted (1777 - 1851) discovered that an electric current produces a magnetic field, and that a magnetic field exerted a force on a current-carrying wire. From 1820 to 1827, French physicist Andre Ampere (1775 - 1836) extended these discoveries and developed the mathematical relationship describing the strength of the magnetic field as a function of current. In 1831, English chemist and physicist Michael Faraday (1791 - 1867) discovered that a changing magnetic field, which he explained in terms of changing magnetic lines of force, produces an electric current in a wire. This was a giant step forward because it was the forerunner of the concept of force fields, which are used to explain all forces in nature today. A computer simulation of the force on an electric charge is given at http://ephysics.physics.ucla.edu/physlets/1.1/e_electric_field.htm.
In 1873, these disparate phenomena and theories were all pulled together into one elegant theory by Scottish physicist James Clerk Maxwell (1831 - 1879). Maxwell’s four equations describing the electromagnetic field are recognized as one of the greatest achievements of 19th century physics. Maxwell was able to calculate the speed of propagation of the electromagnetic field from his equations, and found it to be approximately equal to the speed of light. He then proposed that light is an electromagnetic phenomenon. Because electromagnetic fields can oscillate at any frequency, he concluded that visible light occupied only a very small portion of the frequency spectrum of electromagnetic radiation. The entire spectrum includes radio waves of low-frequency, high-frequency, very-high frequency, ultra-high frequency, and microwaves. At still higher frequencies are infrared radiation, visible light, ultraviolet radiation, x-rays, and gamma rays. All of these are fundamentally the same kind of waves, the only difference between them being the frequency of the radiation (see next section).
Now we ask, what is the electromagnetic field, anyway? Is it a physical object? To answer that question, we must understand what we mean by the term physical object. One definition is that it is anything that carries force, energy, and momentum. By this definition the electromagnetic field is a physical object because it carries force, energy, and momentum. However, this merely defines the electromagnetic field in terms of other things that require their own definitions. Force, energy, and momentum can only be defined in terms of the operations necessary to measure them and these operations require physical objects on which to make the measurements. Thus, all physical objects are defined in terms of other physical objects, so the definition is circular. This is another indication that the concept of objective reality is nothing but a concept (see Section 1.1).