Entropy in Thermodynamics with Real-World Applications

There are many interpretations of entropy as used in physics: measure of disorder, how spread out the energy is, measure of unavailability of energy to do work, etc. All of these interpretations put certain properties on the concept of energy and heat transfer.

As most interpretations of entropy rely heavily on thermodynamic processes, this article will first introduce main thermodynamic concepts and then link them to the concept of entropy.

Table of Contents

Intro to Thermodynamics

We know that heat transfer can do work. In car engine, burning fuel transfers heat into the air by conduction and radiation. Hot air trapped inside the internal combustion engine will press the walls harder, causing expansion and movement in parts of the engine. The movement is transferred to the wheels and brings the rest of the car into motion.

The total amount of energy is constant by law of conservation of energy. No matter how many forms the energy takes, same amount of energy exists in the universe at all points in time.

There are certain limits, however, on the conversion of energy when it changes from one form into another. This limit is called the entropy, and it makes certain energy transfer processes unable to go in reverse. When such processes happen, they lead to increased entropy of the universe.

First Law of Thermodynamics

First law of thermodynamics is important as it relates thermodynamic processes, heat engine operations and internal energy of a system. It is stated as follows:

Change in internal energy of a system equals the heat transferred into the system minus the work done by the system.

In equation form, it is the following:

\[ \Delta U = Q_{in} – W \]

Thermal energy is the average kinetic energy of a system’s particles due to their motion. Motion can be translational, rotational or vibrational such as for gases and liquids. Solid particles have rotational and vibrational motions.

Internal energy is the total energy of a system, including the kinetic and potential energies of its particles. For monatomic gases, translational kinetic energy is the only source of thermal energy of a particle (Figure 1). For monatomic gases, thermal energy of a particle equals average KE of a particle equals 3/2kT. Therefore, internal energy is the sum of thermal energies of all its particles: U=3/2NkT.

Figure 1. Monatomic gas and its internal energy as sum of thermal energies of particles.

Change in internal energy U is independent of path. Heat can be transferred in and work done by/on the system in any order, but the total change in internal energy produced will be the same.

Simple Thermodynamic Processes

There are four types of simple thermodynamic processes:

Isobaric	Constant Pressure P
Isochoric	Constant Volume V
Adiabatic	No Heat Transfer Q = 0
Isothermal	Constant Temperature T

A Pressure-Volume (PV) diagram is used to illustrate all 4 types of simple processes.

Figure 2. Isobaric and Isochoric Processes

On Figure 2, isobaric process is when the gas expands (does work) while maintaining a constant pressure. The gas uses its internal energy U to do expansion work:

\[ PV = NkT \] \[ P = Nk\frac{T}{V} \]

As volume increases, temperature drops, which makes the T/V ratio smaller. To maintain constant pressure there must be a steady supply of heat Q into the system. The heat will lead to larger temperature, and the T/V ratio will stay the same.

Isochoric process is when the gas changes pressure and temperature in such a way as to keep volume constant. On Figure 2, pressure drops so there must corresponding drop in temperature to keep volume constant:

\[ V = Nk\frac{T}{P} \]

Figure 3. Isothermal and Adiabatic Processes.

Isothermal process on Figure 3 keeps temperature constant. As gas expands, its temperature drops leading to lower pressure. There must be heat Q coming in to keep temperature constant. There is no change in internal energy in this process. So, U = Q – W =0 or Q = W, and all incoming heat is converted into work.

Adiabatic infers no heat transfer. Change in internal energy delta U = Work. As gas expands, its temperature drops, leading to lower pressure.

It is possible to define any state of fluid in terms of 2 out of three macroscopic properties: Pressure, Volume and Temperature. Hence, any point on the PV plot defines a unique state that fluid can take.

Heat Engines

The heat engine that is studied the most is the idealized Carnot heat engine. Its working is illustrated in Figure 4.

Figure 4. Carnot engine (yellow) operating between two reservoirs of temperatures Th and Tc.

Carnot engine operates between hot reservoir of temperature Th and cold reservoir of temperature Tc. It converts some of the heat from hot reservoir into work. The rest is transferred into cold reservoir.

Carnot engine employs cyclical processes. A cyclical process brings the system back to its original state at the end of every cycle. Or, in terms of the PV plot, the system transitions from certain values of pressure and volume back to the same values (A-B-C-D-A):

Figure 5. PV diagram for a Carnot cycle. A to B is isothermal, B to C is adiabatic, C to D is isothermal and D to A is adiabatic.

Second Law of Thermodynamics

Second law of thermodynamics places some limitations on how the measured heat transfers.

First Form of Second Law of Thermodynamics

The law that forbids processes going in reverse direction is called the second law of thermodynamics. Its expression is the following:

Heat is spontaneously transferred from hot object to cold object and never spontaneously from cold object to hot object.

Figure 6. Depiction of a heat transfer process.

Many of the processes are irreversible. Heat from hot objects will be transferred into cold objects but will never transfer in reverse. Mechanical energy such as kinetic energy can be converted into heat by friction but the reverse – having a hot stationary object cool off and start moving is impossible. Also, a gas that is concentrated in one corner of a container will expand to fill the entire container uniformly but will never regroup into a corner as it was before.

Second Form of Second Law of Thermodynamics

In terms of heat engines, the second law of thermodynamics also states:

It is impossible in any system for heat transfer to completely convert into work in a cyclical process where the system returns back to its initial state.

In a cyclical process the net heat transfer into the system is the work done by the system. W = Qh – Qc. The problem is that in all processes there is some heat transfer to the environment Qc. So not all heat is converted into work. Efficiency of such an engine is:

\[ \textbf{Eff} = \frac{W}{Q_h} = 1 – \frac{Q_c}{Q_h} \]

There is also another important relationship:

\[ \textbf{(1)} \frac{Q_c}{Q_h} = \frac{T_c}{T_h} \]

This can be proved by:

\[ \Delta U = Q – W = 0 \] \[ Q = W \] \[ W = \sum P \Delta V \] \[ \textbf{Since } PV = NkT \] \[ \textbf{Then} (2P)V = Nk(2T) \]

In other words, if we double temperature T, then PV is also doubled. Doubling of PV leads to doubled work W. Since heat Q is equal to work W, Doubled work leads to doubled heat. So the ratio Tf/Ti (final temperature over initial temperature) equals the ratio Qf/Qi. Equation (1) holds!

Carnot engine efficiency is therefore:

\[ \textbf{Eff} = 1 – \frac{T_c}{T_h} \]

So 100% efficiency is only possible when either the temperature of the cold reservoir is absolute zero: Tc =0, Or when the temperature of the hot reservoir is infinetly large Th -> Inf.

Carnot cycle is a theoretical idealized cycle with maximum efficiency but impractical for real world steam engines due to operational limitations. In practice, the processes in a Carnot cycle are not reversible. They involve dissipative factors, such as friction and turbulence. This increases heat transfer Qc to the environment and reduces work and the efficiency of the engine.

Third Form of Second Law of Thermodynamics

Carnot engine introduces the third expression of the second law of thermodynamics:

Carnot engine is the most efficient engine operating between two temperatures among other engines operating between the same two temperatures. All engines employing only reversible processes and operating between the same two temperatures have maximum efficiency equal to that of a Carnot engine.

Real heat engines are less efficient than Carnot engine since in real engines irreversible processes reduce heat transfer to work (see Figure 7).

Figure 7. In real engines, there will be additional heat generated by friction (Wf) between the piston and the walls of the container, resulting in less work output W

Entropy Definition#1 of Second Law of Thermodynamics

The total entropy of a system either increases or remains constant in any process; it never decreases.

Here are three facts of entropy related to this law:

Entropy is the measure of how spread out the energy is.
When one part of the system decreases in entropy, there must be another part of the system that increases in entropy. Total entropy of the universe either stays constant or increases.
Entropy increases when energy is transferred as heat in an irreversible process. For example, if heat flows spontaneously from a hot object to a cold object, the entropy of the universe increases.

Change in entropy delta s is defined as heat transfer divided by temperature:

\[ \Delta S = (\frac{Q}{T})_{rev} \]

Where Q is the heat transfer, which is positive for heat transferred in and negative for heat transferred out. T is the absolute temperature at which the reversible process takes place.

To find change in entropy of a system of monatomic gas (1 mole of helium, 4 grams per mole) going from temperature Ti to temperature Tf:

\[ \Delta U = \frac{3}{2} N_A k (T_f – T_i) \] \[ N_A = 6.022 * 10^{23} \] \[ k = 1.38 * 10^{-23} \]

Delta U is the amount of heat Q necessary to increase the helium temperature from Ti to Tf.

\[ \Delta S = \int_{T_i}^{T_f} ds = \int_{T_i}^{T_f}\frac{dQ}{T} = \int_{T_i}^{T_f} \frac{1}{T} \frac{3}{2} N_A\, k \, dT \] \[ \Delta S = \frac{3}{2} N_A k \ln(\frac{T_f}{T_i}) \]

When Tf is greater than Ti, change in entropy is positive, meaning heat is gained. When final temperature Tf is less than Ti, change in entropy is negative, meaning heat is lost. To see why heat transfer from hot object to cold object increases entropy of the system (Universe), check this:

Some quantity of hot helium decreases its temperature from 200 K to 150 K while heating up same quantity of helium increasing its temperature from 100 K to 150 K:

\[ \Delta S_{net} = \frac{3}{2} N_A k \ln(\frac{150}{200}) + \frac{3}{2} N_A k \ln(\frac{150}{100}) = \frac{3}{2} N_A k \ln(\frac{150}{200} \frac{150}{100}) = 1.468 \frac{J}{K} \]

Indeed, the entropy of the system increased.

These facts explain why a hot teacup will transfer heat to its colder surroundings and never from its surroundings back to the cup. Total entropy increases because the energy gets spread out. The process is irreversible and hence energy cannot go back in direction.

Also note that there is no work done by heat transfer from one object to another.

Entropy Definition#1 of Second Law of Thermodynamics (Continued)

When heat is transformed into work there must be an increase in entropy of the system, in other words energy gets spread out when work is done.

This is exactly what happens to gas in constricted volume for an isothermal process.

Figure 8. Heat transfer occurs from hot object (red) into chamber with gas. Gas expands doing work while maintaining constant temperature.

Imagine that gas in a chamber receives heat from a hot object. Hot object is placed near gas chamber in such a way that the temperature of the chamber T2 remains constant. Note that heat transfers out of the hot object into the chamber as T1 > T2. Some rocks are removed as the gas chamber expands to establish a gas-rock equilibrium. Pressure from rocks due to gravity remains equal to pressure from gas.

The change in entropy for a hot object is:

\[ \Delta s_1 = \frac{Q_{out}}{\textbf{avg}{(T_{1})}} \]

Note that Qout is negative and avg(T1) is the average temperature of hot object as it transfers heat out.

The change in entropy for gas in chamber:

\[ \Delta s_2 = \frac{Q_{in}}{T_2} \]

Note that Qin is positive. Gas expands by doing work keeping temperature constant (isothermal process). This means all the heat it receives from hot object is converted into work. The total kinetic energy remains constant but gets spread out in the chamber.

Since Qout = -Qin and T2 < avg(T1) we have that:

\[ \Delta S_{net} = \Delta s_1 + \Delta s_2 = Q_{in} (\frac{1}{T_2} – \frac{1}{\textbf{avg}(T_1)}) \] \[ \Delta S_{net} > 0 \textbf{ since } T_2 < \textbf{avg}(T_1) \]

The net entropy of the system increases which is what we would expect.

Entropy Definition#2 of Second Law of Thermodynamics

Entropy in terms of number of states (or number of possible configurations):

\[ S = k \ln \Omega \]

Fact about entropy related to the fifth form:

Entropy of the system increases as more states become available for the particles to take on (more heat means particles can take on more states)

Suppose there is a certain amount of gas trapped in a closed container (box). The closed container is adjacent to another closed container of the same volume V. The wall separates the two containers.

Figure 9. Gas is limited to container of volume V.

The number of all possible microstates of gas atoms in the closed container is the following:

X microstates per gas atom. N atoms overall. Total number of states S (configurations) of atoms:

\[ X*X*…*X = X^N \]

Taking the natural logarithm of the combinations and multiplying by arbitrary constant k allow a scale-friendly treatment of the quantity:

\[ S_i = k \ln X^N \]

Now suppose the wall is removed and gas atoms are free to occupy all available space.

Figure 10. Gas expands as wall is removed, to occupy all available space.

The number of possible microstates per gas atom is doubled:

2*X microstates per gas atom. N atoms overall. Total number of states/configurations of atoms:

\[ S_f = k \ln (2X*2X*…*2X) = k \ln(2X)^N \]

The increase in the number of states/configurations of a system going from state 1 to state 2 (from closed container to a container double size) is following:

\[ \textbf{(1)} S_f – S_i = k \ln(2X) ^ N – k \ln(X^N) = k \ln(2^N) = Nk\ln2 \]

Gas molecules get spread over a larger volume and go from deeply concentrated to evenly spread out.

Now let us see how entropy of a system changes when gas is brought to occupy chamber of twice the initial volume by thermodynamic processes:

Figure 11. Equivalent thermodynamic process for gas to transition from volume V to volume 2V.

Since the path from initial state to final state is an isotherm, the following equation holds:

\[ \Delta U = Q_{in} – W = 0 \]

Work done by system equals heat transferred to system. We can find work by integrating:

\[ \textbf{Work } = \int_{V_i}^{V_f} P dV \]

Because we know the ideal gas law PV = NkT, we substitute pressure as a function of volume:

\[ PV = NkT \] \[ P = \frac{NkT}{V} \] \[ \int_{V_i}^{V_f} P dV = \int_{V_i}^{V_f} \frac{NkT}{V} dV = NkT \ln(\frac{V_f}{V_i}) = NkT \ln(\frac{2V_i}{V_i}) = NkT \ln(2) \]

Finally, we express heat and temperature with following:

\[ W = Q = NkT \ln2 \] \[ \frac{Q}{T} = \Delta S = Nk\ln2 \]

Recall from equation (1):

\[ S_f – S_i = k \ln(2X) ^ N – k \ln(X^N) = k \ln(2^N) = Nk\ln2 \]

Note that:

\[ S_f – S_i = Nk \ln 2 = \Delta S \]

So, both entropy in terms of number of states and entropy in terms of heat transferred to/from object of temperature T are equivalent.

View the following Khan Academy video to learn more: Reconciling thermodynamic and state definitions of entropy

Other Facts About Entropy

Fact #1 – Entropy and Reversible Processes

A purely reversible process (even if work is done) does not increase the entropy of the universe as energy can go in both directions. Entropy increases only when there is a one-way process – energy is used or wasted and becomes unavailable.

In the reversible process of Carnot engine, the full cycle from state A through B, C, D and back into state A have the following changes in entropy:

\[ \Delta S = \frac{-Q_h}{T_h} + \frac{Q_c}{T_c} \]

Substituting the following relationship:

\[ \frac{T_c}{T_h} = \frac{Q_c}{Q_h} \]

Yields

\[ \Delta S = 0 \]

Thus, entropy of the universe stays the same in the reversible process of a Carnot engine.

Fact #2 – Entropy and Unavailability of Energy to Do Work

Also, a measure of unavailability of energy to do work. Increased entropy – less energy is available for work.

As snow is mixed with water, the two bodies will eventually reach thermal equilibrium. When they reach same temperature, you cannot run a heat engine with them.

Fact #3 – Entropy and Measure of Disorder

Entropy is the measure of disorder related to heat transfer, work alone does not inherently increase entropy

For example, melting a block of ice means taking a highly structured and orderly system of water molecules and converting it into a disorderly liquid in which molecules have no fixed position. Thus, the entropy of the system of ice increases.

To give another example, imagine a bicycle wheel driven by pedaling. Pedaling brings the wheel into rotation while static friction converts rotational kinetic energy of the wheel into translational kinetic energy of the bike. Neither pedaling nor static friction do not produce heat, but both do some work. Hence, work does not inherently increase entropy.

Big Picture – Entropy in Universe and Sun-Earth System

Entropy increases in a closed system such as the Universe. However, in isolated systems, such as the Sun and planet Earth, the variation in temperatures led to more energy being available to do work.

Sun is a source of low entropy energy. Energy is low entropy since it is very concentrated in one place – sun. Earth transforms incoming heat from the sun. There are three places this transformed energy can go: heat transfer, work or stored potential energy. E.g. plants take one form of energy – light and convert it into chemical potential energy. This irreversible thermodynamic process is called photosynthesis. Humans take stored energy in the form of food and convert it into heat, work or store it as fat.

The world’s complexity we see today, such as buildings, animals, plants are all results of atoms getting bound together into molecules and further into complex structures. These structures can be thought of as stored potential energy. Everything else is just cosmic dust (unbound atoms and particles) where principles of thermodynamics apply.

You can view the Earth-Sun system as a heat engine operating between hot reservoir – Sun, and cold reservoir – outer space. Earth is the engine that runs in between.

The Sun’s entropy is decreasing since it is giving away heat. Earth’s entropy increases since it receives heat. Some heat is reflected back from Earth into outer space. The rest is converted into kinetic energy of wind and water, potential energy of structured living systems and stored energy in various forms.

The Universe is destined to end at highest entropy. This happens when there is no variation in temperature and everything is spread equally among available space. However, it is a long process, estimated to take 10^100 years.

Conclusion

Study of thermodynamics is crucial for understanding how heat relates to internal energy, how heat engines work and finally, how entropy operates.

The most common principle that relates to entropy is the number of states that a bunch of atoms can be in. The more spread atoms become the larger the entropy. The more concentrated the atoms are (proportionate to available space) – the less the entropy. Greater variation in temperatures of atoms lead to less entropy. Less variation in temperature – more entropy.

Entropy can only increase or stay the same in any process. Thus, entropy is also the amount of energy unavailable to do work. As entropy increases, the amount of resource in the Universe depletes.