Learn › Bash Redirection and File Descriptors › Closing the pipe
Closing the pipe. Try the examples, inspect their output, and explain the result. Practice feedback does not award XP or mastery.
A producer finishes writing, but its consumer still waits. There is no more useful data. Why does the consumer not stop?
End of file, or EOF, is a read result indicating that no more bytes can arrive from the stream in its current lifetime. On a pipe, an empty buffer alone does not establish EOF. Every descriptor referring to the write end must close. A helper that inherited an unused writer can keep a reader waiting.
This chapter requires Linux, Bash, GNU coreutils, and Python 3. Python provides small measuring instruments for kernel behavior. You do not need to write a Python application; follow the opened descriptors and the observations printed by each probe. All examples stay inside the process or the private practice directory.
In your managed Bash practice terminal
Run camp-redirection 10 to enter this chapter's private directory. While this lab machine is running, the same command returns to it after reconnecting; camp-redirection 10 --fresh creates a new one without deleting earlier files. Run the code blocks there and use cd to leave.
The first instrument creates a pipe and duplicates its writer. It closes the original writer, checks whether a read could finish immediately, then closes the duplicate. select checks readiness without consuming data. A zero timeout makes this check immediate. Save the instrument with this complete block:
Before running the saved instrument, predict whether the reader can reach end of file while the duplicate writer remains open. The duplicate makes no write, but its open reference still matters.
cat > writer-lifetime.py <<'PY'
import os, select
r, w = os.pipe()
spare = os.dup(w)
os.close(w)
print("ready with spare:", bool(select.select([r], [], [], 0)[0]))
os.close(spare)
print("after close:", repr(os.read(r, 1)))
os.close(r)
PYTry in the practice terminal: close the last writer reference
python3 writer-lifetime.pyExpected result
ready with spare: False
after close: b''b'' represents an empty byte sequence returned by read: EOF. This is different from a failed read on a descriptor that the reader itself has closed.
| Process moment | Read end | Original write end | Duplicate write end |
|---|---|---|---|
| After creation and duplication | Open | Open | Open, same pipe writer |
| After first close | Open, would wait | Closed | Open |
| After final writer close | Open, reads EOF | Closed | Closed |
A separate process inheriting the duplicate would have the same effect until it closed its copy. A filename is not what keeps the pipe alive; the open references do.
When every read end closes, a later write causes SIGPIPE, a signal normally terminating the writer. If the signal is ignored or handled, the write can instead report the broken-pipe error EPIPE.
head reads one line and exits successfully. In this lab, yes ends with status 141, meaning 128 plus SIGPIPE's signal number 13, after its reader closes.
yes sample | head -n 1; stages=("${PIPESTATUS[@]}"); printf 'producer=%s consumer=%s\n' "${stages[0]}" "${stages[1]}"Expected output
sample
producer=141 consumer=0On this Linux example, Bash reports signal 13 as 128 + 13, or 141. The unlimited producer makes the later write inevitable. A small finite producer might finish before the consumer closes, so its result could differ. Decide whether early termination is intended before treating this status as an incident.
A pipe has finite capacity. A blocking writer waits when it cannot fit more data. The following Linux instrument asks the kernel for this pipe's capacity, fills it once, and checks readiness. It then drains the pipe and checks again.
cat > pipe-capacity.py <<'PY'
import os, fcntl, select
r, w = os.pipe()
# Linux UAPI: F_GETPIPE_SZ = F_LINUX_SPECIFIC_BASE (1024) + 8.
# Some older Python builds omit the name, so retain its Linux command value.
get_pipe_size = getattr(fcntl, "F_GETPIPE_SZ", 1032)
capacity = fcntl.fcntl(w, get_pipe_size)
os.set_blocking(w, False)
written = os.write(w, b"x" * capacity)
assert written == capacity
print("writable when full:", bool(select.select([], [w], [], 0)[1]))
os.read(r, capacity)
print("writable after drain:", bool(select.select([], [w], [], 0)[1]))
os.close(w)
os.close(r)
PYTry in the practice terminal: observe backpressure
python3 pipe-capacity.pyExpected result
writable when full: False
writable after drain: TrueBackpressure is this slowing of a producer when the next stage cannot accept more bytes. It can be healthy flow control. A deadlock occurs when the processes wait in a cycle that none can break. Increasing a buffer is not a general repair for that cycle.
The fallback number 1032 identifies the Linux capacity-query operation, not the pipe's capacity. The kernel still supplies the capacity at runtime. Its value comes from the Linux command definition and Linux command-number base.
The next instrument deliberately waits for data while keeping its own writer open. GNU timeout limits it to one second and adds a one-second forced-termination limit. No background process remains after the command completes.
timeout --kill-after=1s 1s python3 -c 'import os; r,w=os.pipe(); os.read(r,1)'
result=$?
printf 'bounded probe status: %s\n' "$result"The ordinary result is bounded probe status: 124, identifying the expired timeout. A timeout proves that the deadline expired, not which descriptor caused it. This probe's cause is known because its complete ownership is visible in one line.
For an unfamiliar script, record the command, each process's role, the waiting operation, and its open descriptors before terminating the disposable run. Linux /proc/PID/fd and strace -f can help. Match both ends by their pipe identity; seeing only the intended producer has exited is insufficient.
Try this design question before the solution: a helper needs its own stdin to receive a control message, but inherits a data-pipe writer it never uses. Which descriptor must it close, and when? The data reader must see EOF while the helper remains available for the control message.
python3 - <<'PY'
import os, select, subprocess
r, w = os.pipe()
helper = subprocess.Popen(
["python3", "-c",
"import os,sys; os.close(int(sys.argv[1])); sys.stdin.buffer.read(1)",
str(w)],
stdin=subprocess.PIPE, pass_fds=(w,))
os.close(w)
try:
ready = select.select([r], [], [], 2)[0]
assert ready, "a writer is still open"
assert helper.poll() is None, "helper exited before the control message"
print("EOF while helper runs:", os.read(r, 1) == b"")
finally:
helper.communicate(b"x", timeout=2)
os.close(r)
assert helper.returncode == 0, "helper failed"
PYThe worked result is EOF while helper runs: True. The helper closes the unused inherited writer before waiting for control input. Its control pipe remains separate. The parent also closes its writer copy. The two-second checks bound the observation and cleanup without guessing a sleep duration.
Draw the data pipe and the helper's control pipe as separate objects. Mark each owner's close operation. Explain why waiting for the helper before sending its control message would be a second problem. In a bounded disposable variant, retain the unwanted writer and identify the missing close from the failed observation.
Reference: Linux pipe behavior.
Practice Chapter 10. Closing the pipe in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.