
Fathom23 min read
Beyond the magic terminal box
Linux · Terminal · Kernel · Unix · Systems · Shell · Processes
The terminal window is not one process. Six layers from key to kernel, copies of environment, pipes, signals, and who actually holds the TTY—plus commands you can run to prove it.
This lecture assumes you can open a terminal, type a command, and press Enter. It does not assume you already know what a process, a file descriptor, or the kernel is. New words get a definition in the same paragraph.
Words we will use
| Word | Meaning in this lecture |
|---|---|
| Process | One running program. The operating system gives it a number called a PID (process ID). Your window is not automatically “one process.” |
| Kernel | The core of Linux that talks to hardware and creates/kills processes. You do not run it like an app; everything else asks it to do work via system calls (read, write, fork, …). |
| Byte | The smallest unit of data we care about here. The letter p is the byte 0x70. A “character you typed” becomes bytes before any program sees it. |
| File descriptor (fd) | A small integer a process uses as a handle to something it can read or write. By convention: 0 = stdin (input), 1 = stdout (normal output), 2 = stderr (error output). |
| Parent / child | When process A asks the kernel to start process B, A is the parent and B is the child. The child can outlive the thought “I clicked the icon,” but it usually dies if the parent’s session is torn down—details later. |
| Shell | The program that prints a prompt ($ or %) and waits for a line of text. Common names: bash, zsh, fish. It is a process, not the window. |
| Terminal emulator | The window program (Alacritty, GNOME Terminal, Kitty, …). It draws letters on screen and owns the “fake keyboard cable” described below. |
| Signal | A small numbered message the kernel can deliver to a process: “stop,” “hang up,” “continue.” It is not a character in your program’s input—unless the program asked to treat it as one. |
| Job / process group | One or more processes the shell treats as a unit (a pipeline is usually one job). The kernel gives the group a PGID. Foreground vs background is about which group holds the TTY, not about whether the CPU is busy. |
One-sentence crystal
The “terminal” is several programs plus a slice of the kernel: the window draws pixels and owns one end of a fake serial cable (a PTY); the kernel may edit or echo your typing; the shell is a child reading the other end; variables and external commands live in copies of memory, not in one box.
Level 1 — Why “just type” is not enough
A beginner points at the rectangle and says: “That is one program.” Then ordinary failures have no names.
| You only type… | What happens | Without a map | With a map |
|---|---|---|---|
VAR=1 (no export) | Python cannot see VAR | “Linux is random” | Child never inherited it |
vim, then kill it badly | Keys do not echo | Reinstall the OS | Line discipline still raw; reset / stty sane |
long-job & then close the window | Job dies | “Background should live” | SIGHUP; use tmux / nohup |
cmd1 | cmd2 | Hang or empty output | “Pipes are buggy” | Two processes, one kernel buffer |
Ctrl+C | Wrong thing dies, or nothing | “Ctrl+C is broken” | Signal hits the foreground process group |
| Blog command works here, fails in CI | CI has no TTY | “CI is stupid” | isatty / $TERM differ |
ssh then messy vim | Wrong colors | “Server is broken” | $TERM is a dictionary the host lacks |
pwd but no /usr/bin/pwd in ps | Confusion | “ps is lying” | Often a builtin: the shell answered |
You do not need ioctl trivia to use a computer. You need to name the actor when the happy path ends: window, PTY, kernel manners, shell, child, environment copy. Day to day, still just type. The map is for the first unexplained failure—so you do not reboot because echo died (stty had four flags wrong).
The window is a frame. The shell and your commands are the specimen. The frame matters; it is not the cell.
Level 2 — The big picture (architecture)
2.1 Three things people mash into one word
Say these out loud until they feel different:
- Terminal emulator = the window app. It talks to the graphical system (Wayland or X11 on Linux), draws glyphs, and holds the master end of a fake cable.
- PTY (pseudo-terminal) = a pair of devices inside the kernel that pretend to be an old serial terminal (a teletype, historically TTY). One end is master, one end is slave. Bytes written on one end can be read on the other.
- Shell = the interpreter process (
bash/zsh/ …) attached to the slave. It is a customer of the PTY, not the PTY itself.
/dev/tty means: “whatever terminal this process is attached to.” It is an alias, like “my house” rather than a street number.
The Linux console (Ctrl+Alt+F3, device /dev/tty3, …) is a different path: real text console, often no GUI emulator. Same kind of line discipline; different glass.
2.2 Six layers, from your finger to the kernel
Six layers ≠ six processes. Some are parts of one program. Some are kernel code you never “start.”
Numbered in the order a keystroke travels:
| # | Nickname | What it is | Everyday job |
|---|---|---|---|
| 1 | Glass | Display / window | OS delivers key events; later, glyphs get painted |
| 2 | Paint | Emulator’s VT / ANSI engine | Cursor, color, scrollback grid |
| 3 | Master | PTY master fd | Speaking tube the window process reads and writes |
| 4 | Manners | Line discipline n_tty + PTY slave /dev/pts/N | Hold a line, echo, turn Ctrl+C into a signal. Kernel. |
| 5 | Shell | bash / zsh / … | Reads the line; builtin or spawn a child |
| 6 | Core | Kernel VFS / scheduler | getcwd, read, write, CPU time |
You type
→ 1 Glass window
→ 2 Paint emulator logic
→ 3 Master PTY master (still the window process)
║ kernel
→ 4 Manners n_tty + slave
║ userspace
→ 5 Shell read(0)
→ 6 Core if the command needs a syscall (almost always)
The shell is not “below” the kernel in importance. It is a userspace process whose stdin/stdout are the slave. Manners (4) sit on the wire.
2.3 The window is a tree of processes
Claim: one open terminal window is at least two processes, often three.
Process A Terminal emulator “the window.” Lives until you close it.
│
│ The emulator asks the kernel: copy me, then become a shell.
│ (The usual pair of requests is called fork, then execve.
│ We will say what those words mean in Level 3.)
▼
Process B Interactive shell Prints the prompt. Lives until the session ends.
│
│ Only if you run an *external* program (a file such as /usr/bin/ls)
▼
Process C That program Lives only until that program exits.
| Who | Example name you might see in ps | Role | How long it lives |
|---|---|---|---|
| Emulator | alacritty, gnome-terminal-server | Parent; draws the window; holds PTY master | Until the window closes |
| Shell | bash, zsh | Child of emulator; holds PTY slave as stdin/out/err | Until you exit the shell or the session dies |
| External command | /usr/bin/ls, /usr/bin/pwd | Child of the shell | Until that command finishes |
Builtin versus external (say this until it is boring)
- A builtin is a command the shell implements itself. Examples in bash/zsh:
cd,export,jobs, and usuallypwd. No Process C. The work happens inside Process B. - An external command is a file the kernel can run, found via
PATH(a list of directories). Examples:/usr/bin/ls,/usr/bin/grep,/usr/bin/python3. The shell creates a child and replaces that child’s program image with the file, then waits.
type pwd tells you which story your shell uses.
2.4 Variables: two boxes, only one is inherited
A variable is a named string in memory (NAME=value). It is not “a setting of the window.” It belongs to a process.
There are two boxes inside the shell:
| Shell variable | Environment variable | |
|---|---|---|
| Where it lives | Only in this shell’s own table | In the list the kernel will copy to a new program |
| Does a child program see it? | No | Yes (it gets a copy) |
| How you create it | VAR=hello | export VAR or export VAR=hello |
| How you look at many of them | set (prints a lot) | env or printenv |
| Typical use | A loop counter; a temporary flag | PATH, language/locale, API keys |
The one-way rule (memorize this):
- Parent → child: the child receives a copy of the exported list when it starts (
execve). - Child → parent: if the child changes a variable, the parent does not change. There is no “write back.”
That is why “it works when I type it” and “it fails in a script / cron / systemd” so often: those other worlds never saw your unexported GUI-shell variables.
Level 3 — The story of one command
3.1 You type pwd and press Enter
We assume:
- a normal prompt (the line discipline is in canonical / “cooked” mode: wait for Enter);
pwdis a builtin;- therefore only two processes exist: emulator + shell.
Phase A — Down: key → bytes → shell wakes up
- 1 Glass. You press keys. The window system turns that into events (“key p down”) and gives them to the emulator process.
- 2 Paint. The emulator maps them to
p,w,d, then Enter. Enter is often CR (0x0D). Payload:0x70 0x77 0x64 0x0D. - 3 Master. The emulator writes those bytes into its PTY master fd.
- 4 Manners (
n_tty). Echo: copies go back to the master so you see letters before the shell has the whole line. Buffer: the line sits in kernel memory; the shell is still inread(). Enter: CR usually becomes NL (0x0A); the kernel wakes the shell. - 5 Shell.
read(0, …)returnspwd\n. stdin is the slave. The shell parses a builtin and does not create Process C. - 6 Core. The builtin asks
getcwd(or uses its ownPWD—shells differ). The kernel returns a path such as/home/you.
Phase B — Up: path string → pixels
- 5 Shell. Writes
/home/you\nto stdout (fd 1), then a new prompt. fd 1 is the same slave. - 4 Manners. May turn NL into CR+NL (old printers). Bytes move toward the master.
- 3 Master. The emulator reads the master.
- 2 Paint. Updates the grid of cells.
- 1 Glass. Fonts become pixels. You see the path.
If you run /usr/bin/pwd instead
After step 5 the shell says: this is a file, not a builtin.
fork— kernel clones the shell into a child (Process C). Clone means: same open files, same variables for a moment.execve("/usr/bin/pwd", …)— the child throws away the shell program and becomespwd. The environment copy is installed here.waitpid— the original shell sleeps until Process C exits, then prints the next prompt.
While /usr/bin/pwd runs, ps can show three PIDs. When it exits, you are back to two.
3.2 Connecting commands (|, ;, &&, ||)
| What you type | How many programs start (typical) | Same time or one after another? | When does the second run? |
|---|---|---|---|
cmd1 | cmd2 | Two children (plus the shell) | At the same time | Always; they share a pipe |
cmd1 ; cmd2 | One, then the other | One after another | Always, even if the first failed |
cmd1 && cmd2 | One, then maybe the other | One after another | Only if the first succeeded (exit code 0) |
cmd1 || cmd2 | One, then maybe the other | One after another | Only if the first failed (exit code not 0) |
Exit code: every process ends with a small integer. 0 means success by convention. The shell stores the last one in $?.
A pipe is not “do A, then do B.” It is:
- Shell asks the kernel for a pipe: a small in-memory buffer with a write end and a read end.
- Shell starts two children.
- Left command’s stdout is attached to the write end.
- Right command’s stdin is attached to the read end.
- Both run together. If the left produces faster than the right consumes, the buffer fills and the kernel pauses the left. If the right is hungry, it waits. That pause is called back-pressure. You do not implement it; the kernel does.
3.3 Signals, then the steering wheel (foreground and background)
A signal is a kernel-to-process telegram. It has a name (SIGINT) and a default disposition (what happens if the program did not install a handler): die, stop, continue, or ignore. The shell and the line discipline are simply two common senders. You can also send by hand: kill -INT <pid> is the same family of event as Ctrl+C, aimed at a PID you chose.
A table of signals you will actually meet
| Signal | Typical number | How a human usually causes it | Default if unhandled | Plain meaning |
|---|---|---|---|---|
SIGINT | 2 | Ctrl+C (if isig); kill -INT | Terminate | “Interrupt what you are doing now.” Polite stop for interactive work. |
SIGQUIT | 3 | Ctrl+\ | Terminate + core dump | “Quit, and leave a body for the coroner.” |
SIGTSTP | 20 (Linux) | Ctrl+Z | Stop (pause; process still exists) | “Hands off the wheel; freeze.” The job is not dead. |
SIGCONT | 18 | fg, bg, kill -CONT | Continue | “Thaw.” Needed after SIGTSTP. |
SIGTTIN | 21 | Background job reads the TTY | Stop | “You may not read the keyboard back there.” |
SIGTTOU | 22 | Background job writes the TTY when tostop is on | Stop | “You may not scribble on the screen back there.” |
SIGHUP | 1 | Close the window / drop SSH; kill -HUP | Terminate | “The cable was unplugged.” Hang up. |
SIGTERM | 15 | kill <pid> (default); service managers | Terminate | “Please exit cleanly.” Not bound to a key. |
SIGKILL | 9 | kill -9 | Terminate; cannot be caught | “Die immediately.” Last resort, not a workflow. |
SIGWINCH | 28 | Resize the window | Ignore | “The glass changed size; redraw if you care.” |
SIGPIPE | 13 | Write to a pipe whose reader is gone | Terminate | “The other end hung up.” Why cmd1 | head does not run forever. |
SIGCHLD | 17 | A child exits or stops | Ignore (shell uses it) | The shell’s cue to wait and print [1]+ Done. |
Numbers can differ on other Unixes. Trust kill -l on your host.
Two facts beginners mix:
Ctrl+Zis not kill. It pauses.jobsstill lists the job.fgorbgsendsSIGCONT.&is not “run on another computer.” It is “start this job without giving it the steering wheel.” The CPU still runs it. The TTY does not belong to it.
Foreground vs background
Imagine one steering wheel per session (one controlling TTY).
| Foreground | Background | |
|---|---|---|
| How you start it | cmd then Enter (the usual) | cmd & or Ctrl+Z then bg |
| Steering wheel | This job holds it | The shell holds it (you get a prompt back) |
May read the TTY? | Yes | No → SIGTTIN, usually stopped |
Receives Ctrl+C / Ctrl+Z? | Yes (whole process group) | No. Those keys go to whoever is in front—usually the shell, which then starts a new line |
| Prompt | Hidden until the job ends or is stopped | Returns immediately |
| Shell command to swap | — | fg = give wheel back; bg = continue without the wheel |
A pipeline cat | grep foo is one job (one process group). One Ctrl+C is meant to SIGINT both ends.
Closing the window is not Ctrl+C. The emulator drops the PTY master. The kernel sends SIGHUP to the session. Foreground and background jobs both die unless they have been detached from that fate: nohup, disown, or a multiplexer such as tmux (a different engine, with its own PTYs).
Ctrl+C in one sentence: if isig is on, byte 0x03 is not given to the program as a character; the kernel sends SIGINT to the foreground process group.
Shell verbs (Level 4 preview)
long-job & # start in background; prints a job number, e.g. [1] 4521
jobs # list jobs of *this* shell
fg # last job → foreground
fg %1 # job number 1 → foreground
bg %1 # continue job 1 in background (after Ctrl+Z)
kill %1 # SIGTERM the job (the group), not a random PID
disown %1 # this shell will not SIGHUP it on exit (still dies if the TTY vanishes, unless nohup/tmux)
nohup long-job & # ignore SIGHUP; log to nohup.out by default
%1 is a job spec. It is not a PID. ps shows PIDs; jobs shows the shell’s own numbering.
Real situations (how to apply the skill)
Scenario A — Compile or test while you keep the prompt
- Problem:
npm testormake -jtakes minutes. You still need togit status. - Correct:
make -j8 &then work.jobsto see it. If the compiler prints too much, redirect:make -j8 >build.log 2>&1 &. - Wrong: opening five extra GUI windows because you think the prompt is “busy.” The prompt is busy only if the job is foreground.
- Expert note: if the job must outlive this window,
&is not enough. Use tmux (or a systemd unit). Background ≠ immortal.
Scenario B — You started in the foreground and need the prompt back
- Problem: you forgot
&.rsyncis running. You need to check disk. - Correct:
Ctrl+Z(SIGTSTP) →bg(SIGCONTwithout the wheel) →df -h.fgif you must watch rsync again. - Wrong:
Ctrl+C“to get the prompt” and then restart a two-hour copy. - Expert note:
Ctrl+Zis a surgical pause. Experts reach for it the way they reach for a turn signal—not as a crash.
Scenario C — A tool that wants the keyboard (vim, top, a password prompt, ssh)
- Problem: you ran
vim file &ordocker attach … &. - What happens: the editor is background. The first time it reads the TTY,
SIGTTINstops it. The screen looks “frozen” or garbled. - Correct: interactive programs stay foreground.
fgif you already blundered. - Expert note: experts never background a program whose job is to be the viewfinder. They background producers (builds, downloads, servers that log to a file), not consumers of stdin.
Scenario D — Long remote job over a flaky network
- Problem:
ssh hostthen./migrate.pyin the foreground. VPN drops →SIGHUP→ migration dies. - Correct: on the server,
tmux new -s migratethen run the job (see the tmux how-to). Detach. Close the laptop. - Almost correct:
nohup ./migrate.py >migrate.log 2>&1 &— survives hangup, but you have no live TTY to watch; youtail -fthe log. Fine for fire-and-forget. - Wrong:
./migrate.py &inside the SSH shell, then close the laptop. Background still receivesSIGHUPwith the session.
Scenario E — A pipeline you must stop cleanly
- Problem:
journalctl -f | grep --line-buffered error. - Correct: leave it foreground.
Ctrl+Chits the whole group. - Wrong: background it, then
killonlygrep.journalctlmay become an orphan still writing. Preferkill %n(the job) or foreground +Ctrl+C.
Scenario F — A service that should not live in your TTY at all
- Problem:
python app.py &on a server “so it keeps running.” - Correct for a toy: tmux or
nohupplus a log. - Correct for production: a systemd unit (restart policy, journal, user). The TTY is a classroom, not an init system.
- Expert note: experts ask “must a human be attached?” If no, the answer is rarely
&.
How experts think about foreground and background
| Beginner | Expert | |
|---|---|---|
What & means | “Make it survive” / “run it elsewhere” | “Relinquish the TTY. The job still dies on hangup unless I also detach the session.” |
| Default for a new command | Whatever they typed last time | Foreground if they must see or type; background + log if they must keep the prompt; tmux/unit if they must keep the job. |
Ctrl+C / Ctrl+Z | Panic keys | INT = abort this job; TSTP = park this job. Chosen on purpose. |
A stopped job (T) in ps | “It crashed” | “It is waiting for SIGCONT—I probably Ctrl+Z’d it or it hit SIGTTIN.” |
| Many GUI terminals | A substitute for job control | A smell. One shell + jobs/tmux is cheaper to reason about. |
| Who receives the signal | “The thing I am looking at” | The foreground process group of that TTY. They confirm with ps -o pid,pgid,stat,cmd. |
The expert decision procedure (use this until it is muscle):
- Does this program need stdin or a live full screen? → Foreground.
- Do I only need CPU + a log, and I want this prompt? → Background, redirect fds,
jobsto watch. - Must it survive this window / this SSH? → tmux (human will return) or systemd (the machine should own it).
&is not step 3. - Must I stop it? → Prefer the job (
Ctrl+Cin fg, orkill %n) overkill -9of a guessed PID.
Three questions: who holds the wheel, who must outlive the wheel, which telegram you are sending.
3.4 Cooked mode vs raw mode
| Mode | What happens to keys | Who draws the letters you type? | What is Ctrl+C? |
|---|---|---|---|
| Cooked (canonical, default at a bare prompt) | Kernel waits for Enter; handles Backspace for you | Usually the kernel echo | Usually a signal |
| Raw (vim, htop, many full-screen tools) | Each key goes to the program immediately | The program (it may not echo) | Often just the byte 0x03 |
A “broken terminal” after you kill a full-screen program is usually leftover raw + no echo: you type and see nothing; Enter does not behave. The computer is not cursed. You restore cooked manners with reset or stty sane (Level 5).
Level 4 — What you actually type
4.1 Variables
# Local to this shell only. No spaces around = (spaces break the command).
LOCAL_VAR="linux"
# Put it in the inherited list (promote), or create already exported:
export LOCAL_VAR
export GLOBAL_VAR="production"
# Give ONE command a variable without changing the parent shell:
LOG_LEVEL=debug python3 app.py
# Take it out of the inherited list, then delete it entirely
# (declare +x is bash; in zsh use typeset +x):
declare +x GLOBAL_VAR
unset GLOBAL_VAR
Names you will see constantly:
| Name | Typical meaning |
|---|---|
PATH | Directories, separated by :, where the shell looks for external programs |
HOME | Your home directory (/home/you) |
USER / LOGNAME | Your login name |
SHELL | Path of your login shell (/bin/bash, …) |
PWD | Current directory (the shell’s idea of it) |
LANG / LC_ALL | Language and number/date formatting |
Making them survive a new window is just “run these assignments when a shell starts”:
| File | When it usually runs |
|---|---|
~/.bashrc or ~/.zshrc | Each new interactive terminal window (common case on a laptop) |
~/.bash_profile / ~/.profile | Login shells (classic SSH login) |
/etc/profile | System-wide, login shells |
/etc/environment | System-wide KEY=value via PAM—no export keyword |
A systemd service has its own Environment= lines. It will not automatically see variables you exported only in a GUI terminal.
4.2 Asking the line discipline what it is doing
stty -a
You do not need to memorize the C structure. Remember three switches:
| Switch | If on |
|---|---|
icanon | Cooked: wait for Enter |
echo | Show keys as you type (via the kernel) |
isig | Ctrl+C / Ctrl+Z become signals |
4.3 Colours and cursor moves are still bytes
When vim “clears the screen,” it writes ESC sequences (\033…) into the same stdout pipe. Layer 2 (Paint) interprets them. $TERM names a terminfo dictionary. A wrong $TERM over SSH means the host lacks that dictionary, not that “Linux is broken.”
Resizing the window is a separate message (SIGWINCH / window-size ioctl), not something you must type as ESC yourself.
Level 5 — Do this on your machine
Use a throwaway terminal window. After each command, read the output and match it to a sentence above.
# 1) Who am I, and which fake cable is this?
tty # expect something like /dev/pts/3
echo "This shell's PID is $$"
# 2) Family tree of this session (PID, parent PID, terminal, command)
ps -o pid,ppid,tty,cmd
# 3) Line-discipline switches (look for icanon echo isig)
stty -a | head
# 4) Is pwd a builtin or a file?
type pwd
type -a pwd # may show both a builtin and /usr/bin/pwd
pwd # builtin path: usually NO extra process
/usr/bin/pwd # file path: a third process exists briefly
Inheritance (the copy rule):
export DEMO=parent
bash -c 'echo child sees DEMO=$DEMO; DEMO=child; echo child changed it to $DEMO'
echo "parent still has DEMO=$DEMO" # still "parent"
One-shot environment:
unset DEMO
DEMO=only python3 -c 'import os; print("python sees", os.environ.get("DEMO"))'
echo "parent DEMO is [${DEMO-}]" # empty
Operators:
false ; echo "semicolon still runs this"
false && echo "you should NOT see this"
false || echo "or-runs because false failed"
# Two programs at once, kernel buffer between them:
printf 'hello\nworld\n' | grep world
Restore a smashed TTY (optional; type the second line even if you see no echo):
stty raw -echo
stty sane
If the second line is hard to type, run reset and press Enter.
Job control (do this in order):
sleep 300 # foreground: no prompt
# Ctrl+Z # SIGTSTP — process still exists
jobs -l # [1]+ Stopped sleep 300
ps -o pid,pgid,stat,cmd | grep sleep # STAT is T (stopped)
bg # SIGCONT, still no wheel
jobs -l # Running
# try: (sleep 300 &) # already background
# type: fg # take the wheel again
# Ctrl+C # SIGINT — sleep dies
Background cannot steal the keyboard:
cat & # cat wants stdin
jobs -l # often Stopped (tty input) — SIGTTIN
fg # now cat holds the wheel; Ctrl+C to leave
Second window: ps -o pid,pgid,stat,cmd while you Ctrl+Z / bg / fg. Watch STAT (T vs S) and PGID.
Hangup vs background (optional, use a throwaway SSH): sleep 300 & then close the SSH client. The sleep is usually gone. Repeat inside tmux; detach; close the client; the sleep remains. That is the difference & does not teach.
Closing
We started with a naming error (one window, one process) and ended with a tree, a six-layer byte path, and an environment that only travels downward. Ctrl+C, export, |, and a “dead” terminal are stories you can finish once the nouns are long enough.
Open edge: With no window at all—serial getty, machine console—which of the six layers vanish, and which part of n_tty stays the same?