Kiet Nguyen logo
NotesNotesResumeResume
© 2026 Kiet Nguyen
← All notes
Dark elegant terminal window as layered glass plates of emulator, kernel, and shell

Fathom·July 4, 2026·23 min read

Beyond the magic terminal box

Linux · Terminal · Kernel · Unix · Systems · Shell · Processes

The terminal window is not one process. Six layers from key to kernel, copies of environment, pipes, signals, and who actually holds the TTY—plus commands you can run to prove it.

This lecture assumes you can open a terminal, type a command, and press Enter. It does not assume you already know what a process, a file descriptor, or the kernel is. New words get a definition in the same paragraph.

Linux as you meet it on a laptop or server. macOS is close, with small name changes. Check on your machine: tty, stty -a, ps, echo $$. If an output differs, trust your machine.

Words we will use

WordMeaning in this lecture
ProcessOne running program. The operating system gives it a number called a PID (process ID). Your window is not automatically “one process.”
KernelThe core of Linux that talks to hardware and creates/kills processes. You do not run it like an app; everything else asks it to do work via system calls (read, write, fork, …).
ByteThe smallest unit of data we care about here. The letter p is the byte 0x70. A “character you typed” becomes bytes before any program sees it.
File descriptor (fd)A small integer a process uses as a handle to something it can read or write. By convention: 0 = stdin (input), 1 = stdout (normal output), 2 = stderr (error output).
Parent / childWhen process A asks the kernel to start process B, A is the parent and B is the child. The child can outlive the thought “I clicked the icon,” but it usually dies if the parent’s session is torn down—details later.
ShellThe program that prints a prompt ($ or %) and waits for a line of text. Common names: bash, zsh, fish. It is a process, not the window.
Terminal emulatorThe window program (Alacritty, GNOME Terminal, Kitty, …). It draws letters on screen and owns the “fake keyboard cable” described below.
SignalA small numbered message the kernel can deliver to a process: “stop,” “hang up,” “continue.” It is not a character in your program’s input—unless the program asked to treat it as one.
Job / process groupOne or more processes the shell treats as a unit (a pipeline is usually one job). The kernel gives the group a PGID. Foreground vs background is about which group holds the TTY, not about whether the CPU is busy.

One-sentence crystal

The “terminal” is several programs plus a slice of the kernel: the window draws pixels and owns one end of a fake serial cable (a PTY); the kernel may edit or echo your typing; the shell is a child reading the other end; variables and external commands live in copies of memory, not in one box.


Level 1 — Why “just type” is not enough

A beginner points at the rectangle and says: “That is one program.” Then ordinary failures have no names.

You only type…What happensWithout a mapWith a map
VAR=1 (no export)Python cannot see VAR“Linux is random”Child never inherited it
vim, then kill it badlyKeys do not echoReinstall the OSLine discipline still raw; reset / stty sane
long-job & then close the windowJob dies“Background should live”SIGHUP; use tmux / nohup
cmd1 | cmd2Hang or empty output“Pipes are buggy”Two processes, one kernel buffer
Ctrl+CWrong thing dies, or nothing“Ctrl+C is broken”Signal hits the foreground process group
Blog command works here, fails in CICI has no TTY“CI is stupid”isatty / $TERM differ
ssh then messy vimWrong colors“Server is broken”$TERM is a dictionary the host lacks
pwd but no /usr/bin/pwd in psConfusion“ps is lying”Often a builtin: the shell answered

You do not need ioctl trivia to use a computer. You need to name the actor when the happy path ends: window, PTY, kernel manners, shell, child, environment copy. Day to day, still just type. The map is for the first unexplained failure—so you do not reboot because echo died (stty had four flags wrong).

The window is a frame. The shell and your commands are the specimen. The frame matters; it is not the cell.


Level 2 — The big picture (architecture)

2.1 Three things people mash into one word

Say these out loud until they feel different:

  1. Terminal emulator = the window app. It talks to the graphical system (Wayland or X11 on Linux), draws glyphs, and holds the master end of a fake cable.
  2. PTY (pseudo-terminal) = a pair of devices inside the kernel that pretend to be an old serial terminal (a teletype, historically TTY). One end is master, one end is slave. Bytes written on one end can be read on the other.
  3. Shell = the interpreter process (bash / zsh / …) attached to the slave. It is a customer of the PTY, not the PTY itself.

/dev/tty means: “whatever terminal this process is attached to.” It is an alias, like “my house” rather than a street number.

The Linux console (Ctrl+Alt+F3, device /dev/tty3, …) is a different path: real text console, often no GUI emulator. Same kind of line discipline; different glass.

2.2 Six layers, from your finger to the kernel

Six layers ≠ six processes. Some are parts of one program. Some are kernel code you never “start.”

Numbered in the order a keystroke travels:

#NicknameWhat it isEveryday job
1GlassDisplay / windowOS delivers key events; later, glyphs get painted
2PaintEmulator’s VT / ANSI engineCursor, color, scrollback grid
3MasterPTY master fdSpeaking tube the window process reads and writes
4MannersLine discipline n_tty + PTY slave /dev/pts/NHold a line, echo, turn Ctrl+C into a signal. Kernel.
5Shellbash / zsh / …Reads the line; builtin or spawn a child
6CoreKernel VFS / schedulergetcwd, read, write, CPU time
  You type
    → 1 Glass     window
    → 2 Paint     emulator logic
    → 3 Master    PTY master (still the window process)
         ║  kernel
    → 4 Manners   n_tty + slave
         ║  userspace
    → 5 Shell     read(0)
    → 6 Core      if the command needs a syscall (almost always)

The shell is not “below” the kernel in importance. It is a userspace process whose stdin/stdout are the slave. Manners (4) sit on the wire.

2.3 The window is a tree of processes

Claim: one open terminal window is at least two processes, often three.

 Process A   Terminal emulator     “the window.” Lives until you close it.
      │
      │  The emulator asks the kernel: copy me, then become a shell.
      │  (The usual pair of requests is called fork, then execve.
      │   We will say what those words mean in Level 3.)
      ▼
 Process B   Interactive shell     Prints the prompt. Lives until the session ends.
      │
      │  Only if you run an *external* program (a file such as /usr/bin/ls)
      ▼
 Process C   That program          Lives only until that program exits.
WhoExample name you might see in psRoleHow long it lives
Emulatoralacritty, gnome-terminal-serverParent; draws the window; holds PTY masterUntil the window closes
Shellbash, zshChild of emulator; holds PTY slave as stdin/out/errUntil you exit the shell or the session dies
External command/usr/bin/ls, /usr/bin/pwdChild of the shellUntil that command finishes

Builtin versus external (say this until it is boring)

  • A builtin is a command the shell implements itself. Examples in bash/zsh: cd, export, jobs, and usually pwd. No Process C. The work happens inside Process B.
  • An external command is a file the kernel can run, found via PATH (a list of directories). Examples: /usr/bin/ls, /usr/bin/grep, /usr/bin/python3. The shell creates a child and replaces that child’s program image with the file, then waits.

type pwd tells you which story your shell uses.

2.4 Variables: two boxes, only one is inherited

A variable is a named string in memory (NAME=value). It is not “a setting of the window.” It belongs to a process.

There are two boxes inside the shell:

Shell variableEnvironment variable
Where it livesOnly in this shell’s own tableIn the list the kernel will copy to a new program
Does a child program see it?NoYes (it gets a copy)
How you create itVAR=helloexport VAR or export VAR=hello
How you look at many of themset (prints a lot)env or printenv
Typical useA loop counter; a temporary flagPATH, language/locale, API keys

The one-way rule (memorize this):

  1. Parent → child: the child receives a copy of the exported list when it starts (execve).
  2. Child → parent: if the child changes a variable, the parent does not change. There is no “write back.”

That is why “it works when I type it” and “it fails in a script / cron / systemd” so often: those other worlds never saw your unexported GUI-shell variables.


Level 3 — The story of one command

3.1 You type pwd and press Enter

We assume:

  • a normal prompt (the line discipline is in canonical / “cooked” mode: wait for Enter);
  • pwd is a builtin;
  • therefore only two processes exist: emulator + shell.

Phase A — Down: key → bytes → shell wakes up

  1. 1 Glass. You press keys. The window system turns that into events (“key p down”) and gives them to the emulator process.
  2. 2 Paint. The emulator maps them to p, w, d, then Enter. Enter is often CR (0x0D). Payload: 0x70 0x77 0x64 0x0D.
  3. 3 Master. The emulator writes those bytes into its PTY master fd.
  4. 4 Manners (n_tty). Echo: copies go back to the master so you see letters before the shell has the whole line. Buffer: the line sits in kernel memory; the shell is still in read(). Enter: CR usually becomes NL (0x0A); the kernel wakes the shell.
  5. 5 Shell. read(0, …) returns pwd\n. stdin is the slave. The shell parses a builtin and does not create Process C.
  6. 6 Core. The builtin asks getcwd (or uses its own PWD—shells differ). The kernel returns a path such as /home/you.

Phase B — Up: path string → pixels

  1. 5 Shell. Writes /home/you\n to stdout (fd 1), then a new prompt. fd 1 is the same slave.
  2. 4 Manners. May turn NL into CR+NL (old printers). Bytes move toward the master.
  3. 3 Master. The emulator reads the master.
  4. 2 Paint. Updates the grid of cells.
  5. 1 Glass. Fonts become pixels. You see the path.

If you run /usr/bin/pwd instead

After step 5 the shell says: this is a file, not a builtin.

  1. fork — kernel clones the shell into a child (Process C). Clone means: same open files, same variables for a moment.
  2. execve("/usr/bin/pwd", …) — the child throws away the shell program and becomes pwd. The environment copy is installed here.
  3. waitpid — the original shell sleeps until Process C exits, then prints the next prompt.

While /usr/bin/pwd runs, ps can show three PIDs. When it exits, you are back to two.

3.2 Connecting commands (|, ;, &&, ||)

What you typeHow many programs start (typical)Same time or one after another?When does the second run?
cmd1 | cmd2Two children (plus the shell)At the same timeAlways; they share a pipe
cmd1 ; cmd2One, then the otherOne after anotherAlways, even if the first failed
cmd1 && cmd2One, then maybe the otherOne after anotherOnly if the first succeeded (exit code 0)
cmd1 || cmd2One, then maybe the otherOne after anotherOnly if the first failed (exit code not 0)

Exit code: every process ends with a small integer. 0 means success by convention. The shell stores the last one in $?.

A pipe is not “do A, then do B.” It is:

  1. Shell asks the kernel for a pipe: a small in-memory buffer with a write end and a read end.
  2. Shell starts two children.
  3. Left command’s stdout is attached to the write end.
  4. Right command’s stdin is attached to the read end.
  5. Both run together. If the left produces faster than the right consumes, the buffer fills and the kernel pauses the left. If the right is hungry, it waits. That pause is called back-pressure. You do not implement it; the kernel does.

3.3 Signals, then the steering wheel (foreground and background)

A signal is a kernel-to-process telegram. It has a name (SIGINT) and a default disposition (what happens if the program did not install a handler): die, stop, continue, or ignore. The shell and the line discipline are simply two common senders. You can also send by hand: kill -INT <pid> is the same family of event as Ctrl+C, aimed at a PID you chose.

A table of signals you will actually meet

SignalTypical numberHow a human usually causes itDefault if unhandledPlain meaning
SIGINT2Ctrl+C (if isig); kill -INTTerminate“Interrupt what you are doing now.” Polite stop for interactive work.
SIGQUIT3Ctrl+\Terminate + core dump“Quit, and leave a body for the coroner.”
SIGTSTP20 (Linux)Ctrl+ZStop (pause; process still exists)“Hands off the wheel; freeze.” The job is not dead.
SIGCONT18fg, bg, kill -CONTContinue“Thaw.” Needed after SIGTSTP.
SIGTTIN21Background job reads the TTYStop“You may not read the keyboard back there.”
SIGTTOU22Background job writes the TTY when tostop is onStop“You may not scribble on the screen back there.”
SIGHUP1Close the window / drop SSH; kill -HUPTerminate“The cable was unplugged.” Hang up.
SIGTERM15kill <pid> (default); service managersTerminate“Please exit cleanly.” Not bound to a key.
SIGKILL9kill -9Terminate; cannot be caught“Die immediately.” Last resort, not a workflow.
SIGWINCH28Resize the windowIgnore“The glass changed size; redraw if you care.”
SIGPIPE13Write to a pipe whose reader is goneTerminate“The other end hung up.” Why cmd1 | head does not run forever.
SIGCHLD17A child exits or stopsIgnore (shell uses it)The shell’s cue to wait and print [1]+ Done.

Numbers can differ on other Unixes. Trust kill -l on your host.

Two facts beginners mix:

  1. Ctrl+Z is not kill. It pauses. jobs still lists the job. fg or bg sends SIGCONT.
  2. & is not “run on another computer.” It is “start this job without giving it the steering wheel.” The CPU still runs it. The TTY does not belong to it.

Foreground vs background

Imagine one steering wheel per session (one controlling TTY).

ForegroundBackground
How you start itcmd then Enter (the usual)cmd & or Ctrl+Z then bg
Steering wheelThis job holds itThe shell holds it (you get a prompt back)
May read the TTY?YesNo → SIGTTIN, usually stopped
Receives Ctrl+C / Ctrl+Z?Yes (whole process group)No. Those keys go to whoever is in front—usually the shell, which then starts a new line
PromptHidden until the job ends or is stoppedReturns immediately
Shell command to swap—fg = give wheel back; bg = continue without the wheel

A pipeline cat | grep foo is one job (one process group). One Ctrl+C is meant to SIGINT both ends.

Closing the window is not Ctrl+C. The emulator drops the PTY master. The kernel sends SIGHUP to the session. Foreground and background jobs both die unless they have been detached from that fate: nohup, disown, or a multiplexer such as tmux (a different engine, with its own PTYs).

Ctrl+C in one sentence: if isig is on, byte 0x03 is not given to the program as a character; the kernel sends SIGINT to the foreground process group.

Shell verbs (Level 4 preview)

long-job &          # start in background; prints a job number, e.g. [1] 4521
jobs                # list jobs of *this* shell
fg                  # last job → foreground
fg %1               # job number 1 → foreground
bg %1               # continue job 1 in background (after Ctrl+Z)
kill %1             # SIGTERM the job (the group), not a random PID
disown %1           # this shell will not SIGHUP it on exit (still dies if the TTY vanishes, unless nohup/tmux)
nohup long-job &    # ignore SIGHUP; log to nohup.out by default

%1 is a job spec. It is not a PID. ps shows PIDs; jobs shows the shell’s own numbering.

Real situations (how to apply the skill)

Scenario A — Compile or test while you keep the prompt

  • Problem: npm test or make -j takes minutes. You still need to git status.
  • Correct: make -j8 & then work. jobs to see it. If the compiler prints too much, redirect: make -j8 >build.log 2>&1 &.
  • Wrong: opening five extra GUI windows because you think the prompt is “busy.” The prompt is busy only if the job is foreground.
  • Expert note: if the job must outlive this window, & is not enough. Use tmux (or a systemd unit). Background ≠ immortal.

Scenario B — You started in the foreground and need the prompt back

  • Problem: you forgot &. rsync is running. You need to check disk.
  • Correct: Ctrl+Z (SIGTSTP) → bg (SIGCONT without the wheel) → df -h. fg if you must watch rsync again.
  • Wrong: Ctrl+C “to get the prompt” and then restart a two-hour copy.
  • Expert note: Ctrl+Z is a surgical pause. Experts reach for it the way they reach for a turn signal—not as a crash.

Scenario C — A tool that wants the keyboard (vim, top, a password prompt, ssh)

  • Problem: you ran vim file & or docker attach … &.
  • What happens: the editor is background. The first time it reads the TTY, SIGTTIN stops it. The screen looks “frozen” or garbled.
  • Correct: interactive programs stay foreground. fg if you already blundered.
  • Expert note: experts never background a program whose job is to be the viewfinder. They background producers (builds, downloads, servers that log to a file), not consumers of stdin.

Scenario D — Long remote job over a flaky network

  • Problem: ssh host then ./migrate.py in the foreground. VPN drops → SIGHUP → migration dies.
  • Correct: on the server, tmux new -s migrate then run the job (see the tmux how-to). Detach. Close the laptop.
  • Almost correct: nohup ./migrate.py >migrate.log 2>&1 & — survives hangup, but you have no live TTY to watch; you tail -f the log. Fine for fire-and-forget.
  • Wrong: ./migrate.py & inside the SSH shell, then close the laptop. Background still receives SIGHUP with the session.

Scenario E — A pipeline you must stop cleanly

  • Problem: journalctl -f | grep --line-buffered error.
  • Correct: leave it foreground. Ctrl+C hits the whole group.
  • Wrong: background it, then kill only grep. journalctl may become an orphan still writing. Prefer kill %n (the job) or foreground + Ctrl+C.

Scenario F — A service that should not live in your TTY at all

  • Problem: python app.py & on a server “so it keeps running.”
  • Correct for a toy: tmux or nohup plus a log.
  • Correct for production: a systemd unit (restart policy, journal, user). The TTY is a classroom, not an init system.
  • Expert note: experts ask “must a human be attached?” If no, the answer is rarely &.

How experts think about foreground and background

BeginnerExpert
What & means“Make it survive” / “run it elsewhere”“Relinquish the TTY. The job still dies on hangup unless I also detach the session.”
Default for a new commandWhatever they typed last timeForeground if they must see or type; background + log if they must keep the prompt; tmux/unit if they must keep the job.
Ctrl+C / Ctrl+ZPanic keysINT = abort this job; TSTP = park this job. Chosen on purpose.
A stopped job (T) in ps“It crashed”“It is waiting for SIGCONT—I probably Ctrl+Z’d it or it hit SIGTTIN.”
Many GUI terminalsA substitute for job controlA smell. One shell + jobs/tmux is cheaper to reason about.
Who receives the signal“The thing I am looking at”The foreground process group of that TTY. They confirm with ps -o pid,pgid,stat,cmd.

The expert decision procedure (use this until it is muscle):

  1. Does this program need stdin or a live full screen? → Foreground.
  2. Do I only need CPU + a log, and I want this prompt? → Background, redirect fds, jobs to watch.
  3. Must it survive this window / this SSH? → tmux (human will return) or systemd (the machine should own it). & is not step 3.
  4. Must I stop it? → Prefer the job (Ctrl+C in fg, or kill %n) over kill -9 of a guessed PID.

Three questions: who holds the wheel, who must outlive the wheel, which telegram you are sending.

3.4 Cooked mode vs raw mode

ModeWhat happens to keysWho draws the letters you type?What is Ctrl+C?
Cooked (canonical, default at a bare prompt)Kernel waits for Enter; handles Backspace for youUsually the kernel echoUsually a signal
Raw (vim, htop, many full-screen tools)Each key goes to the program immediatelyThe program (it may not echo)Often just the byte 0x03

A “broken terminal” after you kill a full-screen program is usually leftover raw + no echo: you type and see nothing; Enter does not behave. The computer is not cursed. You restore cooked manners with reset or stty sane (Level 5).


Level 4 — What you actually type

4.1 Variables

# Local to this shell only. No spaces around =  (spaces break the command).
LOCAL_VAR="linux"

# Put it in the inherited list (promote), or create already exported:
export LOCAL_VAR
export GLOBAL_VAR="production"

# Give ONE command a variable without changing the parent shell:
LOG_LEVEL=debug python3 app.py

# Take it out of the inherited list, then delete it entirely
# (declare +x is bash; in zsh use typeset +x):
declare +x GLOBAL_VAR
unset GLOBAL_VAR

Names you will see constantly:

NameTypical meaning
PATHDirectories, separated by :, where the shell looks for external programs
HOMEYour home directory (/home/you)
USER / LOGNAMEYour login name
SHELLPath of your login shell (/bin/bash, …)
PWDCurrent directory (the shell’s idea of it)
LANG / LC_ALLLanguage and number/date formatting

Making them survive a new window is just “run these assignments when a shell starts”:

FileWhen it usually runs
~/.bashrc or ~/.zshrcEach new interactive terminal window (common case on a laptop)
~/.bash_profile / ~/.profileLogin shells (classic SSH login)
/etc/profileSystem-wide, login shells
/etc/environmentSystem-wide KEY=value via PAM—no export keyword

A systemd service has its own Environment= lines. It will not automatically see variables you exported only in a GUI terminal.

4.2 Asking the line discipline what it is doing

stty -a

You do not need to memorize the C structure. Remember three switches:

SwitchIf on
icanonCooked: wait for Enter
echoShow keys as you type (via the kernel)
isigCtrl+C / Ctrl+Z become signals

4.3 Colours and cursor moves are still bytes

When vim “clears the screen,” it writes ESC sequences (\033…) into the same stdout pipe. Layer 2 (Paint) interprets them. $TERM names a terminfo dictionary. A wrong $TERM over SSH means the host lacks that dictionary, not that “Linux is broken.”

Resizing the window is a separate message (SIGWINCH / window-size ioctl), not something you must type as ESC yourself.


Level 5 — Do this on your machine

Use a throwaway terminal window. After each command, read the output and match it to a sentence above.

# 1) Who am I, and which fake cable is this?
tty                    # expect something like /dev/pts/3
echo "This shell's PID is $$"

# 2) Family tree of this session (PID, parent PID, terminal, command)
ps -o pid,ppid,tty,cmd

# 3) Line-discipline switches (look for icanon echo isig)
stty -a | head

# 4) Is pwd a builtin or a file?
type pwd
type -a pwd            # may show both a builtin and /usr/bin/pwd

pwd                    # builtin path: usually NO extra process
/usr/bin/pwd           # file path: a third process exists briefly

Inheritance (the copy rule):

export DEMO=parent
bash -c 'echo child sees DEMO=$DEMO; DEMO=child; echo child changed it to $DEMO'
echo "parent still has DEMO=$DEMO"    # still "parent"

One-shot environment:

unset DEMO
DEMO=only python3 -c 'import os; print("python sees", os.environ.get("DEMO"))'
echo "parent DEMO is [${DEMO-}]"     # empty

Operators:

false ; echo "semicolon still runs this"
false && echo "you should NOT see this"
false || echo "or-runs because false failed"

# Two programs at once, kernel buffer between them:
printf 'hello\nworld\n' | grep world

Restore a smashed TTY (optional; type the second line even if you see no echo):

stty raw -echo
stty sane

If the second line is hard to type, run reset and press Enter.

Job control (do this in order):

sleep 300                 # foreground: no prompt
# Ctrl+Z                  # SIGTSTP — process still exists
jobs -l                   # [1]+ Stopped  sleep 300
ps -o pid,pgid,stat,cmd | grep sleep   # STAT is T (stopped)
bg                        # SIGCONT, still no wheel
jobs -l                   # Running
# try:  (sleep 300 &)     # already background
# type:  fg               # take the wheel again
# Ctrl+C                  # SIGINT — sleep dies

Background cannot steal the keyboard:

cat &                     # cat wants stdin
jobs -l                   # often Stopped (tty input) — SIGTTIN
fg                        # now cat holds the wheel; Ctrl+C to leave

Second window: ps -o pid,pgid,stat,cmd while you Ctrl+Z / bg / fg. Watch STAT (T vs S) and PGID.

Hangup vs background (optional, use a throwaway SSH): sleep 300 & then close the SSH client. The sleep is usually gone. Repeat inside tmux; detach; close the client; the sleep remains. That is the difference & does not teach.


Closing

We started with a naming error (one window, one process) and ended with a tree, a six-layer byte path, and an environment that only travels downward. Ctrl+C, export, |, and a “dead” terminal are stories you can finish once the nouns are long enough.

Open edge: With no window at all—serial getty, machine console—which of the six layers vanish, and which part of n_tty stays the same?

Back to notes