Kiet Nguyen logo
NotesNotesResumeResume
© 2026 Kiet Nguyen
← All categories

09

Journal & Logs

  • Last N lines for a unit
  • Follow logs while reproducing
  • Errors since boot / time window; OOM in kernel log
journalctl — core filtersjournalctl — time windowsjournalctl — output formatjournalctl — disk & vacuumjournalctl — other useful

Must-know cold

  • journalctl -u UNIT -n 100 --no-pager
  • journalctl -u UNIT -f (follow)
  • journalctl -u UNIT --since "30 min ago"
  • journalctl -b -p err · journalctl -k (kernel)
  • Always pair with systemctl status UNIT first (sheet 08)

Priority levels (-p)

Definition: syslog-style severities. Lower number = more severe. -p err shows 0–3 (emerg…err), not “err and debug.”

NamePriorityNotes
emerg0Also panic
alert1
crit2
err3Also error
warning4Also warn
notice5
info6
debug7

Commands

journalctl — core filters

Definition: Query the systemd journal by unit, boot, priority, or kernel.

OptionArgumentMeaningExample
(none)—All journal (huge)Avoid bare
-uUNITUnit _SYSTEMD_UNIT= plus systemd/coredump lines about it (not every app file log)journalctl -u nginx
-tSYSLOG_IDSyslog identifierjournalctl -t sshd
-k / --dmesg—Kernel messages (_TRANSPORT=kernel). Implies current boot unless you add -b -1journalctl -k
-b—Current boot onlyjournalctl -b
-b-1 / IDPrevious boot (-1 = boot before last) or boot IDjournalctl -b -1
--list-boots—List boots with IDsjournalctl --list-boots
-nNLast N entries (default 10 if -n has no number). Implied by -fjournalctl -n 50
-f—Follow new entries (starts from the last -n lines)journalctl -f
-e—Jump to end in less. Implies -n 1000 and current boot. Not with --no-pagerjournalctl -e
-r—Reverse (newest first)journalctl -r
-pPRIOThis severity and more important (numerically ≤ N)journalctl -p err
-pFROM..TOInclusive priority rangejournalctl -p err..alert
-g / --grepREGEXPCRE on MESSAGE= only. All-lowercase pattern is case-insensitivejournalctl -g 'OOM|killed'

Flag combos

ComboMeaningExample
-u UNIT -n 100 --no-pagerLast 100 for unitDefault incident
-u UNIT -fLive while restartTwo terminals
-u UNIT -p err -n 50Errors only
-b -p err --no-pagerErrors this boot

journalctl — time windows

Definition: Restrict journal output to a since/until time range.

OptionArgumentMeaningExample
--sinceTIMEStart timejournalctl --since "1 hour ago"
--untilTIMEEnd timejournalctl --until "10:00"
--sinceYYYY-MM-DDFrom datejournalctl --since 2026-08-01
--since"YYYY-MM-DD HH:MM:SS"Precisejournalctl --since "2026-08-01 09:00:00"

TIME forms: yesterday · today · 2 hours ago · 2026-08-01 · 2026-08-01 09:14:00

Flag combos

ComboMeaningExample
-u U --since "30 min ago" --no-pagerRecent unit windowRCA
--since "09:00" --until "09:30" -p errIncident slice

journalctl — output format

Definition: Choose journal presentation (short-iso, json, cat, etc.).

OptionArgumentMeaningExample
-oshortDefault-ish shortjournalctl -o short
-oshort-isoISO timestampsjournalctl -o short-iso
-oshort-preciseHigher precision timejournalctl -o short-precise
-overboseAll fieldsjournalctl -o verbose -n 1
-ojsonJSON linesjournalctl -o json -n 5
-ojson-prettyPretty JSONTickets
-ocatMessage onlyjournalctl -o cat -u U
-x—Extra explanation catalog textjournalctl -x -u U

Flag combos

ComboMeaningExample
-u U -n 20 -o short-iso --no-pagerClean pasteChat/ticket
-u U -o json-pretty -n 5Structured

journalctl — disk & vacuum

Definition: Show journal disk use and reclaim space safely.

OptionArgumentMeaningExample
--disk-usage—Journal disk usejournalctl --disk-usage
--vacuum-sizeSIZEDelete archived files until usage is below SIZE (not the active file)sudo journalctl --vacuum-size=200M
--vacuum-timeTIMEDrop archived files older than TIMEsudo journalctl --vacuum-time=14d
--vacuum-filesNLeave N journal files (still won’t delete the active one)sudo journalctl --vacuum-files=5
--rotate—Archive the current file so vacuum can reclaim itsudo journalctl --rotate --vacuum-size=200M

Flag combos

ComboMeaningExample
--rotate --vacuum-size=200MRotate first, then vacuum (active file included)Disk-full playbook

journalctl — other useful

Definition: User journals, field matches, no-pager, and related filters.

OptionArgumentMeaningExample
--no-pager—Print raw; scripts/SSHAlways in automation
--user—Current user’s journal (needs persistent storage). Default without flags is everything the caller may readjournalctl --user -u app
--user-unitUNITUser unit (_SYSTEMD_USER_UNIT=). Same idea as --user -ujournalctl --user-unit=app
_PID=NField match PIDjournalctl _PID=1234
_COMM=NAMECommand name fieldjournalctl _COMM=sshd
-q—Hide “Journal begins at…” / reboot banners and “can’t read system journal” hintsjournalctl -q
--utc—Show UTC timesjournalctl --utc

Flag combos

ComboMeaningExample
-k --since "1 hour ago" | grep -i oomKernel OOM huntMemory incidents
--user -u lab -fUser service liveLab units

Common recipes

GoalCommand
Unit last 100 linesjournalctl -u nginx -n 100 --no-pager
Follow unitjournalctl -u nginx -f
Last 30 minutesjournalctl -u nginx --since "30 min ago" --no-pager
Previous boot unitjournalctl -b -1 -u nginx --no-pager
System errors this bootjournalctl -b -p err --no-pager
Kernel OOM (this boot)journalctl -k --since "today" --no-pager | grep -iE 'oom|killed process'
Previous-boot kerneljournalctl -k -b -1 --no-pager
Userspace OOM (oomd)journalctl -u systemd-oomd --since "today" --no-pager
Disk used by journaljournalctl --disk-usage
Failed unit deep divesystemctl status U -l --no-pager; journalctl -u U -n 200 --no-pager

Pitfalls

  • Bare journalctl is enormous — always constrain (-u, -b, --since, -n).
  • No persistent journal (Storage=auto and no /var/log/journal) → logs live in /run and die on reboot. -b -1 / --list-boots look empty.
  • -k and -e both imply this boot unless you pass another -b. Previous-boot kernel is journalctl -k -b -1.
  • -p err is emerg…err (≤ 3), not “err and noisier.”
  • -g is PCRE on MESSAGE= only (not unit names). All-lowercase patterns match case-insensitively.
  • -e is pager-only (less) and caps at 1000 lines; do not pair with --no-pager.
  • App may log only to files under /var/log — empty journal ≠ healthy app. -u will not see those.
  • systemd-oomd is a unit, not kernel. journalctl -k misses it.
  • Vacuum only deletes archived files. --disk-usage still counts the active file — --rotate first if you need that space.
  • --user needs persistent storage; without it user journals may be missing.
  • Timezones: --since "09:14" is host local unless --utc. Check timedatectl.
  • Permissions: system journal needs root or membership in systemd-journal / adm / wheel.

For more details, try man <command> in your terminal.

Previous08 systemd ServicesNext10 Cron & Timers