A condensed, practical Bash reference for day-to-day scripting.

Not a Bash tutorial. Just the things that repeatedly matter in real scripts.

Bash-specific features are marked where relevant.

Quotes & Substitution

Pattern Meaning Example
'text' literal, no expansion '$HOME'
"text" expand vars & commands, preserve spaces "hi $USER"
$(cmd) command substitution $(pwd)
$'text' C-style escapes $'\n'

Why quoting matters

name="Tien Du"

mkdir $name

Expands into:

mkdir Tien Du

Creates two arguments instead of one.

Correct:

mkdir "$name"

Notes

  • Single quotes are safest for literal text.
  • Double quotes prevent accidental word splitting.
  • Prefer $(cmd) over old backticks: `cmd`.

Pipes & Redirection

Pattern Meaning Example
`cmd1 cmd2` pipe stdout
`cmd1 & cmd2` pipe stdout+stderr
> overwrite stdout echo hi > x
>> append stdout echo hi >> x
< stdin from file wc -l < x
2> stderr to file cmd 2> err
&> stdout+stderr to file cmd &> all.log
2>&1 stderr to stdout cmd 2>&1
1>&2 stdout to stderr echo hi 1>&2
<<< here-string grep hi <<< "$x"
<<EOF heredoc cat <<EOF
tee pipe + save `echo hi
`> ` force overwrite

File descriptors

FD Meaning
0 stdin
1 stdout
2 stderr

Example:

cmd > out.log 2> err.log
  • stdout goes to out.log
  • stderr goes to err.log

Portable version of &>:

cmd > out.log 2>&1

Process Substitution (Bash)

Pattern Meaning Example
<(cmd) cmd output as file diff <(sort a) <(sort b)
>(cmd) redirect into cmd echo hi > >(sed "s/h/H/")

Useful when a command expects filenames instead of stdin.

Parameter Expansion

Pattern Meaning Example
${v:-x} default if unset/empty ${a:-hi}
${v-x} default if unset ${a-hi}
${v:=x} assign default ${a:=hi}
${v:?msg} error if unset/empty ${a:?bad}
${v:+x} use x if set ${a:+yes}
${v#p} / ${v##p} trim front ${p##*/}
${v%p} / ${v%%p} trim back ${f%%.*}
${v/p/r} replace first ${x/-/_}
${v//p/r} replace all ${x//-/_}
${#v} length ${#s}
${v:pos} substring ${s:2}
${v:pos:len} bounded substring ${s:1:3}

Common patterns

Get filename:

path="/tmp/test.txt"

echo "${path##*/}"

Output:

test.txt

Remove extension:

echo "${path%.*}"

Output:

/tmp/test

Globs & Brace Expansion

Pattern Meaning Example
* wildcard *.txt
? single char a?.txt
[abc] char class file[1-9]
{a,b} alternation {dev,prod}.cfg
{1..5} sequence {1..3}

Example

mkdir -p project/{src,tests,docs}

Empty glob caveat

rm *.tmp

If nothing matches, Bash may pass literal *.tmp.

Safer:

shopt -s nullglob

Or guard inside the loop:

for f in *.tmp; do
  [[ -e "$f" ]] || continue
  rm -- "$f"
done

Links: Hardlink & Symlink

Commands

Command Meaning Example
ln a b create hardlink ln file copy
ln -s a b create symlink ln -s /real/file link
readlink x show symlink target readlink link
readlink -f x resolve real path readlink -f link
unlink x remove link unlink link
ls -li inspect inode ls -li file copy
Feature Hardlink Symlink
Points to inode path
Cross-filesystem no yes
Breaks if target removed no yes
Separate inode no yes
Common use duplicate refs shortcuts

Mental model

Symlink:

  • shortcut/path reference

Hardlink:

  • another name for the same file

Variables & Environment

Pattern Meaning Example
v=hi assign name="tien"
export v export variable export PATH
readonly v constant readonly API_KEY
local v=x function-local inside functions
$? last exit code echo $?

Example

API_KEY="abc"

export API_KEY

python app.py

Child processes only inherit exported variables.

IFS, Reading & Arrays (Bash)

IFS

Pattern Meaning
IFS word splitting chars
IFS=$'\n' newline-only split
IFS=: colon split
IFS= read -r line safe raw line read

Arrays

Pattern Meaning Example
a=(x y z) create array a=(1 2 3)
declare -a a declare array declare -a files
${a[0]} first element ${a[0]}
${a[@]} all elements "${a[@]}"
${#a[@]} array length ${#a[@]}
a+=(x) append a+=(4)
read -ra a split input into array read -ra a <<< "$s"
readarray -t a read lines into array readarray -t a < file.txt
mapfile -t a same as readarray mapfile -t a < file.txt

Why IFS= read -r matters

while IFS= read -r line; do
  echo "$line"
done < file.txt
  • IFS= prevents trimming/splitting
  • -r prevents backslash escaping
  • safest way to read raw lines

Safe iteration

for f in "${files[@]}"; do
  echo "$f"
done

Always quote "${arr[@]}".

Unquoted arrays can split on spaces unexpectedly.

Read file into array

readarray -t lines < file.txt

printf '%s\n' "${lines[@]}"

Common Internal Variables

Variable Meaning
$0 script name
$1..$9 positional args
$# arg count
"$@" safe all args
$* unsafe all args
$$ shell PID
$! last bg PID
$? last exit code
$PWD current dir
$OLDPWD previous dir
$RANDOM random int
$LINENO current line
${BASH_SOURCE[0]} script path

Notes

Always prefer:

"$@"

over:

$*

because it preserves argument boundaries safely.

Functions

Pattern Meaning Example
f() {} function f(){ echo ok; }
function f {} Bash alternative non-POSIX
$1 "$@" function args echo "$1"
return n function exit code return 42

Return vs output

This does not return a string:

return "hello"

return only supports numeric exit codes.

Use stdout instead:

get_name() {
  echo "Tien"
}

name="$(get_name)"

Conditionals & Comparisons

if / elif / else

if [[ $x -gt 10 ]]; then
  echo big
elif [[ $x -gt 5 ]]; then
  echo medium
else
  echo small
fi

case

Usually cleaner than long if-chains.

case "$cmd" in
  start) echo "Starting" ;;
  stop) echo "Stopping" ;;
  *) echo "Unknown" ;;
esac

Pattern matching

[[ "$file" == *.txt ]]

Regex

[[ "$x" =~ ^[0-9]+$ ]]

Regex works only inside [[ ]].

Numeric operators

Operator Meaning
-eq equal
-ne not equal
-gt greater
-lt less
-ge greater/equal
-le less/equal

Examples:

(( a > b ))

[ "$a" -gt "$b" ]

File Tests

Pattern Meaning
-f regular file
-d directory
-e exists
-s size > 0
-r -w -x read/write/exec
-n non-empty string
-z empty string

Example:

[[ -f "$path" ]]

Loops

Pattern Meaning Example
for x in ... iterate list for f in *.txt; do ...; done
while cmd loop while success while read -r l; do ...; done
until cmd loop until success until ping -c1 host; do :; done
break / continue flow control loop control

Example

for f in *.txt; do
  [[ -e "$f" ]] || continue
  echo "$f"
done

Always quote loop variables.

Avoid pipe subshell surprises

This may run the loop in a subshell:

cat file.txt | while read -r line; do
  count=$((count + 1))
done

Prefer:

while IFS= read -r line; do
  count=$((count + 1))
done < file.txt

Arithmetic

Pattern Meaning
((i++)) increment
((i+=2)) add
x=$((a+b)) compute

Example:

count=0

((count+=1))

echo "$count"

find

Command Meaning
find . -name "*.txt" find by name
find . -type f files only
find . -type d directories only
find . -maxdepth 2 limit recursion
find . -mtime -1 modified <1 day
find . -size +100M larger than 100 MB
find . -exec cmd {} \; run per file

Examples

find . -name "*.log"

find . -type f -exec rm -- {} \;

Safe filename handling

Prefer:

find . -print0 | xargs -0

This safely handles:

  • spaces
  • tabs
  • newlines in filenames

xargs

Command Meaning
xargs cmd stdin to args
xargs -0 null-safe mode
xargs -n1 one arg per cmd
xargs -P4 parallel jobs
xargs -I{} placeholder substitution
xargs bash -c run shell snippet

Parallel compression

find . -name "*.fastq" -print0 \
  | xargs -0 -P8 -I{} gzip "{}"

Runs 8 compression jobs in parallel.

Process many files safely

find data -name "*.txt" -print0 \
  | xargs -0 -P4 -I{} bash -c '
      wc -l "$1"
    ' _ {}

Why the _ {} pattern?

  • _ becomes $0 inside bash -c
  • {} becomes $1
  • "$1" keeps filenames safe

Everyday Text Tools

Command Use Example
grep search lines grep -R "TODO" .
sed stream edit sed 's/foo/bar/g' file
awk field processing awk '{print $1}' file
cut select columns cut -f1 sample.tsv
sort sort lines sort names.txt
uniq -c count repeats `sort x
column -t align table column -t file.tsv
head / tail inspect edges tail -f app.log

Useful one-liners

grep -R "pattern" .

awk -F '\t' '{print $1, $3}' file.tsv

sort ids.txt | uniq -c | sort -nr

column -t -s $'\t' file.tsv | less -S

Keep these simple. If parsing becomes complex, switch to Python.

Hidden Characters

Some files look normal but contain hidden characters.

The common one is Windows line ending:

\r\n

Linux expects:

\n

This can break scripts with errors like:

bad interpreter: /usr/bin/env: ‘bash\r’: No such file or directory

Check it:

cat -v script.sh | head

If you see ^M, the file has Windows carriage returns.

Lower-level check:

od -c script.sh | head

Look for:

\r \n

Fix it:

dos2unix script.sh

Or without dos2unix:

sed -i 's/\r$//' script.sh

Useful for scripts, config files, TSV files, manifests, and anything copied from Windows.

Archives & Transfers

Command Meaning
tar -czf out.tar.gz dir/ create gzip tarball
tar -xzf out.tar.gz extract gzip tarball
gzip file compress file
gunzip file.gz decompress file
rsync -av src/ dst/ sync directory
rsync -av --dry-run src/ dst/ preview sync

Inspect before extracting

List the archive first:

tar -tzf archive.tar.gz | less

Then extract into a dedicated directory:

mkdir -p extracted
tar -xzf archive.tar.gz -C extracted

-v only adds verbose output. Compression flags:

Archive Flag
.tar.gz z
.tar.xz J
.tar.bz2 j
.tar none

Undo extraction into the wrong directory

tar has no real undo. The archive can identify extracted paths, but it cannot restore files that were overwritten.

For a .tar.gz extracted into the current directory:

archive="archive.tar.gz"
target="$PWD"

# Remove extracted files and symlinks
tar -tzf "$archive" |
  sed 's#^\./##' |
  while IFS= read -r item; do
    case "$item" in
      ""|/*|../*|*/../*|*/..) continue ;;
    esac

    path="$target/$item"

    if [[ -f "$path" || -L "$path" ]]; then
      rm -v -- "$path"
    fi
  done

# Remove extracted directories only when empty
tar -tzf "$archive" |
  sed 's#^\./##' |
  awk '/\/$/ {
    sub(/\/$/, "")
    print length($0) "\t" $0
  }' |
  sort -rn |
  cut -f2- |
  while IFS= read -r item; do
    case "$item" in
      ""|/*|../*|*/../*|*/..) continue ;;
    esac

    rmdir "$target/$item" 2>/dev/null || true
  done

This preserves unrelated files and non-empty directories. Use tar -tJf for .tar.xz, or tar -tf for an uncompressed .tar.

Examples:

tar -czf results.tar.gz results/

rsync -av --progress data/ backup/data/

Use rsync when copying many files. It resumes better than plain cp.

curl

Pattern Meaning
curl URL print response
curl -L URL follow redirects
curl -o file URL save as file
curl -O URL save using remote filename
curl -fsSL URL fail quietly, show errors, follow redirects
curl -H 'Header: x' URL send header

Examples:

curl -fsSL "https://example.com/file.txt" -o file.txt

curl -H "Authorization: Bearer $TOKEN" "https://api.example.com/items"

Avoid piping remote scripts directly into bash unless you trust the source.

Date & Time

Command Meaning
date current date/time
date +%F YYYY-MM-DD
date +%Y%m%d_%H%M%S timestamp for filenames
sleep 5 wait 5 seconds
time cmd measure runtime

Example:

stamp="$(date +%Y%m%d_%H%M%S)"
log="run_${stamp}.log"

Job Control

Pattern Meaning
cmd & run in background
$! PID of last background job
wait wait for background jobs
jobs list shell jobs
kill PID stop process
nohup cmd & keep running after logout

Example:

long_job_1 &
pid1=$!

long_job_2 &
pid2=$!

wait "$pid1"
wait "$pid2"

For serious parallel work, prefer xargs -P, GNU Parallel, Nextflow, Snakemake, or a job scheduler.

Debugging

Pattern Meaning
bash -n script.sh syntax check only
bash -x script.sh trace execution
set -x enable trace
set +x disable trace
DEBUG=1 ./script.sh opt-in debug mode

Debug toggle:

DEBUG="${DEBUG:-0}"

if [[ "$DEBUG" == 1 ]]; then
  set -x
fi

Dry Run Pattern

Useful before destructive commands.

dry_run=false

run() {
  if "$dry_run"; then
    printf 'DRY RUN:' >&2
    printf ' %q' "$@" >&2
    printf '\n' >&2
  else
    "$@"
  fi
}

run rm -- "$file"

Then wire it to a flag:

case "${1:-}" in
  --dry-run) dry_run=true ;;
esac

Script Safety Flags

set -euo pipefail
Flag Meaning
-e exit on command failure
-u error on unset variables
pipefail pipeline fails if any command fails

Why pipefail matters

Without pipefail:

false | true

Pipeline exits successfully because the last command succeeded.

With:

set -o pipefail

the pipeline fails correctly.

Notes

  • set -e has edge cases and is not perfect error handling.
  • Commands inside if, while, &&, || may behave differently.
  • pipefail prevents hidden failures in pipelines.
  • Combine with traps for easier debugging.

Example:

set -Eeuo pipefail

trap 'echo "error at line $LINENO" >&2' ERR

Argument Parsing

Required and optional positional args

input="${1:?missing input file}"
output="${2:-out.txt}"

Meaning:

  • $1 is required
  • $2 is optional
  • default output is out.txt

Small flag parser

input=""
threads=1

while [[ $# -gt 0 ]]; do
  case "$1" in
    -i|--input)
      input="${2:?missing value for $1}"
      shift 2
      ;;
    -t|--threads)
      threads="${2:?missing value for $1}"
      shift 2
      ;;
    -h|--help)
      echo "usage: $0 -i input.txt [-t threads]"
      exit 0
      ;;
    *)
      echo "unknown argument: $1" >&2
      exit 1
      ;;
  esac
done

[[ -n "$input" ]] || {
  echo "missing --input" >&2
  exit 1
}

This is enough for many small scripts.

Logging & Errors

Log to stderr

log() {
  printf '[%s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*" >&2
}

Example:

log "starting job"
log "input: $input"

Why stderr?

Because stdout can stay clean for real output.

Exit with a message

die() {
  printf 'ERROR: %s\n' "$*" >&2
  exit 1
}

Example:

[[ -f "$file" ]] || die "file not found: $file"

Temporary Files & Cleanup

Bad:

tmp="/tmp/myfile.txt"

Better:

tmpdir="$(mktemp -d)"
trap 'rm -rf "$tmpdir"' EXIT

Then write inside it:

tmp="$tmpdir/work.txt"

This avoids collisions and stale temp files.

For longer scripts, use a cleanup function:

cleanup() {
  [[ -n "${tmpdir:-}" && -d "$tmpdir" ]] && rm -rf "$tmpdir"
}

trap cleanup EXIT

Check Required Commands

need_cmd() {
  command -v "$1" >/dev/null 2>&1 || {
    echo "missing command: $1" >&2
    exit 1
  }
}

need_cmd awk
need_cmd jq
need_cmd samtools

Useful when scripts run on different machines.

Practical Script Template

#!/usr/bin/env bash
set -Eeuo pipefail

log() {
  printf '[%s] %s\n' "$(date '+%Y-%m-%d %H:%M:%S')" "$*" >&2
}

die() {
  printf 'ERROR: %s\n' "$*" >&2
  exit 1
}

cleanup() {
  [[ -n "${tmpdir:-}" && -d "$tmpdir" ]] && rm -rf "$tmpdir"
}

trap cleanup EXIT
trap 'die "failed at line $LINENO"' ERR

tmpdir="$(mktemp -d)"

main() {
  [[ $# -gt 0 ]] || die "usage: $0 <input>"

  local input="$1"

  [[ -f "$input" ]] || die "not a file: $input"

  log "processing $input"

  # real work here
}

main "$@"

This is a good default for small production scripts.

ShellCheck

Run this on serious Bash scripts:

shellcheck script.sh

It catches many common bugs:

  • unquoted variables
  • unsafe loops
  • broken tests
  • unused variables
  • confusing redirects

Common Footguns

Unquoted variables

Bad:

rm $file

Good:

rm -- "$file"

The -- protects against filenames starting with -.

Parsing ls

Bad:

for f in $(ls *.txt); do
  echo "$f"
done

Good:

for f in *.txt; do
  [[ -e "$f" ]] || continue
  echo "$f"
done

Unsafe delete

Bad:

rm -rf "$dir/"

Safer:

[[ -n "$dir" && -d "$dir" ]] || die "bad dir: $dir"
rm -rf -- "$dir"

cd without checking

Bad:

cd "$dir"
rm *.tmp

Good:

cd "$dir" || die "cannot cd into $dir"
rm -- *.tmp

Too much Bash magic

Bash is good glue.

But if the script needs:

  • complex data structures
  • heavy parsing
  • many nested conditions
  • long business logic
  • serious testing

use Python, Go, or another real language instead.

Bash is best for orchestration, not application logic.