Google Style Guides

Shell Style Guide

Authored, revised and maintained by many Googlers.

Table of Contents

Section Contents
Background Which Shell to Use - When to use Shell
Shell Files and Interpreter Invocation File Extensions - SUID/SGID
Environment STDOUT vs STDERR
Comments File Header - Function Comments - Implementation Comments - TODO Comments
Formatting Indentation - Line Length and Long Strings - Pipelines - Control Flow - Case statement - Variable expansion - Quoting
Features and Bugs ShellCheck - Command Substitution - Test, [… ], and [[… ]] - Testing Strings - Wildcard Expansion of Filenames - Eval - Arrays - Pipes to While - Arithmetic - Aliases
Naming Conventions Function Names - Variable Names - Constants and Environment Variable Names - Source Filenames - Use Local Variables - Function Location - main
Calling Commands Checking Return Values - Builtin Commands vs. External Commands
When in Doubt: Be Consistent  

Background

Which Shell to Use

Bash is the only shell scripting language permitted for executables.

Executables must start with #!/bin/bash and minimal flags. Use set to set shell options so that calling your script as bash script_name does not break its functionality.

Restricting all executable shell scripts to bash gives us a consistent shell language that’s installed on all our machines. In particular, this means there is generally no need to strive for POSIX-compatibility or otherwise avoid “bashisms”.

The only exception to the above is where you’re forced to by whatever you’re coding for. For example some legacy operating systems or constrained execution environments may require plain Bourne shell for certain scripts.

When to use Shell

Shell should only be used for small utilities or simple wrapper scripts.

While shell scripting isn’t a development language, it is used for writing various utility scripts throughout Google. This style guide is more a recognition of its use rather than a suggestion that it be used for widespread deployment.

Some guidelines:

Shell Files and Interpreter Invocation

File Extensions

Executables should have a .sh extension or no extension.

Libraries must have a .sh extension and should not be executable.

SUID/SGID

SUID and SGID are forbidden on shell scripts.

There are too many security issues with shell that make it nearly impossible to secure sufficiently to allow SUID/SGID. While bash does make it difficult to run SUID, it’s still possible on some platforms which is why we’re being explicit about banning it.

Use sudo to provide elevated access if you need it.

Environment

STDOUT vs STDERR

All error messages should go to STDERR.

This makes it easier to separate normal status from actual issues.

A function to print out error messages along with other status information is recommended.

err() {
  echo "[$(date +'%Y-%m-%dT%H:%M:%S%z')]: $*" >&2
}

if ! do_something; then
  err "Unable to do_something"
  exit 1
fi

Comments

File Header

Start each file with a description of its contents.

Every file must have a top-level comment including a brief overview of its contents. A copyright notice and author information are optional.

Example:

#!/bin/bash
#
# Perform hot backups of Oracle databases.

Function Comments

Any function that is not both obvious and short must have a function header comment. Any function in a library must have a function header comment regardless of length or complexity.

It should be possible for someone else to learn how to use your program or to use a function in your library by reading the comments (and self-help, if provided) without reading the code.

All function header comments should describe the intended API behaviour using:

Example:

#######################################
# Cleanup files from the backup directory.
# Globals:
#   BACKUP_DIR
#   ORACLE_SID
# Arguments:
#   None
#######################################
function cleanup() {
  …
}

#######################################
# Get configuration directory.
# Globals:
#   SOMEDIR
# Arguments:
#   None
# Outputs:
#   Writes location to stdout
#######################################
function get_dir() {
  echo "${SOMEDIR}"
}

#######################################
# Delete a file in a sophisticated manner.
# Arguments:
#   File to delete, a path.
# Returns:
#   0 if thing was deleted, non-zero on error.
#######################################
function del_thing() {
  rm "$1"
}

Implementation Comments

Comment tricky, non-obvious, interesting or important parts of your code.

This follows general Google coding comment practice. Don’t comment everything. If there’s a complex algorithm or you’re doing something out of the ordinary, put a short comment in.

TODO Comments

Use TODO comments for code that is temporary, a short-term solution, or good-enough but not perfect.

This matches the convention in the C++ Guide.

TODOs should include the string TODO in all caps, followed by the name, e-mail address, or other identifier of the person with the best context about the problem referenced by the TODO. The main purpose is to have a consistent TODO that can be searched to find out how to get more details upon request. A TODO is not a commitment that the person referenced will fix the problem. Thus when you create a TODO, it is almost always your name that is given.

Examples:

# TODO(mrmonkey): Handle the unlikely edge cases (bug ####)

Formatting

While you should follow the style that’s already there for files that you’re modifying, the following are required for any new code.

Indentation

Indent 2 spaces. No tabs.

Use blank lines between blocks to improve readability. Indentation is two spaces. Whatever you do, don’t use tabs. For existing files, stay faithful to the existing indentation.

Exception: The only exception for using tabs is for the body of <<- tab-indented here-document.

Line Length and Long Strings

Maximum line length is 80 characters.

If you have to write literal strings that are longer than 80 characters, this should be done with a here document or an embedded newline if possible.

Words that are longer than 80 chars and can’t sensibly be split are ok, but where possible these items should be on a line of their own, or factored into a variable. Examples include file paths and URLs, particularly where string-matching them (such as grep) is valuable for maintenance.

# DO use 'here document's
cat <<END
I am an exceptionally long
string.
END

# Embedded newlines are ok too
long_string="I am an exceptionally
long string."

long_file="/i/am/an/exceptionally/loooooooooooooooooooooooooooooooooooooooooooooooooooong_file"

long_string_with_long_file="i am including an exceptionally \
/very/long/file\
 in this long string."

# Long file converted into a shorter variable name with cleaner line breaking.
long_string_alt="i am an including an exceptionally ${long_file} in this long\
 string"
# Just because a line contains an exception doesn't mean the rest of the
# line shouldn't be wrapped like usual.

bad_long_string_with_long_file="i am including an exceptionally /very/long/file in this long string."

Pipelines

Pipelines should be split one per line if they don’t all fit on one line.

If a pipeline all fits on one line, it should be on one line.

If not, it should be split at one pipe segment per line with the pipe on the newline and a 2 space indent for the next section of the pipe. \ should be consistently used to indicate line continuation. This applies to a chain of commands combined using | as well as to logical compounds using || and &&.

# All fits on one line
command1 | command2

# Long commands
command1 \
  | command2 \
  | command3 \
  | command4

This helps readability when distinguishing a pipeline from a regular long command continuation, particularly if the line is using both.

Comments will need to precede the whole pipeline. If the comment and pipeline are large and complex, then it is worth considering moving low level details of them aside by using a helper function.

Control Flow

Put ; then and ; do on the same line as the if, for, or while.

Control flow statements in shell are a bit different, but we follow the same principles as with braces when declaring functions. That is: ; then and ; do should be on the same line as the if/for/while/until/select. else should be on its own line and closing statements (fi and done) should be on their own line vertically aligned with the opening statement.

Example:

# If inside a function remember to declare the loop variable as
# a local to avoid it leaking into the global environment:
local dir
for dir in "${dirs_to_cleanup[@]}"; do
  if [[ -d "${dir}/${SESSION_ID}" ]]; then
    log_date "Cleaning up old files in ${dir}/${SESSION_ID}"
    rm "${dir}/${SESSION_ID}/"* || error_message
  else
    mkdir -p "${dir}/${SESSION_ID}" || error_message
  fi
done

Although it is possible to omit in "$@" in for loops we recommend consistently including it for clarity.

for arg in "$@"; do
  echo "argument: ${arg}"
done

Case statement

The matching expressions are indented one level from the case and esac. Multiline actions are indented another level. In general, there is no need to quote match expressions. Pattern expressions should not be preceded by an open parenthesis. Avoid the ;& and ;;& notations.

case "${expression}" in
  a)
    variable="…"
    some_command "${variable}" "${other_expr}" …
    ;;
  absolute)
    actions="relative"
    another_command "${actions}" "${other_expr}" …
    ;;
  *)
    error "Unexpected expression '${expression}'"
    ;;
esac

Simple commands may be put on the same line as the pattern and ;; as long as the expression remains readable. This is often appropriate for single-letter option processing. When the actions don’t fit on a single line, put the pattern on a line on its own, then the actions, then ;; also on a line of its own. When on the same line as the actions, use a space after the close parenthesis of the pattern and another before the ;;.

verbose='false'
aflag=''
bflag=''
files=''
while getopts 'abf:v' flag; do
  case "${flag}" in
    a) aflag='true' ;;
    b) bflag='true' ;;
    f) files="${OPTARG}" ;;
    v) verbose='true' ;;
    *) error "Unexpected option ${flag}" ;;
  esac
done

Variable expansion

In order of precedence: Stay consistent with what you find; quote your variables; prefer "${var}" over "$var".

These are strongly recommended guidelines but not mandatory regulation. Nonetheless, the fact that it’s a recommendation and not mandatory doesn’t mean it should be taken lightly or downplayed.

They are listed in order of precedence.

Prefer brace-delimiting all other variables.

# Section of *recommended* cases.

# Preferred style for 'special' variables:
echo "Positional: $1" "$5" "$3"
echo "Specials: !=$!, -=$-, _=$_. ?=$?, #=$# *=$* @=$@ \$=$$ …"

# Braces necessary:
echo "many parameters: ${10}"

# Braces avoiding confusion:
# Output is "a0b0c0"
set -- a b c
echo "${1}0${2}0${3}0"

# Preferred style for other variables:
echo "PATH=${PATH}, PWD=${PWD}, mine=${some_var}"
while read -r f; do
  echo "file=${f}"
done < <(find /tmp)
# Section of *discouraged* cases

# Unquoted vars, unbraced vars, brace-delimited single letter
# shell specials.
echo a=$avar "b=$bvar" "PID=${$}" "${1}"

# Confusing use: this is expanded as "${1}0${2}0${3}0",
# not "${10}${20}${30}
set -- a b c
echo "$10$20$30"

NOTE: Using braces in ${var} is not a form of quoting. “Double quotes” must be used as well.

Quoting

# 'Single' quotes indicate that no substitution is desired.
# "Double" quotes indicate that substitution is required/tolerated.

# Simple examples

# "quote command substitutions"
# Note that quotes nested inside "$()" don't need escaping.
flag="$(some_command and its args "$@" 'quoted separately')"

# "quote variables"
echo "${flag}"

# Use arrays with quoted expansion for lists.
declare -a FLAGS
FLAGS=( --foo --bar='baz' )
readonly FLAGS
mybinary "${FLAGS[@]}"

# It's ok to not quote internal integer variables.
if (( $# > 3 )); then
  echo "ppid=${PPID}"
fi

# "never quote literal integers"
value=32
# "quote command substitutions", even when you expect integers
number="$(generate_number)"

# "prefer quoting words", not compulsory
readonly USE_INTEGER='true'

# "quote shell meta characters"
echo 'Hello stranger, and well met. Earn lots of $$$'
echo "Process $$: Done making \$\$\$."

# "command options or path names"
# ($1 is assumed to contain a value here)
grep -li Hugo /dev/null "$1"

# Less simple examples
# "quote variables, unless proven false": ccs might be empty
git send-email --to "${reviewers}" ${ccs:+"--cc" "${ccs}"}

# Positional parameter precautions: $1 might be unset
# Single quotes leave regex as-is.
grep -cP '([Ss]pecial|\|?characters*)$' ${1:+"$1"}

# For passing on arguments,
# "$@" is right almost every time, and
# $* is wrong almost every time:
#
# * $* and $@ will split on spaces, clobbering up arguments
#   that contain spaces and dropping empty strings;
# * "$@" will retain arguments as-is, so no args
#   provided will result in no args being passed on;
#   This is in most cases what you want to use for passing
#   on arguments.
# * "$*" expands to one argument, with all args joined
#   by (usually) spaces,
#   so no args provided will result in one empty string
#   being passed on.
#
# Consult
# https://www.gnu.org/software/bash/manual/html_node/Special-Parameters.html and
# https://mywiki.wooledge.org/BashGuide/Arrays for more

(set -- 1 "2 two" "3 three tres"; echo $#; set -- "$*"; echo "$#, $@")
(set -- 1 "2 two" "3 three tres"; echo $#; set -- "$@"; echo "$#, $@")

Features and Bugs

ShellCheck

The ShellCheck project identifies common bugs and warnings for your shell scripts. It is recommended for all scripts, large or small.

Command Substitution

Use $(command) instead of backticks.

Nested backticks require escaping the inner ones with \ . The $(command) format doesn’t change when nested and is easier to read.

Example:

# This is preferred:
var="$(command