How Aho, Weinberger and Kernighan Created AWK

AWK appeared in 1977 at Bell Labs, and its name is simply the initials of its three authors: Alfred Aho, Peter Weinberger and Brian Kernighan. The story of how Aho, Weinberger and Kernighan created AWK is a very Unix story: three researchers wanted a small language for the many one-off jobs of slicing, counting and reformatting text that were too fiddly for sed and grep and too small to justify a C program.

How Aho, Weinberger and Kernighan created AWK

Each author brought something. Aho had written egrep and worked on the theory of regular expressions and parsing. Weinberger had a database background. Kernighan was interested in tools that made programming easier for non-specialists. The goal was a language where a useful program could be one or two lines long.

They settled on a model in which the program is a list of pattern-action pairs. AWK reads input one record (usually one line) at a time, splits it into fields, and runs each action whose pattern matches. The first version shipped with Version 7 Unix in 1979. A substantially enlarged version, sometimes called "new awk" or nawk, added user-defined functions, multiple input streams and computed regular expressions around 1985. In 1988 the three authors published The AWK Programming Language, and a second edition followed in 2023.

Design and key features

• Implicit input loop: no need to open files or read lines yourself.
• Automatic field splitting: $1, $2 and so on hold the fields, $0 the whole line, NF the field count, NR the record number.
• Regular expressions as first-class patterns, for example /error/ { print }.
• Associative arrays: arrays indexed by strings, years before most mainstream languages had hash tables built in.
• Typeless values: a value is treated as a number or a string depending on context.
• BEGIN and END blocks that run before the first and after the last record.
• C-like syntax for expressions, printf and control flow.

A short AWK example

A one-liner such as awk -F: '$3 >= 1000 { print $1 }' /etc/passwd prints the user names of regular accounts. Here is a small script that totals bytes per HTTP status code in a web server access log in common log format:

# usage: awk -f status.awk access.log
{ bytes[$9] += $10 }
END {
    for (code in bytes)
        printf "%s %d\n", code, bytes[code]
}

The whole program is the associative array bytes, keyed by the status code in field 9.

Implementations: one true awk, gawk, mawk and busybox

AWK is also specified by POSIX, so a script that sticks to the standard runs on any compliant Unix-like system.

Where AWK is used

AWK lives inside shell scripts, Makefiles, build systems and cron jobs. System administrators use it to summarise logs, extract columns from command output (ps, df, ls -l), reformat CSV-like data and generate reports. Bioinformaticians use it for large tab-separated files because it streams data and starts instantly. The authors have often said they were surprised to see people write programs thousands of lines long in a language designed for one-liners.

Influence and legacy

Larry Wall created Perl in 1987 partly to go beyond what AWK and sed could do, and Perl's associative arrays, regex-heavy style and $ variables show the debt clearly. AWK also influenced Tcl, Lua and the AMPL modelling language. Aho went on to share the 2020 Turing Award with Jeffrey Ullman for work on compilers and algorithms, and Kernighan co-wrote The C Programming Language with Dennis Ritchie.

Is AWK still used today?

Every day. It is installed by default on Linux, macOS and the BSDs, it is in POSIX, and it remains the quickest way to answer "sum column 5 of this file" from a terminal. The tale of how Aho, Weinberger and Kernighan created AWK, a tiny pattern-action language meant for one-liners, explains why it has outlasted many bigger scripting languages: it does one job and does it with almost no ceremony.

Frequently asked questions

What does AWK stand for?

AWK stands for the surnames of its creators: Alfred Aho, Peter Weinberger and Brian Kernighan. It is not an acronym for a phrase, and the language is usually written in capitals while the command is awk.

What is the difference between awk, nawk and gawk?

The original 1977 awk was small. nawk is the expanded Bell Labs version from the mid-1980s that added functions and other features, and it is the basis of POSIX awk. gawk is the GNU implementation, compatible with POSIX but with many extensions such as network I/O and loadable modules.

Should I learn AWK or Python for text processing?

They solve different sizes of problem. AWK is ideal for short, streaming jobs on columnar text right in the shell. For anything that needs libraries, complex data structures or long-term maintenance, Python is usually the better choice, and many people use both.