Procmail Quick Reference Guide

I'm not sure if this is useful to anyone. Except for small tangential remarks, all of this information is available straight from the manual. Thus, I might not spend a lot of time maintaining this. (Feel free to make changes to your own copy. Send me diffs. Thanks.) If there's sufficient interest, I may generate a PostScript rendering or something -- mail me your comments!

This Quick Reference is a companion document to the Procmail FAQ which is available from http://www.iki.fi/era/procmail/mini-faq.html and a number of mirrors. There is also a link collection with pointers to several good introductory documents and a whole slew of other interesting stuff.

This version should be reasonably accurate for version 3.11pre7. Bug fixes and new features in newer versions have not yet been incorporated, sorry. (Feel free to volunteer ...)

This Lynx-friendly rendering of the Quick Reference was kindly contributed by Holger Wahlen. Thanks!

My original tables rendering is not available on-line anymore.

Disclaimers and Assorted Remarks

A lot of Procmail's defaults can conveniently be changed with compile-time options. Your local default values may vary from what is listed here; your local documentation will contain the actual defaults used by your Procmail (unless someone foolishly recompiled the Procmail binary but didn't remake the manuals at the same time).

Sorely missing from this compilation are the options you can pass to Procmail (not sure if anybody knows what they do :-) and the signals. These are described in the manual.

While other parts of the manual are obviously duplicated here, you should not simply skip the manuals. At the risk of stating the obvious, some of the stuff in the manuals is only in the manuals, and some of it is important.

(Unfortunately, it turns out that some of the stuff here is not in the manuals. Perhaps this can change.)

Bet you thought this is where I was going to say, "if you lose mail, it's not my fault". Well, it isn't. And, this is copyrighted material and so on.

Top-level elements

Comments
# comment ...
Older versions are a bit picky about where they accept comments and whitespace. Never put a comment on the same line as a regular expression.
Assignments
Variable names are customarily upper case. Procmail's built-in pseudovariables are presented below.
VAR1=value VAR2="other value"
There can be many assignments on the same line. If the value contains spaces, you need to use single or double quotes.
VAR
The same as VAR= or VAR="". A lone token on a line is often a mistake; a good lint should warn you about this.
Recipes
recipes consist of three parts that are explained in detail below:
  1. Colon line (in its simplest form just :0)
  2. Conditions (optional -- leave out to do actions unconditionally)
  3. Action line
Note that there must be exactly one colon line for each action line, and exactly one action line for each colon line. If you want more than one action, group them under a set of braces.
Substitutions
Literal elements can be replaced with variable substitutions or command substitutions in most places.
$VAR
Substitute $VAR with the value of the variable VAR.
You can also say ${VAR} to make the variable's name unambiguous (i.e. ${VAR}iable expands to the value of VAR followed by the literal text "iable", whereas $VARiable looks for a variable called VARiable).
Notice that this will not happen in condition lines unless you include the $ modifier.
$\VAR
Substitute $\VAR with a version of the variable's value where all regular expression special characters have been backslash-escaped. This makes the interpolated value suitable for literal matching in condition lines.
Curiously, the substituted value will always begin with an empty pair of parentheses; this is to protect against problems with leading backslashes in the interpolated value. (See the FAQ for details.)
${VAR:-value}
${VAR-value}
Interpolation with default value: Substitute the expression with the value of VAR if it is set, otherwise with the literal replacement text value.
VAR is considered to be set if it has a non-null value with ${VAR:-value}, and merely if it has some value with just ${VAR-value}
The shell's other replacement substitution mechanisms, besides ${VAR:-value} and ${VAR:+value} and their colonless variants, are not understood by Procmail.
${VAR:+value}
${VAR+value}
Interpolation with alternate value: Substitute the expression with the literal replacement text value if VAR is set, otherwise substitute with nothing.
VAR is considered to be set if it has a non-null value with ${VAR:+value}, and merely if it has some value with just ${VAR+value}
The shell's other replacement substitution mechanisms, besides ${VAR:-value} and ${VAR:+value} and their colonless variants, are not understood by Procmail.
`cmd1 | cmd2`
Substitute `cmd1 | cmd2` with the output of this shell command pipeline, just like the same construct works in the shell.
Notice that these are not normal apostrophes (ASCII 39) but backticks (ASCII 96).

Colon line

The "colon line" starts every recipe. The first character is always a colon, and the second character is the number zero. (Other numbers are possible but this is only useful for backwards compatibility. See the FAQ if you're historically inclined.)

Locking

If you wonder about locking, the FAQ has an explanation for newbies.
:0 flags
No locking.
You don't need locking on forwarding recipes and often not on pipelines (except when they end up saving to a file).
:0 flags:
Let Procmail figure out lock file to use.
This should basically always be used when saving to a file, either directly or via a pipeline with >>
:0 flags:lockfile
Name a lock file to use. Alternatively, use the LOCKFILE pseudovariable to create a regional lock.
If you're a newbie, you probably don't need named locks.

Flags

Flags affect how the following recipe is executed, in various ways.

In the absence of any flags, Procmail's default is as if you had used the flags Hhb

A, a, B, b, c, D, E, e, f, H, h, i, r, W, w

Scope of matching

H
Condition lines examine the headers of the message (default).
B
Condition lines examine the body of the message (off by default).
Lone B turns off H; use HB to examine both headers and body.

Notice the difference between "scope of matching" and "scope of action."

Scope of action

h
Action line gets fed the headers of the message.
b
Action line gets fed the body of the message.
Default is both headers and body. Lone h turns off b and vice versa; use hb to explicitly turn on both.

Notice the difference between "scope of matching" and "scope of action."

Flow control

c
Clone message and execute the action(s) in a subprocess if the conditions match. The parent process continues with the original message after the clone process finishes.
A
Execute this recipe if the previous recipe's conditions were met.
a
Execute this recipe if the previous recipe's conditions were met and its action(s) were completed successfully.
E
Execute this recipe if the previous recipe's conditions were not met.
e
Execute this recipe if the previous recipe's conditions were met, but its action(s) couldn't be completed.

Execution mode

f
Feed the message to the pipeline on the action line if the conditions are met, and continue processing with the output of the pipeline (replacing the original message).
This is called "filter mode" in the Procmail documentation; some find this misleading, because everything Procmail does is "filtering". Here we are talking about a filter in the general Unix sense: think of this as if the mail message you are processing would be piped through whatever is on the action line before it actually gets piped on to Procmail (or at least the rest of your .procmailrc. This is of course only done if the condition(s) are satisified). In other words, run a program which transforms its input into something else, and use the results.
i
Suppress error checking when writing to a pipeline. This is typically used to get rid of SIGPIPE errors when the pipeline doesn't eat all of the input Procmail wants to feed it. (This is a FAQ.)
r
Raw mode: Don't do any "fixing" of the original message when writing it out (such as adding a final newline if the message didn't have one originally).
w
Wait for the program in the action line to finish before continuing. Otherwise, Procmail will spawn off the program and leave it executing on its own.
W
Like w, but additionally suppresses any "program failure" messages from the action pipeline.

Case sensitivity

Procmail normally matches without regard for case.
D
Pay attention to character case when matching: "a" is treated as distinct from "A" and so on. Some of the special macros are always matched case-insensitively.

Conditions

If you want to execute an action unconditionally, just leave out the condition lines completely:
	# Save all remaining messages to DEFAULT
	:0:
	$DEFAULT
Something like this is frequently seen as the last recipe in a complicated brace expression, or the last recipe in an rc file, to take care of all messages which haven't been filed already by some earlier recipe.

The following informal exposition fails to fully convey the essentially recursive definition of what is a condition. Basically, the available modifiers can be stacked to unlimited depths (although you would be hard pressed to find any practical need for more than one of each, at the most).

Tests with regular expressions

Details of procmail's regular expression syntax are presented below.
* regex
Test a part of the message (usually just the headers, in the absence of a B flag) against regex.
* variable ?? regex
Test the value of variable against regex. Various pseudovariables are listed below.

Test size of message part

* > number
True if message is bigger than number bytes.
* < number
True if message is smaller than number bytes.
This construct was broken in versions prior to 3.11pre4.

You can mix ?? with these to compare sizes of variables' values. (That's the number of bytes occupied by a variable's value, not arithmetic comparison of strings representing numbers. Use scoring for that.

Also note that this is not the number of lines in the message, it's a raw byte count. You can use scoring to count the number of lines, too.

Finally, note that < and > have a special meaning when used in scoring recipes.)

Test exit code of external program

* ? shell command (pipeline)
The program gets the current processed message as its standard input (but is of course free to ignore it); the condition is successful if the program returns a zero exit code. If the command is a pipeline with several commands in it, the exit code is whatever the last program in the pipeline returns (at least on systems worthy of the Unix name).
Messages printed to standard error by the pipeline are displayed in the log.

Negation

* ! condition
Succeeds if condition fails.

Substitution in conditions

* $ condition
Subject condition to variable and backtick substitution before actually evaluating the condition. In particular, this will resolve any references to variables ($VAR) which will otherwise be interpreted literally (because $ is a regular-expression character which you normally don't want Procmail to tamper with). Incidentally, quoted strings will also have their quotes stripped and backticks will be evaluated, too. (In other words, quotes have to be backslash-escaped and ordinarily backslash-escaped literal characters need to have their backslashes doubled.)
You can stack multiple $ flags to force multiple substitution passes.

Scoring

* x^y condition
Scoring: If the condition is true, add a number to the total score. x is the number to add on the first match; y influences by how much the score should be adjusted for subsequent matches. When y is zero, only add x the first time the condition matches. An empty condition always matches. The final score is in the $? pseudovariable and the action is taken if the final score is positive.
There are lots of variations which depend on the values of the parameters and the nature of the condition. See the procmailsc(5) manual page for details and examples.

Action line

If you want to do more than one action, you can't just stack them one after the other, you need multiple recipes (possibly unconditional, and/or grouped in a pair of braces) and a colon line (and optionally conditions, of course) for each.

Also note that flags which affect the action line are not actually taking effect until the action is actually attempted; in particular, a c flag doesn't generate a clone of the message until its conditions have all been met.

Forward to other address(es)

! addr1@site.net addr2@else.com addr3 ...
Most MTA:s should allow you to forward to more than one address. Try with or without commas between the addresses. (Sendmail eats both.)
The ! action is functionally equivalent to the pipeline | $SENDMAIL "$SENDMAILFLAGS"
! -t
This special case says to read the recipient data from the message's actual headers. Might not work with cheap Sendmail imitations.

Feed to a shell command pipeline

| cmd1 foo | cmd2 -opt >>file
The pipeline is expected to save the message somewhere; you can play with the recipe's flags to tell Procmail otherwise. With >> syntax, Procmail can figure out a lock file. (Always use locking when writing to a file.)
VAR=| cmd1 | cmd2 ...
The output of the command pipeline can be assigned to a variable. This makes the recipe non-delivering.
Notice that this syntax is only allowed on the action line, not in a plain assignment. (Use backticks with plain assigments to achieve the same effect.)

Save to a folder

/path/to/filename
Save to a plain file; if there is no path given, MAILDIR is used as the directory.
Always use locking when writing to a plain file (except /dev/null).
directory/.
Save to an MH folder. Note the trailing /.
You can list several MH folders at the same time. Only one file will actually be written, the rest will be hard links.
directory/
Save to a directory. The trailing slash is not strictly necessary.
You can list several directories at the same time. Only one file will actually be written, the rest will be hard links.

Compound recipe

Instead of a single action line, an entire block of recipes can be used when the condition matches:
	{
	    # ... more recipes
	}
The stuff between the braces can be any valid Procmail construct (as in, anything mentioned in "Top-level elements" above).

Note well that an action that is a variable assignment always has to go inside a set of braces:

	{ VAR=value }
Just VAR=value without the braces would save to a folder named VAR=value. (Yes, that is a valid file name under Unix.)

An empty set of braces is a valid action if you want a recipe to not do anything (for example, in the "then" part of an "if-then-else" series of recipes, perhaps) but you have to remember to put in at least one whitespace character (tab, space, newline) between the braces.


Regular expressions

Literal strings

Procmail's parsing of backslashes at beginning of line is hairy. The examples here simply avoid getting into a situation where a regular expression starts with a backslash.
<moo;foo/bar:baz&qu-ux!sna@fu>
The literal string <moo;foo/bar:baz&qu-ux!sna@fu>; note that none of the characters here need to be escaped in any fashion. (An exclamation mark needs to be disambiguated at the beginning of a line, though, to distinguish it from the negation operator.)
()\\ \[ \| \(
The literal string \ [ | ( . The empty parentheses are often necessary before a leading backslash in order to avoid ambiguities.
()\[Procmail\]
The literal string [Procmail]. This is just a corollary of the previous example. Literal round parentheses and vertical bars also need to be escaped.
(moo )
The literal string moo followed by a literal space. Trailing whitespace is usually ignored, so you have to do something like this to explicitly tell Procmail you want a literal space.
Control characters
Procmail doesn't know about \t and \n and other such Perlisms. You can safely embed any sort of bare control character other than newline in Procmail recipes, though.

Anchoring

Without anchoring, Procmail doesn't look at the context of the match. For instance, [abcd]* matches the string "zzzz" (because "zzzz" does indeed contain "zero or more" of, well, anything), but ^[abcd]*$ doesn't. With the anchors, you get to say the matching line mustn't contain anything else besides zero or more of "a", "b", "c", and/or "d".
^
Beginning of line.
$
End of line.
^^
Beginning or end of string (i.e., of the search space). To say the full header of a message mustn't contain anything except a From: header line, for example, you'd say ^^From:.*^^.

Alternation

bar|moo
Either "bar" or "moo". Obviously you could have more complicated regular expressions on either side of the alternation bar.
[abcd]
Any of the characters inside the brackets. This is called a character class.
[]()[]
Tricky example: A character class naming any one of the characters ] ( ) [. Note that you don't need backslash escapes inside a character class.
[^abcd]
Complement a class: This matches any one character except (newline or) a, b, c, or d
[a-z]
Character range: this matches any one character in the class consisting of all the characters between a and z, inclusive.
Recall that Procmail is normally not paying attention to character case, so this matches A through Z, too (unless you use the D flag to prevent that).
[^][^()-]
Trick example: character class which matches any one character except close square bracket (ASCII code 93), open square bracket (ASCII code 91), caret (^), round parentheses, or minus (-).
If you want to include a literal minus in a character class, it has to be the first character in that class (after any ^ class negation operator) or the last character in that class.
If you want to include a literal closing bracket in a character class, it has to be the first character in that class (after any ^ class negation operator).
[class]
Trick example: This is identical to the class [acls], i.e. it matches any one of the characters inside the brackets, but only once (regardless of how many times you put it in the brackets).
This is often hard for newbies.

Wildcards

.
Any single character, except newline.

Repetition

Notably absent is the {m,n} construct found in many modern extended regular expression implementations.
o*
Zero or more o:s (i.e., "" or "o" or "oo" or "ooo" or "oooo" or ...)
o+
One or more o:s (i.e., "o" or "oo" or "ooo" or "oooo" or...)
o?
Zero or one o:s (i.e., "" or "o")
[abcd]*
Any string with zero or more a:s, b:s, c:s, and/or d:s (i.e. "", "ab", "baaa", "caddadac",...)
(bar|moo)*
Zero or more of "bar" or "moo" (i.e. "", "moo", "bar", "moomoo", "moobar", "barmoo", "moomoomoobar",...)
(bar|moo)+
One or more of "bar" or "moo" (i.e. "moo", "bar", "moomoo", "moobar", "barmoo", "moomoomoobar",...)

Special Procmail constructs

\<
A shorthand for the character class [^a-zA-Z0-9_] except it can also match newlines.
This is an incompatible imitation of the "word end boundary" operator found in some extended regular expression implementations. For example, note that \< and \> are actually identical.
\>
A shorthand for the character class [^a-zA-Z0-9_] except it can also match newlines.
This is an incompatible imitation of the "word end boundary" operator found in some extended regular expression implementations. For example, note that \< and \> are actually identical.
\/
Start grabbing stuff into MATCH. This is a substitute for real backreferences as found in many extended regular expression implementations. Procmail allows for exactly one backref that remembers everything between \/ and the end of the match; don't try to put more than one of these in a recipe.
The \/ operator changes Procmail's matching behavior after \/ from stingy to greedy. See the FAQ for details.

Macros

Procmail has some handy built-in shorthands which expand into complicated regular expressions. (These are not true macros, they expand only when used as part of a regular expression.)
^TO
This covers most headers which can carry your address in them, such as To:, Apparently-To:, Cc:, Resent-To:, etc. You should perhaps use ^TO_ instead.
Note that there is no corresponding ^FROM macro.
^TO_
This is a better bounded version of ^TO which unfortunately doesn't exist in older versions of Procmail (it was introduced in version 3.11pre4). Unless backwards compatibility is an issue, you should probably prefer ^TO_ over ^TO.
Note that there is no corresponding ^FROM_ macro.
^FROM_MAILER
This should cover most messages from mailer programs (bounces etc.).
^FROM_DAEMON
This is an expanded version of ^FROM_MAILER which covers a broader range of machine-generated messages, notably including most mailing lists and other (benign) bulk e-mail.

Variables

Curiously, even those special pseudovariables which are primarily useful for their associated side effects (LOG, SHIFT, etc) can be referenced, so you can see what the last value you assigned to them was. This is only marginally useful for most of us, but comes as a pleasant surprise in situations where this is actually useful.

$#, $$, $-, $1 etc., $=, $?, $@, $_, B, COMSAT, DEFAULT, DELIVERED, DROPPRIVS, EXITCODE, H, HOST, INCLUDERC, LASTFOLDER, LINEBUF, LOCKEXT, LOCKFILE, LOCKSLEEP, LOCKTIMEOUT, LOG, LOGABSTRACT, LOGFILE, MAILDIR, MATCH, MSGPREFIX, NORESRETRY, ORGMAIL, SENDMAIL, SENDMAILFLAGS, SHELL, SHELLFLAGS, SHELLMETAS, SHIFT, SUSPEND, TIMEOUT, TRAP, UMASK, VERBOSE

Mailboxes

MAILDIR
The directory in which Procmail will be executing and where output files will be written unless you specify an explicit pathname.
In other words, assigning a directory to MAILDIR causes Procmail to switch its current directory. (Assigning a nonexistent directory will produce an error message in the log and Procmail will stay wherever it already was. The default MAILDIR is the value of $HOME.)
MSGPREFIX
Base file name used when writing a message to a directory (in a new file, one per message). The default is msg. (including the trailing dot; you get files called stuff like msg.HND where HND is a more or less random string tacked on to make a unique file name).
MH files have their own naming convention.
DEFAULT
The default mailbox where mail will be written unless some recipe directs Procmail to save it elsewhere. (You shouldn't normally need to fiddle with this.)
ORGMAIL
If DEFAULT cannot be written to, Procmail tries the file name this variable points to as a last resort. (You shouldn't fiddle with this unless you think you know what you are doing.)

Programs to use

Procmail has reasonable defaults written in at compile time. Mostly these shouldn't need to be changed.
SHELL
Which shell to use when spawning external programs. This should always explicitly be set to point to a sh-compatible shell. If your login shell is a csh derivative, you need to set this to /bin/sh. (SHELLMETAS determines when exactly a shell is thought to be needed.)
SHELLFLAGS
Options used when starting a shell.
SENDMAIL
Where to find Sendmail on this system. (Usually not to be fiddled with.)
SENDMAILFLAGS
Options used when starting a sendmail.

System interaction

UMASK
What mode to give created files. man umask for details.
SHELLMETAS
If a shell pipeline contains any character in SHELLMETAS, a shell is used to run the pipeline, on the theory that the command is too complicated for Procmail to parse. If the command line doesn't really need to get parsed by the shell, temporarily set SHELLMETAS to something else (but do save the old value) before the recipe.
TRAP
A shell snippet to execute after delivery.
EXITCODE
What return code to pass back to the calling program. Sendmail, for example, pays a lot of attention to the exit code. This can be used to control, e.g., what kind of bounce message exactly Sendmail generates when a message can't be delivered.
The exit code of a program called up by Procmail is in $?

Logging

LOGFILE
Where Procmail should write its log. If this is unset, logging messages go to standard error (and are neither saved anywhere nor seen by anyone, except when you run Procmail interactively).
LOG
Assigning something to this variable writes out the assigned value to the LOGFILE.
Often, you want to include a final newline in the string you log:
	LOG="Moo!
"
(Yes, that's LOG=, open quote, text, newline, closing quote.)
VERBOSE
Set to yes to get very detailed logging.
LOGABSTRACT
Which events exactly get logged. Set to all to have every delivery logged. This is slightly broken in 3.11pre7.
COMSAT
Controls what kind of comsat(8) notices Procmail generates, if any.

Procmail's state

HOST
Which host we're on. Set to a bogus value to terminate the present toplevel .rc file immediately (meaning if you are in a file INCLUDERC=ed from .procmailrc, both the current file and .procmailrc are skipped. If there are other file name arguments passed to Procmail on the command line, those will still be executed).
DELIVERED
If the current message was already delivered, this is set to yes. You can set it to yes to make Procmail pretend it was.
LASTFOLDER
Where the current message was last saved (folder or command).
MATCH
Whatever the latest \/ operator grabbed.
$=
Whatever the result of the latest scoring recipe was.
$1, $2, ...; $@; $#
Procmail's command-line arguments, like in the shell: $1 is the first command-line argument, etc.; $@ contains all arguments, and $# contains the number of arguments.
See also the SHIFT pseudovariable.
$$
The current Procmail's PID. This is the same as in the shell.
$?
The exit code of the previous shell command. This is the same as in the shell.
$_
The name of the current rcfile.
$-
An alias for LASTFOLDER
Note that $= and $@ can't be used directly; you have to assign the value to another variable before it can be used for anything useful.

Message contents

H
The headers of the processed message.
B
The body of the processed message.
These are primarily useful for constrained conditions such as H ?? regex or B > number when the recipe has a H or B flag that restricts conditions to the other part of the message. Use HB or BH to examine both headers and body.

Locking

LOCKFILE
Assigning to this variable creates a regional lock file which endures until LOCKFILE is assigned to again (just LOCKFILE assigns the empty string and removes any lock).
LOCKEXT
The file name extension used when creating a lock file. All the programs which have to share a lock obviously need to agree on what extension to use. The default, .lock, is usually fine and shouldn't be changed unless you have special needs.
LOCKSLEEP
When Procmail wants to lock a file which is already locked, it goes to sleep and wakes up and tries again after LOCKSLEEP seconds.
LOCKTIMEOUT
How long Procmail must wait in vain before it is allowed to decide the existing lock on a file must have been stale, and proceed to break the lock.

Error handling

TIMEOUT
How long to wait for a child before giving up on it.
NORESRETRY
How many times to wait for a serious resource shortage condition to, uh, just go away. Default is try four times before bouncing a message as undeliverable.
SUSPEND
How long to wait between NORESRETRY retries. Default is 16 seconds.

Miscellaneous

LINEBUF
How long recipe lines Procmail is prepared to cope with. If you need to process really big regular expressions or grab lots of data into MATCH, you'll need to increase this.
Note that exceeding LINEBUF can make Procmail crash. (The manual blandly states that "behavior is undefined" but we all know what that means in practice!)
SHIFT
You can shift down Procmail's command line arguments (much like the shell builtin shift) by assigning a positive number to this pseudovariable.
INCLUDERC
Assigning a file name to this variable will cause Procmail to load that file and start processing the recipes in it, like a subroutine. When the included file is processed, Procmail continues in the present file where it left off.
DROPPRIVS
When Procmail is executing as setuid or setgid, setting this to yes will make it drop all its special privileges.

Random notes

The plural of "recipe" is "recipes". Both forms have only one occurrence of the letter i.

Valid HTML 4.0!

$Id: quickref.html,v 1.2 2009/01/07 12:04:03 gsutter Exp $