6.4 Getting Mail

Reading mail with a newsreader—isn’t that just plain WeIrD? But of course.

6.4.1 Mail in a Newsreader

If you are used to traditional mail readers, but have decided to switch to reading mail with Gnus, you may find yourself experiencing something of a culture shock.

Gnus does not behave like traditional mail readers. If you want to make it behave that way, you can, but it’s an uphill battle.

Gnus, by default, handles all its groups using the same approach. This approach is very newsreaderly—you enter a group, see the new/unread messages, and when you read the messages, they get marked as read, and you don’t see them any more. (Unless you explicitly ask for them.)

In particular, you do not do anything explicitly to delete messages.

Does this mean that all the messages that have been marked as read are deleted? How awful!

But, no, it means that old messages are expired according to some scheme or other. For news messages, the expire process is controlled by the news administrator; for mail, the expire process is controlled by you. The expire process for mail is covered in depth in Expiring Mail.

What many Gnus users find, after using it a while for both news and mail, is that the transport mechanism has very little to do with how they want to treat a message.

Many people subscribe to several mailing lists. These are transported via SMTP, and are therefore mail. But we might go for weeks without answering, or even reading these messages very carefully. We may not need to save them because if we should need to read one again, they are archived somewhere else.

Some people have local news groups which have only a handful of readers. These are transported via NNTP, and are therefore news. But we may need to read and answer a large fraction of the messages very carefully in order to do our work. And there may not be an archive, so we may need to save the interesting messages the same way we would personal mail.

The important distinction turns out to be not the transport mechanism, but other factors such as how interested we are in the subject matter, or how easy it is to retrieve the message if we need to read it again.

Gnus provides many options for sorting mail into “groups” which behave like newsgroups, and for treating each group (whether mail or news) differently.

Some users never get comfortable using the Gnus (ahem) paradigm and wish that Gnus should grow up and be a male, er, mail reader. It is possible to whip Gnus into a more mailreaderly being, but, as said before, it’s not easy. People who prefer proper mail readers should try VM instead, which is an excellent, and proper, mail reader.

I don’t mean to scare anybody off, but I want to make it clear that you may be required to learn a new way of thinking about messages. After you’ve been subjected to The Gnus Way, you will come to love it. I can guarantee it. (At least the guy who sold me the Emacs Subliminal Brain-Washing Functions that I’ve put into Gnus did guarantee it. You Will Be Assimilated. You Love Gnus. You Love The Gnus Mail Way. You Do.)

6.4.2 Getting Started Reading Mail

It’s quite easy to use Gnus to read your new mail. You just plonk the mail back end of your choice into gnus-secondary-select-methods, and things will happen automatically.

For instance, if you want to use nnml (which is a “one file per mail” back end), you could put the following in your ‘~/.gnus.el’ file:

(setq gnus-secondary-select-methods '((nnml "")))

Now, the next time you start Gnus, this back end will be queried for new articles, and it will move all the messages in your spool file to its directory, which is ‘~/Mail/’ by default. The new group that will be created (‘mail.misc’) will be subscribed, and you can read it like any other group.

You will probably want to split the mail into several groups, though:

(setq nnmail-split-methods
      '(("junk" "^From:.*Lars Ingebrigtsen")
        ("crazy" "^Subject:.*die\\|^Organization:.*flabby")
        ("other" "")))

This will result in three new nnml mail groups being created: ‘nnml:junk’, ‘nnml:crazy’, and ‘nnml:other’. All the mail that doesn’t fit into the first two groups will be placed in the last group.

This should be sufficient for reading mail with Gnus. You might want to give the other sections in this part of the manual a perusal, though. Especially see section Choosing a Mail Back End and see section Expiring Mail.

6.4.3 Splitting Mail

The nnmail-split-methods variable says how the incoming mail is to be split into groups.

(setq nnmail-split-methods
  '(("mail.junk" "^From:.*Lars Ingebrigtsen")
    ("mail.crazy" "^Subject:.*die\\|^Organization:.*flabby")
    ("mail.other" "")))

This variable is a list of lists, where the first element of each of these lists is the name of the mail group (they do not have to be called something beginning with ‘mail’, by the way), and the second element is a regular expression used on the header of each mail to determine if it belongs in this mail group. The first string may contain ‘\\1’ forms, like the ones used by replace-match to insert sub-expressions from the matched text. For instance:

("list.\\1" "From:.* \\(.*\\)-list@majordomo.com")

In that case, nnmail-split-lowercase-expanded controls whether the inserted text should be made lowercase. See section Fancy Mail Splitting.

The second element can also be a function. In that case, it will be called narrowed to the headers with the first element of the rule as the argument. It should return a non-nil value if it thinks that the mail belongs in that group.

The last of these groups should always be a general one, and the regular expression should always be ‘""’ so that it matches any mails that haven’t been matched by any of the other regexps. (These rules are processed from the beginning of the alist toward the end. The first rule to make a match will “win”, unless you have crossposting enabled. In that case, all matching rules will “win”.) If no rule matched, the mail will end up in the ‘bogus’ group. When new groups are created by splitting mail, you may want to run gnus-group-find-new-groups to see the new groups. This also applies to the ‘bogus’ group.

If you like to tinker with this yourself, you can set this variable to a function of your choice. This function will be called without any arguments in a buffer narrowed to the headers of an incoming mail message. The function should return a list of group names that it thinks should carry this mail message.

This variable can also be a fancy split method. For the syntax, see Fancy Mail Splitting.

Note that the mail back ends are free to maul the poor, innocent, incoming headers all they want to. They all add Lines headers; some add X-Gnus-Group headers; most rename the Unix mbox From<SPACE> line to something else.

The mail back ends all support cross-posting. If several regexps match, the mail will be “cross-posted” to all those groups. nnmail-crosspost says whether to use this mechanism or not. Note that no articles are crossposted to the general (‘""’) group.

nnmh and nnml makes crossposts by creating hard links to the crossposted articles. However, not all file systems support hard links. If that’s the case for you, set nnmail-crosspost-link-function to copy-file. (This variable is add-name-to-file by default.)

If you wish to see where the previous mail split put the messages, you can use the M-x nnmail-split-history command. If you wish to see where re-spooling messages would put the messages, you can use gnus-summary-respool-trace and related commands (see section Mail Group Commands).

Header lines longer than the value of nnmail-split-header-length-limit are excluded from the split function.

By default, splitting does not decode headers, so you can not match on non-ASCII strings. But it is useful if you want to match articles based on the raw header data. To enable it, set the nnmail-mail-splitting-decodes variable to a non-nil value. In addition, the value of the nnmail-mail-splitting-charset variable is used for decoding non-MIME encoded string when nnmail-mail-splitting-decodes is non-nil. The default value is nil which means not to decode non-MIME encoded string. A suitable value for you will be undecided or be the charset used normally in mails you are interested in.

By default, splitting is performed on all incoming messages. If you specify a directory entry for the variable mail-sources (see section Mail Source Specifiers), however, then splitting does not happen by default. You can set the variable nnmail-resplit-incoming to a non-nil value to make splitting happen even in this case. (This variable has no effect on other kinds of entries.)

Gnus gives you all the opportunity you could possibly want for shooting yourself in the foot. Let’s say you create a group that will contain all the mail you get from your boss. And then you accidentally unsubscribe from the group. Gnus will still put all the mail from your boss in the unsubscribed group, and so, when your boss mails you “Have that report ready by Monday or you’re fired!”, you’ll never see it and, come Tuesday, you’ll still believe that you’re gainfully employed while you really should be out collecting empty bottles to save up for next month’s rent money.

6.4.4 Mail Sources

Mail can be gotten from many different sources—the mail spool, from a POP mail server, from a procmail directory, or from a maildir, for instance.

6.4.4.1 Mail Source Specifiers

You tell Gnus how to fetch mail by setting mail-sources (see section Fetching Mail) to a mail source specifier.

Here’s an example:

(pop :server "pop3.mailserver.com" :user "myname")

As can be observed, a mail source specifier is a list where the first element is a mail source type, followed by an arbitrary number of keywords. Keywords that are not explicitly specified are given default values.

The mail-sources is global for all mail groups. You can specify an additional mail source for a particular group by including the group mail specifier in mail-sources, and setting a mail-source group parameter (see section Group Parameters) specifying a single mail source. When this is used, mail-sources is typically just (group); the mail-source parameter for a group might look like this:

(mail-source . (file :path "home/user/spools/foo.spool"))

This means that the group’s (and only this group’s) messages will be fetched from the spool file ‘/user/spools/foo.spool’.

The following mail source types are available:

file

Get mail from a single file; typically from the mail spool.

Keywords:

:path

The file name. Defaults to the value of the MAIL environment variable or the value of rmail-spool-directory (usually something like ‘/usr/mail/spool/user-name’).

:prescript

:postscript

Script run before/after fetching mail.

An example file mail source:

(file :path "/usr/spool/mail/user-name")

Or using the default file name:

(file)

If the mail spool file is not located on the local machine, it’s best to use POP or IMAP or the like to fetch the mail. You can not use ange-ftp file names here—it has no way to lock the mail spool while moving the mail.

If it’s impossible to set up a proper server, you can use ssh instead.

(setq mail-sources '((file :prescript "ssh host bin/getmail >%t")))

The ‘getmail’ script would look something like the following:

#!/bin/sh # getmail - move mail from spool to stdout # flu@iki.fi MOVEMAIL=/usr/lib/emacs/20.3/i386-redhat-linux/movemail TMP=$HOME/Mail/tmp rm -f $TMP; $MOVEMAIL $MAIL $TMP >/dev/null && cat $TMP

Alter this script to fit the ‘movemail’ and temporary file you want to use.

directory

Get mail from several files in a directory. This is typically used when you have procmail split the incoming mail into several files. That is, there is a one-to-one correspondence between files in that directory and groups, so that mail from the file ‘foo.bar.spool’ will be put in the group foo.bar. (You can change the suffix to be used instead of .spool.) Setting nnmail-scan-directory-mail-source-once to non-nil forces Gnus to scan the mail source only once. This is particularly useful if you want to scan mail groups at a specified level.

There is also the variable nnmail-resplit-incoming, if you set that to a non-nil value, then the normal splitting process is applied to all the files from the directory, Splitting Mail.

Keywords:

:path

The name of the directory where the files are. There is no default value.

:suffix

Only files ending with this suffix are used. The default is ‘.spool’.

:predicate

Only files that have this predicate return non-nil are returned. The default is identity. This is used as an additional filter—only files that have the right suffix and satisfy this predicate are considered.

:prescript

:postscript

Script run before/after fetching mail.

An example directory mail source:

(directory :path "/home/user-name/procmail-dir/" :suffix ".prcml")

pop

Get mail from a POP server.

Keywords:

:server

The name of the POP server. The default is taken from the MAILHOST environment variable.

:port

The port number of the POP server. This can be a number (e.g., ‘:port 1234’) or a string (e.g., ‘:port "pop3"’). If it is a string, it should be a service name as listed in ‘/etc/services’ on Unix systems. The default is ‘"pop3"’. On some systems you might need to specify it as ‘"pop-3"’ instead.

:user

The user name to give to the POP server. The default is the login name.

:password

The password to give to the POP server. If not specified, the user is prompted.

:program

The program to use to fetch mail from the POP server. This should be a format-like string. Here’s an example:

fetchmail %u@%s -P %p %t

The valid format specifier characters are:

‘t’

The name of the file the mail is to be moved to. This must always be included in this string.

‘s’

The name of the server.

‘P’

The port number of the server.

‘u’

The user name to use.

‘p’

The password to use.

The values used for these specs are taken from the values you give the corresponding keywords.

:prescript

A script to be run before fetching the mail. The syntax is the same as the :program keyword. This can also be a function to be run.

One popular way to use this is to set up an SSH tunnel to access the POP server. Here’s an example:

(pop :server "127.0.0.1" :port 1234 :user "foo" :password "secret" :prescript "nohup ssh -f -L 1234:pop.server:110 remote.host sleep 3600 &")

:postscript

A script to be run after fetching the mail. The syntax is the same as the :program keyword. This can also be a function to be run.

:function

The function to use to fetch mail from the POP server. The function is called with one parameter—the name of the file where the mail should be moved to.

:authentication

This can be either the symbol password or the symbol apop and says what authentication scheme to use. The default is password.

:leave

Non-nil if the mail is to be left on the POP server after fetching. Only the built-in pop3-movemail program (the default) supports this keyword.

If this is a number, leave mails on the server for this many days since you first checked new mails. In that case, mails once fetched will never be fetched again by the UIDL control. If this is nil (the default), mails will be deleted on the server right after fetching. If this is neither nil nor a number, all mails will be left on the server, and you will end up getting the same mails again and again.

The pop3-uidl-file variable specifies the file to which the UIDL data are locally stored. The default value is ‘~/.pop3-uidl’.

Note that POP servers maintain no state information between sessions, so what the client believes is there and what is actually there may not match up. If they do not, then you may get duplicate mails or the whole thing can fall apart and leave you with a corrupt mailbox.

If the :program and :function keywords aren’t specified, pop3-movemail will be used.

Here are some examples for getting mail from a POP server.

Fetch from the default POP server, using the default user name, and default fetcher:

(pop)

Fetch from a named server with a named user and password:

(pop :server "my.pop.server" :user "user-name" :password "secret")

Leave mails on the server for 14 days:

(pop :server "my.pop.server" :user "user-name" :password "secret" :leave 14)

Use ‘movemail’ to move the mail:

(pop :program "movemail po:%u %t %p")

maildir

Get mail from a maildir. This is a type of mailbox that is supported by at least qmail and postfix, where each file in a special directory contains exactly one mail.

Keywords:

:path

The name of the directory where the mails are stored. The default is taken from the MAILDIR environment variable or ‘~/Maildir/’.

:subdirs

The subdirectories of the Maildir. The default is ‘("new" "cur")’.

You can also get mails from remote hosts (because maildirs don’t suffer from locking problems).

Two example maildir mail sources:

(maildir :path "/home/user-name/Maildir/" :subdirs ("cur" "new"))

(maildir :path "/user@remotehost.org:~/Maildir/" :subdirs ("new"))

imap

Get mail from a IMAP server. If you don’t want to use IMAP as intended, as a network mail reading protocol (i.e., with nnimap), for some reason or other, Gnus let you treat it similar to a POP server and fetches articles from a given IMAP mailbox. See section Using IMAP, for more information.

Keywords:

:server

The name of the IMAP server. The default is taken from the MAILHOST environment variable.

:port

The port number of the IMAP server. The default is ‘143’, or ‘993’ for TLS/SSL connections.

:user

The user name to give to the IMAP server. The default is the login name.

:password

The password to give to the IMAP server. If not specified, the user is prompted.

:stream

What stream to use for connecting to the server, this is one of the symbols in imap-stream-alist. Right now, this means ‘gssapi’, ‘kerberos4’, ‘starttls’, ‘tls’, ‘ssl’, ‘shell’ or the default ‘network’.

:authentication

Which authenticator to use for authenticating to the server, this is one of the symbols in imap-authenticator-alist. Right now, this means ‘gssapi’, ‘kerberos4’, ‘digest-md5’, ‘cram-md5’, ‘anonymous’ or the default ‘login’.

:program

When using the ‘shell’ :stream, the contents of this variable is mapped into the imap-shell-program variable. This should be a format-like string (or list of strings). Here’s an example:

ssh %s imapd

Make sure nothing is interfering with the output of the program, e.g., don’t forget to redirect the error output to the void. The valid format specifier characters are:

‘s’

The name of the server.

‘l’

User name from imap-default-user.

‘p’

The port number of the server.

The values used for these specs are taken from the values you give the corresponding keywords.

:mailbox

The name of the mailbox to get mail from. The default is ‘INBOX’ which normally is the mailbox which receives incoming mail.

:predicate

The predicate used to find articles to fetch. The default, ‘UNSEEN UNDELETED’, is probably the best choice for most people, but if you sometimes peek in your mailbox with a IMAP client and mark some articles as read (or; SEEN) you might want to set this to ‘1:*’. Then all articles in the mailbox is fetched, no matter what. For a complete list of predicates, see RFC 2060 section 6.4.4.

:fetchflag

How to flag fetched articles on the server, the default ‘\Deleted’ will mark them as deleted, an alternative would be ‘\Seen’ which would simply mark them as read. These are the two most likely choices, but more flags are defined in RFC 2060 section 2.3.2.

:dontexpunge

If non-nil, don’t remove all articles marked as deleted in the mailbox after finishing the fetch.

An example IMAP mail source:

(imap :server "mail.mycorp.com" :stream kerberos4 :fetchflag "\\Seen")

group

Get the actual mail source from the mail-source group parameter, See section Group Parameters.

Common Keywords

Common keywords can be used in any type of mail source.

Keywords:

:plugged

If non-nil, fetch the mail even when Gnus is unplugged. If you use directory source to get mail, you can specify it as in this example:

(setq mail-sources '((directory :path "/home/pavel/.Spool/" :suffix "" :plugged t)))

Gnus will then fetch your mail even when you are unplugged. This is useful when you use local mail and news.

6.4.4.2 Function Interface

Some of the above keywords specify a Lisp function to be executed. For each keyword :foo, the Lisp variable foo is bound to the value of the keyword while the function is executing. For example, consider the following mail-source setting:

(setq mail-sources '((pop :user "jrl"
                          :server "pophost" :function fetchfunc)))

While the function fetchfunc is executing, the symbol user is bound to "jrl", and the symbol server is bound to "pophost". The symbols port, password, program, prescript, postscript, function, and authentication are also bound (to their default values).

See above for a list of keywords for each type of mail source.

6.4.4.3 Mail Source Customization

The following is a list of variables that influence how the mail is fetched. You would normally not need to set or change any of these variables.

mail-source-crash-box

File where mail will be stored while processing it. The default is
‘~/.emacs-mail-crash-box’.

mail-source-delete-incoming

If non-nil, delete incoming files after handling them. If t, delete the files immediately, if nil, never delete any files. If a positive number, delete files older than number of days (the deletion will only happen when receiving new mail). You may also set mail-source-delete-incoming to nil and call mail-source-delete-old-incoming from a hook or interactively. mail-source-delete-incoming defaults to 10 in alpha Gnusae and 2 in released Gnusae. See section Gnus Development.

mail-source-delete-old-incoming-confirm

If non-nil, ask for confirmation before deleting old incoming files. This variable only applies when mail-source-delete-incoming is a positive number.

mail-source-ignore-errors

If non-nil, ignore errors when reading mail from a mail source.

mail-source-directory

Directory where incoming mail source files (if any) will be stored. The default is ‘~/Mail/’. At present, the only thing this is used for is to say where the incoming files will be stored if the variable mail-source-delete-incoming is nil or a number.

mail-source-incoming-file-prefix

Prefix for file name for storing incoming mail. The default is ‘Incoming’, in which case files will end up with names like ‘Incoming30630D_’ or ‘Incoming298602ZD’. This is really only relevant if mail-source-delete-incoming is nil or a number.

mail-source-default-file-modes

All new mail files will get this file mode. The default is #o600.

mail-source-movemail-program

If non-nil, name of program for fetching new mail. If nil, movemail in exec-directory.

6.4.4.4 Fetching Mail

The way to actually tell Gnus where to get new mail from is to set mail-sources to a list of mail source specifiers (see section Mail Source Specifiers).

If this variable is nil, the mail back ends will never attempt to fetch mail by themselves.

If you want to fetch mail both from your local spool as well as a POP mail server, you’d say something like:

(setq mail-sources
      '((file)
        (pop :server "pop3.mail.server"
             :password "secret")))

Or, if you don’t want to use any of the keyword defaults:

(setq mail-sources
      '((file :path "/var/spool/mail/user-name")
        (pop :server "pop3.mail.server"
             :user "user-name"
             :port "pop3"
             :password "secret")))

When you use a mail back end, Gnus will slurp all your mail from your inbox and plonk it down in your home directory. Gnus doesn’t move any mail if you’re not using a mail back end—you have to do a lot of magic invocations first. At the time when you have finished drawing the pentagram, lightened the candles, and sacrificed the goat, you really shouldn’t be too surprised when Gnus moves your mail.

6.4.5 Mail Back End Variables

These variables are (for the most part) pertinent to all the various mail back ends.

nnmail-read-incoming-hook

The mail back ends all call this hook after reading new mail. You can use this hook to notify any mail watch programs, if you want to.

nnmail-split-hook

Hook run in the buffer where the mail headers of each message is kept just before the splitting based on these headers is done. The hook is free to modify the buffer contents in any way it sees fit—the buffer is discarded after the splitting has been done, and no changes performed in the buffer will show up in any files. gnus-article-decode-encoded-words is one likely function to add to this hook.

nnmail-pre-get-new-mail-hook

nnmail-post-get-new-mail-hook

These are two useful hooks executed when treating new incoming mail—nnmail-pre-get-new-mail-hook (is called just before starting to handle the new mail) and nnmail-post-get-new-mail-hook (is called when the mail handling is done). Here’s and example of using these two hooks to change the default file modes the new mail files get:

(add-hook 'nnmail-pre-get-new-mail-hook (lambda () (set-default-file-modes #o700))) (add-hook 'nnmail-post-get-new-mail-hook (lambda () (set-default-file-modes #o775)))

nnmail-use-long-file-names

If non-nil, the mail back ends will use long file and directory names. Groups like ‘mail.misc’ will end up in directories (assuming use of nnml back end) or files (assuming use of nnfolder back end) like ‘mail.misc’. If it is nil, the same group will end up in ‘mail/misc’.

nnmail-delete-file-function

Function called to delete files. It is delete-file by default.

nnmail-cache-accepted-message-ids

If non-nil, put the Message-IDs of articles imported into the back end (via Gcc, for instance) into the mail duplication discovery cache. The default is nil.

nnmail-cache-ignore-groups

This can be a regular expression or a list of regular expressions. Group names that match any of the regular expressions will never be recorded in the Message-ID cache.

This can be useful, for example, when using Fancy Splitting (see section Fancy Mail Splitting) together with the function nnmail-split-fancy-with-parent.

6.4.6 Fancy Mail Splitting

If the rather simple, standard method for specifying how to split mail doesn’t allow you to do what you want, you can set nnmail-split-methods to nnmail-split-fancy. Then you can play with the nnmail-split-fancy variable.

Let’s look at an example value of this variable first:

;; Messages from the mailer daemon are not crossposted to any of
;; the ordinary groups.  Warnings are put in a separate group
;; from real errors.
(| ("from" mail (| ("subject" "warn.*" "mail.warning")
                   "mail.misc"))
   ;; Non-error messages are crossposted to all relevant
   ;; groups, but we don't crosspost between the group for the
   ;; (ding) list and the group for other (ding) related mail.
   (& (| (any "ding@ifi\\.uio\\.no" "ding.list")
         ("subject" "ding" "ding.misc"))
      ;; Other mailing lists…
      (any "procmail@informatik\\.rwth-aachen\\.de" "procmail.list")
      (any "SmartList@informatik\\.rwth-aachen\\.de" "SmartList.list")
      ;; Both lists below have the same suffix, so prevent
      ;; cross-posting to mkpkg.list of messages posted only to
      ;; the bugs- list, but allow cross-posting when the
      ;; message was really cross-posted.
      (any "bugs-mypackage@somewhere" "mypkg.bugs")
      (any "mypackage@somewhere" - "bugs-mypackage" "mypkg.list")
      ;; People…
      (any "larsi@ifi\\.uio\\.no" "people.Lars_Magne_Ingebrigtsen"))
   ;; Unmatched mail goes to the catch all group.
   "misc.misc")

This variable has the format of a split. A split is a (possibly) recursive structure where each split may contain other splits. Here are the possible split syntaxes:

group

If the split is a string, that will be taken as a group name. Normal regexp match expansion will be done. See below for examples.

(field value [- restrict […] ] split [invert-partial])

The split can be a list containing at least three elements. If the first element field (a regexp matching a header) contains value (also a regexp) then store the message as specified by split.

If restrict (yet another regexp) matches some string after field and before the end of the matched value, the split is ignored. If none of the restrict clauses match, split is processed.

The last element invert-partial is optional. If it is non-nil, the match-partial-words behavior controlled by the variable nnmail-split-fancy-match-partial-words (see below) is be inverted. (New in Gnus 5.10.7)

(| split …)

If the split is a list, and the first element is | (vertical bar), then process each split until one of them matches. A split is said to match if it will cause the mail message to be stored in one or more groups.

(& split …)

If the split is a list, and the first element is &, then process all splits in the list.

junk

If the split is the symbol junk, then don’t save (i.e., delete) this message. Use with extreme caution.

(: function arg1 arg2 …)

If the split is a list, and the first element is ‘:’, then the second element will be called as a function with args given as arguments. The function should return a split.

For instance, the following function could be used to split based on the body of the messages:

(defun split-on-body () (save-excursion (save-restriction (widen) (goto-char (point-min)) (when (re-search-forward "Some.*string" nil t) "string.group"))))

The buffer is narrowed to the header of the message in question when function is run. That’s why (widen) needs to be called after save-excursion and save-restriction in the example above. Also note that with the nnimap backend, message bodies will not be downloaded by default. You need to set nnimap-split-download-body to t to do that (see section Client-Side IMAP Splitting).

(! func split)

If the split is a list, and the first element is !, then split will be processed, and func will be called as a function with the result of split as argument. func should return a split.

nil

If the split is nil, it is ignored.

In these splits, field must match a complete field name.

Normally, value in these splits must match a complete word according to the fundamental mode syntax table. In other words, all value’s will be implicitly surrounded by \<...\> markers, which are word delimiters. Therefore, if you use the following split, for example,

(any "joe" "joemail")

messages sent from ‘joedavis@foo.org’ will normally not be filed in ‘joemail’. If you want to alter this behavior, you can use any of the following three ways:

1. You can set the nnmail-split-fancy-match-partial-words variable to non-nil in order to ignore word boundaries and instead the match becomes more like a grep. This variable controls whether partial words are matched during fancy splitting. The default value is nil.

   Note that it influences all value’s in your split rules.

2. value beginning with .* ignores word boundaries in front of a word. Similarly, if value ends with .*, word boundaries in the rear of a word will be ignored. For example, the value "@example\\.com" does not match ‘foo@example.com’ but ".*@example\\.com" does.
3. You can set the invert-partial flag in your split rules of the ‘(field value …)’ types, aforementioned in this section. If the flag is set, word boundaries on both sides of a word are ignored even if nnmail-split-fancy-match-partial-words is nil. Contrarily, if the flag is set, word boundaries are not ignored even if nnmail-split-fancy-match-partial-words is non-nil. (New in Gnus 5.10.7)

field and value can also be Lisp symbols, in that case they are expanded as specified by the variable nnmail-split-abbrev-alist. This is an alist of cons cells, where the CAR of a cell contains the key, and the CDR contains the associated value. Predefined entries in nnmail-split-abbrev-alist include:

from

Matches the ‘From’, ‘Sender’ and ‘Resent-From’ fields.

to

Matches the ‘To’, ‘Cc’, ‘Apparently-To’, ‘Resent-To’ and ‘Resent-Cc’ fields.

any

Is the union of the from and to entries.

nnmail-split-fancy-syntax-table is the syntax table in effect when all this splitting is performed.

If you want to have Gnus create groups dynamically based on some information in the headers (i.e., do replace-match-like substitutions in the group names), you can say things like:

(any "debian-\\b\\(\\w+\\)@lists.debian.org" "mail.debian.\\1")

In this example, messages sent to ‘debian-foo@lists.debian.org’ will be filed in ‘mail.debian.foo’.

If the string contains the element ‘\\&’, then the previously matched string will be substituted. Similarly, the elements ‘\\1’ up to ‘\\9’ will be substituted with the text matched by the groupings 1 through 9.

Where nnmail-split-lowercase-expanded controls whether the lowercase of the matched string should be used for the substitution. Setting it as non-nil is useful to avoid the creation of multiple groups when users send to an address using different case (i.e., mailing-list@domain vs Mailing-List@Domain). The default value is t.

nnmail-split-fancy-with-parent is a function which allows you to split followups into the same groups their parents are in. Sometimes you can’t make splitting rules for all your mail. For example, your boss might send you personal mail regarding different projects you are working on, and as you can’t tell your boss to put a distinguishing string into the subject line, you have to resort to manually moving the messages into the right group. With this function, you only have to do it once per thread.

To use this feature, you have to set nnmail-treat-duplicates and nnmail-cache-accepted-message-ids to a non-nil value. And then you can include nnmail-split-fancy-with-parent using the colon feature, like so:

(setq nnmail-treat-duplicates 'warn     ; or delete
      nnmail-cache-accepted-message-ids t
      nnmail-split-fancy
      '(| (: nnmail-split-fancy-with-parent)
          ;; other splits go here
        ))

This feature works as follows: when nnmail-treat-duplicates is non-nil, Gnus records the message id of every message it sees in the file specified by the variable nnmail-message-id-cache-file, together with the group it is in (the group is omitted for non-mail messages). When mail splitting is invoked, the function nnmail-split-fancy-with-parent then looks at the References (and In-Reply-To) header of each message to split and searches the file specified by nnmail-message-id-cache-file for the message ids. When it has found a parent, it returns the corresponding group name unless the group name matches the regexp nnmail-split-fancy-with-parent-ignore-groups. It is recommended that you set nnmail-message-id-cache-length to a somewhat higher number than the default so that the message ids are still in the cache. (A value of 5000 appears to create a file some 300 kBytes in size.) When nnmail-cache-accepted-message-ids is non-nil, Gnus also records the message ids of moved articles, so that the followup messages goes into the new group.

Also see the variable nnmail-cache-ignore-groups if you don’t want certain groups to be recorded in the cache. For example, if all outgoing messages are written to an “outgoing” group, you could set nnmail-cache-ignore-groups to match that group name. Otherwise, answers to all your messages would end up in the “outgoing” group.

6.4.7 Group Mail Splitting

If you subscribe to dozens of mailing lists but you don’t want to maintain mail splitting rules manually, group mail splitting is for you. You just have to set to-list and/or to-address in group parameters or group customization and set nnmail-split-methods to gnus-group-split. This splitting function will scan all groups for those parameters and split mail accordingly, i.e., messages posted from or to the addresses specified in the parameters to-list or to-address of a mail group will be stored in that group.

Sometimes, mailing lists have multiple addresses, and you may want mail splitting to recognize them all: just set the extra-aliases group parameter to the list of additional addresses and it’s done. If you’d rather use a regular expression, set split-regexp.

All these parameters in a group will be used to create an nnmail-split-fancy split, in which the field is ‘any’, the value is a single regular expression that matches to-list, to-address, all of extra-aliases and all matches of split-regexp, and the split is the name of the group. restricts are also supported: just set the split-exclude parameter to a list of regular expressions.

If you can’t get the right split to be generated using all these parameters, or you just need something fancier, you can set the parameter split-spec to an nnmail-split-fancy split. In this case, all other aforementioned parameters will be ignored by gnus-group-split. In particular, split-spec may be set to nil, in which case the group will be ignored by gnus-group-split.

gnus-group-split will do cross-posting on all groups that match, by defining a single & fancy split containing one split for each group. If a message doesn’t match any split, it will be stored in the group named in gnus-group-split-default-catch-all-group, unless some group has split-spec set to catch-all, in which case that group is used as the catch-all group. Even though this variable is often used just to name a group, it may also be set to an arbitrarily complex fancy split (after all, a group name is a fancy split), and this may be useful to split mail that doesn’t go to any mailing list to personal mail folders. Note that this fancy split is added as the last element of a | split list that also contains a & split with the rules extracted from group parameters.

It’s time for an example. Assume the following group parameters have been defined:

nnml:mail.bar:
((to-address . "bar@femail.com")
 (split-regexp . ".*@femail\\.com"))
nnml:mail.foo:
((to-list . "foo@nowhere.gov")
 (extra-aliases "foo@localhost" "foo-redist@home")
 (split-exclude "bugs-foo" "rambling-foo")
 (admin-address . "foo-request@nowhere.gov"))
nnml:mail.others:
((split-spec . catch-all))

Setting nnmail-split-methods to gnus-group-split will behave as if nnmail-split-fancy had been selected and variable nnmail-split-fancy had been set as follows:

(| (& (any "\\(bar@femail\\.com\\|.*@femail\\.com\\)" "mail.bar")
      (any "\\(foo@nowhere\\.gov\\|foo@localhost\\|foo-redist@home\\)"
           - "bugs-foo" - "rambling-foo" "mail.foo"))
   "mail.others")

If you’d rather not use group splitting for all your mail groups, you may use it for only some of them, by using nnmail-split-fancy splits like this:

(: gnus-group-split-fancy groups no-crosspost catch-all)

groups may be a regular expression or a list of group names whose parameters will be scanned to generate the output split. no-crosspost can be used to disable cross-posting; in this case, a single | split will be output. catch-all is the fall back fancy split, used like gnus-group-split-default-catch-all-group. If catch-all is nil, or if split-regexp matches the empty string in any selected group, no catch-all split will be issued. Otherwise, if some group has split-spec set to catch-all, this group will override the value of the catch-all argument.

Unfortunately, scanning all groups and their parameters can be quite slow, especially considering that it has to be done for every message. But don’t despair! The function gnus-group-split-setup can be used to enable gnus-group-split in a much more efficient way. It sets nnmail-split-methods to nnmail-split-fancy and sets nnmail-split-fancy to the split produced by gnus-group-split-fancy. Thus, the group parameters are only scanned once, no matter how many messages are split.

However, if you change group parameters, you’d have to update nnmail-split-fancy manually. You can do it by running gnus-group-split-update. If you’d rather have it updated automatically, just tell gnus-group-split-setup to do it for you. For example, add to your ‘~/.gnus.el’:

(gnus-group-split-setup auto-update catch-all)

If auto-update is non-nil, gnus-group-split-update will be added to nnmail-pre-get-new-mail-hook, so you won’t ever have to worry about updating nnmail-split-fancy again. If you don’t omit catch-all (it’s optional, equivalent to nil), gnus-group-split-default-catch-all-group will be set to its value.

Because you may want to change nnmail-split-fancy after it is set by gnus-group-split-update, this function will run gnus-group-split-updated-hook just before finishing.

6.4.8 Incorporating Old Mail

Most people have lots of old mail stored in various file formats. If you have set up Gnus to read mail using one of the spiffy Gnus mail back ends, you’ll probably wish to have that old mail incorporated into your mail groups.

Doing so can be quite easy.

To take an example: You’re reading mail using nnml (see section Mail Spool), and have set nnmail-split-methods to a satisfactory value (see section Splitting Mail). You have an old Unix mbox file filled with important, but old, mail. You want to move it into your nnml groups.

Here’s how:

1. Go to the group buffer.
2. Type G f and give the file name to the mbox file when prompted to create an nndoc group from the mbox file (see section Foreign Groups).
3. Type SPACE to enter the newly created group.
4. Type M P b to process-mark all articles in this group’s buffer (see section Setting Process Marks).
5. Type B r to respool all the process-marked articles, and answer ‘nnml’ when prompted (see section Mail Group Commands).

All the mail messages in the mbox file will now also be spread out over all your nnml groups. Try entering them and check whether things have gone without a glitch. If things look ok, you may consider deleting the mbox file, but I wouldn’t do that unless I was absolutely sure that all the mail has ended up where it should be.

Respooling is also a handy thing to do if you’re switching from one mail back end to another. Just respool all the mail in the old mail groups using the new mail back end.

6.4.9 Expiring Mail

Traditional mail readers have a tendency to remove mail articles when you mark them as read, in some way. Gnus takes a fundamentally different approach to mail reading.

Gnus basically considers mail just to be news that has been received in a rather peculiar manner. It does not think that it has the power to actually change the mail, or delete any mail messages. If you enter a mail group, and mark articles as “read”, or kill them in some other fashion, the mail articles will still exist on the system. I repeat: Gnus will not delete your old, read mail. Unless you ask it to, of course.

To make Gnus get rid of your unwanted mail, you have to mark the articles as expirable. (With the default key bindings, this means that you have to type E.) This does not mean that the articles will disappear right away, however. In general, a mail article will be deleted from your system if, 1) it is marked as expirable, AND 2) it is more than one week old. If you do not mark an article as expirable, it will remain on your system until hell freezes over. This bears repeating one more time, with some spurious capitalizations: IF you do NOT mark articles as EXPIRABLE, Gnus will NEVER delete those ARTICLES.

You do not have to mark articles as expirable by hand. Gnus provides two features, called “auto-expire” and “total-expire”, that can help you with this. In a nutshell, “auto-expire” means that Gnus hits E for you when you select an article. And “total-expire” means that Gnus considers all articles as expirable that are read. So, in addition to the articles marked ‘E’, also the articles marked ‘r’, ‘R’, ‘O’, ‘K’, ‘Y’ (and so on) are considered expirable. gnus-auto-expirable-marks has the full list of these marks.

When should either auto-expire or total-expire be used? Most people who are subscribed to mailing lists split each list into its own group and then turn on auto-expire or total-expire for those groups. (See section Splitting Mail, for more information on splitting each list into its own group.)

Which one is better, auto-expire or total-expire? It’s not easy to answer. Generally speaking, auto-expire is probably faster. Another advantage of auto-expire is that you get more marks to work with: for the articles that are supposed to stick around, you can still choose between tick and dormant and read marks. But with total-expire, you only have dormant and ticked to choose from. The advantage of total-expire is that it works well with adaptive scoring (see section Adaptive Scoring). Auto-expire works with normal scoring but not with adaptive scoring.

Groups that match the regular expression gnus-auto-expirable-newsgroups will have all articles that you read marked as expirable automatically. All articles marked as expirable have an ‘E’ in the first column in the summary buffer.

By default, if you have auto expiry switched on, Gnus will mark all the articles you read as expirable, no matter if they were read or unread before. To avoid having articles marked as read marked as expirable automatically, you can put something like the following in your ‘~/.gnus.el’ file:

(remove-hook 'gnus-mark-article-hook
             'gnus-summary-mark-read-and-unread-as-read)
(add-hook 'gnus-mark-article-hook 'gnus-summary-mark-unread-as-read)

Note that making a group auto-expirable doesn’t mean that all read articles are expired—only the articles marked as expirable will be expired. Also note that using the d command won’t make articles expirable—only semi-automatic marking of articles as read will mark the articles as expirable in auto-expirable groups.

Let’s say you subscribe to a couple of mailing lists, and you want the articles you have read to disappear after a while:

(setq gnus-auto-expirable-newsgroups
      "mail.nonsense-list\\|mail.nice-list")

Another way to have auto-expiry happen is to have the element auto-expire in the group parameters of the group.

If you use adaptive scoring (see section Adaptive Scoring) and auto-expiring, you’ll have problems. Auto-expiring and adaptive scoring don’t really mix very well.

The nnmail-expiry-wait variable supplies the default time an expirable article has to live. Gnus starts counting days from when the message arrived, not from when it was sent. The default is seven days.

Gnus also supplies a function that lets you fine-tune how long articles are to live, based on what group they are in. Let’s say you want to have one month expiry period in the ‘mail.private’ group, a one day expiry period in the ‘mail.junk’ group, and a six day expiry period everywhere else:

(setq nnmail-expiry-wait-function
      (lambda (group)
       (cond ((string= group "mail.private")
               31)
             ((string= group "mail.junk")
               1)
             ((string= group "important")
               'never)
             (t
               6))))

The group names this function is fed are “unadorned” group names—no ‘nnml:’ prefixes and the like.

The nnmail-expiry-wait variable and nnmail-expiry-wait-function function can either be a number (not necessarily an integer) or one of the symbols immediate or never.

You can also use the expiry-wait group parameter to selectively change the expiry period (see section Group Parameters).

The normal action taken when expiring articles is to delete them. However, in some circumstances it might make more sense to move them to other groups instead of deleting them. The variable nnmail-expiry-target (and the expiry-target group parameter) controls this. The variable supplies a default value for all groups, which can be overridden for specific groups by the group parameter. default value is delete, but this can also be a string (which should be the name of the group the message should be moved to), or a function (which will be called in a buffer narrowed to the message in question, and with the name of the group being moved from as its parameter) which should return a target—either a group name or delete.

Here’s an example for specifying a group name:

(setq nnmail-expiry-target "nnml:expired")

Gnus provides a function nnmail-fancy-expiry-target which will expire mail to groups according to the variable nnmail-fancy-expiry-targets. Here’s an example:

 (setq nnmail-expiry-target 'nnmail-fancy-expiry-target
       nnmail-fancy-expiry-targets
       '((to-from "boss" "nnfolder:Work")
         ("subject" "IMPORTANT" "nnfolder:IMPORTANT.%Y.%b")
         ("from" ".*" "nnfolder:Archive-%Y")))

With this setup, any mail that has IMPORTANT in its Subject header and was sent in the year YYYY and month MMM, will get expired to the group nnfolder:IMPORTANT.YYYY.MMM. If its From or To header contains the string boss, it will get expired to nnfolder:Work. All other mail will get expired to nnfolder:Archive-YYYY.

If nnmail-keep-last-article is non-nil, Gnus will never expire the final article in a mail newsgroup. This is to make life easier for procmail users.

By the way: That line up there, about Gnus never expiring non-expirable articles, is a lie. If you put total-expire in the group parameters, articles will not be marked as expirable, but all read articles will be put through the expiry process. Use with extreme caution. Even more dangerous is the gnus-total-expirable-newsgroups variable. All groups that match this regexp will have all read articles put through the expiry process, which means that all old mail articles in the groups in question will be deleted after a while. Use with extreme caution, and don’t come crying to me when you discover that the regexp you used matched the wrong group and all your important mail has disappeared. Be a man! Or a woman! Whatever you feel more comfortable with! So there!

Most people make most of their mail groups total-expirable, though.

If gnus-inhibit-user-auto-expire is non-nil, user marking commands will not mark an article as expirable, even if the group has auto-expire turned on.

The expirable marks of articles will be removed when copying or moving them to a group in which auto-expire is not turned on. This is for preventing articles from being expired unintentionally. On the other hand, to a group that has turned auto-expire on, the expirable marks of articles that are copied or moved will not be changed by default. I.e., when copying or moving to such a group, articles that were expirable will be left expirable and ones that were not expirable will not be marked as expirable. So, even though in auto-expire groups, some articles will never get expired (unless you read them again). If you don’t side with that behavior that unexpirable articles may be mixed into auto-expire groups, you can set gnus-mark-copied-or-moved-articles-as-expirable to a non-nil value. In that case, articles that have been read will be marked as expirable automatically when being copied or moved to a group that has auto-expire turned on. The default value is nil.

6.4.10 Washing Mail

Mailers and list servers are notorious for doing all sorts of really, really stupid things with mail. “Hey, RFC 822 doesn’t explicitly prohibit us from adding the string wE aRe ElItE!!!!!1!! to the end of all lines passing through our server, so let’s do that!!!!1!” Yes, but RFC 822 wasn’t designed to be read by morons. Things that were considered to be self-evident were not discussed. So. Here we are.

Case in point: The German version of Microsoft Exchange adds ‘AW: ’ to the subjects of replies instead of ‘Re: ’. I could pretend to be shocked and dismayed by this, but I haven’t got the energy. It is to laugh.

Gnus provides a plethora of functions for washing articles while displaying them, but it might be nicer to do the filtering before storing the mail to disk. For that purpose, we have three hooks and various functions that can be put in these hooks.

nnmail-prepare-incoming-hook

This hook is called before doing anything with the mail and is meant for grand, sweeping gestures. It is called in a buffer that contains all the new, incoming mail. Functions to be used include:

nnheader-ms-strip-cr

Remove trailing carriage returns from each line. This is default on Emacs running on MS machines.

nnmail-prepare-incoming-header-hook

This hook is called narrowed to each header. It can be used when cleaning up the headers. Functions that can be used include:

nnmail-remove-leading-whitespace

Clear leading white space that “helpful” listservs have added to the headers to make them look nice. Aaah.

(Note that this function works on both the header on the body of all messages, so it is a potentially dangerous function to use (if a body of a message contains something that looks like a header line). So rather than fix the bug, it is of course the right solution to make it into a feature by documenting it.)

nnmail-remove-list-identifiers

Some list servers add an identifier—for example, ‘(idm)’—to the beginning of all Subject headers. I’m sure that’s nice for people who use stone age mail readers. This function will remove strings that match the nnmail-list-identifiers regexp, which can also be a list of regexp. nnmail-list-identifiers may not contain \\(..\\).

For instance, if you want to remove the ‘(idm)’ and the ‘nagnagnag’ identifiers:

(setq nnmail-list-identifiers '("(idm)" "nagnagnag"))

This can also be done non-destructively with gnus-list-identifiers, See section Article Hiding.

nnmail-remove-tabs

Translate all ‘TAB’ characters into ‘SPACE’ characters.

nnmail-ignore-broken-references

Some mail user agents (e.g., Eudora and Pegasus) produce broken References headers, but correct In-Reply-To headers. This function will get rid of the References header if the headers contain a line matching the regular expression nnmail-broken-references-mailers.

nnmail-prepare-incoming-message-hook

This hook is called narrowed to each message. Functions to be used include:

article-de-quoted-unreadable

Decode Quoted Readable encoding.

6.4.11 Duplicates

If you are a member of a couple of mailing lists, you will sometimes receive two copies of the same mail. This can be quite annoying, so nnmail checks for and treats any duplicates it might find. To do this, it keeps a cache of old Message-IDs: nnmail-message-id-cache-file, which is ‘~/.nnmail-cache’ by default. The approximate maximum number of Message-IDs stored there is controlled by the nnmail-message-id-cache-length variable, which is 1000 by default. (So 1000 Message-IDs will be stored.) If all this sounds scary to you, you can set nnmail-treat-duplicates to warn (which is what it is by default), and nnmail won’t delete duplicate mails. Instead it will insert a warning into the head of the mail saying that it thinks that this is a duplicate of a different message.

This variable can also be a function. If that’s the case, the function will be called from a buffer narrowed to the message in question with the Message-ID as a parameter. The function must return either nil, warn, or delete.

You can turn this feature off completely by setting the variable to nil.

If you want all the duplicate mails to be put into a special duplicates group, you could do that using the normal mail split methods:

(setq nnmail-split-fancy
      '(| ;; Messages duplicates go to a separate group.
        ("gnus-warning" "duplicat\\(e\\|ion\\) of message" "duplicate")
        ;; Message from daemons, postmaster, and the like to another.
        (any mail "mail.misc")
        ;; Other rules.
        [...] ))

Or something like:

(setq nnmail-split-methods
      '(("duplicates" "^Gnus-Warning:.*duplicate")
        ;; Other rules.
        [...]))

Here’s a neat feature: If you know that the recipient reads her mail with Gnus, and that she has nnmail-treat-duplicates set to delete, you can send her as many insults as you like, just by using a Message-ID of a mail that you know that she’s already received. Think of all the fun! She’ll never see any of it! Whee!

6.4.12 Not Reading Mail

If you start using any of the mail back ends, they have the annoying habit of assuming that you want to read mail with them. This might not be unreasonable, but it might not be what you want.

If you set mail-sources and nnmail-spool-file to nil, none of the back ends will ever attempt to read incoming mail, which should help.

This might be too much, if, for instance, you are reading mail quite happily with nnml and just want to peek at some old (pre-Emacs 23) Rmail file you have stashed away with nnbabyl. All back ends have variables called back-end-get-new-mail. If you want to disable the nnbabyl mail reading, you edit the virtual server for the group to have a setting where nnbabyl-get-new-mail to nil.

All the mail back ends will call nn*-prepare-save-mail-hook narrowed to the article to be saved before saving it when reading incoming mail.

6.4.13 Choosing a Mail Back End

Gnus will read the mail spool when you activate a mail group. The mail file is first copied to your home directory. What happens after that depends on what format you want to store your mail in.

There are six different mail back ends in the standard Gnus, and more back ends are available separately. The mail back end most people use (because it is possibly the fastest) is nnml (see section Mail Spool).

6.4.13.1 Unix Mail Box

The nnmbox back end will use the standard Un*x mbox file to store mail. nnmbox will add extra headers to each mail article to say which group it belongs in.

Virtual server settings:

nnmbox-mbox-file

The name of the mail box in the user’s home directory. Default is ‘~/mbox’.

nnmbox-active-file

The name of the active file for the mail box. Default is ‘~/.mbox-active’.

nnmbox-get-new-mail

If non-nil, nnmbox will read incoming mail and split it into groups. Default is t.

6.4.13.2 Babyl

The nnbabyl back end will use a Babyl mail box to store mail. nnbabyl will add extra headers to each mail article to say which group it belongs in.

Virtual server settings:

nnbabyl-mbox-file

The name of the Babyl file. The default is ‘~/RMAIL’

nnbabyl-active-file

The name of the active file for the Babyl file. The default is ‘~/.rmail-active’

nnbabyl-get-new-mail

If non-nil, nnbabyl will read incoming mail. Default is t

6.4.13.3 Mail Spool

The nnml spool mail format isn’t compatible with any other known format. It should be used with some caution.

If you use this back end, Gnus will split all incoming mail into files, one file for each mail, and put the articles into the corresponding directories under the directory specified by the nnml-directory variable. The default value is ‘~/Mail/’.

You do not have to create any directories beforehand; Gnus will take care of all that.

If you have a strict limit as to how many files you are allowed to store in your account, you should not use this back end. As each mail gets its own file, you might very well occupy thousands of inodes within a few weeks. If this is no problem for you, and it isn’t a problem for you having your friendly systems administrator walking around, madly, shouting “Who is eating all my inodes?! Who? Who!?!”, then you should know that this is probably the fastest format to use. You do not have to trudge through a big mbox file just to read your new mail.

nnml is probably the slowest back end when it comes to article splitting. It has to create lots of files, and it also generates NOV databases for the incoming mails. This makes it possibly the fastest back end when it comes to reading mail.

Virtual server settings:

nnml-directory

All nnml directories will be placed under this directory. The default is the value of message-directory (whose default value is ‘~/Mail’).

nnml-active-file

The active file for the nnml server. The default is ‘~/Mail/active’.

nnml-newsgroups-file

The nnml group descriptions file. See section Newsgroups File Format. The default is ‘~/Mail/newsgroups’.

nnml-get-new-mail

If non-nil, nnml will read incoming mail. The default is t.

nnml-nov-is-evil

If non-nil, this back end will ignore any NOV files. The default is nil.

nnml-nov-file-name

The name of the NOV files. The default is ‘.overview’.

nnml-prepare-save-mail-hook

Hook run narrowed to an article before saving.

nnml-use-compressed-files

If non-nil, nnml will allow using compressed message files. This requires auto-compression-mode to be enabled (see (emacs)Compressed Files section ‘Compressed Files’ in The Emacs Manual). If the value of nnml-use-compressed-files is a string, it is used as the file extension specifying the compression program. You can set it to ‘.bz2’ if your Emacs supports it. A value of t is equivalent to ‘.gz’.

nnml-compressed-files-size-threshold

Default size threshold for compressed message files. Message files with bodies larger than that many characters will be automatically compressed if nnml-use-compressed-files is non-nil.

If your nnml groups and NOV files get totally out of whack, you can do a complete update by typing M-x nnml-generate-nov-databases. This command will trawl through the entire nnml hierarchy, looking at each and every article, so it might take a while to complete. A better interface to this functionality can be found in the server buffer (see section Server Commands).

6.4.13.4 MH Spool

nnmh is just like nnml, except that is doesn’t generate NOV databases and it doesn’t keep an active file or marks file. This makes nnmh a much slower back end than nnml, but it also makes it easier to write procmail scripts for.

Virtual server settings:

nnmh-directory

All nnmh directories will be located under this directory. The default is the value of message-directory (whose default is ‘~/Mail’)

nnmh-get-new-mail

If non-nil, nnmh will read incoming mail. The default is t.

nnmh-be-safe

If non-nil, nnmh will go to ridiculous lengths to make sure that the articles in the folder are actually what Gnus thinks they are. It will check date stamps and stat everything in sight, so setting this to t will mean a serious slow-down. If you never use anything but Gnus to read the nnmh articles, you do not have to set this variable to t. The default is nil.

6.4.13.5 Maildir

nnmaildir stores mail in the maildir format, with each maildir corresponding to a group in Gnus. This format is documented here: http://cr.yp.to/proto/maildir.html and here: http://www.qmail.org/man/man5/maildir.html. nnmaildir also stores extra information in the ‘.nnmaildir/’ directory within a maildir.

Maildir format was designed to allow concurrent deliveries and reading, without needing locks. With other back ends, you would have your mail delivered to a spool of some kind, and then you would configure Gnus to split mail from that spool into your groups. You can still do that with nnmaildir, but the more common configuration is to have your mail delivered directly to the maildirs that appear as group in Gnus.

nnmaildir is designed to be perfectly reliable: C-g will never corrupt its data in memory, and SIGKILL will never corrupt its data in the filesystem.

nnmaildir stores article marks and NOV data in each maildir. So you can copy a whole maildir from one Gnus setup to another, and you will keep your marks.

Virtual server settings:

directory

For each of your nnmaildir servers (it’s very unlikely that you’d need more than one), you need to create a directory and populate it with maildirs or symlinks to maildirs (and nothing else; do not choose a directory already used for other purposes). Each maildir will be represented in Gnus as a newsgroup on that server; the filename of the symlink will be the name of the group. Any filenames in the directory starting with ‘.’ are ignored. The directory is scanned when you first start Gnus, and each time you type g in the group buffer; if any maildirs have been removed or added, nnmaildir notices at these times.

The value of the directory parameter should be a Lisp form which is processed by eval and expand-file-name to get the path of the directory for this server. The form is evaled only when the server is opened; the resulting string is used until the server is closed. (If you don’t know about forms and eval, don’t worry—a simple string will work.) This parameter is not optional; you must specify it. I don’t recommend using "~/Mail" or a subdirectory of it; several other parts of Gnus use that directory by default for various things, and may get confused if nnmaildir uses it too. "~/.nnmaildir" is a typical value.

target-prefix

This should be a Lisp form which is processed by eval and expand-file-name. The form is evaled only when the server is opened; the resulting string is used until the server is closed.

When you create a group on an nnmaildir server, the maildir is created with target-prefix prepended to its name, and a symlink pointing to that maildir is created, named with the plain group name. So if directory is "~/.nnmaildir" and target-prefix is "../maildirs/", then when you create the group foo, nnmaildir will create ‘~/.nnmaildir/../maildirs/foo’ as a maildir, and will create ‘~/.nnmaildir/foo’ as a symlink pointing to ‘../maildirs/foo’.

You can set target-prefix to a string without any slashes to create both maildirs and symlinks in the same directory; in this case, any maildirs found in directory whose names start with target-prefix will not be listed as groups (but the symlinks pointing to them will be).

As a special case, if target-prefix is "" (the default), then when you create a group, the maildir will be created in directory without a corresponding symlink. Beware that you cannot use gnus-group-delete-group on such groups without the force argument.

directory-files

This should be a function with the same interface as directory-files (such as directory-files itself). It is used to scan the server’s directory for maildirs. This parameter is optional; the default is nnheader-directory-files-safe if nnheader-directory-files-is-safe is nil, and directory-files otherwise. (nnheader-directory-files-is-safe is checked only once when the server is opened; if you want to check it each time the directory is scanned, you’ll have to provide your own function that does that.)

get-new-mail

If non-nil, then after scanning for new mail in the group maildirs themselves as usual, this server will also incorporate mail the conventional Gnus way, from mail-sources according to nnmail-split-methods or nnmail-split-fancy. The default value is nil.

Do not use the same maildir both in mail-sources and as an nnmaildir group. The results might happen to be useful, but that would be by chance, not by design, and the results might be different in the future. If your split rules create new groups, remember to supply a create-directory server parameter.

6.4.13.6 Group parameters

nnmaildir uses several group parameters. It’s safe to ignore all this; the default behavior for nnmaildir is the same as the default behavior for other mail back ends: articles are deleted after one week, etc. Except for the expiry parameters, all this functionality is unique to nnmaildir, so you can ignore it if you’re just trying to duplicate the behavior you already have with another back end.

If the value of any of these parameters is a vector, the first element is evaluated as a Lisp form and the result is used, rather than the original value. If the value is not a vector, the value itself is evaluated as a Lisp form. (This is why these parameters use names different from those of other, similar parameters supported by other back ends: they have different, though similar, meanings.) (For numbers, strings, nil, and t, you can ignore the eval business again; for other values, remember to use an extra quote and wrap the value in a vector when appropriate.)

expire-age

An integer specifying the minimum age, in seconds, of an article before it will be expired, or the symbol never to specify that articles should never be expired. If this parameter is not set, nnmaildir falls back to the usual nnmail-expiry-wait(-function) variables (the expiry-wait group parameter overrides nnmail-expiry-wait and makes nnmail-expiry-wait-function ineffective). If you wanted a value of 3 days, you could use something like [(* 3 24 60 60)]; nnmaildir will evaluate the form and use the result. An article’s age is measured starting from the article file’s modification time. Normally, this is the same as the article’s delivery time, but editing an article makes it younger. Moving an article (other than via expiry) may also make an article younger.

expire-group

If this is set to a string such as a full Gnus group name, like

"backend+server.address.string:group.name"

and if it is not the name of the same group that the parameter belongs to, then articles will be moved to the specified group during expiry before being deleted. If this is set to an nnmaildir group, the article will be just as old in the destination group as it was in the source group. So be careful with expire-age in the destination group. If this is set to the name of the same group that the parameter belongs to, then the article is not expired at all. If you use the vector form, the first element is evaluated once for each article. So that form can refer to nnmaildir-article-file-name, etc., to decide where to put the article. Even if this parameter is not set, nnmaildir does not fall back to the expiry-target group parameter or the nnmail-expiry-target variable.

read-only

If this is set to t, nnmaildir will treat the articles in this maildir as read-only. This means: articles are not renamed from ‘new/’ into ‘cur/’; articles are only found in ‘new/’, not ‘cur/’; articles are never deleted; articles cannot be edited. ‘new/’ is expected to be a symlink to the ‘new/’ directory of another maildir—e.g., a system-wide mailbox containing a mailing list of common interest. Everything in the maildir outside ‘new/’ is not treated as read-only, so for a shared mailbox, you do still need to set up your own maildir (or have write permission to the shared mailbox); your maildir just won’t contain extra copies of the articles.

directory-files

A function with the same interface as directory-files. It is used to scan the directories in the maildir corresponding to this group to find articles. The default is the function specified by the server’s directory-files parameter.

distrust-Lines:

If non-nil, nnmaildir will always count the lines of an article, rather than use the Lines: header field. If nil, the header field will be used if present.

always-marks

A list of mark symbols, such as ['(read expire)]. Whenever Gnus asks nnmaildir for article marks, nnmaildir will say that all articles have these marks, regardless of whether the marks stored in the filesystem say so. This is a proof-of-concept feature that will probably be removed eventually; it ought to be done in Gnus proper, or abandoned if it’s not worthwhile.

never-marks

A list of mark symbols, such as ['(tick expire)]. Whenever Gnus asks nnmaildir for article marks, nnmaildir will say that no articles have these marks, regardless of whether the marks stored in the filesystem say so. never-marks overrides always-marks. This is a proof-of-concept feature that will probably be removed eventually; it ought to be done in Gnus proper, or abandoned if it’s not worthwhile.

nov-cache-size

An integer specifying the size of the NOV memory cache. To speed things up, nnmaildir keeps NOV data in memory for a limited number of articles in each group. (This is probably not worthwhile, and will probably be removed in the future.) This parameter’s value is noticed only the first time a group is seen after the server is opened—i.e., when you first start Gnus, typically. The NOV cache is never resized until the server is closed and reopened. The default is an estimate of the number of articles that would be displayed in the summary buffer: a count of articles that are either marked with tick or not marked with read, plus a little extra.

6.4.13.7 Article identification

Articles are stored in the ‘cur/’ subdirectory of each maildir. Each article file is named like uniq:info, where uniq contains no colons. nnmaildir ignores, but preserves, the :info part. (Other maildir readers typically use this part of the filename to store marks.) The uniq part uniquely identifies the article, and is used in various places in the ‘.nnmaildir/’ subdirectory of the maildir to store information about the corresponding article. The full pathname of an article is available in the variable nnmaildir-article-file-name after you request the article in the summary buffer.

6.4.13.8 NOV data

An article identified by uniq has its NOV data (used to generate lines in the summary buffer) stored in .nnmaildir/nov/uniq. There is no nnmaildir-generate-nov-databases function. (There isn’t much need for it—an article’s NOV data is updated automatically when the article or nnmail-extra-headers has changed.) You can force nnmaildir to regenerate the NOV data for a single article simply by deleting the corresponding NOV file, but beware: this will also cause nnmaildir to assign a new article number for this article, which may cause trouble with seen marks, the Agent, and the cache.

6.4.13.9 Article marks

An article identified by uniq is considered to have the mark flag when the file ‘.nnmaildir/marks/flag/uniq’ exists. When Gnus asks nnmaildir for a group’s marks, nnmaildir looks for such files and reports the set of marks it finds. When Gnus asks nnmaildir to store a new set of marks, nnmaildir creates and deletes the corresponding files as needed. (Actually, rather than create a new file for each mark, it just creates hard links to ‘.nnmaildir/markfile’, to save inodes.)

You can invent new marks by creating a new directory in ‘.nnmaildir/marks/’. You can tar up a maildir and remove it from your server, untar it later, and keep your marks. You can add and remove marks yourself by creating and deleting mark files. If you do this while Gnus is running and your nnmaildir server is open, it’s best to exit all summary buffers for nnmaildir groups and type s in the group buffer first, and to type g or M-g in the group buffer afterwards. Otherwise, Gnus might not pick up the changes, and might undo them.

6.4.13.10 Mail Folders

nnfolder is a back end for storing each mail group in a separate file. Each file is in the standard Un*x mbox format. nnfolder will add extra headers to keep track of article numbers and arrival dates.

Virtual server settings:

nnfolder-directory

All the nnfolder mail boxes will be stored under this directory. The default is the value of message-directory (whose default is ‘~/Mail’)

nnfolder-active-file

The name of the active file. The default is ‘~/Mail/active’.

nnfolder-newsgroups-file

The name of the group descriptions file. See section Newsgroups File Format. The default is ‘~/Mail/newsgroups’

nnfolder-get-new-mail

If non-nil, nnfolder will read incoming mail. The default is t

nnfolder-save-buffer-hook

Hook run before saving the folders. Note that Emacs does the normal backup renaming of files even with the nnfolder buffers. If you wish to switch this off, you could say something like the following in your ‘.emacs’ file:

(defun turn-off-backup () (set (make-local-variable 'backup-inhibited) t)) (add-hook 'nnfolder-save-buffer-hook 'turn-off-backup)

nnfolder-delete-mail-hook

Hook run in a buffer narrowed to the message that is to be deleted. This function can be used to copy the message to somewhere else, or to extract some information from it before removing it.

nnfolder-nov-is-evil

If non-nil, this back end will ignore any NOV files. The default is nil.

nnfolder-nov-file-suffix

The extension for NOV files. The default is ‘.nov’.

nnfolder-nov-directory

The directory where the NOV files should be stored. If nil, nnfolder-directory is used.

If you have lots of nnfolder-like files you’d like to read with nnfolder, you can use the M-x nnfolder-generate-active-file command to make nnfolder aware of all likely files in nnfolder-directory. This only works if you use long file names, though.

6.4.13.11 Comparing Mail Back Ends

First, just for terminology, the back end is the common word for a low-level access method—a transport, if you will, by which something is acquired. The sense is that one’s mail has to come from somewhere, and so selection of a suitable back end is required in order to get that mail within spitting distance of Gnus.

The same concept exists for Usenet itself: Though access to articles is typically done by NNTP these days, once upon a midnight dreary, everyone in the world got at Usenet by running a reader on the machine where the articles lay (the machine which today we call an NNTP server), and access was by the reader stepping into the articles’ directory spool area directly. One can still select between either the nntp or nnspool back ends, to select between these methods, if one happens actually to live on the server (or can see its spool directly, anyway, via NFS).

The goal in selecting a mail back end is to pick one which simultaneously represents a suitable way of dealing with the original format plus leaving mail in a form that is convenient to use in the future. Here are some high and low points on each:

nnmbox

UNIX systems have historically had a single, very common, and well-defined format. All messages arrive in a single spool file, and they are delineated by a line whose regular expression matches ‘^From_’. (My notational use of ‘_’ is to indicate a space, to make it clear in this instance that this is not the RFC-specified ‘From:’ header.) Because Emacs and therefore Gnus emanate historically from the Unix environment, it is simplest if one does not mess a great deal with the original mailbox format, so if one chooses this back end, Gnus’ primary activity in getting mail from the real spool area to Gnus’ preferred directory is simply to copy it, with no (appreciable) format change in the process. It is the “dumbest” way to move mail into availability in the Gnus environment. This makes it fast to move into place, but slow to parse, when Gnus has to look at what’s where.

nnbabyl

Once upon a time, there was the DEC-10 and DEC-20, running operating systems called TOPS and related things, and the usual (only?) mail reading environment was a thing called Babyl. I don’t know what format was used for mail landing on the system, but Babyl had its own internal format to which mail was converted, primarily involving creating a spool-file-like entity with a scheme for inserting Babyl-specific headers and status bits above the top of each message in the file. Rmail was Emacs’s first mail reader, it was written by Richard Stallman, and Stallman came out of that TOPS/Babyl environment, so he wrote Rmail to understand the mail files folks already had in existence. Gnus (and VM, for that matter) continue to support this format because it’s perceived as having some good qualities in those mailer-specific headers/status bits stuff. Rmail itself still exists as well, of course, and is still maintained within Emacs. Since Emacs 23, it uses standard mbox format rather than Babyl.

Both of the above forms leave your mail in a single file on your file system, and they must parse that entire file each time you take a look at your mail.

nnml

nnml is the back end which smells the most as though you were actually operating with an nnspool-accessed Usenet system. (In fact, I believe nnml actually derived from nnspool code, lo these years ago.) One’s mail is taken from the original spool file, and is then cut up into individual message files, 1:1. It maintains a Usenet-style active file (analogous to what one finds in an INN- or CNews-based news system in (for instance) ‘/var/lib/news/active’, or what is returned via the ‘NNTP LIST’ verb) and also creates overview files for efficient group entry, as has been defined for NNTP servers for some years now. It is slower in mail-splitting, due to the creation of lots of files, updates to the nnml active file, and additions to overview files on a per-message basis, but it is extremely fast on access because of what amounts to the indexing support provided by the active file and overviews.

nnml costs inodes in a big way; that is, it soaks up the resource which defines available places in the file system to put new files. Sysadmins take a dim view of heavy inode occupation within tight, shared file systems. But if you live on a personal machine where the file system is your own and space is not at a premium, nnml wins big.

It is also problematic using this back end if you are living in a FAT16-based Windows world, since much space will be wasted on all these tiny files.

nnmh

The Rand MH mail-reading system has been around UNIX systems for a very long time; it operates by splitting one’s spool file of messages into individual files, but with little or no indexing support—nnmh is considered to be semantically equivalent to “nnml without active file or overviews”. This is arguably the worst choice, because one gets the slowness of individual file creation married to the slowness of access parsing when learning what’s new in one’s groups.

nnfolder

Basically the effect of nnfolder is nnmbox (the first method described above) on a per-group basis. That is, nnmbox itself puts all one’s mail in one file; nnfolder provides a little bit of optimization to this so that each of one’s mail groups has a Unix mail box file. It’s faster than nnmbox because each group can be parsed separately, and still provides the simple Unix mail box format requiring minimal effort in moving the mail around. In addition, it maintains an “active” file making it much faster for Gnus to figure out how many messages there are in each separate group.

If you have groups that are expected to have a massive amount of messages, nnfolder is not the best choice, but if you receive only a moderate amount of mail, nnfolder is probably the most friendly mail back end all over.

nnmaildir

For configuring expiry and other things, nnmaildir uses incompatible group parameters, slightly different from those of other mail back ends.

nnmaildir is largely similar to nnml, with some notable differences. Each message is stored in a separate file, but the filename is unrelated to the article number in Gnus. nnmaildir also stores the equivalent of nnml’s overview files in one file per article, so it uses about twice as many inodes as nnml. (Use df -i to see how plentiful your inode supply is.) If this slows you down or takes up very much space, a non-block-structured file system.

Since maildirs don’t require locking for delivery, the maildirs you use as groups can also be the maildirs your mail is directly delivered to. This means you can skip Gnus’ mail splitting if your mail is already organized into different mailboxes during delivery. A directory entry in mail-sources would have a similar effect, but would require one set of mailboxes for spooling deliveries (in mbox format, thus damaging message bodies), and another set to be used as groups (in whatever format you like). A maildir has a built-in spool, in the new/ subdirectory. Beware that currently, mail moved from new/ to cur/ instead of via mail splitting will not undergo treatment such as duplicate checking.

nnmaildir stores article marks for a given group in the corresponding maildir, in a way designed so that it’s easy to manipulate them from outside Gnus. You can tar up a maildir, unpack it somewhere else, and still have your marks.

nnmaildir uses a significant amount of memory to speed things up. (It keeps in memory some of the things that nnml stores in files and that nnmh repeatedly parses out of message files.) If this is a problem for you, you can set the nov-cache-size group parameter to something small (0 would probably not work, but 1 probably would) to make it use less memory. This caching will probably be removed in the future.

Startup is likely to be slower with nnmaildir than with other back ends. Everything else is likely to be faster, depending in part on your file system.

nnmaildir does not use nnoo, so you cannot use nnoo to write an nnmaildir-derived back end.

This document was generated on January 25, 2015 using texi2html 1.82.