Fetchmail

Fetchmail is a wonderful program which might just solve a lot of issues that people have when they want to copy data from the IMAP server, whether it be because they're leaving the department and want a copy of their mail, or if you wish to use a mail client which can't access the IMAP server directly (though that is an unsupported path, so you're on your own).

This article or section needs to be updated. Parts of this article or section have been identified as no longer being up to date. Please update the article to reflect recent events, and remove this template when finished. Last update: Is this still relevant or should it be removed entirely?

Running Fetchmail

Fetchmail is not installed on linux machines in the department by default, so you would have to request it be installed first. Before you run it, however, be sure to read and understand all the information here. Running it improperly can cause mail loops, deletion of all mail on the server, and many other Bad Things (some of which are a minor nuisance, others a major problem and unrecoverable).

NOTE:

This is important, and bears repeating. If you do not understand what you're doing with Fetchmail, you CAN and most likely WILL cause problems ranging from losing your email to infinite loops which can crash a mail server. If you're not sure, ask us to have a look at your setup. While Fetchmail isn't actively supported here, we'd rather you wait to get an answer from us than blindly run the program and then ask us to restore your entire mailbox.

.fetchmailrc

First, you will need a .fetchmailrc file. This tells fetchmail what do to in order to get your mail. If you just want to download a copy of your mail to a local file, this .fetchmailrc file doesn't need very much information:

poll mail proto IMAP
keep
fetchall
mda "/usr/bin/procmail -t -m my.pmrc"

These lines translate to:

• Contact the server "mail", using the IMAP protocol
• Keep all mail on the server, don't delete it (just download a copy)
• Grab messages already read, as well as the unread messages
• Feed the downloaded messages into Procmail with the arguments:
  • -t - Fail softly (report an error to Fetchmail if there's a problem)
  • -m my.pmrc - Act as a mail filter, using the Procmail recipe my.pmrc instead of the default ~/.procmailrc

NOTES:

1. Because of the 'keep' and 'fetchall' commands, this will cause *all* your mail to be downloaded *every time* you run the program. It will also mark all messages as 'read' on the server when it downloads them. While I could've written the file differently so you could use this all the time, I chose this set of commands in case someone doesn't read the warnings, so their mail isn't deleted from the server unexpectedly. If, however, you're reading this much and would like to change the behavior, read on to the bottom of this section and I'll explain how you can do so.
2. This set of commands will only download mail in your inbox, not any subfolders of the inbox. I will explain momentarily how you can tell it to gather mail from subfolders as well as the inbox.
3. I'll repeat, when this command runs it will mark all your mail as 'read'. Therefore, if you run fetchmail with this RC script, delete the resultant output file(s) and then login to the mail server with Pine, you will have no unread messages. They will still be on the server, so all is not lost; just the status of the mails.

my.pmrc

The my.pmrc file is listed here. You can also see other examples of Procmail recipes you might use with Fetchmail.

Advanced topics

NOTE:

Make sure you understand the implications of what you're doing; deleting all your mail off the server and asking for it to be replaced may be impossible, so always test your script with all destructive options off first (namely, use the flag '-k' to fetchmail to force it to keep all mail, regardless of what you said in the .fetchmailrc script).

Only fetch unread mail

If you wish to keep your mail on the server but pull unread messages from it (and flag them as read on the server), remove the 'fetchall' keyword from the .fetchmailrc file. This tells fetchmail to only download unread mail, but mark it as read when it does. In keeping with this, you can then run 'fetchmail -c' to only check for unread mail on the server, and not try to download it.

Download & delete

If you wish to be brave, and remove all your mail from the server keeping it only in a file, then remove the 'keep' directive from the .fetchmailrc file. This tells fetchmail to delete the messages as it downloads them. If you still have the 'fetchall' keyword there, then mail that you've read before on the server (through an IMAP client, such as Pine or Roundcube) will also be downloaded; if you remove 'fetchall', then only mail that is unread will be pulled down and deleted from the server.

Use other sources

You can tell fetchmail to get your email from other sources as well, by using other 'poll' directives naming other servers. For example you could tell it to get your @princeton.edu mail from OIT's IMAP server. 'man fetchmail' should give you all the information you need to figure this out.

Preconnect condition

You can tell fetchmail that there's a preconnect condition that must be met before it can connect to the named server. Why would you do this? Let's say you want to pull all your unread mail from our server, while at home. But you don't want to use SSL to do it, you'd rather use SSH to tunnel the connection:

1. Tell fetchmail that instead of polling the server 'mail.astro.princeton.edu', it has to poll "localhost port 9000".
2. Set a preconnect condition to do "ssh -L 9000:mail.astro.princeton.edu:143 coma.astro.princeton.edu 'sleep 60'".

   This tells SSH to connect to coma, and run the command 'sleep 60', which keeps the pipe open just enough for you to tunnel through it. Next, it will forward the local port 9000, through the SSH tunnel to coma, and from there connect to port 143 on the mail server (which is the IMAP port).

Now your username and password will be encrypted from the local machine to coma, and will only travel in cleartext from coma to the mail server (which being on the same network means it's pretty safe). This is a great trick to use if you have to get mail with POP3, which is not usually SSL-aware (so it's an unencrypted connection), but you have a shell account on the POP3 server or another machine on its local network.

Not just for inboxes

If you want to pull mail from more than one folder on the IMAP server, use the 'folders' keyword in the .fetchmailrc file. The keyword tells the program to get mail from the folders listed. Then list the folders you want to retrieve after the keyword, separated with commas. One thing to remember is that Cyrus IMAP keeps all folders as subfolders of your inbox, so if you want to get your inbox and two folders (we'll call them foo and bar), you need to specify "folders INBOX, INBOX.foo, INBOX.bar".

If you want to get *really* tricky, then tell procmail to deliver normally to $MAILDIR/$FOLDER. Edit the 'mda' line in your .fetchmailrc and add the '-p' flag to the call to procmail (this preserves the environment when you call the program). Then do this with a bash script (untested, use at your own risk!):

for F in inbox inbox.foo inbox.bar ; do
  export FOLDER=$F
  fetchmail -r $FOLDER
done

This will set $FOLDER to inbox and run fetchmail, gathering your inbox and putting the messages into ~/mail/inbox; then it will run again, getting mail from INBOX.foo, and putting it into the file ~/mail/inbox.foo, and then again with bar, etc. You could do this listing all your mail folders in the "for" line, and it will pull down *all* your mail from the entire server. It will ask for your password every time, but there is a way around that in the .fetchmailrc file (an exercise left for the reader, since it's very insecure to have your password in a file like that). Also be very sure that you know what you're doing if you do this; if you have the "fetchall" keyword in your .fetchmailrc file, and no "keep", you'll not only be copying all your messages to local files but also deleting them from the server, which is not reversible.