Hypermail

Contents:

• What is Hypermail?
• Usage
• Command-Line Options
  • Input and Output Options
  • Archive Interface Options
  • Updating Options
  • Miscellaneous Options
• Configuration Options
• Order of Options Processing
• Other Things
• Running Hypermail Automatically
• Getting Help With Hypermail
• Getting Hypermail Software
• Credits
• See Also

What is Hypermail?

Each HTML file that is generated for a message contains (where applicable):

• the subject of the article,
• the name and email address of the sender,
• the date the article was sent,
• links to the next and previous messages in the archive,
• a link to the message the article is in reply to, and
• a link to the message next in the current thread.

In addition, Hypermail will convert references in each message to email addresses and URLs to hyperlinks so they can be selected. Email addresses can be converted to mailto: URLs or links to a CGI mail program.

To complement each set of HTML messages, four index files are created which sort the articles by date received, thread, subject, and author. Each entry in these index files are links to the individual articles and provide a bird's-eye view of every archived message.

Hypermail was originally developed and designed by Tom Gruber for Enterprise Integration Technologies (EIT) in Common Lisp. It was later rewritten in C by Kevin Hughes while at EIT. Hypermail is now being maintained by Peter McCluskey <pcm@rahul.net>.

To see what Hypermail can do, take a look at these Hypermail-produced archives:

• Hypermail Development Archives
• WU-FTPD Mailing list Archives
• FAQ Maintainers Archives

Usage

Usage: hypermail [options]

Options:
  -a URL        : URL to other archives
  -A            : Maintain an mbox archive
  -b URL        : URL to archive information
  -c file       : Configuration file to read in
  -d dir        : The directory to save HTML files in
  -g            : Build a GDBM header cache
  -i            : Read messages from standard input
  -l label      : What to name the output archive
  -m mbox       : Mail archive to read in
  -M            : Use metadata
  -n listaddr   : The submission address of the list
  -o keyword=val: Set config item
  -p            : Show progress
  -s htmlsuffix : HTML file suffix (.html, .htm, ..)
  -t            : Use Tables
  -T            : Use index tables
  -u            : Append all input messages
  -v            : Show configuration variables only
  -V            : Show version information and exit
  -x            : Overwrite previous messages
  -X            : Write haof XML files
  -0 number     : Delete messages
  -1            : Read only one mail from input
  -L lang       : Specify language to use (de en es fi fr is pl pt sv no el gr ru it )

Using the flags -h, or -? with Hypermail will display this usage summary.

Command-Line Options

-i,
-m  "mailbox",
-d  "directory",
-c  "file"

To tell Hypermail what mailbox to read in, use the -m option. If articles will be sent to Hypermail through standard input, use the -i option. Note that the -m and -i options can't be used together! By default, Hypermail will look for a file called mbox to read its articles in from.

The -d option specifies the directory to put the HTML files and index files that are created into. If the directory doesn't exist, a new one will be created with the name that is specified. If the -d option isn't used, Hypermail will look for a directory with the same name as the mailbox or will create one if needed.

   example 1: hypermail -m "wu-ftpd" -d "/wu-ftpd"
   example 2: cat "/var/spool/mail/wu-ftpd" | hypermail -i

1. This example reads the articles in wu-ftpd and will save the output in the /wu-ftpd directory.

2. This reads the file /var/spool/mail/wu-ftpd from standard input and will save the output in a directory called archive in the same directory Hypermail was run from.

Note that Hypermail can only read messages in the UNIX mailbox format! Such archives are typically RFC 2822 mail messages appended to each other that look similar to this:

   From john@foo.com  Mon Jan  1 00:01:30 1994
   Date: Mon, 1 Jan 1994 00:01:15 PDT
   From: john@foo.com
   To: everyone@foo.com
   Subject: Hello, world!

   Hi, everyone, just saying hello!

   From someone.else@foo.com  Mon Jan  1 00:02:00 1994
   Date: Mon, 1 Jan 1994 00:01:45 PDT
   ...

The messages are typically separated by lines in this format:

   From wu-ftpd@wugate.wustl.edu  Fri Jul  1 00:18:20 1994

The -c option tells Hypermail to read in settings from a configuration file. By default, the program will attempt to read settings from a file called .hmrc in the user's home directory if it exists.

In the configuration file, variables are set in the following manner:

variable = number
variable = "string"

Archive Interface Options:

-l  "label",
-b  "About URL",
-a  "Other Archives URL"

The -l option tells Hypermail what to call the archive - the name that is specified will be in the title of the index pages so users know what sort of messages are being archived.

The -a option includes a link labelled "Other mail archives" in the index pages to any specified URL. This way users who are looking at the archive have the opportunity to go to pointers to other mail archives. By default, this will be a pointer to the parent directory in which the archive files reside.

The -b option includes a link labelled "About this archive" in the index pages to any specified URL. This way users who are looking at the archive have the opportunity to go to information about the archive.

   example: hypermail -l "WU-FTPD Development Archives"
            -a "http://www.landfield.com/wu-ftpd/"
            -b "http://www.landfield.com/wu-ftpd/mail-archive/"

In the index files for the archive, the above setting will produce something like this:

(top of page)

WU-FTPD Archives

• About this archive
• Other mail archives

(list of indexed articles below)

Updating Options:

-x,
-u

The -x option tells Hypermail to explicitly overwrite any previous HTML files that may exist. Use this option only when it is desirable to completely rewrite the entire archive.

The -u option tells Hypermail to add message(s) to the end of the existing HTML file archive and integrate them into it by links and cross-references. All archive index files will be regenerated to include the new message.

Hypermail used to require that you only send it one message at a time when using the -u option, but it should now work reasonably when given mailboxes containing multiple messages.

When using the -u option, don't send any messages that Hypermail has already processed. If you want Hypermail to recognize that some messages are old messages that shouldn't be added to the archive again, send it a mailbox with a complete set of messages and avoid the -u option.

   example 1: cat "one.letter" | hypermail -i -u -d "/wu-ftpd/mail-archives"
   example 2: hypermail -u -m "one.letter" -d "/wu-ftpd/mail-archives"
   example 3: hypermail -m "mailbox" -d "/wu-ftpd/mail-archives" -x
   example 4: hypermail -m "mailbox" -d "/wu-ftpd/mail-archives"

1. This tells Hypermail to take the article it receives from standard input and integrate it with the archive under the wu-ftpd/mail-archives directory. If no archive exists, a new one will be created with the specified letter as the first file of the archive.

2. This does the same thing, except that the letter is read in from a file that contains only that letter.

3. With these options, Hypermail will read in the articles from mailbox and write over any existing files in the wu-ftpd/mail-archives directory if they exist. If no archive exists, a new one will be created.

4. With these options, Hypermail will read in the articles from mailbox and only write new articles - it will not overwrite any existing archive files.

Note that no matter what options are specified, the index files are always rewritten. The date when Hypermail was last run is included in index pages, so it's easy to tell when the archive was last updated.

Miscellaneous Options

-p
-v
-V

The -p option shows a progress report as Hypermail reads in and writes out messages - the number of files that Hypermail is reading and writing and the file names of the directory and files created are shown. This information is written to standard output.

The -v option shows the configuration variables and their values that Hypermail would use if it was run with the same configuration file and command line options. This is useful when starting up a new list or modifying a list configuration file. Once the information is displayed, Hypermail terminates and no actual processing occurs.

The -V option prints the Hypermail version information. Once the information is displayed, Hypermail terminates and not actual processing occurs.

The -0 option list message numbers that should be deleted from the html archive. The mbox is not changed. It is equivalent to the delete_msgnum option.

Configuration Options

   setenv HM_MBOX /home/john/my_mailbox
   setenv HM_FILEMODE 0600

In the configuration file, variables must be in lowercase and separated by their values with an equals (=) sign. Blank lines and lines beginning with the # character are skipped:

   mbox = "/home/john/my_mailbox"
   filemode = 0600

Below is a list of the more important configuration variables. For a complete list, see hmrc.html.

HM_LABEL "label name"

Define this as the default label to put in archives.

HM_ARCHIVES "URL"

This will create a link in the archived index pages to the specified URL. Define as "NONE" to omit such a link. See also custom_archives.

HM_HMAIL "list submission address"

This is the email address used to send a new message to a hypermail archive. "NONE" means don't use it. Since this is different for each hypermail archive, you should probably leave it set to "NONE" here, and let it be specified at runtime by command-line parameters in the list specific configfile.
See also newmsg_command and replymsg_command.

HM_DIR "directory"

This is the default directory that Hypermail will look for when creating and updating archives. If defined as "NONE", the directory name will be the same name as the mailbox read in.

HM_MBOX "filename"

This is the default mailbox to read messages in from. Define this with a value of "NONE" to read from standard input as the default.

HM_STRIPSUBJECT "text"

A string to be stripped from all subject lines. Helps unclutter mailing lists which add tags to subject lines.

HM_FOLDER_BY_DATE = "strftime-date-format"

This string causes the messages to be put in subdirectories by date. The string will be passed to strftime(3) to generate subdirectory names based on message dates. Suggested values are "%y%m" or "%b%y" for monthly subdirectories, "%Y" for yearly, "%G/%V" for weekly. Do not alter this for an existing archive without removing the old html files. If you use this and update the archive incrementally (e.g. with -u), you must use the usegdbm option.
See also monthly_index.

HM_ISODATE boolean_number

Set this to On to display article received dates in YYYY-MM-DD HH:MM:SS format. If used with the gmtime option, a Z will be inserted between the DD and HH.
See also eurodate and dateformat.

HM_LANGUAGE "language-id"

This is a two-letter string specifying the default language to use, or a longer string specifying a language and locale. Set this the value of the language table you wish to use when running and generating archives. See also iso2022jp and eurodate.

Current supported languages, with their default locales:

de (de_DE) - German
en (en_US) - English
es (es_ES) - Spanish
fi (fi_FI) - Finnish
fr (fr_FR) - French
el (el) - Greek
gr (el_GR) - Greek
is (is_IS) - Icelandic
no (no_NO) - Norwegian
pl (pl_PL) - Polish
pt (pt_BR) - Brazilian Portuguese
ru (ru_RU) - Russian
sv (sv_SE) - Swedish

The directory /usr/share/i18n/locales on many systems has the locale codes that are available on that system.

HM_INCREMENT -1, 0, or 1
Define as 1 to append all input messages to the end of existing archives.
Define as 0 for it to read a mailbox that corresponds to the entire archive. If there are any existing html messages, it will figure out which ones at the end of the mailbox are new, and add only those that haven't been converted yet.
Define as -1 to have hypermail figure out whether the input is entirely new messages to be appended or whether it contains messages that are already in the archive. A value of -1 cannot be used with the mbox_shortened option or with the -i command line option or with mbox = NONE.

HM_APPEND boolean_number

Set this to On to maintain a parallel mbox archive. The file name defaults to mbox in the directory specified by -d or dir.
See also append_filename and txtsuffix.

HM_SHOWHTML 0, 1, or 2

Define as 1 to show the articles in a proportionally-spaced font rather than a fixed-width (monospace) font. Setting this option to 1 also tells Hypermail to attempt to italicize quoted passages in articles.

Define as 2 for more complex conversion to html similar to that in txt2html.pl. Showhtml = 2 will normally produce nicer looking results than showhtml = 1, and showhtml = 0 will look pretty dull, but 1 and 2 run risks of altering the appearance in undesired ways.

HM_LINKQUOTES boolean_number

Set this to On to create fine-grained links from quoted text to the text where the quote originated. It also improves the threads index file by more accurately matching messages with replies. Note that this may be rather cpu intensive (see the searchbackmsgnum option to alter the performance).

HM_ABOUT "URL"

This will create a link in the archived index pages to the specified URL. Define as "NONE" to omit such a link.

HM_MAILTO address

The address of the contact point that is put in the HTML header line

<LINK REV=made HREF=mailto:MAILTO>

The <LINK...> header can be disabled by default by setting HM_MAILTO to "NONE".

HM_INDEXTABLE boolean_number

Setting this variable to 1 will tell Hypermail to generate a message index Subject/Author/Date listings using a table format. Set to 0 if you want the standard Hypermail index page look and feel.

HM_FILTER_OUT expression Delete messages with headers matching regular expressions (PCRE syntax). See also filter_require, filter_out_full_body, and filter_require_full_body.

HM_DOMAINADDR "domainname"

Set this to the domainname you want added to a mail address appearing in the RFC2822 field which lack a hostname. When the list resides on the same host as the user sending the message, it is often not required of the MTA to domain-ize these addresses for delivery. In such cases, Hypermail will add the DOMAINADDR to the email address. If defined as NONE, this feature is turned off.

HM_USEMETA [ 0 | 1 ]

This option allows you to use metadata to store the content type of a MIME attachments and, later on, when a user browses the attachment, send back this information in the HTTP Content-Type header. When set to 1, the Content-Type header of a MIME attachment will be stored in a metadata file. Let us say that the MIME attachments for a message are stored in directory att-num. The metadata for those attachments will then be stored in directory att-num/.meta. If a MIME attachment is stored in file att-file, its metadata will be stored in file att-file.meta. This convention is directly compatible with the Apache server handling of metadata.

HM_REVERSE boolean_number

Defining this variable as 1 will reverse-sort the article entries in the date and thread index files by the date they were received. That is, the most recent messages will appear at the top of the index rather than the other way around.

HM_MHTMLHEADERFILE "path"

Define path as the path to a file containing valid HTML formatting statements that you wish to included at the top of every message page. Hypermail will print this file as the header of the message so make sure it contains <HTML>, <HEAD>, and <BODY> and other statements that suit your local customized needs.
See also ihtmlheaderfile, ihtmlfooterfile, and mhtmlfooterfile.

HM_CONFIGFILE "filename"

This is the default configuration file to read settings in from. This can only be specified as an environment variable. If the first character is "~", Hypermail will look for the file under the current user's home directory.

Order of Options Processing

Settings are processed in this order:

1. From the program's hard-wired internal defaults (specified in options.h),
2. From runtime environment variables,
3. From the configuration file,
4. From command-line options.

Early versions of Hypermail read the command line before reading the configuration file.

Other Things

• date.html is the index of articles sorted by the date they were received by the system's mail daemon.
• thread.html is the index of articles sorted by thread first, then the date they were received.
• subject.html is the index of articles sorted by subject. Any "Re:" prefixes in front of subjects will have been stripped out.
• author.html is the index of articles sorted by the first word of the author's name. If the author's name can't be determined, their email address will be substituted.
• One of the above files will be called index.html and is the default index that users can go to when entering the archive.

Sorting: In the date and thread index files, note that these lists are sorted by the date the articles were received by the system's mail daemon, not by the date they were written on. The order of articles in the date index may not necessarily match the order in which the article files are written and linked together. Because of this, it is a good idea to make sure the mailbox is sorted by date with the most recent messages towards the bottom.

Running Hypermail automatically: All that's needed to start archiving email messages is to set up Hypermail to do incremental updates in your /etc/aliases file (assuming that you use sendmail or something that works like it to deliver mail). Here's what an entry might look like (the last line is one unbroken line):

#
# WU-FTPD Mailing List Archives
#
wulist: "|/usr/local/bin/hypermail -i -u -d /wu-ftpd/mail-archive -l \"WU-FTPD Mailing List Archive\""

If you use qmail instead of sendmail, you probably want to create a file /var/qmail/alias/.qmail-<mylistemailaddress> containing something like this:

|/usr/local/bin/hypermail -i -u -d /wu-ftpd/mail-archive -l \"WU-FTPD Mailing List Archive\"

If you are running Linux kernel version 2.4 or higher, dnotify looks like it provides another way to automate Hypermail.

Including HTML in messages: One can include formatted HTML in message bodies by enclosing the HTML with the <HTML> tag (in either uppercase or lowercase). This tag must be on a line by itself:

   This text will not be parsed...
   <html>
   this text will be parsed as HTML.
   </html>
   This text will not be parsed...

There is no limit to how often the <HTML> tag can be used in an article.

Getting Help With Hypermail

If you are are looking for more information on Hypermail and its features and developmental status, check out SourceForge: Project Info - hypermail and hypermail-project.org. Additional documentation and the hypermail development list archives are available there.

Getting Hypermail Software

Hypermail is available free of charge under GNU Public License. More details about the GPL are available at http://www.fsf.org/copyleft/gpl.html.

Currently, SourceForge: Project Info - hypermail has the most recent version.

The Hypermail Development Center also has beta development versions of hypermail available from time to time.

Credits

I'd also like to thank Kevin Hughes for developing the initial C version of Hypermail. Kevin also provided a great deal of assistance with restarting the current hypermail development.

There are a great deal of people that also contributed to Hypermail's development. The Hypermail Development Center Credits page is an attempt to let you know just who they are.

Hypermail development is currently being fostered by <Peter McCluskey>.

See Also

hypermail(1),   hmrc(4),   Hypermail List Configuration File.   and   Customizing Hypermail Pages and Adding a Search Engines to your Hypermail Archive

Please send any feature requests, bug fixes, and comments on Hypermail to <hypermail@hypermail-project.org>. In order to minimize spam on the list, you must subscribe to the list (at least temporarily) in order to send mail to it. You may subscribe to the list by sending a message with the word "subscribe" in the Subject: field to hypermail-request@hypermail-project.org.

Last updated Sep 2, 2006