Re: Hypermail incremental mode

From: Daniel Stenberg <>
Date: Mon, 12 Apr 1999 13:30:00 +0200
Message-ID: <>

John Finlay wrote:

> At the time, I didn't discover any others who were working on hypermail
> so I just sat on those fixes.

We were many in exactly the same position. :-)

> Now that I have discovered that others are
> still interested in hypermail, I'm trying to come up to speed on the
> latest version(s) and their features.

You're very welcome to our team! We need more people with skill and some time over to make improvements.

> I see that there have been a number of useful enhancements to hypermail
> and am wondering about upgrading to a newer version but want to make
> sure that someone has already added in fixes to make the incremental
> mode robust and to make hypermail performant.

In my eyes, the current version 2 is not any longer comparable with 1.02 when it comes to features, such as MIME decoding etc. So I wouldn't even consider to use 1.02 today.

> 1. added archive locking

This exists in v2. In the recent 2a18 you're now able to specify how long to wait for a lock before deliberatly breaking it.

> 2. added an optional file that contained the summary info of the archive
> to avoid opening all the archive files during an update. [this removed a
> big performance hit on large archives]

This does not exist, and I have not added any such on purpose. I believe this is left for discussions whether we should add one or not. I'm in favour of a solution where such a file is used if existent, but where the HTML files are scanned and that info is used if the "cache" is for some reason not there.

> 3. fixed up the crossindexing of threads, etc. [more reliable indexing]

The entire thread indexing was rewritten for 2a18.

> 4. cleaned up the print.c routines. [rationalized these]

They are still a bit weird but modified a lot since 1.02.

> 5. fixed lots of string handling errors in string.c [source of
> coredumps]

Since beyond 2.0b3 all string handling is dynamic and should not cause any buffer overflows anymore. Instead we got a bunch of NULL referings that have been and will be taken care of.

> 6. changed the addhash function and introduced a mail struct to avoid
> replicating each header string 4 times. [big saving on memory usage with
> large archives]

Done in 2a18.

> 7. eliminated the on-the-fly sorting of messages by using qsort on the
> tables. [big performance enhancement for large archives]

I surely wouldn't mind if you added that functionality to the 2a18 package and sent me a patch for it! ;-)

> 8. fixed up the date routines which seemed to broken in a number of ways

The date routines are pretty much modified for 2a18 too. Paul Haldane has been friendly enough to shoulder the date stuff for now so I'm hoping you can coordinate your ideas with his new stuff.

I'm awaiting some further fixes in that area.

> 9. fixed lots of string handling errors in the parse.c routines.

I believe parse.c has been modified pretty much since 1.02. If you had specific problems with 1.02 I think the best way would be to test those cases against 2a18 to see if they're corrected or not.

> I took a quick look at 20b3 and it seems not to have fixes equivalent to
> 1, 2, 6, or 7. I didn't check for 3, 4, 5, 8, or 9 but I would hope
> that these have been fixed since they are a source of coredumps.

The beta 3 is dated August '98 and is by all measures an old version.

> After a quick look at 2a18, I noticed fixes equivalent to 1 and 6. I
> didn't see fixes equivalent to 2 or 7. I didn't check for the rest
> though I hope these are fixed.

You're correct.

> - is my analysis of the current releases wrt my concerns somewhat
> correct?

It seems so, yes.

> - does anyone else care about/use incremental mode of hypermail?

The bug report flow shows that many people use hypermail in incremental mode, yes. I also believe that one of the main reason people use hypermail at all and not any competitor, is speed. We must keep hypermail speedy or else we have no reason to exist at all.

> - does anyone care about large archives or is the current practice to
> slice the archives by month or week to avoid the inherent problems?

I think it depends a little on what "large archives" mean. I think hypermail should be made to work nicely and swiftly with archives containing at least a couple of thousand mails, if that is a large archive or not by your means I don't know.

> - does the 2a18 represent the current development effort? Is 20b3 an
> earlier dead release?

Yes and yes. Have a look at It is my hypermail page that blurbs a little about what versions that do what and it also explains how to get the very latest source off the CVS server.

It has turned out that I am the current maintainer of the developing since Kent Landfield has simply vanished.

I do not wish to take credit for all of the hypermail v2 developing, but I am collecting all the patches I get and add them into the archive and I also publish tar-balls of the development version with some random intervals ;-)

We do need and appriciate all help and support we can get with developing hypermail v2 into a stable and fast version. Right now, I tend to prioritize fixes that improve stability, instead of adding more and more fancy features before a stable v2 has been released. We also need a lot of help with docs and setting up a decent web page (Kent is the maintainer of and it is terribly outdated).

 / Daniel

(please don't write me at this address, my real address is Received on Mon 12 Apr 1999 01:34:33 PM GMT

This archive was generated by hypermail 2.3.0 : Sat 13 Mar 2010 03:46:11 AM GMT GMT