Re: Getting started

From: Elliot Lee <sopwith_at_redhat.com_at_hypermail-project.org>
Date: Thu, 23 Apr 1998 00:46:52 -0400 (EDT)
Message-ID: <Pine.LNX.3.96.980423002135.9222A-100000_at_lacrosse.redhat.com>

On Wed, 22 Apr 1998, Guido van Rossum wrote:

> I'm not sure how pipermail compares to hypermail in detail, but I know
> that Andrew was aware of several hypermail restrictions and coded
> pipermail to avoid these. Andrew, would you care to comment?

To be honest, I haven't tried pipermail specifically, but I have run most of the other mail -> html converters out there, and only hypermail had the speed needed. archive.redhat.com has 4.5M mailbox files that generate 2500 messages, and although even hypermail *starts* to choke on them, nothing else gets close.

Writing it in C gives a few advantages:

• For a speed-intensive task like this, nothing else gives you the control needed (don't get me wrong - Python is cool, just not something I'd personally choose to use here)
• We can mmap() in the mailbox file and speed up parsing a lot.
• Other cool speedups (writev() for writing out the messages, for example) that require low-level manipulations.
• There's probably more, I'm just not thinking of it right now.

I would suggest using 'glib' (a set of utility classes for C, i.e. sane string manipulation, hash tables, tree, linked list, etc.). It would give us much better memory management, among other things, as well as GString - the infinitely-long buffer :)

Perhaps we must rewrite hypermail for sanity, though - existing code has no concept of "buffer overflow" no matter how hard you try ;-)

• Elliot Chicken Little was right.

Received on Thu 23 Apr 1998 06:49:23 AM GMT