[hypermail] performance

 srose_at_direct.ca (Scott Rose) writes:
>On an entirely different note, I have code that improves the performance
>of hypermail, particularly in the case of large archives- hypermail
>opens each file in an archive to build new indices each time a message
>arrives when you run in message-at-a-time mode, and it's desperately
>expensive. My approach uses a GDBM index so that a whole lot less I/O
>has to take place. I've been waiting for 2.0 to ship before bringing
>this up again... I mentioned it to Kent a year or so ago. Any interest?
>It should be generalized beyond GDBM to be most widely useful...

 Could we see this code?

 I'm looking at hypermail's performance on a 4500 message mbox, and the biggest performance problems I see are O(N^2) algorithms, not I/O problems. I've found that crude balancing of the datelist tree speeds up addheader dramatically (it went from taking 1/3 of the time to nearly zer time). Repeated iterations through replylist and threadlist are the next biggest cpu wastes, and I'm wondering what to do about them.

-- 
------------------------------------------------------------------------------
Peter McCluskey          | Fed up with democracy's problems? Examine Futarchy:
http://www.rahul.net/pcm | http://hanson.gmu.edu/futarchy.pdf or .ps