I'm perpetually mystified by the human propensity for self-delusion: we seem to be able to talk ourselves into the most ridiculous things, whether through hopelessly naive optimism, stupidity or selective memory. A wiser head than mine (George Santayana, if I recall correctly) once summed it up quite well, saying "Those who do not learn from the mistakes of history are doomed to repeat them".

From the preceding paragraph, you should be able to glean the idea that I'm not best pleased with myself at the moment. I've got myself into a quagmire, and like all quagmires, it's wider and stickier than I ever expected. I'm talking about the changes to the internal object interface used by Pegasus Mail.

When I originally took this on, it was a logical, worthwhile futureproofing modification that was only going to take a week or two to finish: that was nearly two months ago. Now, don't get me wrong - the "worthwhile", "logical" and "futureproofing" parts are all still completely valid, but seventeen years of doing this should have taught me that it was never going to take only a week or two - especially when the addressbooks became involved.

Pegasus Mail's legacy addressbooks are a source of unending grief for me: they're some of the oldest data structures in the program, dating from the DOS version. Back in the DOS days, programmers had to be really, really careful about how they used memory, because there wasn't much of it available. As a result, the addressbook code is incredibly contorted in places to allow the smallest possible memory use, even though in the days of gigabytes of RAM, the notion is now totally irrelevant. Unfortunately, slanting the entire development of a piece of code towards one priority like this is quite pernicious, because it debilitates almost every other part of the code. The result is that the legacy addressbook code in Pegasus Mail is fragile, awkward, convoluted and unbearably tedious in places, and I really don't like working on it very much.

Now, Pegasus Mail v5 will have a new addressbook format which will, I believe, be an absolute market leader - much more an information manager than an addressbook, its capabilities will make it enormously useful to almost everyone, unlike the antiquated and minimalist excuse for an addressbook we have at the moment. The design for the new format is done, and quite a lot of the code is even written: the problem is that I can't abandon the existing format - I'd get lynched - so I have to make sure that the two formats co-exist, if only to provide an easy migration path for existing users. This means that I have had to overhaul the old addressbook code once more so it can use the new interface I've designed.

The update on the old code is now mostly done, fortunately - although it's been quite a trial. There's still a bit to do though, so it's back to the salt mines for me. *sigh*.

Cheers!

-- David --

Do you really have to maintain both, or could you just provide a migration tool?

I've done migration tools in the past, and believe me, they're far more work than this. My last experience with a migration tool was so painful that I'll never willingly do it again.

-- David --

One thought would be to allow someone else in the community to do the migration tool. Hopefully this would be a nice separate utility and so would not directly effect pegasus. I would assume that all you would have to do is give this other person (or persons) the data structures for the new address books and you could go back to the more fun stuff.

It's not quite that simple, unfortunately, for two main reasons:

1: You always end up spending quite a lot of time explaining things to the people doing the work and handling clarifications (although I have to admit that Lukas Gebauer wrote most of his tools practically without contacting me at all).

2: In this case, we're dealing with a transition from some really simple data structures to some really high-end, powerful (and hence more complex) data structures. This requires extra libraries and documentation that I don't have in a releasable form at this point.

In the long run, it's actually much quicker for me to do this type of thing myself. Things might be different if we had an Open Source environment, but that's another discussion completely.

David,

I don't think yo need to maitain the old address book. People might not know better but once they'll have seen the new code they'll wonder how in the world they managed so far.

I would simply import the old address books into the new format. It should be relatively easy to map the fields: you're going from a very simple to a more complex structure so there shouldn't be too many errors.

You could also let the user do the mapping: decide globally or per addressbook which old field matches the new ones.

Of course, you'll probably create a backup if the original addressbook files!

Oh it was only a thought.

One thing that you might do is if need be support both versions of the address books in the next version, but announce that the next version will not support the old versions. Then as long as you can use the mass drag and drop method of address book record transfer between books to create new books in the new format, just let everyone during the next period of time move them over to the new format and then while this will not help now, it will help with the next version as you can then abandon this old format and simplify the code.

Regarding the "squeezing" of code to available memory, don't you miss the "ahhh" gratifying sense of "elegance" that a perfectly piece of compacted program could elicit... or is that just the anal part of me?