Developer's musings

0
-1
closed
David Harris posted Nov 20 '12 at 1:35 pm

Just a short update for now - I realize I have been more than usually quiet and remote recently, and for that, I apologize - I have had a significant amount on my plate.

Over the course of this year, I have completely rewritten my TCP/IP code (the part of the program that actually connects to the Internet), to make it more modular, more reliable, and more maintainable. A huge part of this process has involved moving away from the third-party package I previously used to handle SSL/TLS secure connections and instead using the industry-standard OpenSSL suite. With this code complete and tested, I am now well into reworking the various Mercury protocol modules to use it. This has a number of ramifications: firstly, SSL support should be much more reliable, and should work correctly with a much wider range of external clients and servers. Secondly, it will be much easier to keep the SSL components of Mercury up-to-date, and only a single point of change will be necessary at my end to update all the modules, compared with the way it is now, where I have to update each module separately. Thirdly, you will be able to use "real" certificates for SSL, and Mercury will be able to generate the necessary signing requests you can give to certification authorities (CAs). A useful side-effect of the process is also that TCP-based operations in Mercury will be somewhat faster than they have previously been.

Other developments in Mercury are proceeding, most specifically the separation of interface from server process, and the reworking of the help system to use my own help engine: these processes, however, are taking longer than I would like, and it seemed best to bring out a version with the new SSL code as soon as possible. As a result, as soon as the Mercury protocol modules are all converted over to the

new TCP/IP libraries, I'll be giving them to my test team for a short

round of testing, the aim being to release an interim cut of Mercury

(tentatively to be called v4.8), either late in December or early in

January. Other developments will follow in due course over the subsequent months. Licensees need not be worried about this update - all existing licenses will continue to work with it without change.

Again, my apologies for the lack of regular updates: I keep saying I'll get better at it, yet I never seem to be... <slightly embarrassed grin>

All my very best to you all.

Cheers!

-- David --

0
-1
closed
David Harris posted Jul 21 '12 at 5:09 am

 Sometimes when you're in a hole, it can seem as if the effort required to climb out of it is just more than you'll ever be able to manage, as though the task is just too hard, with too little reward at the end, and all you want to do is to give up and  rest. I found myself in such a hole at the start of 2007.

Put simply, I was completely burned-out. I had been working on Pegasus Mail for 18 years, Mercury for 14, and my fortunes were in head-long decline. Money was in short supply, collections of old source code needed huge amounts of  updating, and I had simply answered too much mail.

I probably need to explain what I mean by "answered too much mail". From the earliest days of Pegasus Mail (1989 was when it sent its first ever message), I've always kept an open mailbox: unlike the lead developers at large corporate software firms, I don't have a carefully-guarded e-mail address - the same address that people could use to mail me in 1994 (the only time I've ever updated it) can still be used today, and I make no attempt to hide it. The price of this has been a veritable deluge of mail over the years, with everything that accompanies that - there has been praise, there has been hate, I've made friends and occasionally lost them, but above all, there have been bug reports.

Now, for a software developer, bug reports are like accusations of failure: when you're as involved in your work as I am, each report is sharp and painful like a paper cut - it's very hard to pull back emotionally and see them as the positive things they actually are (because they allow you to make your work better). Each one knocks you back a little, and after 17 years of dealing with them, I had reached a point where I was almost phobic about them. At the start of 2007, I was in a mental and emotional place where I actually *dreaded* opening my new mail folder. For someone who has always regarded communication as being the gift I most wanted to give people, this was an ironic and unnatural position to be in.

January 2007, then, was when I faced a kind of "mini crisis" in my life - not quite a breakdown, but it was a very, very unhappy time. Somehow, though, with help from my friends, my very dedicated beta test teams and from a hardcore group of committed users in the Pegasus Mail and Mercury communities, I clawed my way out of the pit. But to do so, I had to make some changes, and to accept that there were things I just couldn't do any more: in particular, I simply couldn't handle the same volume of mail that I had up until then.

So this, then, is my apology if you have found me unresponsive in the last few years. I am deeply conscious that I do not communicate with my  community as well as I either should, or would wish to. Cutting down the amount of mail I handle has been an essential part of the process of easing myself back into a working state, but however unavoidable it might have been, it's still an adjustment I don't like. Increasingly, I am relying on my beta test team and supporters to ensure that things I really need to know get escalated to me through channels I can sustain. That's one reason why this community site is such a god-send for me - even if I don't participate here as much as I might wish to, people I trust implicitly *are* participating all the time, and they regularly refer issues to me, thereby reducing my communication burden, for which I am enormously grateful.

Five years on, and the start of 2007 now seems mercifully distant. After a long period of time where I didn't, I am now actually enjoying doing the work again. The dismal process of updating and modernizing old, clunky code is now mostly done, and new, more interesting projects give me something to look forward to each day. No matter how quiet I may seem to be, rest assured that I am anything but idle: good things are happening in Dunedin - indeed, it's hard for me to tell you how frustrating it is that the time required to bring them to completion seems so great. I guess this is just the inevitable evolution of software: user expectations are now so much higher than they were ten years ago, and the environments richer, more complex and more difficult: doing even very small, seemingly simple tasks can now take a depressingly long time. I am, however, very excited by the things I'm producing at present, and can't wait until they're ready for me to be able to show them to you.

The only real problem I haven't resolved from the dark days of 2007 is that of funding. Initiatives like the Pegasus Mail Thousand have basically failed because of my personality - I simply don't like asking for help, because I don't like the idea of being a nag or a nuisance, and because a core part of the New Zealand psyche is the idea of not being pushy, not promoting yourself. I am, in a sense, my own worst enemy.

I've never been very good at the whole "let's make money out of this" aspect of software - it's not the primary reason I do any of this: my clear preference would be not to have to ask for anything - to make Pegasus Mail and Mercury into the pure gifts I would like them to be. But I have to eat, as well, and that's getting harder and harder. In spite of my own inertia and poor fund-raising skills, though, there are still quite a few people who, even without prompting or reminders, still send donations for Pegasus Mail: I see every donation, and the morale boost they provide is as valuable as the money they provide: my heart-felt thanks go out to you all. I still live in the quaint, naive hope that some day a corporation or foundation might offer some kind of sponsorship - after all, the amounts required are actually pretty tiny - but I guess this is fantasy in 2012. I'm clearly going to have to do something, but at this stage, I don't know what it is.

So, there it is, straight from the heart. I live for the day when I can again offer something exciting and of real value to my community - after all, they're - no, "you're" the number one reason I keep doing this.

All my very best to you,

-- David --

0
-1
closed
David Harris posted Feb 3 '12 at 2:55 am

I received this message today, completely unsolicited, from Keith Harris, a long-time user of Pegasus Mail.

-------------------- cut here --------------------

Your

program fails sadly now big time.


I add an

email to a distribution list and also to the safe sender list but it still comes

up as junk mail.


This is no

longer a program anyone wants on their system.


I have

tried to stay true to you but if this crap keeps on gettiing worse, as it is, I

will move away and no longer recommend you.


David, the

recommendations I personally made to you were never

implemented.


V 6.43

includes NONE of my recommendations.


Get it

right sunshine or get lost.


Keith

[Harris]

-------------------- cut here --------------------


I am very old-school, and it shocks me to think that *anyone* could send a message like this - it was like a physical blow.

Nonetheless, it brings home to me forcefully the fact that I may be simply flogging a dead horse. The last four years have been terribly difficult for me, doing dreary code modermization and surviving on almost no income, purely because I believed there were still users out there who valued my work and for whom it continued to provide benefits. But maybe I'm simply wasting everyone's time continuing work on Pegasus Mail? There are days when I really feel like I'm living in a total vacuum, and it's now nearly impossible for me to tell whether or not I'm actually doing anyone any good any more.

If you feel the time has come when I no longer serve a useful purpose, when my work no longer has enough worth to continue labouring on it, now is certainly the time to tell me - if I'm just prolonging everyone's misery, for Heaven's sake, let me know.

-- David --

0
-1
closed
David Harris posted Jun 15 '11 at 6:29 am

With the release of Mercury v4.73, it became clear that it was time to start work on major developments within Mercury that are long overdue. The most obvious and immediately necessary of these is the complete separation of the user interface from the working processes at the core of the program. Under Windows 7, Windows Server 2008 and, we presume, upcoming versions of Windows, service processes can no longer have user interfaces that interact with the desktop, which makes configuring Mercury in these environments a bit fiddly. What is needed is a user interface that can run anywhere, connecting to the working processes to configure them: this would allow you to configure the copy of Mercury running on the server in your machine room from the comfort of your own desktop PC, or from home if there was an after-hours problem. Allied with this is the need to manage mailing lists remotely as well - I am building the new remote facilities in such a way that there will be a simple application allowing list moderators to manage just the lists for which they are responsible from anywhere on the Internet. For unlicensed copies of Mercury, the program will continue to run as an application with the full user interface built-in - and indeed, if you have a license, you can run the program as an application with its full UI *and* still configure it remotely. This ability to have the UI or not as you please is quite powerful and took quite a bit of effort.

After quite a bit of development over the last few months, Mercury is now in a state where the separation of UI and working core is complete. It is now possible to run Mercury as a service with no user interface, and configure it using the full user interface running as a separate process on the same machine: from this, it is a fairly short step to allow the configuration to be fully remote as well. After that, it's just making the consoles for the various processes available remotely and the job will be effectively done. One of the things that makes the process more time-consuming is the need to ensure that very high levels of security apply to the process, so everything is getting bullet-proofed and double-checked to that end.

In other areas, Mercury, like Pegasus Mail, will be migrating to use the industry-standard OpenSSL library for its secure communications, which should improve interoperability and will bring the new ability to use "real" SSL certificates (i.e, ones issued by CAs such as Verisign). As I remarked in a recent Pegasus Mail blog posting, the development of the new common message store for both programs is proceeding well and should finally come to fruition this year. A side-effect of the new message store will be that it will be possible for me to rework large parts of the MercuryI IMAP server that are currently limited by the old message store. The aim is to produce faster, more reliable IMAP service with support for a number of the more important modern IMAP extensions, such as SORT and IDLE.

Well that's it for now - time to get back to working on any of the thousands of things I seem to have piled up on my desktop. All my very best to you all!

Cheers!

-- David --

0
-1
closed
David Harris posted Jun 15 '11 at 6:07 am

Pegasus Mail v4.62 is now in the late stages of preparation for release: it primarily works around some issued introduced by Microsoft with the roll-out of Internet Explorer 9, but has a range of small bug fixes as well. It should be out within a couple of weeks of this posting.

In the longer term, the main focus in Pegasus Mail is now moving onto new functionality, in particular the new Pegasus Mail contact manager, which is an ultra-powerful replacement for the now antique Pegasus Mail addressbook system. This code has been developing over the last few months and, like so many things these days, has ended up being quite a bit more work than I anticipated, mostly because once it became clear how capable it was going to be, it also became necessary to start devising a user interface that would make the most of it. I've ended up writing a lot of new UI code, including some stuff that is actually pure out-and-out fun... I probably need to explain that. As software projects get larger, you spend more and more of your time fixing bugs, modernizing code, and generally doing background tasks that end users will seldom even be aware is there; in all honesty, this process is humdrum and quite soul-destroying, but it's unavoidable if the program is to remain viable. The real pleasure in developing an application like Pegasus Mail is in the addition of new features that offer direct, visible benefits to my community - that's when working on it is fun again. The new contact manager definitely falls into the "fun" category - in one mad orgy of development, the program is going to go from having an antiquated addressbook that is barely usable in the modern world to having a contact manager that many sites may be able to use as a full-blown CRM system.

The other major forthcoming change to the program, the new backing message store, has been on hold for a little while as I fought fires in other places, but development is now proceeding on it again: I'm now working extensively on the IMAP portion of the message store. IMAP users should find the new message store much faster at most tasks, because it will be able to take advantage of IMAP extensions and features that were not previously possible, and it will also be possible to use Pegasus Mail as an IMAP-only mail program if you wish. This will be accompanied by new SSL code using the industry-standard OpenSSL libraries, for greater stability and compatibility over secured connections. I feel like I've been talking about this message store forever, but when I look at it and realize that I've written over 100,000 lines of highly-organized code already, I realize that in its own right, it is already a considerably larger project than many full-blown software packages out there. It's frustrating putting in so much effort without feeling like I have much to show you yet, but the work will be worthwhile in the end.

On a slightly gloomier note, looming over all of this is the shadow of considerable financial difficulty... *sigh*. I cannot deny that the last couple of years have been financially disastrous in almost every way, and it's a constant struggle to find ways of making ends meet. I'll be sending out a plea to the Pegasus Mail Thousand soon and hope that as many as people as possible will be able to respond so I can carry on doing what I've now been doing for over 21 years.

It's Winter in New Zealand now, and cccccooooolllldddd.... To all of my antipodean colleagues, wrap up and keep warm, and to those of you lucky enough to be basking in the warmth of northern climes, have fun in the sun!

Cheers to you all!

-- David --

0
-1

In response to considerable user demand, I have begun work on a significant change to the way Mercury manages domain names. In all versions of Mercury up to and including the current ones, it has been possible to have multiple domain names serviced by a single Mercury instance, but all such domains shared a single user database (so, if you had a user called "john", then that user would appear in all the domains serviced by Mercury). Over time, it has become desirable to allow different domains to have their own users and mailing lists, separate from other domains on the server. There has also been growing demand for specialized types of domain, such as "MX Domains", where a Mercury instance can act as a secondary mail server for a domain that is not otherwise local (i.e, has no local users on the Mercury system). Finally, with a move to truly separate domains, it makes sense to allow domains to have their own specific filters, content and spam controls, whitelists and other mail processing controls, something not possible in the current implementation.

Providing these and other new capabilities will require considerable internal restructuring of the way Mercury works, and as an unavoidable consequence, certain internal Mercury API calls (the special functions made available to developers of third party expansion and plugin modules) are going to be affected. It is my aim to keep the disruption to the interfaces as small as possible, but there is inevitably going to be the possibility that some some existing third-party Mercury Daemons might require some adjustments. If you are a developer of Mercury Daemons, I would like to invite you to join the Mercury beta testing process so you can be kept informed of the changes well before the new code sees public release - please mail me directly, using my address if you would like to take part.

0
-1
closed
David Harris posted Feb 11 '10 at 1:44 am

The very earliest mail message I still have that was sent by Pegasus Mail is dated February 13th 1990: it is from me, to me, has the subject line "A little test", and a body that simply says "what are you doing for dinner tonight?". I know there were earlier messages than this, but because I had no sense of posterity, I never made any attempt to keep them - I really didn't think Pegasus Mail would be around for more than a couple of years, just like any other piece of software.

So, a bit like a puppy whose actual birthday is not known for sure, I have always celebrated Pegasus Mail's birthday on February 13th, except on years when that was a Friday (because I'm a little superstitious). Most years, the celebration consists of nothing much more than me thinking "Gosh, have I REALLY been doing this for xx years?!?", but this year seems worthy of a slightly more public observation - this year, Pegasus Mail turns 20. Scary though it seems, I have users of my program who were not yet born when it sent its first message. Even scarier is the thought that when I started doing this I was 28, and now I'm 48: I've spent not far off half my life working on Pegasus Mail and its constellation of related programs, documents, interfaces and technologies.

It's amazing to me, sitting here in my sunny little office in Dunedin, New Zealand, to think of everything that has happened in those 20 years - the people who have been involved, some who have stayed and some who have moved on; the lows and highs (and there have been plenty of both); the changes, both to the programs and to the world they live in... In fact, it's those changes that always astound me most, I think: when I started, the Internet was barely known: in New Zealand, "The Internet" had just recently moved from being a single 9600bps (that's "bits per second") leased line serving the entire country to an incredible 64Kbps ("kilobits per second") line - still serving the whole country... E-mail was the most exciting new thing I'd ever seen, the World Wide Web didn't exist at all, there was no such thing as a "URL", and the idea of "spam" was a kind of silly joke that nobody thought would amount to anything more than a mild, passing nuisance... In 1990, I was still occasionally using a modem to access BIX (the Byte Information Exchange), which was a "bulletin board" - an idea now totally lost in history. These things seem unreal now - distant and almost incomprehensible: indeed, there are days when I feel like Methuselah, impossibly old.

How different the world is now. E-mail is like bread - we consume it without even thinking about it any more, the World-Wide-Web has become the largest corporate battleground in history and most of the Western World spends a sizable portion of each day immersed in it; URLs are now more commonly-seen than phone numbers, spam constitutes more than 90% of all the mail circulating the world, and nobody under 30 would probably have even the faintest idea what a "modem" or a "bulletin board" were.

In almost any other field of human endeavour, 20 years is not a long time - it's not even a single human generation... But in the Internet, 20 years is near to eternity, and for 20 years, Pegasus Mail has been out there riding the clouds, pulling me along with it like a slightly bewildered Bellerophon. The program has always seemed to have something of a personal existence all of its own - there are times when I have really felt as though I was just along for the ride (probably more frequently as the program has become older). I rather like that, though - the sense that it's not just an outgrowth of me, but something that reflects the nature of a community of users - there's something personal about that, and I've always felt that many of those who have used Pegasus Mail for a long time feel it too.

So now we're about to enter the third decade of Pegasus Mail. Will it make it to 30? Who knows, but I really hope so: I still enjoy working on it, and even though the Pegasus Mail Thousand hasn't had entirely the response I might have hoped for, I'll keep working on Pegasus Mail and Mercury for as long as I can. I hope you'll join me for the ride, so we can watch together how the world will change again, but for now, if you are of a drinking disposition, think about raising a glass to Pegasus Mail on February 13th, as it celebrates this major milestone in its life: I certainly will be.

All my very best wishes to you all,

-- David Harris --
February 11th 2010.

0
-1
closed
David Harris posted Dec 24 '09 at 11:58 am

Just a short post to wish everyone a wonderful festive season - have a Happy Christmas, and let's look forward to a prosperous New Year - in fact, "New Decade"! (Scary, isn't it).

Pegasus Mail v4.52. with fixes for Windows 7, is now ready, and will be out between Christmas and New Year. In the New Year itself, January is likely to be quiet on the public front, but things should start heating up in February.

To everyone who has supported me this year, especially my beta test team and all those who have signed up for the Pegasus Mail Thousand (about 500 so far), my thanks for all your help and backing: please accept my warmest wishes and have a wondeful break.

Cheers!

-- David --

0
-1
closed
David Harris posted Oct 20 '09 at 3:00 pm

Well! It's been a while since I've written here - in fact, I admit I'm a little shocked how long... Oops: and I had promised myself I wouldn't have these long absences any more.

In my defense, I should say that I have been far from idle. Since the release of Pegasus Mail v4.51 and Mercury/32 v4.72, I've begun work on what is probably the single biggest change in the core innards of the programs since they were introduced: I am rewriting the Mail Store. Let me clarify that: the move from my old Borland C compiler to Visual C was a huge effort, but it was mostly about porting and tidying the existing code: what I am doing now is rewriting the deepest core of the program from the ground up.

The "Mail Store" is that part of the program that manages folders, messages, attachments - in fact, anything low-level to do with any mail message you can read in your copy of Pegasus Mail. The Mail Store currently in use in Pegasus Mail is very old - much of it has not changed markedly since the Windows version of the program was originally released in 1993. While it works quite well, it has become increasingly difficult to maintain, and nearly impossible to expand: it also does not lend itself to being made available to third-party developers (such as extensions writers). More pertinently, though, the Pegasus Mail code was not an entirely happy port to Mercury - many of the lingering performance issues associated with the MercuryI IMAP server can be directly attributed to the age of the Mail Store code.

The aim of the Mail Store rewrite is to have one common library used by both programs, and to apply the lessons I have learned in nearly 20 years of writing this type of program to producing something that will have industrial strength and high performance. This development is the basis for all the good things I have planned for both programs - a high-speed webmail interface for Mercury, new secure and encrypted folder formats for both programs, and much faster, more robust IMAP support that can take advantage of IMAP extensions when they are available (something the current code cannot easily do). The new Mail Store will also be directly available to extension and Daemon developers - for the first time, they will have access to the Mail Store that is exactly equal to that enjoyed by Pegasus Mail and Mercury themselves.

The process is going excellently, and is now well advanced. The target I had originally set myself was to have the new Mail Store essentially finished and ready to bolt into Pegasus Mail by the end of this year: past experience told me that I was probably being optimistic (it's symptomatic of how difficult the industry has become that I can't remember the last time I was able to hit a deadline I set myself) - but to my surprise, it's actually looking as though I may be on schedule this time. Last week, the new code initialized itself, then scanned and read a new mail folder for the first time: this week, it has successfully performed a comprehensive MIME parse on a new mail message... Each day brings new achievements, and a sense of satisfaction that things are finally being done in a modern, future-proof way.

Well, that will do for now: as soon as I can, I'll write another post here about The Pegasus Mail Thousand and what it's going to mean to the sustainability of the programs, and I'll make it my goal to post much more regular progress updates on what's happening behind the scenes.

All my best to you all!

Cheers!

-- David --

0
-1
closed
David Harris posted May 20 '09 at 4:34 pm

Just a short note to let you know that Mercury/32 v4.71 is ready for release. We're currently trying to get a problem sorted out with Symantec - their SEP product is incorrectly identifying the help files in the Mercury archive as viral, which results in failed installations. Why is it always Symantec who mess us around, I wonder? This is the third time in three years, yet no other vendor of A/V products has incorrectly tagged us in that time. Oh well...

Once the problem with Symantec is sorted, the release archive will be made publicly available. The big new feature in v4.71 is the ability to run as a native Windows Service, which is a major step forward for usability in server-based environments.

Release announcements will be made here and in the forums as soon as the archive is available. A formal release of Pegasus Mail v4.5 (with some cool new capabilities) should be out very soon as well.

Cheers!

-- David --

0
-1
closed
David Harris posted Feb 2 '09 at 3:13 am

Just a short posting to let you know that there should be a minor update release of Mercury/32 v4.6, probably with the version number v4.7, late in February.

The changes will not be gigantic - mostly bug fixes and small incremental improvements, such as allowing IMAP connections via direct-connect SSL.

We have begun a significant redevelopment process on some of the core innards of Mercury - most specifically the folder management layer and the user database subsystem: these changes, along with increased Windows integration, native service support, and increased web management capabilities will be the focus of our work this year (although I do have one "surprise" feature coming up in the next few months that I believe many sites will find extremely useful).

While I hate putting in shameless plugs, times are very hard, and all license purchases for Mercury are gratefully received at the moment. Furthermore, if you know of people you think might benefit from Mercury, please introduce them to it - the more the word gets around, the more likely it is that the program can continue to thrive.

Cheers!

-- David --

0
-1
closed
David Harris posted Feb 2 '09 at 3:10 am

Just a short note to let you know that after a complicated December/January period, we're now reviewing bug reports in the Pegasus Mail v4.51 public beta with the aim of getting a formal release out in the next few weeks.

My thanks to everyone who has taken time to report issues, and to Michael in der Wiesche for acting as the report overseer during this period.

When we're ready for a formal release, announcements will be made prominently here.

Cheers!

-- David --

0
-1
closed
David Harris posted Sep 3 '08 at 2:31 pm

A public beta of Pegasus Mail v4.5 is now available for anyone who

wishes to do so to try out. The beta is very complete and stable, but

is provided without formal technical support - you should almost

certainly apply due diligence testing to it before using it in a

production environment.


You can download the public beta from , or a full German version of the beta from http://download-us.pmail.com/w32-451-de-pb1.exe (and yes, we realize they appear to have different version numbers).


Peter Stromblad will post an announcement telling you about new

beta-related forums that will be made available in the community in the

near future - using these forums, you can discuss the beta and provide

feedback if you wish.


Please read the "What's new" notes in the beta installer or help file for information on the v4.5 release.


Cheers to all!


-- David --

0
-1
closed
David Harris posted Aug 11 '08 at 3:42 am

Just a short update on the status of WinPMail v4.5.

I had hoped to have the program released by now, but we've run into a late problem that I feel we have to resolve before a release can take place.

It is currently my intention to release a v4.5 public beta to this community as soon as the code is next in a stable state - hopefully before the end of this week. Once the public beta has had a chance to circulate a little and we're sure there are no remaining serious issues standing in the way of a full release, v4.5 will be made available on an unrestricted public basis.

Cheers!

-- David --

0
-1
closed
David Harris posted Jul 21 '08 at 6:15 am

I've had a number of people ask me if I could put together a blog posting describing what I'm doing at the moment, and what plans there are for Pegasus Mail and Mercury in the short-to-medium term.

Well, the first thing I'll tell you is that my current priority is to get Pegasus Mail v4.5 out the door. On the face of it, it won't look like much has happened, and I suspect many people will be rather disappointed by v4.5, but in reality, more than 50% of the entire codebase of the program has been modified in some way during the shift from Borland C++ to Visual C++; it's been a huge, largely unrewarding job that simply had to be done. In the process, though, we've caught and fixed literally hundreds of small and medium-sized bugs that have been present in the program for periods ranging from recent history to more than a decade.

A huge amount of effort this year has gone into releasing Mercury/32 v4.6 - once again, there was a lot of code modernization going on that doesn't yield obvious visible benefits, but which simply had to be done.

But not everything is porting code and fixing bugs. I've also been spending a lot of time familiarizing myself with new technologies that will have significant bearing on both Mercury and Pegasus Mail:

  • For Mercury, I've been spending a lot of time learning about CSS and XHTML, because the future of Mercury is clearly heavily web-oriented.
  • I've been spending a lot of time learning about SQL and particularly SQLite, because it's clear that both Pegasus Mail and Mercury are going to need robust database facilities in future.
  • I've developed my own object programming interface for both programs: this is a key technology that will have a huge bearing on future development by making it easier to extend the programs' capabilities without side-effects.
  • I'm well into developing a complete replacement for the fundamental message store used by both Pegasus Mail and Mercury. This new component, called MailStore,  is easily the single most important development in either product in more than a decade, and should be in a working state towards the end of this year.
  • I've spent a lot of time this year working on documentation - the bane of all programmers... The Mercury/32 Daemon Developer Kit in particular was a huge amount of work that was vastly overdue, and I do hope to start seeing new Mercury Daemons appearing before much longer.
  • Finally, I've spent a lot of time developing tools - either because I was forced to do so (as in the case of the new help system, which I was forced to do by Microsoft), or because I believe they'll be important in upcoming releases of either program.
I realize I'm not being quite as specific as I'm sure many of you would like: I'm not saying that "X feature will be working by August", or that "Y feature is now operational": I promise that as I get to a position where I can make reasonable statements about new developments I will do so, but in truth, the last two years have been a gigantic, painful, tedious process of modernization and tooling up for new capabilities, rather than actually working on those capabilities per se.

I also realize that I've been somewhat of a stranger to the forums this year: I apologize for that, and will say honestly that my absence has largely been a result of tech-support burnout... Doing technical support is a very soul-destroying process, because you're typically seeing people at their worst and most stressed, and nobody really likes spending all of their time dealing with the bad aspects of their work. As a result, spending too long doing technical support is a recipe for depression and loss of motivation, and I've been doing technical support at some level for nearly two decades now. It's my aim to become a more regular participant in the forums, although I will be consciously and actively trying to resist the self-destructive urge to solve every problem I see posted. I'd like to get back into the forums so I can enjoy interacting with the people who use my work without always burning myself out trying to fix all their problems.

So, there you have it - a slightly longer, and maybe slightly more candid posting than I intended, but I hope it serves to give you a little insight into where I am at the moment.

Cheers to you all!

-- David --

0
-1

I've resisted doing anything very much with Windows Vista until now, but eventually reality catches up with fantasy, so I finally decided it was time to get going. So, I've now got Windows Vista UItimate running here on a laptop (a Dell XPS1530), and over the next few weeks will be taking a bit of time to familiarize myself with its ins and outs.

First impressions usually don't really mean too much - the most I'd usually say about them is that they're a useful indicator of the things you *liked* in previous versions of the OS. In this case, there are a few first impressions that come to mind.

  • Vista seems to want to connect to the Net for *everything*. My Dell came with a TV tuner card, and Windows even wants to connect to the Net before I can use that. Now, I accept that the Net is pretty much omnipresent now, but even so, there are still going to be plenty of times when you want to use a laptop disconnected.
  • UAC... Well, I guess that about says it all. I've used this machine for probably three hours so far, and I'm already SO frustrated with the unending UAC prompts that I'm going to turn them off. This is a case of Microsoft getting an important feature so totally wrong that I don't even know why they bothered.
  • The Windows "sidebar"... I've thought and thought, and I just can't see what use it has. None of the gadgets that comes with the system really seems to have any point to me at all, and I struggle to imagine anything that I would want there. It looks to me like a feature looking for a justification... But of course, other people's mileage may vary.
  • The arbitrary changes to the "Start" menu - in particular the ugly new "programs" menu with its odd scrollbar and strange positioning - these seem like changes that will only serve to frustrate and annoy users used to the way XP worked. Of course, many times you think this, then a few months later (after you've become used to the new approach) you wonder how you ever lived without it... But I don't think this will be one of those cases.
  • The arbitrary changes to where everything is really annoy me - control panels are changed, things have been moved, simple tasks have become enormously difficult (for example, I actually had to open the machine up by removing two torx screws to find the WLAN card's MAC address so I could tell my WLAN about it - something that was a simple right-click under XP).
  • I don't know if I'm ever going to get used to the new Window close/minimize/maximize controls - they just look as though someone has positioned them carelessly during window layout, the way they are (for those few of you who might not have encountered Vista yet, these controls are now half the height of the caption bar and anchored to the top edge of the bar - they look really out of place to me).
  • The look of the system is quite slick, but doesn't really seem complete to me somehow... I can't quite put my finger on it yet, but I think possibly it seems a little "over-stylized" to me - it seems kind of "busy". I'm sure I'll get used to that, though.

As an aside, the XPS1530 is a pretty cool piece of kit - slim and light, attractive to look at, yet still with everything you could ever want on board. I'd have no problems recommending it, regardless of what I end up thinking of Vista.

As I use the system more and develop clearer feelings about the issues, I may write more here on the subject.

Cheers!

-- David --

0
-1
closed
David Harris posted Feb 20 '08 at 12:20 am

Mercury/32 v4.6 is now late in testing, and I thought I'd give you a preview of some the new capabilities it holds.

  • Threaded core:  The core module (the central part of the program that handles mail delivery and routing) is now somewhat multithreaded, resulting in throughput performance increases of anything up to 350% on busy queues.
  • MercuryP (the POP3 server) has been totally overhauled. It is now vastly faster, and a number of problems have been fixed (in particular, an issue arising from a long-standing bug in Windows where message UIDs would change after a daylight savings time change, resulting in mail being downloaded again). It also now includes the short-term blacklisting capabilities found in MercuryI and MercuryS for handling pests and brute-force password crackers.
  • MercuryP now supports login-time filtering: simply by adding any of several filter criteria to the username you use to login to MercuryP, you can control the mail it will present to you. This means that if you only want to see unread urgent mail from "bob" on your PDA, you can now do this, just by adding the string "(urgent,new,from=bob)" to your login name.
  • Mercury now includes a new commandline mail generator called MSendTo: MSendTo can generate mail in a wide range of formats and writes directly to the Mercury queue. It will be very useful for system administrators and anyone wanting to send mail under programmatic control.
  • New consoles: the Mercury protocol modules now have a new console design and interface. I know this doesn't sound like much, and you won't notice a lot of difference, but it's a huge step towards making the Mercury user interface remotable.
  • The MercuryB MLSS mailing list subscription management interface has been totally overhauled. It now offers important new features like the ability to change your status for all lists on the server and to retrieve your list password by e-mail, and has numerous improvements and fixes. A mailing list moderator utility is in development but probably won't be available until the next release.
  • MercuryI (the IMAP server) has had a number of fixes, and now caches the inbox, resulting in a huge performance increase when opening that folder.
  • Lots of small bug fixes.
  • Update notifications and advisories: licensed copies of Mercury will be able to receive notifications of new updates and advisories on security and other issues. License uptake on Mercury has been fair since the licenses became available, but it could definitely be better, and I'm hoping this feature might act as a motivator.

It's possible that this feature list may expand prior to release, but right now, release is itself a feature. I hope to get v4.6 out the door soon.

Cheers!

-- David --

0
-1
closed
David Harris posted Jan 1 '08 at 11:29 am

Well. 2008... Amazing.

It's been 2008 for about 23 hours now, and I admit I'm still wondering what happened to 2007 - it seemed to be over almost before it began.

Many years ago, Queen Elizabeth, in one of her Christmas Broadcasts, described the year she had just had as her "Annus Horribilis", a term that has been widely reused, recycled and abused since then. Although I don't much like resorting to cliché, I have to admit that her little latinate touch was just the right term to describe the way I feel about 2007. Put simply, 2007 was, overall, not a great year for me - it started out badly, and got worse. BUT, and this is a big but, towards the end of the year, it also began to get better, and I now allow myself to feel some quiet hope that the road ahead is finally clearer and less treacherous.

2007 was a year of rebuilding for me - in almost every way. For Pegasus Mail and Mercury, the year was about massive reconstruction, followed by a process of consolidation - making right, then making good. Although frustrating, many of the processes I've had to go through with both products this year were necessary and unavoidable, but with them largely complete, the way is open for smoother development in future. Expect to see more releases in 2008 than there have been in the last two years combined, as we finally begin moving towards Pegasus Mail v5, and as Mercury gets more and more robust and refined.

No blog reflecting on 2007 would be complete without mentioning the work of Peter Stromblad in setting up this community: I believe this site has created a wonderful forum for both programs, and its existence has certainly eased the burden of technical support (which was getting very hard to cope with in previous years) on me and my team. I realize that I've been quiet for the last couple of months, after an initial burst of postings - I expect to get back into more regular attendance in the course of the next few weeks.

Finally, I'd like to thank everyone who has believed in Pegasus Mail and Mercury during the hardest year of their long lives, but especially my beta test team, without whom I simply could not function at all.

So then, in these final minutes of January 1st 2008, I'd like to offer you all my warmest wishes for the New Year: let's get this show on the road - I'm looking forward to enjoying the journey. :-)

Cheers!

-- David --

0
-1
closed
David Harris posted Oct 24 '07 at 4:44 pm

I've been doing this (writing software) for most of my 46 years, and professionally for well over 20 now: one notion that imprints itself more and more forcibly upon me with each passing year is how much more complicated everything seems to get.

I suppose it is in the nature of the human animal to elaborate: caves become tents, which become mud huts, then grow into drafty stone castles, and eventually into towering skyscrapers... Or, bones become sharpened sticks, which become arrows and spears, then muskets, rifles, machine guns until now the US military is testing the "vomit cannon" (a weapon that induces mass nausea by sound waves). With each innovation, every incremental step along the path from humble hamlet to sparkling skyline, things get more complex... One seldom-understood effect of this is that knowledge becomes more and more specialized: any chimp can wield a bone as a weapon, but how many single people would know how to build a vomit cannon from scratch? As tasks get more complex, they progressively surpass the ability of any individual to encompass them in their entirety. The way humans deal with this problem is by specializing - by splitting the task into smaller, more manageable pieces then splicing them together at the end.

The problem with dealing with complexity in this way is that it also introduces complication. Fred Brooks famously stated that "Adding manpower to a late software project makes it later", which can be taken as a general comment on the complication that arises when you increase the number of people involved in any task (not just software tasks). Different people approach problems in different ways, have different priorities, and differing sets of skills: considering this, it always seems miraculous to me that *any* large software project ever yields results at all, let alone working well.

Pegasus Mail is now around 460,000 lines of C code, and Mercury around 200,000. By commercial standards, these are middle-sized projects, and well beyond the optimum size for a single developer. With size comes complexity and complication - nothing in these programs is easy any more - indeed, it makes me weep when I compare the productivity that I could achieve when I was writing the early DOS versions of Pegasus Mail back in 1990 with the almost glacial progress I can manage now. Features that I could have added in a day back then would take a month now, and the depth of knowledge and background required to do them is vastly greater. I think I'm a better programmer now than I was in 1990, but it takes me ten times longer to achieve anything, and it's hard to describe how frustrating that is.

I must admit that when I think of Windows Vista with its tens of millions of lines of code, I can scarcely credit that it works: and the more I consider it, the more a kind of visceral, almost feral mistrust arises in me - how can I trust something so complex that no single human being is capable of fathoming it in its entirety? Would I trust my life to it? My reputation? My identity? Honestly, it's beginning to scare the hell out of me that we're becoming so dependent on this type of technology.

Perhaps I'm being silly, wistful and naive, but wouldn't it be nice if things were simpler? "Simple" is the lost virtue of this age, and I can't help but worry that we're going to miss it far more than we realize.

Cheers!

-- David --

35
101
3
Actions
Hide topic messages
Enable infinite scrolling
Previous
12
Next
All posts under this topic will be deleted ?
Pending draft ... Click to resume editing
Discard draft