Community Discussions and Support
More on Mercury IMAP problems

I've basically given up on Mercury IMAP. IMAP problems are the #1 reason I'm having to migrate our company from Mercury to Exchange. Granted we're probably a worst-case small business scenario for any IMAP server.

* 50 employees, half of whom carry iPhones or iPads for email.

* Outlook 2007, 2010, 2013, and 2016 are all in use around the company. And the CEO carries a MacBook.

* Half our workforce works from home or travels 75%+.

* We have staff who assume it should be possible to have slashes and ampersands in folder names, and deeply nested folders.

It's perfectly typical for me to see some employee mailboxes with 10 connections to their mailbox, given the way they "flip" between iPhone/iPad and Outlook client throughout the day, or attempt to use email while traveling/in flight on dodgy airline WiFi. 

Most other Mercury features have been flawless: POP3, forwarding, aliases, mailing lists, you name it.

IMAP with mobile devices + Outlook clients is a disaster. I reconstruct a mailbox each month, on average.

<p><span style="font-size: 13.3333px;">I've basically given up on Mercury IMAP. </span>IMAP problems are the #1 reason I'm having to migrate our company from Mercury to Exchange. Granted we're probably a worst-case small business scenario for any IMAP server.</p><p>* 50 employees, half of whom carry iPhones or iPads for email.</p><p>* Outlook 2007, 2010, 2013, and 2016 are all in use around the company. And the CEO carries a MacBook.</p><p>* Half our workforce works from home or travels 75%+.</p><p>* We have staff who assume it should be possible to have slashes and ampersands in folder names, and deeply nested folders.</p><p>It's perfectly typical for me to see some employee mailboxes with 10 connections to their mailbox, given the way they "flip" between iPhone/iPad and Outlook client throughout the day, or attempt to use email while traveling/in flight on dodgy airline WiFi. </p><p>Most other Mercury features have been flawless: POP3, forwarding, aliases, mailing lists, you name it. </p><p>IMAP with mobile devices + Outlook clients is a disaster. I reconstruct a mailbox each month, on average.</p>

We are struggling with issues related to Mercury and IMAP connections.  For a long time have observed that Mercury appears to have problems with more than 1 simultaneous connections to the IMAP server.  Today I observe behavior that somewhat explains what is going on.  This time it happens to me, however looking back explains what I have seen with other users many times.  At one point earlier in the day I had connections with a Windows PC and at the same time with an Android tablet.  I see the familiar 'Folder in user by other connections' message from Thunderbird on the PC.  Next I power the Android tablet all the way off.  After a long time (hours) still see the same error even though I am sure I have the only IMAP client connecting to Mercury.  For the sake of test, I disconnect my connection to the Internet at this end such that my IP address no longer exists and no connection is possible for over 15 minutes.  FWIW the TCPIP timeout in Mercury is set at 120 sec, I tried decreasing it to 60 seconds for test, still the same.  Now, at least 2 hours after powering the Android tablet off and in the mean time totally disconnecting this end from the Internet (I am remote from the server), the Current connections window in Mercury still shows the connection.  Looks very much like Mercury is not capable of dealing with dropped connections correctly. 

At this point help is desperately needed.  David, Peter, anyone suggestions please!

Gus

<p>We are struggling with issues related to Mercury and IMAP connections.  For a long time have observed that Mercury appears to have problems with more than 1 simultaneous connections to the IMAP server.  Today I observe behavior that somewhat explains what is going on.  This time it happens to me, however looking back explains what I have seen with other users many times.  At one point earlier in the day I had connections with a Windows PC and at the same time with an Android tablet.  I see the familiar 'Folder in user by other connections' message from Thunderbird on the PC.  Next I power the Android tablet all the way off.  After a long time (hours) still see the same error even though I am sure I have the only IMAP client connecting to Mercury.  For the sake of test, I disconnect my connection to the Internet at this end such that my IP address no longer exists and no connection is possible for over 15 minutes.  FWIW the TCPIP timeout in Mercury is set at 120 sec, I tried decreasing it to 60 seconds for test, still the same.  Now, at least 2 hours after powering the Android tablet off and in the mean time totally disconnecting this end from the Internet (I am remote from the server), the Current connections window in Mercury still shows the connection.  Looks very much like Mercury is not capable of dealing with dropped connections correctly.  </p><p>At this point help is desperately needed.  David, Peter, anyone suggestions please!</p><p>Gus </p>

All I can do is confirm the problem.  Our outside sales and installation folks connect via IMAP with iDevices but want to use the local Pegasus Mail client when in the office but the mailbox remains locked even after the iDevice is turned off.  Users have not choice but to ignore the lock warning.  There has been one instance of a hosed mailbox that I suspect was the result of doing this.

Edit:  I also regularyly make IMAP connections using Pegasus Mail and have made IMAP connections via Thunderbird.  Those connections seem to close fine although they are made in the evenings and on weekends so it is many hours  before a local Pegasus Mail connection is attempted.

<p>All I can do is confirm the problem.  Our outside sales and installation folks connect via IMAP with iDevices but want to use the local Pegasus Mail client when in the office but the mailbox remains locked even after the iDevice is turned off.  Users have not choice but to ignore the lock warning.  There has been one instance of a hosed mailbox that I suspect was the result of doing this.</p><p>Edit:  I also regularyly make IMAP connections using Pegasus Mail and have made IMAP connections via Thunderbird.  Those connections seem to close fine although they are made in the evenings and on weekends so it is many hours  before a local Pegasus Mail connection is attempted. </p>

Please note that there are two different timeouts in play here:

TCP/IP timeout

The length of time in seconds that MercuryI should wait for data on a connection before assuming that the connection is no longer valid and aborting it.

Idle timeout

The length of time Mercury should allow connected clients to remain connected between commands before deciding that they have hung and terminating the connection. MercuryI uses the idle timeout when it is not explicitly expecting a response or a command from the client, whereas the TCP/IP timeout (see above) is used when MercuryI is explicitly waiting for a response from the client. The TCP/IP timeout will almost always be significantly shorter than the idle timeout. You cannot set the idle timeout to a value shorter than 30 minutes, because of the explicit requirements of the IMAP4 protocol definition (see RFC3501).

If a device closes the IMAP connection to the server properly the mailbox lock should immediately be removed. If however a device just silently disappears (due to connections problems or bad manners) the idle timeout should remove the connection after the stipulated time.

If, when using v4.80, there are cases when the idle timeout isn't behaving that way please provide as complete information as possible, including log files, and we will look into it.

 

<p>Please note that there are two different timeouts in play here:</p><blockquote><p><b><i>TCP/IP timeout</i></b></p><p><i>The length of time in seconds that MercuryI should wait for data on a connection before assuming that the connection is no longer valid and aborting it.</i></p><p><b><i>Idle timeout</i></b></p><p><i>The length of time Mercury should allow connected clients to remain connected between commands before deciding that they have hung and terminating the connection. MercuryI uses the idle timeout when it is not explicitly expecting a response or a command from the client, whereas the TCP/IP timeout (see above) is used when MercuryI is explicitly waiting for a response from the client. The TCP/IP timeout will almost always be significantly shorter than the idle timeout. You cannot set the idle timeout to a value shorter than 30 minutes, because of the explicit requirements of the IMAP4 protocol definition (see RFC3501).</i></p></blockquote><p>If a device closes the IMAP connection to the server properly the mailbox lock should immediately be removed. If however a device just silently disappears (due to connections problems or bad manners) the idle timeout should remove the connection after the stipulated time.</p><p>If, when using v4.80, there are cases when the idle timeout isn't behaving that way please provide as complete information as possible, including log files, and we will look into it.</p><p> </p>

Idle timeout is set to the minimum 30 min.  Which log files would you like?  What other info?

 

 

<p>Idle timeout is set to the minimum 30 min.  Which log files would you like?  What other info? </p><p> </p><p>  </p>

If possible please do a controlled test (where you actually have access to all involved devices) and let us see the standard IMAP log for the time of the test plus IMAP session logs for the problematic connections. Screenshots of the console window can be helpful as well if there is additional information. Other than that just describe what was expected to happen and where that failed.

<p>If possible please do a controlled test (where you actually have access to all involved devices) and let us see the standard IMAP log for the time of the test plus IMAP session logs for the problematic connections. Screenshots of the console window can be helpful as well if there is additional information. Other than that just describe what was expected to happen and where that failed.</p>

I tried to run a quick test on Sunday while doing other projects.  This time the connection from the tablet closed correctly when it was powered down.  I have seen this happen enough that I am sure it will happen again, just a matter of catching it.  I did collect logs and at least learned from them that the Thunderbird message 'Folder in use by other connections' is happening due to failing to expunge due to other connections.  This could explain another problem we have been seeing from some time.  If for whatever reason Mercury crashes while a user is in this state where they have moved or deleted a bunch of stuff from their inbox and expunge fails, messages that were moved come back in the inbox and there will ultimately end up being multiple copies in the trash or whatever folder they were moved to.  The last updates to Mercury did reduce the number of crashes, however we are still seeing at least one crash per day.

On a related note, when more than one IMAP client is connected and this is happening, the second one to connect is not able to get new messages in the inbox.  We observe this happening with many different IMAP clients, Android, IOS, Thunderbird on PC or Mac, Outlook on PC, all behave the same.

I did capture logs of the test I ran on Sunday, has the two clients problem, but shutting down the tablet did not hang the connection this time.  I can get you these logs as long as we can find a way to send them without posting them for the public to see on the forum.

I did try to run a test this morning (Monday) again, however with session logging enabled in the  IMAP server we had 700Mbytes of logs in half an hour.  At that rate, I can not afford to leave logging on for too long or we run into disk space problems.  It would sure be nice if there was a way to limit logging to a specific IP address like can be done in SMTP server.  This also makes it a bit difficult to determine exactly which logs to look at.  For the moment, I grep them all looking for the existence of a specific IP address.

Things I remember from the day the connection hung last week are 1.  The connection remains active in the 'Current Connections and status' window on the Mercury IMAP4 server window.  2.  running netstat on the Mercury box shows an established connection to the public static IP address at this end, even an hour after everything at this end has been disconnected from the Internet.

Looks like this is really two problems, probably related.  1.  Remote connections occasionally not disconnecting when the should and 2.  Inability to connect with more than one IMAP client at the same time.

Gus

<p>I tried to run a quick test on Sunday while doing other projects.  This time the connection from the tablet closed correctly when it was powered down.  I have seen this happen enough that I am sure it will happen again, just a matter of catching it.  I did collect logs and at least learned from them that the Thunderbird message 'Folder in use by other connections' is happening due to failing to expunge due to other connections.  This could explain another problem we have been seeing from some time.  If for whatever reason Mercury crashes while a user is in this state where they have moved or deleted a bunch of stuff from their inbox and expunge fails, messages that were moved come back in the inbox and there will ultimately end up being multiple copies in the trash or whatever folder they were moved to.  The last updates to Mercury did reduce the number of crashes, however we are still seeing at least one crash per day.</p><p>On a related note, when more than one IMAP client is connected and this is happening, the second one to connect is not able to get new messages in the inbox.  We observe this happening with many different IMAP clients, Android, IOS, Thunderbird on PC or Mac, Outlook on PC, all behave the same.</p><p>I did capture logs of the test I ran on Sunday, has the two clients problem, but shutting down the tablet did not hang the connection this time.  I can get you these logs as long as we can find a way to send them without posting them for the public to see on the forum.</p><p>I did try to run a test this morning (Monday) again, however with session logging enabled in the  IMAP server we had 700Mbytes of logs in half an hour.  At that rate, I can not afford to leave logging on for too long or we run into disk space problems.  It would sure be nice if there was a way to limit logging to a specific IP address like can be done in SMTP server.  This also makes it a bit difficult to determine exactly which logs to look at.  For the moment, I grep them all looking for the existence of a specific IP address. </p><p>Things I remember from the day the connection hung last week are 1.  The connection remains active in the 'Current Connections and status' window on the Mercury IMAP4 server window.  2.  running netstat on the Mercury box shows an established connection to the public static IP address at this end, even an hour after everything at this end has been disconnected from the Internet. </p><p>Looks like this is really two problems, probably related.  1.  Remote connections occasionally not disconnecting when the should and 2.  Inability to connect with more than one IMAP client at the same time. </p><p>Gus </p>

It seems to me that the crashes you mention are the main problem here. We haven't seen any crashes at all with v4.80 on our main server (except maybe when I try a new version of a daemon).

I'll send you contact details for logs. Waiting for something to happen on a busy server with session logging switched on is as you say not really ideal. If possible it's better to try to provoke an error by connecting from one or more clients. 

<p>It seems to me that the crashes you mention are the main problem here. We haven't seen any crashes at all with v4.80 on our main server (except maybe when I try a new version of a daemon).</p><p><span style="font-size: 10pt;">I'll send you contact details for logs. Waiting for something to happen on a busy server with session logging switched on is as you say not really ideal. If possible it's better to try to provoke an error by connecting from one or more clients. </span></p>
live preview
enter atleast 10 characters
WARNING: You mentioned %MENTIONS%, but they cannot see this message and will not be notified
Saving...
Saved
With selected deselect posts show selected posts
All posts under this topic will be deleted ?
Pending draft ... Click to resume editing
Discard draft