Community Discussions and Support
Mercury GUI terminated due to MS Server 2016 April updates?

Since the last MS updates of April, which we've installed on our MS Server 2016 on weekend, Mercury GUI has been terminated automatically different times.


No any error messages found, neither in Mercury logs nor Windows event logs. Has anybody an idea?


The Mercury service will be started on each server restart. And on next opportunity we quit the service and start the Mercury GUI manually to permanently see the Mercury "dashboard" on the server desktop. This worked for years.


Since the last MS updates of April, which we've installed on our MS Server 2016 on weekend, Mercury GUI has been terminated automatically different times. No any error messages found, neither in Mercury logs nor Windows event logs. Has anybody an idea? The Mercury service will be started on each server restart. And on next opportunity we quit the service and start the Mercury GUI manually to permanently see the Mercury "dashboard" on the server desktop. This worked for years.
edited Apr 24 '23 at 10:55 am

If Mercury is crashing it's usually because some other process is interfering with disk access or network access. Check if any crash dump files (extension .dmp) has been created in the main Mercury directory or the dumps sub-directory.


Other than that I suppose it could be a problem if the service instance of Mercury for some reason hasn't been completely shut down or is trying to restart while the regular program instance is running.


If Mercury is crashing it's usually because some other process is interfering with disk access or network access. Check if any crash dump files (extension .dmp) has been created in the main Mercury directory or the dumps sub-directory. Other than that I suppose it could be a problem if the service instance of Mercury for some reason hasn't been completely shut down or is trying to restart while the regular program instance is running.
edited Apr 24 '23 at 4:08 pm

Hi Rolf,
Thanks for patiently answering here in this forum. Appreciated.


If Mercury is crashing it's usually because some other process is interfering with disk access or network access. Check if any crash dump files (extension .dmp) has been created in the main Mercury directory or the dumps sub-directory.

No *.dmp files available, except one of 2014 and another one of 2022.


Other than that I suppose it could be a problem if the service instance of Mercury for some reason hasn't been completely shut down or is trying to restart while the regular program instance is running.

I'm aware of the issues of different simultaneously working instances of Mercury. I've made such experiences years ago and that's why I'm always thoroughly checking the task manager before manually restarting Mercury with GUI. Further the Windows service of Mercury is not set to "automatic restart after errors" or something like that.


But another thought ... The one and only difference of yesterday/today was that we presently have one streetworker at Singapore. He is logging into our VPN, getting a local IP and is then connecting to Mercury via Thunderbird and/or Roundcube. And I saw his Mercury/Pmail user name within the MercuryI GUI window always directly when restarting Mercury.
But since he logged out from VPN no further crashes or teminations have been occured today afternoon.
Tomorrow morning I'm going to carry out a little test session with him. He should made different mail operations with TB and Roundcube and I will see if anything happens ...
I will report.


Hi Rolf, Thanks for patiently answering here in this forum. Appreciated. [quote="pid:55339, uid:2278"]If Mercury is crashing it's usually because some other process is interfering with disk access or network access. Check if any crash dump files (extension .dmp) has been created in the main Mercury directory or the dumps sub-directory.[/quote] No *.dmp files available, except one of 2014 and another one of 2022. [quote="pid:55339, uid:2278"]Other than that I suppose it could be a problem if the service instance of Mercury for some reason hasn't been completely shut down or is trying to restart while the regular program instance is running.[/quote] I'm aware of the issues of different simultaneously working instances of Mercury. I've made such experiences years ago and that's why I'm always thoroughly checking the task manager before manually restarting Mercury with GUI. Further the Windows service of Mercury is not set to "automatic restart after errors" or something like that. But another thought ... The one and only difference of yesterday/today was that we presently have one streetworker at Singapore. He is logging into our VPN, getting a local IP and is then connecting to Mercury via Thunderbird and/or Roundcube. And I saw his Mercury/Pmail user name within the MercuryI GUI window always directly when restarting Mercury. But since he logged out from VPN no further crashes or teminations have been occured today afternoon. Tomorrow morning I'm going to carry out a little test session with him. He should made different mail operations with TB and Roundcube and I will see if anything happens ... I will report.
edited Apr 24 '23 at 6:26 pm

Good Morning Rolf,
After further investigation I found a most lately saved file "deadlock.fcd.log" in Mercury folder. Therein different single sections with time code are listed which could match to the recent termination times of Mercury. Such a section looks like as follows:



Critical Section deadlock detected, Sun, 23 Apr 2023, 10:47:59
Owner ID of thread attempting to enter section: 1541 (0x605)
Caller's private reference value: 0 (0x0)
Owner ID of thread marked as owning section: 1541 (0x605)
Owner's private reference value: 0 (0x0)
Current / Lock count / recursion count: 1 / -2 (0xFFFFFFFE) / 1 (0x1)
Owning thread's system thread ID: 4420 (0x1144)
Timeout interval defined for section (ms): 1200000
Difference between entry and current ticks: 1201156
IDs of most recent owners: 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605)
REFs of most recent owners: 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0)


Unfortunately I didn't find anything about such deadlock file in the Mercury Manual. Could you kindly give a short explanation about the meaning of this file please?


Further we've made some tests with our colleague in Singapore. After establishing the VPN tunnel he carried out different IMAP mail operations with Thunderbird. It has been shown that various IMAP connections from his IP address remain permanently (in MercuryI status window), even if he has already quit Thunderbird again.
So it could indeed have something to do with competing sessions or login attempts, if certain IMAP sessions remain open, because e.g. the VPN tunnel is on strike and Thunderbird reconnects anyway.
Further I saw also entries from his IP address in the status window marked with [Not logged in].


How long Mercury is keeping (dead) IMAP connection (or is showing them in the status window) although the mail client is not longer available and disconnected already?


Good Morning Rolf, After further investigation I found a most lately saved file "deadlock.fcd.log" in Mercury folder. Therein different single sections with time code are listed which could match to the recent termination times of Mercury. Such a section looks like as follows: __ Critical Section deadlock detected, Sun, 23 Apr 2023, 10:47:59 Owner ID of thread attempting to enter section: 1541 (0x605) Caller's private reference value: 0 (0x0) Owner ID of thread marked as owning section: 1541 (0x605) Owner's private reference value: 0 (0x0) Current / Lock count / recursion count: 1 / -2 (0xFFFFFFFE) / 1 (0x1) Owning thread's system thread ID: 4420 (0x1144) Timeout interval defined for section (ms): 1200000 Difference between entry and current ticks: 1201156 IDs of most recent owners: 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605), 1541 (0x605) REFs of most recent owners: 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0), 0 (0x0) __ Unfortunately I didn't find anything about such deadlock file in the Mercury Manual. Could you kindly give a short explanation about the meaning of this file please? Further we've made some tests with our colleague in Singapore. After establishing the VPN tunnel he carried out different IMAP mail operations with Thunderbird. It has been shown that various IMAP connections from his IP address remain permanently (in MercuryI status window), even if he has already quit Thunderbird again. So it could indeed have something to do with competing sessions or login attempts, if certain IMAP sessions remain open, because e.g. the VPN tunnel is on strike and Thunderbird reconnects anyway. Further I saw also entries from his IP address in the status window marked with [Not logged in]. How long Mercury is keeping (dead) IMAP connection (or is showing them in the status window) although the mail client is not longer available and disconnected already?
edited Apr 25 '23 at 11:22 am

Deadlock files are diagnostic files created by Mercury when a work thread has received permission to change specific global data but failed to complete this action and release the lock within specified time. I'll forward the information to David who might be able to interpret it. We may need the rest of the file as well, if so we'll get back to you about it.


Deadlock files are diagnostic files created by Mercury when a work thread has received permission to change specific global data but failed to complete this action and release the lock within specified time. I'll forward the information to David who might be able to interpret it. We may need the rest of the file as well, if so we'll get back to you about it.

Hi Rolf,


As follows please find the complete file for further investigations. Please unpack the zip file. Thanks


deadlock.fcd.log.zip


Hi Rolf, As follows please find the complete file for further investigations. Please unpack the zip file. Thanks [deadlock.fcd.log.zip](serve/attachment&path=6448102473f0d)
edited Apr 25 '23 at 6:39 pm

David has now had a look at the log file and responds that this indicates mailbox folder timeouts. Such timeouts are rare but can happen during IMAP access if a folder is damaged or contains very many (60 000 or more) messages. If you can identify the mailbox the mbxmaint commandline tool or mbxmaint_ui GUI version can be used to check and repair folders. (Both these programs can be found in the main Mercury directory.)


David has now had a look at the log file and responds that this indicates mailbox folder timeouts. Such timeouts are rare but can happen during IMAP access if a folder is damaged or contains very many (60 000 or more) messages. If you can identify the mailbox the mbxmaint commandline tool or mbxmaint_ui GUI version can be used to check and repair folders. (Both these programs can be found in the main Mercury directory.)

Hi Rolf,
Thanks for the reply and your hints regarding possible folder damages. I will check this tomorrow, when I'm back in the office. Generally I'm aware of this and the available maintenance tools. Mail folders of my users will be checked regarding their size (should not exceed 2GB) and damages regularly. But until now we didn't experience any folder damages when working with IMAP clients, but only in connection with Pmail. But the affected user is also accessing via Pmail, when he is in the office. Insofar ...
Anyway, let me check the folder consistancy then I will report.


Greetings to David


edit:
Just checked all mail folders of the affected user. Only one older folder of 2018 was faulty and has been repaired by maintenance tool. Don't know whether the user accessed this folder. He is actually on his flight back.


Hi Rolf, Thanks for the reply and your hints regarding possible folder damages. I will check this tomorrow, when I'm back in the office. Generally I'm aware of this and the available maintenance tools. Mail folders of my users will be checked regarding their size (should not exceed 2GB) and damages regularly. But until now we didn't experience any folder damages when working with IMAP clients, but only in connection with Pmail. But the affected user is also accessing via Pmail, when he is in the office. Insofar ... Anyway, let me check the folder consistancy then I will report. Greetings to David edit: Just checked all mail folders of the affected user. Only one older folder of 2018 was faulty and has been repaired by maintenance tool. Don't know whether the user accessed this folder. He is actually on his flight back.
edited Apr 26 '23 at 5:23 pm

Provisional conclusion.


Since my colleague is back from Singapore and no one else has accessed MercuryI through VPN tunnel from Singapore, Mercury is running stable again without any automatic shutdowns or crashs although lot of other users are permanently accessing MercuryI.


In the meantime I've also updated the VPN Client of the affected user and have repaired one corrupt mail folder of 2018 (as already reported).


Now I'm waiting for the next journey of a colleague to Asia to see what happens. I will report ...


Provisional conclusion. Since my colleague is back from Singapore and no one else has accessed MercuryI through VPN tunnel from Singapore, Mercury is running stable again without any automatic shutdowns or crashs although lot of other users are permanently accessing MercuryI. In the meantime I've also updated the VPN Client of the affected user and have repaired one corrupt mail folder of 2018 (as already reported). Now I'm waiting for the next journey of a colleague to Asia to see what happens. I will report ...
live preview
enter atleast 10 characters
WARNING: You mentioned %MENTIONS%, but they cannot see this message and will not be notified
Saving...
Saved
With selected deselect posts show selected posts
All posts under this topic will be deleted ?
Pending draft ... Click to resume editing
Discard draft