Community Discussions and Support
[Mercury Feature Suggestion] Workaround for Outlook Mobile IMAP STARTTLS bug

The mobile versions of Microsoft Outlook have broken support for using STARTTLS during an IMAP connection. After sending the first STARTTLS command and receiving a reply, they send a second, spurious STARTTLS command. This gets misinterpreted as a bizzare, implausibly high TLS version by OpenSSL, which causes it to cut the connection. Outlook then reports an alleged "certificate error".


Outlook Mobile is still able to connect using the legacy port which uses TLS from the beginning. However, if you connect to the normal port, if it detects STARTTLS support it will attempt to use it even if you attempt to configure it not to.


There's a Python mail suite called Twisted that includes an IMAP server implementation and TLS implementation in pure-ish Python. I was able to build a toy server and patch it so that if received a TLS version of 63.48 (for example), instead of immediately disconnecting it tried to reinterpret that packet as a cleartext IMAP command and, if it was STARTTLS, send another affirmative reply and restart TLS from there. With this patch, Thunderbird, Outlook desktop, Outlook mobile, and Twisted's client libraries were all able to connect and run without issue.


I know this is awful, that OpenSSL goes out of its way to make it hard to support this -- and with good reason -- and that Microsoft should fix their application to comply with the IMAP specification. However, I've reported this problem to them several times and they won't fix it. There is no MercuryI Events support in the DDK (and its internals are specifically and explicitly not documented, unlike most of the rest of Mercury) so I can't fix this on my end; if I'm to get this to work, I'll need it in Mercury itself.


(I'm posting this here because the Feature Suggestions forum doesn't allow new topics, I don't think this is within the scope of a technical support incident, and Mercury's licensing system is down anyway so I couldn't file one even if I was wrong.)


The mobile versions of Microsoft Outlook have broken support for using STARTTLS during an IMAP connection. After sending the first STARTTLS command and receiving a reply, they send a second, spurious STARTTLS command. This gets misinterpreted as a bizzare, implausibly high TLS version by OpenSSL, which causes it to cut the connection. Outlook then reports an alleged "certificate error". Outlook Mobile is still able to connect using the legacy port which uses TLS from the beginning. However, if you connect to the normal port, if it detects STARTTLS support it will attempt to use it even if you attempt to configure it not to. There's a Python mail suite called Twisted that includes an IMAP server implementation and TLS implementation in pure-ish Python. I was able to build a toy server and patch it so that if received a TLS version of 63.48 (for example), instead of immediately disconnecting it tried to reinterpret that packet as a cleartext IMAP command and, if it was STARTTLS, send another affirmative reply and restart TLS from there. With this patch, Thunderbird, Outlook desktop, Outlook mobile, and Twisted's client libraries were all able to connect and run without issue. I know this is awful, that OpenSSL goes out of its way to make it hard to support this -- and with good reason -- and that Microsoft should fix their application to comply with the IMAP specification. However, I've reported this problem to them several times and they won't fix it. There is no MercuryI Events support in the DDK (and its internals are specifically and explicitly not documented, unlike most of the rest of Mercury) so I can't fix this on my end; if I'm to get this to work, I'll need it in Mercury itself. (I'm posting this here because the Feature Suggestions forum doesn't allow new topics, I don't think this is within the scope of a technical support incident, and Mercury's licensing system is down anyway so I couldn't file one even if I was wrong.)

I've corrected access rules for Mercury suggestions, and I'll let David know about this post.


I've corrected access rules for Mercury suggestions, and I'll let David know about this post.

From your description of it, this is something that would actually require changes to OpenSSL - it's not something I can fix or workaround in Mercury; by the time the spurious STARTTLS comes in, I've already called OpenSSL to handle the handshake.


I make it a point of absolute policy not to modify anything in OpenSSL - I'm not a cryptographer, so any changes I made would be ill-informed at best, dangerous to security at worst.


I'll have a look on the OpenSSL forums and see if there is anything there that I could implement. It's about time for me to look at bringing out an updated OpenSSL build anyway.


From your description of it, this is something that would actually require changes to OpenSSL - it's not something I can fix or workaround in Mercury; by the time the spurious STARTTLS comes in, I've already called OpenSSL to handle the handshake. I make it a point of absolute policy not to modify anything in OpenSSL - I'm not a cryptographer, so any changes I made would be ill-informed at best, dangerous to security at worst. I'll have a look on the OpenSSL forums and see if there is anything there that I could implement. It's about time for me to look at bringing out an updated OpenSSL build anyway.

Windows Server 2019


I just changed out an expired certificate in Mercury (both GoDaddy multi-url certs) and now I'm getting "no common encryption method" when I try to use STARTTLS
in either POP or SMTP.


I'm on Mercury 4.91, and have made sure that the only OpenSSL in my system is Version 3.1 and it's on my Path. Does Mercury have a built-in version of OpenSSL? And if so, how can it be updated?


Only way to access mail right now is to disable STARTTLS which most users don't want to do, so I need to get this fixed quickly. Thanks for any pointers.


Windows Server 2019 I just changed out an expired certificate in Mercury (both GoDaddy multi-url certs) and now I'm getting "no common encryption method" when I try to use STARTTLS in either POP or SMTP. I'm on Mercury 4.91, and have made sure that the only OpenSSL in my system is Version 3.1 and it's on my Path. Does Mercury have a built-in version of OpenSSL? And if so, how can it be updated? Only way to access mail right now is to disable STARTTLS which most users don't want to do, so I need to get this fixed quickly. Thanks for any pointers.

Hi Bill,


Mercury comes with it's own version of Openssl. there is no need to have a different one installed. Upgrading would be nice, but I can't help you with that.


Just install your new certs and don't refer to your installed OpenSSL. That would be my advise right now. Maybe others know more ?


Johannes


Hi Bill, Mercury comes with it's own version of Openssl. there is no need to have a different one installed. Upgrading would be nice, but I can't help you with that. Just install your new certs and don't refer to your installed OpenSSL. That would be my advise right now. Maybe others know more ? Johannes

Thanks Johannes for the response.


The OpenSSL on the machine is required for other processes. The certs are installed and I'm unclear why the "no common encryption" method error is occurring.


I had a version of Thunderbird that was a few versions out of date, so updated it after the errors started, but the new version has the same error.


How can I debug this further? I tried session logging in both POP and SMTP and the error is the same -
OK Begin SSL/TLS negotiation...<cr><lf>
12:36:33.102: [!] OpenSSL reported error -1/1 during handshake - diagnostics follow:
12:36:33.102: [!] -------------------------------------------------------------------------
12:36:33.102: [!] error:1417A0C1 : SSL routines:tls_post_process_client_hello:no shared cipher
12:36:33.102: [!] -------------------------------------------------------------------------
Any debugging suggestions?


Thanks Johannes for the response. The OpenSSL on the machine is required for other processes. The certs are installed and I&#039;m unclear why the &quot;no common encryption&quot; method error is occurring. I had a version of Thunderbird that was a few versions out of date, so updated it after the errors started, but the new version has the same error. How can I debug this further? I tried session logging in both POP and SMTP and the error is the same - OK Begin SSL/TLS negotiation...&lt;cr&gt;&lt;lf&gt; 12:36:33.102: [!] OpenSSL reported error -1/1 during handshake - diagnostics follow: 12:36:33.102: [!] ------------------------------------------------------------------------- 12:36:33.102: [!] error:1417A0C1 : SSL routines:tls_post_process_client_hello:no shared cipher 12:36:33.102: [!] ------------------------------------------------------------------------- Any debugging suggestions?
edited Oct 25 '23 at 6:39 pm

Hi Bill,


2 things here, Mercury want's it's own openssl as mentioned and for Thunderbird, so I have minimal to no experince with it, but seems not compatible with OpenSSl 3, maybe only with OpenSSL1.... not sure right now which version of the openssl1. line is current.


Looks to me, you have to make a choice in regards the application which needs OpenSSL 3 and the mail system, server as even the client.


Johannes


Hi Bill, 2 things here, Mercury want&#039;s it&#039;s own openssl as mentioned and for Thunderbird, so I have minimal to no experince with it, but seems not compatible with OpenSSl 3, maybe only with OpenSSL1.... not sure right now which version of the openssl1. line is current. Looks to me, you have to make a choice in regards the application which needs OpenSSL 3 and the mail system, server as even the client. Johannes

Well, I appreciate the thought, though the following is true.


  • The machine had both OpenSSL 1.1 and OpenSSL 3.1 installed. The Path pointed to 3.1. T-bird accounts set up with STARTTLS.
  • All worked great with Mercury 4.91 with the old cert and T-bird 113.
  • I updated the cert and it all failed with "no shared cipher"
  • I upgraded T-bird to 115, "no shared cipher"
  • I moved OpenSSL 1.1. out of the way so that only 3.1 was accessible, no change
  • I restored OpenSSL 1.1 and pointed the path to it. No change
  • (stopping and starting Mercury after every change).
  • re-created the cert with references from the provider, - no difference

In between all these I changed T-bird to no security, tested to confirm connection and send/retrieve, then changed back to STARTTLS. Restarted T-bird multiple times.


I've run out of things I know to test, or configurations to alter. Hoping for further suggestions. Thanks.


Well, I appreciate the thought, though the following is true. - The machine had both OpenSSL 1.1 and OpenSSL 3.1 installed. The Path pointed to 3.1. T-bird accounts set up with STARTTLS. - All worked great with Mercury 4.91 with the old cert and T-bird 113. - I updated the cert and it all failed with &quot;no shared cipher&quot; - I upgraded T-bird to 115, &quot;no shared cipher&quot; - I moved OpenSSL 1.1. out of the way so that only 3.1 was accessible, no change - I restored OpenSSL 1.1 and pointed the path to it. No change - (stopping and starting Mercury after every change). - re-created the cert with references from the provider, - no difference In between all these I changed T-bird to no security, tested to confirm connection and send/retrieve, then changed back to STARTTLS. Restarted T-bird multiple times. I&#039;ve run out of things I know to test, or configurations to alter. Hoping for further suggestions. Thanks.

I found this comment about someone having a similar problem with Exim:


Some old mail servers which communicate with yours might not like modern signature algorithm such as EC 256 bits (ecdsa-with-SHA256, ecdsa-with-SHA384, ecdsa-with-SHA512), hence the error might occur.


So you will need to re-create a certificate for a hostname at least. If you use free certificates from Let's Encrypt, then run the following command as root to create a certificate with a sha256WithRSAEncryption signature algorithm:


So I tried having Mercury generate a self-signed certificate, assuming that Mercury might generate a certificate it would like, but still got the same error. Seems like it might be a mis-match between T-bird and Mercury, but then why was it working with the old cert and not the new? Totally confused.


I found this comment about someone having a similar problem with Exim: Some old mail servers which communicate with yours might not like modern signature algorithm such as EC 256 bits (ecdsa-with-SHA256, ecdsa-with-SHA384, ecdsa-with-SHA512), hence the error might occur. So you will need to re-create a certificate for a hostname at least. If you use free certificates from Let&#039;s Encrypt, then run the following command as root to create a certificate with a sha256WithRSAEncryption signature algorithm: So I tried having Mercury generate a self-signed certificate, assuming that Mercury might generate a certificate it would like, but still got the same error. Seems like it might be a mis-match between T-bird and Mercury, but then why was it working with the old cert and not the new? Totally confused.

Hi Bill,


that bears the question: what has changed in addition to the new Certificate?
On my system, as an example, I have no installation of OpenSSL at all.


And I just tried with TB 115.3.1 to access Mercury (SMTP & IMAP) with one of my accounts using STARTTLS and had no problems accessing per IMAP and sending out per SMTP.


Johannes


Hi Bill, that bears the question: what has changed in addition to the new Certificate? On my system, as an example, I have no installation of OpenSSL at all. And I just tried with TB 115.3.1 to access Mercury (SMTP &amp; IMAP) with one of my accounts using STARTTLS and had no problems accessing per IMAP and sending out per SMTP. Johannes

Bill, I'm replying here because I'm not sure you're receiving my mail from the Mercury support account (I've sent two messages regarding this issue now, one of which bounced with no error diagnostic).


Mercury uses its own build of OpenSSL, which is compiled and linked into TCPIP.DLL. There is no issue using it in conjunction with other copies of OpenSSL that might be installed on the system. The build used in Mercury v4.91 is OpenSSL 1.1.1k, which is not the most recent, but is recent enough that there is no chance of protocols having changed in a way that would render it unsuitable (it has full support for TLS 1.2 and TLS 1.3, and a full suite of standard and EC encryption/keying modules).


What is most likely happening is that you have not installed the certificate where Mercury expects to find it, and it is either loading the old one still, or not loading a certificate at all. To be clear on this, Mercury expects a certificate to exist in two files, in a location specified in the SSL configuration dialog for the Mercury module that uses it. The first file should contain the actual certificate, should have the extension .PEM, and must be in PEM format - let's call this file "server.pem". The second file should contain the private key for the certificate: it must have the same name part as the certificate file, and must have the extension ".prk.pem". So, in our example here, the private key should be called "server.prk.pem". Depending on how your CA provides the certificate, you may be required to do a little editing to separate the two files.


So... You need to check is that the certificate is in the correct format, that it and its private key have the expected names, and that it's in the location where Mercury is configured to look for it. If these factors are all correct, there is no reason why it should not work correctly.


-- David --


Bill, I&#039;m replying here because I&#039;m not sure you&#039;re receiving my mail from the Mercury support account (I&#039;ve sent two messages regarding this issue now, one of which bounced with no error diagnostic). Mercury uses its own build of OpenSSL, which is compiled and linked into TCPIP.DLL. There is no issue using it in conjunction with other copies of OpenSSL that might be installed on the system. The build used in Mercury v4.91 is OpenSSL 1.1.1k, which is not the most recent, but is recent enough that there is no chance of protocols having changed in a way that would render it unsuitable (it has full support for TLS 1.2 and TLS 1.3, and a full suite of standard and EC encryption/keying modules). What is most likely happening is that you have not installed the certificate where Mercury expects to find it, and it is either loading the old one still, or not loading a certificate at all. To be clear on this, Mercury expects a certificate to exist in two files, in a location specified in the SSL configuration dialog for the Mercury module that uses it. The first file should contain the actual certificate, should have the extension .PEM, and must be in PEM format - let&#039;s call this file &quot;server.pem&quot;. The second file should contain the private key for the certificate: it must have the same name part as the certificate file, and must have the extension &quot;.prk.pem&quot;. So, in our example here, the private key should be called &quot;server.prk.pem&quot;. Depending on how your CA provides the certificate, you may be required to do a little editing to separate the two files. So... You need to check is that the certificate is in the correct format, that it and its private key have the expected names, and that it&#039;s in the location where Mercury is configured to look for it. If these factors are all correct, there is no reason why it should not work correctly. -- David --

Thank you very much, David. I was playing whack-a-mole with attacking ip addresses and accidentally got a range wrong, which is why you couldn't respond.


And, egg on my face, I forgot that the key had to be named the same thing. I must have glossed over that in the help file multiple times.


All is working fine now, sorry for the trouble everyone.


Thank you very much, David. I was playing whack-a-mole with attacking ip addresses and accidentally got a range wrong, which is why you couldn&#039;t respond. And, egg on my face, I forgot that the key had to be named the same thing. I must have glossed over that in the help file multiple times. All is working fine now, sorry for the trouble everyone.
live preview
enter atleast 10 characters
WARNING: You mentioned %MENTIONS%, but they cannot see this message and will not be notified
Saving...
Saved
With selected deselect posts show selected posts
All posts under this topic will be deleted ?
Pending draft ... Click to resume editing
Discard draft