lost connection with 127.0.0.1[127.0.0.1] while sending end of data #1523
Comments
|
I haven't had time to debug this either, but I've seen the issue of double sent emails in similar circumstances. It only started with version 0.40 - nothing else changed for us, if that helps in any way. I'll update if I get time to look into this more. |
|
@sgrimmett I also think that this started from v0.40 but not 100% sure as I didn't scan logs before. |
|
@MasterPCUK the only thing I can think of without actually digging is that it is related to graylisting - this changed (from what I recall) when moving up to .40. For the life of me, though, I can't find where I read that! |
|
We dropped a custom package and are using the stock postgrey package now. Very little in Mail-in-a-Box changed in v0.40 (it's 40, not 4 --- the number just goes up by 1 each release). If there's a configuration problem, it might be more likely to be in the Ubuntu defaults that might have changed from 14.04 to 18.04. |
|
I'm not sure if we should suspect graylisting as in my case: postgrey[7874]: action=pass |
|
Thanks Josh - updated my typo to 0.40! It's probably not greylisting if stock and based on MasterPCUK's comment... just throwing out what I recall being different from 0.3x versions. Will advise if I find anything further related to this issue. |
|
A little update. I created 4 email address for one domain. Email arrived in all four mailboxes, but got saved only for first two, getting delivered twice for user3 and user4. Could this be something to do with RCPT TO in postfix and or timing issue? Below log with time codes. |
|
|
|
Hello. I had a lot of troubles this week caused by this bug. I finally found this thread and your workaround seems to work well (The CPU has higher loads when we receive a mail, but at least it works). I have commented on all the symptoms in this other thread. https://discourse.mailinabox.email/t/cant-receive-mails/4624 In my case, we have about 120 users in the mail-in-a-box server. The server with version 14.04 of ubuntu worked perfectly. This saturday we migrate to 18.04 and the troubles started at monday (when the users begins to send emails). A large number of these emails have 3 or more recipients. In fact, in some of our accounts, we have rules on roundcube to forward the incoming mail to a big number of users (12). So, the queues begins to grow very quickly. I reinstalled the backup another time this night to be sure that the migration was correct, but in 40 minutes we have 500 messages in the queue again. When we use your workaround the queues were reduced gradually. Now we havn't got any mail in the server's queue. You may have a lot of information of our troubles in the link of the forum, but I am at your disposal if you want me to do some test on my server to try to solve this problem. In my opinion, it is important that the next version of mail-in-a-box has this problem corrected (because this trouble has to affect a lot more people). |
|
Hi folks. I did a little Googling and didn't see any documentation suggesting I tried sending my box an email with To: containing two local users, and the mail was delivered fine. Before making a change to |
|
No problem. I will try lmtp_destination_recipient_limit = 10 and i will keep an eye on the queues to see what happens. I tell you in a while what happens in these circumstances. |
|
@JoshData on my box, the issue is not sending an email to two local users, it is when there is more than 2. I switched to lmtp_destination_recipient_limit = 1, and my issues were gone. I'm just tried lmtp_destination_recipient_limit = 10.. and got double messages still. I haven't had time to dig into the logs to see if they differ, but will post when I have a chance to do so. For now, I'm sticking with lmtp_destination_recipient_limit = 1, until I have more time to explore further. |
|
OK. I made the test. I sended a message to 5 email destination with the "=10" parameter. When i change this parameter i made a "sudo service postfix restart" an then i send the message. One of the recipients have roules in roundcube to resend it to more persons. I wait 4 minutes and the message still in queque. So i change again the parameter to "=1" and restart postfix again and the message come out of the queue quickly. Then, i send another message in same conditions. The new message with the "=1" parameter come out in a few seconds. If you need me to do more tests, tell me. @sgrimmett says anything about double messages. This days many people complained about repeated emails lot of times. Since I change the parameter to "=1" we haven't got this trouble. Regards. |
|
@JoshData This issue only happens when mail is sent to three or more recipients in TO field. Can you please try that? |
|
In the log posted above, the issue seems to be that Dovecot is not sending proper 250 responses for recipients 3 and 4, even though it says it has stored the mail. Hence Postfix doesn't know they were delivered and will retry again next time. spampd times out because it is waiting for responses about the last 2 recipients (code). If spampd wasn't in the mix, postfix would just disconnect when Dovecot does and the end result would be the same (postfix would retry the last 2 recipients later). From LMTP protocol RFC §4.2:
For the record, Right now the only change I might see in spampd for this issue is that it disconnect from the sender (postfix) if the destination (Dovecot) hangs up. Instead of waiting for the replies that will obviously never come and then timing out. Would not solve the delivery issue at all, but it might better represent where the actual problem lies. |
|
I got the same "lost connection with 127.0.0.1[127.0.0.1] while sending end of data" issue with mail delivering more than 2 times for email with more than 3 cc lists. I am using MIAB 0.40 in Ubuntu 18.4 with 200GB spaces in DO |
|
Actually I just noticed in the log that Postfix sent a So now I'm not sure what exactly postfix is expecting. Dovecot does in fact happily quit, though not before sending one more 250 response through. Which postfix still seems to process... even after having sent It might be useful to see what happens if spampd is taken out of the loop (for a test) and postfix was set up to deliver directly to dovecot. Should be just a matter of changing the port postfix delivers to... but I don't know anything about MIAB setup. The only possibly relevant postfix setting I could find offhand is:
Not clear to me what response it would be waiting for. The |
|
Hello I took spampd out of the loop and delivered mail straight to dovecot. |
|
@MasterPCUK Could you try this version? I just commented out the "patch for LMTP" code where it waits for all recipients to be ack'd. Keep the debugging on please. Thanks.
|
|
@mpaperno |
Thank you... @MasterPCUK .... its working for now.. |
|
Strange... does it (spampd or postfix) still time out? Or does spampd ever log "Closed connections" message at the end? Also a bit confusing because we can't see the Not understanding right now why Dovecot would send 3 250 responses in the "no-spampd" test but only 2 with spampd involved... if that's in fact what is going on. Or if Postfix just assumes the rest were delivered after Dovecot disconnects. It is possible spampd prevents Postfix from detecting when the receiving side closed the socket (which isn't trivial to detect). |
|
@mpaperno As far as I can see, LMTP patch code is never hit. |
|
Okay... that could explain things. :) But why does it not detect the protocol? Can you see what
says? Could stick that just before the What version of spampd was originally installed in this setup? 2.42? |
|
@mpaperno Yes, original version was 2.42. |
|
Wow... I'm not sure that ever worked. Wonder how that "LMTP patch" was tested :) This is the original code (LMTP patch restored) with the protocol detection (hopefully) working. Simple regex change in SpamPD::Server::chat(). |
|
@mpaperno Yes, I can confirm this working. Log: |
|
👍 Thanks for all the testing! I'll have a new official |
|
@mpaperno Fantastic! Thank you for your help! |
|
Looks like Debian or Ubuntu was patching it to work, and they must have removed the patch somewhere between Ubuntu 14.04 and now. |
…a-box/mailinabox#1523); Fix Warning for "Use of uninitialized value in string" (closes #22).
|
https://github.com/mpaperno/spampd/releases/tag/2.53 I doubt the old Ubuntu releases will be updated, at least not by default. There's an auto-updated list of Linux distros and which versions of spampd they package on https://github.com/mpaperno/spampd/blob/master/readme.md . As you can see, they're not quick to update :) I'd recommend MIAB install/update spampd directly from the GH repo instead of relying on distro versions. After proper testing, of course ;-)
Huh? They'd have to publish the changes somewhere... never mind maybe file a bug report. Anyway, the LMTP patch was added in v2.40, and you can see from that list that Ubuntu 14.04 is still using spampd v2.30. So it either somehow worked before the "LMTP patch" (which seems unlikely based on above tests), or maybe no one used multi-delivery/lmtp... or something else changed. |
|
The Debian patch is at https://sources.debian.org/patches/spampd/2.30-23/52-fix-multidest-lmtp.patch/. (This took me a long time to find.) The patch was removed in the latest version. |
|
@JoshData Sorry, actually I think you're right... I looked at the old spampd release notes and the LMTP patch did come upstream. I found the version Debian Stretch uses, and it's patched with the LMTP fix. At first it looked like the same fix they submitted to me upstream (which got included in 2.40), but there's one crucial difference in that regex check which breaks detection of lmtp in the first place. So I guess after they updated to 2.42 (in Ubuntu 18.04) they dropped the patch, but no one ever reported the new issue until now. |
|
I'm currently testing latest spampd https://github.com/mpaperno/spampd/releases/tag/2.53 on a live Mail-In-A-Box v0.40. So far it works as expected. It would be great if other could also test and let us know the results. |
|
Thanks everyone!! I'm adding |
|
I dropped a note to the maintainer of the Debian I also reviewed the remaining patches which the Debian package applies, and there are two which change the default settings of Add startup options to configure like patched Debian/Ubuntu package default settings:
Or patch: Or use patched version: |
|
OK. I'm testing the complete patch. I made the following. In my home folder: wget https://github.com/mpaperno/spampd/archive/2.53.zip An then i commented the line who I added before, #lmtp_destination_recipient_limit = 1 I saved the changes an i made sudo shutdown -r now I made all this 10 minutes ago and i made some tests (And all my users are sending and receiving mails) and it seems that all works well. In a cople of hours i'll say you if all remains working. @JoshData Perhaps is better test the complete patch instead of doing only lmtp_destination_recipient_limit = 1 . If this patch is working, it can be added to 0.41and resolve all the troubles at same time. |
|
Since the change of my previous post, it seems that all works normally. The mails with multiple recipients get out of the queues fast and all mail are arriving. |
|
6 hours from apply the patch and all works perfect. All mail is sent and received correctly and there are no mails in queue. As soon as they enter in the queue, they go out. Seems that it solves the trouble (Note: my MIAB has 120 users and it does a hard work dialy, so I think is a good test). My last stats of received mail are this.
|
|
Updated I have no idea how to get Ubuntu to update their older releases, or if it's even possible. |
|
Has anyone considered requesting a backport? 2.53 is available in 20.04. Although I've successfully built a PPA for myself a loooong time ago, I've never requested a backport. |
Hello
I tired to debug this my self, but I can't.
This happens every time with incoming emails with three or more recipients.
So in comes and email to recipient=info@myhost.tld, with CC's to print@myhost.tld, sales@myhost.tld.
lmtp saves message for info@myhost.tld and for print@myhost.tld.
But for sales@myhost.tld message get's sent twice and this error.
I am really lost. At first I and another person suspected spampd as seen here:
mpaperno/spampd#23
But that does not seem to be the case.
Here is log related to this issue.
Not sure where to look, so I hope that some one can help to figure this one out.
The text was updated successfully, but these errors were encountered: