Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Characters being stripped from emails since upgrade from 2.1.2 to 2.3.0 #2816

Closed
leemathewbrooks opened this issue Feb 23, 2024 · 21 comments · Fixed by #2812
Closed

Characters being stripped from emails since upgrade from 2.1.2 to 2.3.0 #2816

leemathewbrooks opened this issue Feb 23, 2024 · 21 comments · Fixed by #2812
Labels
bug star-issue This is an issue which is an excellent example of reporting an issue - just flagged as an example

Comments

@leemathewbrooks
Copy link

Describe the bug

I have a bizarre issue wherein HTML appears to be getting mangled in seemingly random emails.
Viewing an affected message renders fine in the backend (with no missing characters), however downloading the email from the backend, or viewing the resulting email in Outlook etc. results in broken source code. See below:

The message source is correct in the "raw-xxxx-xx-xx" database table for a given message, so the issue seems to be occurring once it's being parsed either for download or delivery.

It would appear that the first character from any attributes wrapped in quotes is being lost, but I'm at a loss as to why.
The same template is used for numerous emails (the source is a WooCommerce website) and share the same header/footer HTML etc, which in turn isn't affected.

I upgraded to 2.3.0 from I believe 2.1.2 a few days ago, at which point the issue began.
I don't believe the email HTML itself is the problem, however I'll gladly be proven otherwise.

Click tracking is turned off, though open tracking is enabled. Toggling these still produces the issue.

I'm happy to provide any further information/run tests etc to get this nailed.

Original message source

<!DOCTYPE html>
<html lang="en-GB">
   <head>
      <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
      <meta content="width=device-width, initial-scale=1.0" name="viewport">
      <title>The Whole Shebang</title>
      <style type="text/css">@media screen and (max-width: 600px){#header_wrapper{padding: 27px 36px !important; font-size: 24px;}#body_content_inner{font-size: 10px !important;}}</style>
   </head>
   <body leftmargin="0" marginwidth="0" topmargin="0" marginheight="0" offset="0" style="background-color: #f7f7f7; padding: 0; text-align: center;" bgcolor="#f7f7f7">
      <table width="100%" id="outer_wrapper" style="background-color: #f7f7f7;" bgcolor="#f7f7f7">
         <tr>
            <td>
               <!-- Deliberately empty to support consistent sizing and layout across multiple email clients. -->
            </td>
            <td width="600">
               <div id="wrapper" dir="ltr" style="margin: 0 auto; padding: 70px 0; width: 100%; max-width: 600px; -webkit-text-size-adjust: none;" width="100%">
                  <table border="0" cellpadding="0" cellspacing="0" height="100%" width="100%">
                     <tr>
                        <td align="center" valign="top">
                           <div id="template_header_image">
                              <p style="margin-top: 0;"><img src="https://thewhole-shebang.co.uk/wp-content/uploads/2023/03/wsi.jpg" alt="The Whole Shebang" style="border: none; display: inline-block; font-size: 14px; font-weight: bold; height: auto; outline: none; text-decoration: none; text-transform: capitalize; vertical-align: middle; max-width: 100%; margin-left: 0; margin-right: 0;" border="0"></p>
                           </div>
                           <table border="0" cellpadding="0" cellspacing="0" width="100%" id="template_container" style="background-color: #fff; border: 1px solid #dedede; box-shadow: 0 1px 4px rgba(0,0,0,.1); border-radius: 3px;" bgcolor="#fff">
                              <tr>
                                 <td align="center" valign="top">
                                    <!-- Header -->
                                    <table border="0" cellpadding="0" cellspacing="0" width="100%" id="template_header" style='background-color: #3c3c3c; color: #fff; border-bottom: 0; font-weight: bold; line-height: 100%; vertical-align: middle; font-family: "Helvetica Neue",Helvetica,Roboto,Arial,sans-serif; border-radius: 3px 3px 0 0;' bgcolor="#3c3c3c">
                                       <tr>
                                          <td id="header_wrapper" style="padding: 36px 48px; display: block;">
                                             <h1 style='font-family: "Helvetica Neue",Helvetica,Roboto,Arial,sans-serif; font-size: 30px; font-weight: 300; line-height: 150%; margin: 0; text-align: left; text-shadow: 0 1px 0 #636363; color: #fff; background-color: inherit;' bgcolor="inherit">Password Reset Request</h1>
                                          </td>
                                       </tr>
                                    </table>
                                    <!-- End Header -->
                                 </td>
                              </tr>
                              <tr>
                                 <td align="center" valign="top">
                                    <!-- Body -->
                                    <table border="0" cellpadding="0" cellspacing="0" width="100%" id="template_body">
                                       <tr>
                                          <td valign="top" id="body_content" style="background-color: #fff;" bgcolor="#fff">
                                             <!-- Content -->
                                             <table border="0" cellpadding="20" cellspacing="0" width="100%">
                                                <tr>
                                                   <td valign="top" style="padding: 48px 48px 32px;">
                                                      <div id="body_content_inner" style='color: #636363; font-family: "Helvetica Neue",Helvetica,Roboto,Arial,sans-serif; font-size: 14px; line-height: 150%; text-align: left;' align="left">
                                                         <p style="margin: 0 0 16px;">Hi XXXX,</p>
                                                         <p style="margin: 0 0 16px;">Someone has requested a new password for the following account on The Whole Shebang:</p>
                                                         <p style="margin: 0 0 16px;">Username: XXXX</p>
                                                         <p style="margin: 0 0 16px;">If you didn't make this request, just ignore this email. If you'd like to proceed:</p>
                                                         <p style="margin: 0 0 16px;">
                                                            <a class="link" href="https://thewhole-shebang.co.uk/my-account/lost-password/?key=XXXX&amp;id=XXXX" style="font-weight: normal; text-decoration: underline; color: #3c3c3c;">		Click here to reset your password	</a>
                                                         </p>
                                                         <p style="margin: 0 0 16px;">Thanks for reading.</p>
                                                      </div>
                                                   </td>
                                                </tr>
                                             </table>
                                             <!-- End Content -->
                                          </td>
                                       </tr>
                                    </table>
                                    <!-- End Body -->
                                 </td>
                              </tr>
                           </table>
                        </td>
                     </tr>
                     <tr>
                        <td align="center" valign="top">
                           <!-- Footer -->
                           <table border="0" cellpadding="10" cellspacing="0" width="100%" id="template_footer">
                              <tr>
                                 <td valign="top" style="padding: 0; border-radius: 6px;">
                                    <table border="0" cellpadding="10" cellspacing="0" width="100%">
                                       <tr>
                                          <td colspan="2" valign="middle" id="credit" style='border-radius: 6px; border: 0; color: #8a8a8a; font-family: "Helvetica Neue",Helvetica,Roboto,Arial,sans-serif; font-size: 12px; line-height: 150%; text-align: center; padding: 24px 0;' align="center">
                                             <p style="margin: 0 0 16px;">The Whole Shebang</p>
                                          </td>
                                       </tr>
                                    </table>
                                 </td>
                              </tr>
                           </table>
                           <!-- End Footer -->
                        </td>
                     </tr>
                  </table>
               </div>
            </td>
            <td>
               <!-- Deliberately empty to support consistent sizing and layout across multiple email clients. -->
            </td>
         </tr>
      </table>
      <p class='ampimg' style='display:none;visibility:none;margin:0;padding:0;line-height:0;'><img src='https://click.shop.thewhole-shebang.co.uk/img/nuknvj/MJSVRz7XKdnO' alt=''></p>
   </body>
</html>

Delivered message source

<!DOCTYPE html>
<html lang="n-GB&quot;">
   <head>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <meta content="idth vice-width," initial-scale="0&quot;" name="iewport&quot;">
      <title>The Whole Shebang</title>
      <style type="ext/css&quot;">@media screen and (max-width: 600px){#header_wrapper{padding: 27px 36px !important; font-size: 24px;}#body_content_inner{font-size: 10px !important;}}</style>
   </head>
   <body leftmargin=" marginwidth=" topmargin=" marginheight=" offset=" style=ackground-color: #f7f7f7; padding: 0; text-align: center;" bgcolor="f7f7f7&quot;">
      <table width="00%&quot;" id="uter_wrapper&quot;" style="ackground-color:" #f7f7f7;?="" bgcolor="f7f7f7&quot;">
      <tr>
         <td>
            <!-- Deliberately empty to support consistent sizing and layout across multiple email clients. -->
         </td>
         <td width="00&quot;">
            <div id="rapper&quot;" dir="tr&quot;" style="argin:" 0auto;padding:70px0;width:100%;max-width:600px;-webkit-text-size-adjust:none;?="" width="00%&quot;">
            <table border=" cellpadding=" cellspacing=" height=00%" width="00%&quot;">
               <tr>
                  <td align="enter&quot;" valign="op&quot;">
                     <div id="emplate_header_image&quot;">
                        <p style="argin-top:" 0;?=""><img src="ttps://thewhole-shebang.co.uk/wp-content/uploads/2023/03/wsi.jpg&quot;" alt="he" WholeShebang?="" style="order:" none;display:inline-block;font-size:14px;font-weight:bold;height:auto;outline:none;text-decoration:none;text-transform:capitalize;vertical-align:middle;max-width:100%;margin-left:0;margin-right:0;?="" border="&gt;&lt;/p&gt;									&lt;/div&gt;
                        &lt;table border=" cellpadding=" cellspacing=" width="00%&quot;" id="emplate_container&quot;" style="ackground-color:" #fff;border:1pxsolid#dedede;box-shadow:01px4pxrgba(0,0,0,.1);border-radius:3px;?="" bgcolor="fff&quot;">
               <tr>
               <td align="enter&quot;" valign="op&quot;">
               <!-- Header -->
               <table border=" cellpadding=" cellspacing=" width=00%" id="emplate_header&quot;" style="ackground-color:" #3c3c3c;color:#fff;border-bottom:0;font-weight:bold;line-height:100%;vertical-align:middle;font-family:?HelveticaNeue?,Helvetica,Roboto,Arial,sans-serif;border-radius:3px3px00;?="" bgcolor="3c3c3c&quot;">
               <tr>
               <td id="eader_wrapper&quot;" style="adding:" 36px48px;display:block;?="">
               <h1 style="ont-family:" ?HelveticaNeue?,Helvetica,Roboto,Arial,sans-serif;font-size:30px;font-weight:300;line-height:150%;margin:0;text-align:left;text-shadow:01px0#636363;color:#fff;background-color:inherit;?="" bgcolor="nherit&quot;">Password Reset Request</h1>
               </td>
               </tr>
            </table>
            <!-- End Header -->
         </td>
      </tr>
      <tr>
      <td align="enter&quot;" valign="op&quot;">
      <!-- Body -->
      <table border=" cellpadding=" cellspacing=" width=00%" id="emplate_body&quot;">
      <tr>
      <td valign="op&quot;" id="ody_content&quot;" style="ackground-color:" #fff;?="" bgcolor="fff&quot;">
      <!-- Content -->
      <table border=" cellpadding=0" cellspacing=" width=00%">
      <tr>
      <td valign="op&quot;" style="adding:" 48px48px32px;?="">
      <div id="ody_content_inner&quot;" style="olor:" #636363;font-family:?HelveticaNeue?,Helvetica,Roboto,Arial,sans-serif;font-size:14px;line-height:150%;text-align:left;?="" align="eft&quot;">
      <p style="argin:" 0016px;?="">Hi XXXX,</p>
      <p style="argin:" 0016px;?="">Someone has requested a new password for the following account on The Whole Shebang:</p>
      <p style="argin:" 0016px;?="">Username: XXXX</p>
      <p style="argin:" 0016px;?="">If you didn't make this request, just ignore this email. If you'd like to proceed:</p>
      <p style="argin:" 0016px;?="">
      <a class="ink&quot;" href="ttps://thewhole-shebang.co.uk/my-account/lost-password/?key=XXXX&amp;id=" style="ont-weight:" normal;text-decoration:underline;color:#3c3c3c;?="">		Click here to reset your password	</a>
      </p>
      <p style="argin:" 0016px;?="">Thanks for reading.</p>
      </div>
      </td>
      </tr>
      </table>
      <!-- End Content -->
      </td>
      </tr>
      </table>
      <!-- End Body -->
      </td>
      </tr>
      </table>
      </td>
      </tr>
      <tr>
         <td align="enter&quot;" valign="op&quot;">
            <!-- Footer -->
            <table border=" cellpadding=0" cellspacing=" width=00%" id="emplate_footer&quot;">
               <tr>
                  <td valign="op&quot;" style="adding:" 0;border-radius:6px;?="">
                  <table border=" cellpadding=0" cellspacing=" width=00%">
                     <tr>
                        <td colspan=" valign=iddle" id="redit&quot;" style="order-radius:" 6px;border:0;color:#8a8a8a;font-family:?HelveticaNeue?,Helvetica,Roboto,Arial,sans-serif;font-size:12px;line-height:150%;text-align:center;padding:24px0;?="" align="enter&quot;">
                        <p style="argin:" 0016px;?="">The Whole Shebang</p>
                        </td>
                     </tr>
                  </table>
                  </td>
               </tr>
            </table>
            <!-- End Footer -->
         </td>
      </tr>
      </table>
      </div>
      </td>
      <td>
         <!-- Deliberately empty to support consistent sizing and layout across multiple email clients. -->
      </td>
      </tr>
      </table>
      <p class="mpimg'" style="isplay:none;visibility:none;margin:0;padding:0;line-height:0;'"><img src="ttps://click.shop.thewhole-shebang.co.uk/img/nuknvj/QJR0eufbCuYa'" alt=""></p>
   </body>
</html>

To Reproduce

I'm able to consistently reproduce the issue with these particular emails (password resets), however other emails appear to be working fine. Please see above.

Expected behaviour

Characters not stripped.

Environment details

  • Version 2.3.0
@catphish
Copy link
Contributor

catphish commented Feb 23, 2024

That is interesting. Would you be able to provide a full (uncorrupted) copy of the raw email?

@leemathewbrooks
Copy link
Author

Hi @catphish, the "original message source" above is the uncorrupted version, with the "delivered message source" being the corrupted output that is ultimately delivered.

@catphish
Copy link
Contributor

catphish commented Feb 23, 2024

Hi @catphish, the "original message source" above is the uncorrupted version, with the "delivered message source" being the corrupted output that is ultimately delivered.

Sorry, I wasn't clear in my email, I meant the full raw email (with headers etc). I know you can't download it without corruption, but if you can get a copy somehow that would be very helpful.

@leemathewbrooks
Copy link
Author

@catphish Sure, give me 10 :)

@leemathewbrooks
Copy link
Author

@catphish Ok so it may not be so simple. I can capture the SMTP transcript between the sending webserver and Postal if that's what you're after. It is however formatted terribly, so I apologize in advance.

235 Granted for the-whole-shebang/outbound-1 MAIL FROM:<noreply@shop.thewhole-shebang.co.uk> 250 OK RCPT TO:<lee@ssrv.co> 250 OK DATA 354 Go ahead X-Mailer: Postman SMTP 2.8.11 for WordPress (https://wordpress.org/plugins/post-smtp/) Content-Type: text/html; charset=UTF-8 From: The Whole Shebang <noreply@shop.thewhole-shebang.co.uk> To: lee@ssrv.co Reply-To: sales@thewhole-shebang.co.uk Message-Id: <ace3094f5a6d973465f37e75ba371ae0@thewhole-shebang.co.uk> Subject: Password Reset Request for The Whole Shebang Date: Fri, 23 Feb 2024 21:27:08 +0000 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline MIME-Version: 1.0 <!DOCTYPE html>=0A<html lang=3D"en-GB">=0A=09<head>=0A=09=09<meta http-e= quiv=3D"Content-Type" content=3D"text/html; charset=3DUTF-8">=0A=09=09<m= eta content=3D"width=3Ddevice-width, initial-scale=3D1.0" name=3D"viewpo= rt">=0A=09=09<title>The Whole Shebang</title>=0A=09<style type=3D"text/c= ss">@media screen and (max-width: 600px){#header_wrapper{padding: 27px 3= 6px !important; font-size: 24px;}#body_content_inner{font-size: 10px !im= portant;}}</style>=0A</head>=0A=09<body leftmargin=3D"0" marginwidth=3D"= 0" topmargin=3D"0" marginheight=3D"0" offset=3D"0" style=3D"background-c= olor: #f7f7f7; padding: 0; text-align: center;" bgcolor=3D"#f7f7f7">=0A= =09=09<table width=3D"100%" id=3D"outer_wrapper" style=3D"background-col= or: #f7f7f7;" bgcolor=3D"#f7f7f7">=0A=09=09=09<tr>=0A=09=09=09=09<td><!-= - Deliberately empty to support consistent sizing and layout across mult= iple email clients. --></td>=0A=09=09=09=09<td width=3D"600">=0A=09=09= =09=09=09<div id=3D"wrapper" dir=3D"ltr" style=3D"margin: 0 auto; paddin= g: 70px 0; width: 100%; max-width: 600px; -webkit-text-size-adjust: none= ;" width=3D"100%">=0A=09=09=09=09=09=09<table border=3D"0" cellpadding= =3D"0" cellspacing=3D"0" height=3D"100%" width=3D"100%">=0A=09=09=09=09= =09=09=09<tr>=0A=09=09=09=09=09=09=09=09<td align=3D"center" valign=3D"t= op">=0A=09=09=09=09=09=09=09=09=09<div id=3D"template_header_image">=0A= =09=09=09=09=09=09=09=09=09=09<p style=3D"margin-top: 0;"><img src=3D"ht= tps://thewhole-shebang.co.uk/wp-content/uploads/2023/03/wsi.jpg" alt=3D"= The Whole Shebang" style=3D"border: none; display: inline-block; font-si= ze: 14px; font-weight: bold; height: auto; outline: none; text-decoratio= n: none; text-transform: capitalize; vertical-align: middle; max-width:= 100%; margin-left: 0; margin-right: 0;" border=3D"0"></p>=09=09=09=09= =09=09=09=09=09</div>=0A=09=09=09=09=09=09=09=09=09<table border=3D"0" c= ellpadding=3D"0" cellspacing=3D"0" width=3D"100%" id=3D"template_contain= er" style=3D"background-color: #fff; border: 1px solid #dedede; box-shad= ow: 0 1px 4px rgba(0,0,0,.1); border-radius: 3px;" bgcolor=3D"#fff">=0A= =09=09=09=09=09=09=09=09=09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09<t= d align=3D"center" valign=3D"top">=0A=09=09=09=09=09=09=09=09=09=09=09= =09<!-- Header -->=0A=09=09=09=09=09=09=09=09=09=09=09=09<table border= =3D"0" cellpadding=3D"0" cellspacing=3D"0" width=3D"100%" id=3D"template= _header" style=3D'background-color: #3c3c3c; color: #fff; border-bottom:= 0; font-weight: bold; line-height: 100%; vertical-align: middle; font-f= amily: "Helvetica Neue",Helvetica,Roboto,Arial,sans-serif; border-radius= : 3px 3px 0 0;' bgcolor=3D"#3c3c3c">=0A=09=09=09=09=09=09=09=09=09=09=09= =09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09<td id=3D"header_= wrapper" style=3D"padding: 36px 48px; display: block;">=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09<h1 style=3D'font-family: "Helvetica Ne= ue",Helvetica,Roboto,Arial,sans-serif; font-size: 30px; font-weight: 300= ; line-height: 150%; margin: 0; text-align: left; text-shadow: 0 1px 0 #= 636363; color: #fff; background-color: inherit;' bgcolor=3D"inherit">Pas= sword Reset Request</h1>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09</t= d>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09= =09=09=09=09=09=09</table>=0A=09=09=09=09=09=09=09=09=09=09=09=09<!-- En= d Header -->=0A=09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09= =09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09<tr>=0A=09=09=09=09= =09=09=09=09=09=09=09<td align=3D"center" valign=3D"top">=0A=09=09=09=09= =09=09=09=09=09=09=09=09<!-- Body -->=0A=09=09=09=09=09=09=09=09=09=09= =09=09<table border=3D"0" cellpadding=3D"0" cellspacing=3D"0" width=3D"1= 00%" id=3D"template_body">=0A=09=09=09=09=09=09=09=09=09=09=09=09=09<tr>= =0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09<td valign=3D"top" id=3D"bo= dy_content" style=3D"background-color: #fff;" bgcolor=3D"#fff">=0A=09=09= =09=09=09=09=09=09=09=09=09=09=09=09=09<!-- Content -->=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09<table border=3D"0" cellpadding=3D"20"= cellspacing=3D"0" width=3D"100%">=0A=09=09=09=09=09=09=09=09=09=09=09= =09=09=09=09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09= =09<td valign=3D"top" style=3D"padding: 48px 48px 32px;">=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09=09=09=09<div id=3D"body_content_inner"= style=3D'color: #636363; font-family: "Helvetica Neue",Helvetica,Roboto= ,Arial,sans-serif; font-size: 14px; line-height: 150%; text-align: left;= ' align=3D"left">=0A=0A<p style=3D"margin: 0 0 16px;">Hi lee,</p>=0A<p s= tyle=3D"margin: 0 0 16px;">Someone has requested a new password for the= following account on The Whole Shebang:</p>=0A<p style=3D"margin: 0 0 1= 6px;">Username: lee</p>=0A<p style=3D"margin: 0 0 16px;">If you didn't m= ake this request, just ignore this email. If you'd like to proceed:</p>= =0A<p style=3D"margin: 0 0 16px;">=0A=09<a class=3D"link" href=3D"https:= //thewhole-shebang.co.uk/my-account/lost-password/?key=XXXX= XXXX&id=3D3" style=3D"font-weight: normal; text-decoration: underli= ne; color: #3c3c3c;">=09=09Click here to reset your password=09</a>=0A</= p>=0A=0A<p style=3D"margin: 0 0 16px;">Thanks for reading.</p>=0A=09=09= =09=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09</div>=0A=09=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09= =09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09= =09</table>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09<!-- End Cont= ent -->=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09<= /table>=0A=09=09=09=09=09=09=09=09=09=09=09=09<!-- End Body -->=0A=09=09= =09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09=09</tr>= =0A=09=09=09=09=09=09=09=09=09</table>=0A=09=09=09=09=09=09=09=09</td>= =0A=09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09<tr>=0A=09=09=09=09= =09=09=09=09<td align=3D"center" valign=3D"top">=0A=09=09=09=09=09=09=09= =09=09<!-- Footer -->=0A=09=09=09=09=09=09=09=09=09<table border=3D"0" c= ellpadding=3D"10" cellspacing=3D"0" width=3D"100%" id=3D"template_footer= ">=0A=09=09=09=09=09=09=09=09=09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09= =09<td valign=3D"top" style=3D"padding: 0; border-radius: 6px;">=0A=09= =09=09=09=09=09=09=09=09=09=09=09<table border=3D"0" cellpadding=3D"10"= cellspacing=3D"0" width=3D"100%">=0A=09=09=09=09=09=09=09=09=09=09=09= =09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09<td colspan=3D"2"= valign=3D"middle" id=3D"credit" style=3D'border-radius: 6px; border: 0;= color: #8a8a8a; font-family: "Helvetica Neue",Helvetica,Roboto,Arial,sa= ns-serif; font-size: 12px; line-height: 150%; text-align: center; paddin= g: 24px 0;' align=3D"center">=0A=09=09=09=09=09=09=09=09=09=09=09=09=09= =09=09<p style=3D"margin: 0 0 16px;">The Whole Shebang</p>=0A=09=09=09= =09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09=09= =09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09</table>=0A=09=09= =09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09=09</tr>= =0A=09=09=09=09=09=09=09=09=09</table>=0A=09=09=09=09=09=09=09=09=09<!--= End Footer -->=0A=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09<= /tr>=0A=09=09=09=09=09=09</table>=0A=09=09=09=09=09</div>=0A=09=09=09=09= </td>=0A=09=09=09=09<td><!-- Deliberately empty to support consistent si= zing and layout across multiple email clients. --></td>=0A=09=09=09</tr>= =0A=09=09</table>=0A=09</body>=0A</html>=0A . 250 OK

The headers Postal captures are below - if it's something else you need, let me know and I'll try and get it.

Screenshot 2024-02-23 213918

As you mentioned, I can't capture the client headers (i.e from Outlook) and have a non-corrupted email. I can send the email via other means outside of Postal and capture that, though I'm not sure if that'll be of any use to you?

Lee

@catphish
Copy link
Contributor

Can you pull the text out of the database manually? You mentioned you could read it there.

@leemathewbrooks
Copy link
Author

I can give you the message source, but it doesn't include the headers, just the body I'm afraid and that's what's posted above.
Postal is receiving and storing the message from the source fine it seems. It's only when it's then delivering the message/presenting for download it gets mangled.

image

I'm not sure which table Postal stores the header data in internally, but I imagine it'll just be as per the screenshot in my previous comment.

Any logging we can enable?

@catphish
Copy link
Contributor

I can give you the message source, but it doesn't include the headers, just the body

This is actually extremely useful. I have a hypothesis. We will look into it and let you know.

@leemathewbrooks
Copy link
Author

Great! If you need anything else, I'll be glad to help :)

@catphish
Copy link
Contributor

We've now identified the problem. When the link tracking system modifies the email, we only save the updated message body, and ignore any changes to the headers.
In this case:

  1. The original body is encoded with "Quoted-printable" encoding (the weird =0A=09=09=09=09=09=09=09=09=09=09 stuff seen in the SMTP trace.
  2. The modified version is not encoded, just plain HTML (as seen in your database).
  3. The headers still indicate that the message is quoted printable, but it's not. This causes it to appear mangled.

If you look at the raw .eml file in a text editor, you will see there is no corruption, just clean HTML, but a header saying it's quoted-printable, when it's not.

We will be working on a fix ASAP. In the meantime, you have two workarounds available:

  1. Send the emails as multipart with HTML and plain text (this is usually a good idea anyway)
  2. Disable click/view tracking so that the email is not rewritten.

Thanks to @adamcooke who did the hard work on reproducing this.
Fixed in 2834e2c

@adamcooke
Copy link
Contributor

adamcooke commented Feb 23, 2024

As @catphish said, I've pushed a fix for this and released 2.3.1 which includes that fix. Let us know how you get on with that.

Assuming it works, this issue probably sets a record for the quickest resolved issue in Postal history so congratulations. I can't guarantee any future issues will be so lucky but you never know 🤞

@leemathewbrooks
Copy link
Author

Hi @catphish, @adamcooke

Okay so that makes sense. We're using a Wordpress plugin on this website to generate the emails - if it's generating garbage headers then I'll file a bug with them too.

Unfortunately given the limitations of the plugin I'm unable to change the origin headers without butchering the plugin, which I'd like to avoid (as above, I'll raise a bug report with the plugin author and see if that can be resolved).

In the interim I've set Wordpress to just send the plain text version of this particular email so it's not a showstopper right now thankfully. I'm reluctant to disable the open tracking as it's come in handy a bunch of times when customers are complaining about missed deliveries etc, but it's good to know it's an option if this particular issue becomes more widespread in the interim.

I'd just like to thank you for your exceptional support with this. I raised the issue an hour ago (at 9pm no less) and you've already identified the issue, given me a workaround and are working on a permafix, which I think is absolutely incredible; even more so for an open source project that I can clearly see is a labour of love. I'd love to give something back as a token of my thanks, though I can't see a donate link or anything of the sort.

Thanks for all your help. I've been using Postal for a year now and it's been rock solid!

@leemathewbrooks
Copy link
Author

image

I assume it may take some time for this to filter down?

Lee

@adamcooke
Copy link
Contributor

Yeh, I might have spoke a bit soon. It seems like the build didn't finish. It's currently building the image so it should be there soon.

@leemathewbrooks
Copy link
Author

No worries - I'll try again in 15 and will let you know if the issue is resolved.

Thanks again
Lee

@catphish
Copy link
Contributor

catphish commented Feb 23, 2024

Okay so that makes sense. We're using a Wordpress plugin on this website to generate the emails - if it's generating garbage headers then I'll file a bug with them too.

No - the original email is perfectly valid. It's Postal messing up the headers.

It's just unusual (but perfectly valid) that the Wordpress plugin generates emails with HTML only. Usually you'd send both HTML and plain text together in the same email (multipart).

@leemathewbrooks
Copy link
Author

So I've got the new version installed, re-tested the email and this is now working flawlessly :)

@adamcooke
Copy link
Contributor

adamcooke commented Feb 23, 2024

@pawprintdigital The image has made its way in to the container registry now so you should be good to upgrade now.

Edit: I see you've already discovered this on your own. Good to hear it works!

@leemathewbrooks
Copy link
Author

leemathewbrooks commented Feb 23, 2024

There is one final thing..

After enabling click tracking, the reset link no longer works.
Appreciate this is a seperate issue (and I think there's another open issue surrounding & / & encoding), but for example..

Without click tracking, the output link is

https://thewhole-shebang.co.uk/my-account/lost-password/?key=XXXXX&id=3

With the click tracking, the resulting link is:

https://click.shop.thewhole-shebang.co.uk/nuknvj/XXXXX
Which directs to:
https://thewhole-shebang.co.uk/my-account/lost-password/?key=XXXXX&amp;id=3

The broken & breaks the reset flow. It's not a major issue as I can live without the click tracking for the time being, but if you're currently in the god-mode fixing mood... ;)

Happy to create a new issue for this if I can't find an existing open issue for it.

Lee

@adamcooke adamcooke added the star-issue This is an issue which is an excellent example of reporting an issue - just flagged as an example label Feb 23, 2024
@adamcooke
Copy link
Contributor

Happy to create a new issue for this if I can't find an existing open issue for it.

There is an issue for that #907. It's quite an old one which escaped our attention. It'll be fixed in v3 which will be available soon.

@leemathewbrooks
Copy link
Author

Great, thanks.
I knew I'd seen it somewhere.

I'll leave this thread alone now as it's all resolved.
Thanks again both.

Lee

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug star-issue This is an issue which is an excellent example of reporting an issue - just flagged as an example
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants