Email, spam, multipart-alternative mime

[JackGoa]

Well, from the research I've done, you need to send out a plain text version of your email with the html version. So, an HTML version with tags, as well as a version without tags.

If you don't mind, with my limited knowledge, can I refer you to this article instead: http://www.enewsletterpro.com/articles/multi_part_mime_messages.asp

 

[cheesegrits]

I may have found a way to fix this, working on it.

-- hugh

 

[cheesegrits]

https://github.com/Fabrik/fabrik/commit/732f54ac6b04dc0698f4ac07bdd3259703b5245a

I've forced setting an AltBody, with tags stripped, and added base64 encoding.

I've done some basic testing, and it's working OK to my gmail. It arrives as a multipart like this ...

This is a multi-part message in MIME format.
--b1_845f3548af5733cb372c5fc11902f87b
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64

YmxhaCBibGFoIE9uZVR3bw0KDQpib2xkDQoNCi0tLQ0KVGhpcyBlbWFpbCBo
YXMgYmVlbiBjaGVja2VkIGZvciB2aXJ1c2VzIGJ5IEF2YXN0IGFudGl2aXJ1
cyBzb2Z0d2FyZS4NCmh0dHBzOi8vdTIzMjU5NTkuY3Quc2VuZGdyaWQubmV0
L3dmL2NsaWNrP3Vwbj14d3N5LTJGWHBPazN2VVlhN29DakEydXdkN1BxQUpI
MFR2TWd0U0I0dVdOLTJGZU02elhqbDdnUFF6cjRncFZFR1VoRl9VVlRaVThX
eW10b1hFTkNCNzZScHJrcWRiT2l0QWszbmF0VGRsSnctMkJkdE55a1lmOTZY
VFJROXZIVGg1WDNsNDlrRzUxaHRBbmkxaDhDRVZaRC0yRk9ZZVFoWGw0LTJC
NndUWjI4c2FyUEt1TlVudUtEMk05bkQ4eDM1TlFWdWZRUlZ1R3R3LTJCVmR5
QWpXTWN2WnBlbEhsVTgtMkJCTEhYQWY4bmRvWkNXTm9rYkNZLTJCd2VPNkRr
ZldIVk5zM0N1eERQaFdzQWxMamRNUFdJRmh4ZFNBb3VNVWMtMkYzN2ZkSWND
NFBGS2tRSUIyWEczU2FMZXMtM0QNCg==

--b1_845f3548af5733cb372c5fc11902f87b
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64

YmxhaCBibGFoIDx1bD48bGk+T25lPC9saT48bGk+VHdvPC9saT48L3VsPg0K
PGJyIC8+DQo8Yj5ib2xkPC9iPjxiciAvPg0KPHRhYmxlIHN0eWxlPSJib3Jk
ZXItdG9wOiAxcHggc29saWQgI2FhYWJiNjsiPg0KCTx0cj4NCiAgICAgIDx0
ZCBzdHlsZT0id2lkdGg6IDU1cHg7IHBhZGRpbmctdG9wOiAxOHB4OyI+PGEg
aHJlZj0iaHR0cHM6Ly91MjMyNTk1OS5jdC5zZW5kZ3JpZC5uZXQvd2YvY2xp
Y2s/dXBuPXh3c3ktMkZYcE9rM3ZVWWE3b0NqQTJ1MjJVRFNHLTJCZkZBLTJG
SC0yQldUZ2NMVW15UXQwMDZsYUVnNUdaaVZEcDVYd3VnSHVybmM1SGZNdGlE
NlB1d2JFWmlmMkdwNUZhT2lQbHBrZUE5QnVwMTlhNmdYSTEwS3RobnpqNDRL
M005RW4wMWg4ZnkxVkRTVjZiYTVjOXc1ekFHelNQWXNsMnMyZExPQ1MtMkY5
eHdLdGs2cnl1eTRZZ3l3OGF0MUFsVTJMdHFCMmhfVVZUWlU4V3ltdG9YRU5D
Qjc2UnBya3FkYk9pdEFrM25hdFRkbEp3LTJCZHROeWtZZjk2WFRSUTl2SFRo
NVgzbDQ5a0c1MWh0QW5pMWg4Q0VWWkQtMkZPWWVUOTdPenpicUxOOXF0YTRz
LTJCVVFTTEQ2OXV2V0NQcEZRM1d5N2VKZXpnck9adW0tMkJoNWYzdEdWRnRl
b2dBTWpnZGlEYWNTYjNJOXE5cGxSVmZVMHhvSllxenZvaE9JSjRuWUhFSmpa
bUdLaDVCRjJTU0RoRFZNcFczcC0yRlktMkZrejhYdzdzZEhSd09RTU45cGFK
T1ZuclRxZy0zRCIgdGFyZ2V0PSJfYmxhbmsiPjxpbWcgc3JjPSJodHRwczov
L2lwbWNkbi5hdmFzdC5jb20vaW1hZ2VzLzIwMTYvaWNvbnMvaWNvbi1lbnZl
bG9wZS10aWNrLXJvdW5kLW9yYW5nZS12MS5wbmciIC8+PC9hPjwvdGQ+DQoJ
CTx0ZCBzdHlsZT0id2lkdGg6IDQ3MHB4OyBwYWRkaW5nLXRvcDogMjBweDsg
Y29sb3I6ICM0MTQyNGU7IGZvbnQtc2l6ZTogMTNweDsgZm9udC1mYW1pbHk6
IEFyaWFsLCBIZWx2ZXRpY2EsIHNhbnMtc2VyaWY7IGxpbmUtaGVpZ2h0OiAx
OHB4OyI+VGhpcyBlbWFpbCBoYXMgYmVlbiBzZW50IGZyb20gYSB2aXJ1cy1m
cmVlIGNvbXB1dGVyIHByb3RlY3RlZCBieSBBdmFzdC4gPGEgaHJlZj0iaHR0
cHM6Ly91MjMyNTk1OS5jdC5zZW5kZ3JpZC5uZXQvd2YvY2xpY2s/dXBuPXh3
c3ktMkZYcE9rM3ZVWWE3b0NqQTJ1MjJVRFNHLTJCZkZBLTJGSC0yQldUZ2NM
VW15UXQwMDZsYUVnNUdaaVZEcDVYd3VnSHVybmM1SGZNdGlENlB1d2JFWmlm
MkdwNUZhT2lQbHBrZUE5QnVwMTlhNmdYSTEwS3RobnpqNDRLM005RW4wMWg4
ZnkxVkRTVjZiYTVjOXc1ekFHelNQWXNsMnMyZExPQ1MtMkY5eHdLdGs2cnl1
eTRZZ3l3OGF0MUFsVTJMdHFCMmhfVVZUWlU4V3ltdG9YRU5DQjc2UnBya3Fk
Yk9pdEFrM25hdFRkbEp3LTJCZHROeWtZZjk2WFRSUTl2SFRoNVgzbDQ5a0c1
MWh0QW5pMWg4Q0VWWkQtMkZPWWVVVXRCd1ZmclFaemUwNWh1LTJGTjlkMnhM
WEU0QTVJcHJhc1NQNW1qSTVkYzBmMHpkRVZGOHNlV282ME9Ud3dDQ3ByUVFS
UGNJeGxsSFBCMUNGUGt6S3BrMFhXbFdMWXBLLTJCNG9HeFE5b2RFaWo2T0l2
RVN3NVlPZ0swR1BKQzB4Y3A3S0ZNRmZoa0E5Z0VCMG80VWthVFcwLTNEIiB0
YXJnZXQ9Il9ibGFuayIgc3R5bGU9ImNvbG9yOiAjNDQ1M2VhOyI+d3d3LmF2
YXN0LmNvbTwvYT4NCgkJPC90ZD4NCgk8L3RyPg0KPC90YWJsZT48aW1nIHNy
Yz0iaHR0cHM6Ly91MjMyNTk1OS5jdC5zZW5kZ3JpZC5uZXQvd2Yvb3Blbj91
cG49VVZUWlU4V3ltdG9YRU5DQjc2UnBya3FkYk9pdEFrM25hdFRkbEp3LTJC
ZHROeWtZZjk2WFRSUTl2SFRoNVgzbDQ5a0c1MWh0QW5pMWg4Q0VWWkQtMkZP
WWVkREdkRlpCcWV1YVNvcWM2WVotMkJCZXBkQ0dyLTJCYTNyVTJZRFZYQ1FM
bmtxTi0yRnc2Q2h6SjJuRmpMLTJGQTkwZ3QwSkh1MWc3aENPTjJ1cEYwQjVL
N2twaWk4NTYycFBMSFdrVVhBS3k0ckhQSGNyZURtTmtERWNsMzh0VU5BWngy
R2IwWFRZeE5qTjJ3ZUlXY09RbTZYSDk3Yy0zRCIgYWx0PSIiIHdpZHRoPSIx
IiBoZWlnaHQ9IjEiIGJvcmRlcj0iMCIgc3R5bGU9ImhlaWdodDoxcHggIWlt
cG9ydGFudDt3aWR0aDoxcHggIWltcG9ydGFudDtib3JkZXItd2lkdGg6MCAh
aW1wb3J0YW50O21hcmdpbi10b3A6MCAhaW1wb3J0YW50O21hcmdpbi1ib3R0
b206MCAhaW1wb3J0YW50O21hcmdpbi1yaWdodDowICFpbXBvcnRhbnQ7bWFy
Z2luLWxlZnQ6MCAhaW1wb3J0YW50O3BhZGRpbmctdG9wOjAgIWltcG9ydGFu
dDtwYWRkaW5nLWJvdHRvbTowICFpbXBvcnRhbnQ7cGFkZGluZy1yaWdodDow
ICFpbXBvcnRhbnQ7cGFkZGluZy1sZWZ0OjAgIWltcG9ydGFudDsiLz4K

--b1_845f3548af5733cb372c5fc11902f87b--

-- hugh

 

[cheesegrits]

BTW, that's also through Send Grid, which I really would recommend using. If you use your own local server for sending, you are at the mercy of how well your host maintains their reputation. Sending through Send Grid, you know the relays you are using are always going to have a high reputation score, as they live or die by their reputation score. And it's free up to 12,000 emails a month. And it's very simple to set up, just get a Send Grid account, then change your normal J! SMTP configuration:

https://sendgrid.com/docs/Integrate/Open_Source_Apps/joomla.html

-- hugh

 

[cheesegrits]

The other advantage of using Send Grid is, you have a control panel where you can easily see how well your mail is being delivered - what's bouncing, what's getting opened, etc.

 

[troester]

Great.
I have this "high spam score" problem, too. I'm using a php plugin - but now I can copy

 

[cheesegrits]

I'm thinking I might add a helper for that. We send mail from a number of places - this, the list email, the cron email, etc. We never built a helper before, because until now we've just been doing ...

$res = $mailer->sendMail( $emailFrom, $emailFromName, $email, $thisSubject, $thisMessage, $htmlEmail, $cc, $bcc, $thisAttachments, $returnPath, $returnPathName);

But to add the AltBody, I have to do it the long hand way. For some reason, when I tried adding the AltBody before doing $mailer->sendMail() it didn't work.

Anyway, I'll move all that into one of our helpers, then you can just call that.

-- hugh

 

[cheesegrits]

OK, I added a FabrikWorker::sendmail() helper, takes the exact same args as the J! sendMail(), but if necessary, adds the AltBody.

https://github.com/Fabrik/fabrik/commit/db34b8a1dae9796a1f2415a469643d687e3342b0

@troester - you should just be able to use this instead of $mailer->sendMail() in your plugin.

-- hugh

 

[felixkat]

@cheesegrits From what I've read, and it's been quite a lot in the last couple of hours, the AltBody has to be done in a certain sequence.... and this sequence changed at some point.

@JackGoa The options we had were to send in plain text or to send HTML, however we were forcing the HTML option through for whatever reason.

As per your link what we need is a multipart html/text email which is what Hugh has implemented. The issue comes down to Joomla as it's not something they have in their code despite it being supported by PHPMailer which is what Joomla is using to send the mail.

It was requested by somebody back in 2007 and bumped in 2010 but went stale.

https://developer.joomla.org/joomlacode-archive/issue-4760.html

Somebody was going to put a PR through 2 years ago.... but I guess that didn't happen either.

http://stackoverflow.com/questions/...how-to-add-text-plain-when-sending-html-email

Anyway.... As per Hugh's last commit, it should be working now.

 

[JackGoa]

It's 00:24am here now. Been nursing a few vodka martinis. Will be tackling this in the morning.

I have to install the latest Github first, right? (*another dumb question?)

the world is not as it appears - tapatalk

 

[felixkat]

Yes the latest Github should fix what you need.... it won't cure a hangover though.

 

[JackGoa]

Hehe. Damn. Thanks for all the assistance. Wish I had more knowledge so I could actually physically contribute to Fabrik. Where can I buy the 'I love Fabrik' tshirt?

Will revert back in the morning. Is it Easter weekend that side of the world too?

the world is not as it appears - tapatalk

 

[cheesegrits]

The options we had were to send in plain text or to send HTML, however we were forcing the HTML option through for whatever reason.

We force HTML because even if you don't put HTML markup in your template / msg, any rendered elements will pretty much always contain HTML. So we nl2br the msg, and treat it as HTML, period.

-- hugh

 

[troester]

In addition here a link to spam scores
http://www.futurequest.net/docs/SA/

and the scores the mail server I had problems with is/was using

FREEMAIL_FORGED_REPLYTO=2.503
HTML_MESSAGE=0.001
HTML_MIME_NO_HTML_TAG=0.635 --------- maybe this can be added (putting <html> tags around the text if not there)
MIME_HTML_ONLY=1.105 ------------------------- this should be fixed with the new helper
RCVD_IN_DNSWL_NONE=-0.0001
RCVD_IN_MSPIKE_H4=-0.01
RCVD_IN_MSPIKE_WL=-0.01
TO_NO_BRKTS_HTML_ONLY=1.999 ----------- anybody has an idea what this is? formatting of the TO address?
XPRIO=1.999 ------------ -------------------------------I think this setting is nonsense if priority is normal or less, but ...;
Joomla/phpmailer WAS always adding a priority, is fixed in J!3.5!!! (default is null now)

 

[cheesegrits]

Let me know what changes when you use the new helper.

I can fix the "no HTML tag" by adding one if it doesn't exist.

Not sure I can do anything about the "no to brackets" as formatting the To happens in the php mailer class.



Sent from my HTC One using Tapatalk

 

[JackGoa]

I just installed that new update and I am glad to report a 10/10 spam score! For proof, see attached screenshot

The received emails display properly. Checked in MS Outlook, Gmail, Gmail for Android and Inbox for Android. Not sure how to check the plain text version, so can't report on how that displays.

Guys and gals! You're all awesome! Thanks to all for helping out! This enables me to create more automation and make less phone calls for this client! Hugh, you da bomb! I owe you big time by now. It's coming

 

[JackGoa]

HTML_MIME_NO_HTML_TAG=0.635 --------- maybe this can be added (putting <html> tags around the text if not there)

@troester I fixed this by toggling my editor 'off' and adding DOCTYPE and <html> and <body> tags manually so they don't get stripped out when I save the article. Obviously I'm using an article template for the email being sent.

 

[troester]

@cheesegrits: with the new mail helper it's coming below the spam limit on my "critical" server, even with Joomla3.4.8

I fixed this by toggling my editor 'off' and adding DOCTYPE and <html> and <body> tags manually

Yes, that's what I have done, too.
It seems to be no difference with/without <body> tag (at least on my server).
But it would be better to have it automatically around the HTML part.

I think I've got the TO_NO_BRKTS: it's reported if there's only a TO email, no TO name.
phpMailer only creates <> if there's an array(name,email); it's sufficient to have a space as name
$rc = is_array($recipient) ? $recipient : array(' ',$recipient);
$mailer->addRecipient($rc);

So I think the spam scores can still be improved
by adding <html> tags

and may be by adding a default ' ' TO name (no idea if a blank name will rise the spam score in other spam algorithms)

 

[JackGoa]

Ja, I was not really sure whether the <body> and DOCTYPE declaration was needed, so I looked at some email templates I downloaded and just copied what they did.

I've got a reply-to and from, email and name set, so did not get any notices about that.

 

[cheesegrits]

This commit should fix the html tag and to name (TO_NO_BRKT) issues.

https://github.com/Fabrik/fabrik/commit/dfa992f70ccfc290a1336fd1924ab66fb1702cb7

-- hugh