Posts going thru Gmail have funny  symbol

[Coleen_Astalos]

Pedro,
I think I've figured it out.  You're right it's the encoding.  Earlier in this thread your said to make sure that the Gmail encoding was set to "default text" (which mine was).  What I've found is that the encoding needs to be set to "UTF-8" instead.  I've found that when I send my emails via Gmail with it set to UTF-8 that they show up in Outlook with charset=unicode and the message is displayed correctly with no  symbols.

I'm working with a few of my other users on Gmail whose emails always have the  symbol to see if them changing their Gmail to UTF-8 will solve the problem.  I'll let you know.

Coleen

[Coleen_Astalos]

Oh, the other thing I should have mentioned as I know we initially tried setting the encoding in Gmail to UTF-8 and it didn't work.  

But I found a weird solution here: https://discussions.apple.com/message/22300841#22300841 
which basically had the person using Apple Mail to insert a Dingbat character such as this: ☂ into their email signature to force the message to be sent in unicode format.

I thought - no way is this going to work - but I tried it anyways and it did work!  

I then went back and fiddled around with my Gmail send mail settings (not the encoding - but whether it sends out the emails via Gmail or via my normal ISP) and it seems to work even without the ☂ added to the email.  So I'm not sure whether it was really sending in UTF-8 format when I was testing before.  Long explanation, but I wanted to make sure you knew that I DID try sending it in utf-8 when you asked me to originally.

Anyway, that's why I've asked a few of my users that always have emails with this problem to change their encoding in Gmail to utf-8 so I can see if that solves the problem.  

Coleen

[Coleen_Astalos]

Ok, so as I originally reported, the setting the encoding in Gmail to utf-8 does NOT solve the problem.  I had several members test for me.

But adding a dingbat symbol like the ☂ somewhere in the email does.  

When Nabble creates the email to send out, what does it set the character encoding to?
Coleen

[Pedro]

The Nabble encoding is utf-8. I'm not sure yet what is the problem, because it doesn't happen to me.

[Pedro]

Please, subscribe your accounts here: http://pedro-3.1104361.n5.nabble.com/Coleen-Subscription-f31.html ,
and make posts with your gmail account. Use the exclamation point in order to try to reproduce the bug. Remember to not copy and paste messages.

[Coleen_Astalos]

How is this testing going?  I just posted a few more samples to your test site.
Coleen

[Pedro]

I'm trying different approaches. But I couldn't reproduce this bug yet. Can you test on another pc and browser?

[Coleen_Astalos]

Pedro, I can easily reproduce it over and over again with these two conditions:

1)      The email is sent from a Gmail account to Nabble

2)      The email sent from Nabble is viewed in Outlook 2007

 

Many of my members have the same problem.  So it's not just me.  I get the same results on my laptop (viewing the emails Outlook) and also if I send my email from Gmail using multiple browsers..  

 

But even if yours is displaying correctly in your email program and you can't see the Âs, you should still be able to see the difference in encoding - which  is what is driving this problem.

 

Here is a test that clearly shows the problem occurs when going through Nabble.  If I send an email to both the forum and myself from my Gmail account.  The copy I get directly to myself is properly encoded with UTF-8 and displays fine.  The copy I get that goes thru Nabble is encoded with us-ascii (Windows) and has the  symbols.

 

So let me do that same test with you.  I'm going to post a new email to the forum AND copy you on it as well.

Look at the HTML for the email that comes directly to you (I'm sure you know this, but to see the HTML in Gmail select the down arrow next to the reply arrow and select "Show Original"), search for charset.  You will see the encoding is as follows:

Content-Type: text/plain; charset=UTF-8

 

Now look at the HTML for the copy that went thru Nabble - you will see the encoding is as follows:

Content-Type: text/plain; charset=us-ascii

 

So Gmail sent it as UTF-8 (I can see that in my sent email folder on Gmail) and when Nabble gets the email, processes it and sends it out again, it sends it out with charset=us-ascii.   If this is corrected to send out in UTF-8 then this problem should go away.  I'm actually surprised you've not run across this problem more frequently with the different languages you support.  But maybe Nabble sets the charset properly for those different languages (as it does seem to set it correctly to utf-8 when including a special character somewhere in the email)

 

I certainly have learned a lot about encoding from this problem.  This article: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky <http://www.joelonsoftware.com/articles/Unicode.html> has good explanation of why you'd see different things in different email programs.

 

Let me know what you see from these tests.

Coleen

 

Coleen Astalos, Stampin' Up! Demonstrator
Alamosa, CO, [hidden email]
See my website at http://alamosa.stampinup.net
Are you a demonstrator? Join me on SUDSOL - Your 24/7 Upline!

 

From: Pedro <Nabble> [via Nabble Support] [mailto:[hidden email]]
Sent: Wednesday, January 22, 2014 1:18 PM
To: Coleen_Astalos
Subject: RE: Posts going thru Gmail have funny  symbol

 

I'm trying different approaches. But I couldn't reproduce this bug yet. Can you test on another pc and browser?

My test forum.

 

---

If you reply to this email, your message will be added to the discussion below:

http://support.nabble.com/Posts-going-thru-Gmail-have-funny-A-symbol-tp7589092p7589219.html

To unsubscribe from Posts going thru Gmail have funny  symbol, click here.
NAML

[Pedro]

I received a post from your forum called "It worked!" . Is this working?

[Coleen_Astalos]

Pedro,
I was able to strip off the  symbol (which is actually the non breaking space character & nbsp; ) in the "send_subscription_email" macro by changing the <html_part.html_part/> to this:
 <html_part.> 
    <n.regex_replace_all text="[n.local_node.html_part/]" pattern="\u00a0" replacement=" "/> 
 </html_part.> 

I had to play around a bit to figure what representation of that character would work (as there are multiple).  

This solution to strip off those characters is working for me at this point as that was the only funky character I was consistently seeing.

I don't think this is the "right" way to fix it as it only solves the problem for the   character.  Other characters above #127 that are not defined in us-ascii may still come across wrong since the encoding is being set in Nabble to us-ascii instead of utf-8.   It is interesting to note that Nabble does handle setting the encoding to utf-8 correctly if there is a special character in the original email (such as the dingbat symbol).  
Coleen

[Pedro]

Very clever, I was going in the other direction. I'was trying to remove it when the email arrives.