I’ve been working on a (albeit small) PHP script over the past couple of days which sends an HTML email out to 23 locales – including Russia and Poland. Whilst this seemed simple at face value, there were a couple of brick walls I hit when delivering this email, hopefully this article will help those that are lost.

Email Subject Lines

This one is just a quick FYI, but UTF-8 characters are not supported in email subjects, same as special characters such as trademark or copyright etc. So keep your subject lines as clean as possible.

PHP Mail Headers

This was my first stumbling block. I used the following headers for my HTML email:

$headers = 'From: ' . $fromAdd . "\n";
$headers .= 'Reply-To: ' . $fromAdd . "\n";
$headers .= 'MIME-Version: 1.0' . "\n";
$headers .= "Content-type: text/html; charset=UTF-8\r\n";
$headers .= "Content-Transfer-Encoding: 8bit\r\n";

This worked great, GMail received the mail, even Lotus Notes worked fine – however I had one problem – Outlook. Outlook converted the email to plain text, and wouldn’t render the HTML email. I was stumped – I couldn’t understand why the mail wouldn’t work in Outlook, and Google was no help either. I knew it must be something to do with the headers, but couldn’t figure out what. Then I got it:

$headers .= "Content-Type: text/html; charset=UTF-8\r\n";

This line was incorrect. Outlook will only use the header if “Content-Type:” is written in this way. I had seen suggestions that “Content-Type :” would resolve delivery issues, but it didn’t. So pay special attention to the case sensitive nature of the headers when setting them for Outlook email!

Supporting Russian and Polish in UTF-8

My second problem (which took much longer to solve) was the UTF-8 charset required to support Russian and Polish characters. I setup my MySQL database with all columns set to UTF-8_bin collation, and also set the database and table itself to UTF-8.

I then entered the locale information into the database, everything looked good (via PHPMyAdmin), with the data being saved as blobs. However when retrieving the data out of the database with my PHP script, both the Russian and Polish content would be replaced with:

“????????????”

So I setup a header for the PHP script to show the content in a web page:

<META http-equiv="Content-Type" content="text/html; charset=utf-8">

This still didn’t work. English was fine, French – no worries – but not Polish or Russian. I made sure my browser was on UTF-8 – check – all fine – but no dice!

Next on my list was to try and decode the UTF-8 data into ISO-8859-1 using the PHP function utf8_decode(). Again, even after decoding the UTF-8 data it made no difference to the output on the page.

UTF-8 – The Solution

So after literally hours of combing the internet for help, someone in my team dug up a link for me regarding MySQL connections defaulting to latin1 collation. This was the answer to all my problems. What was happening was that despite all the collation in the database being set to UTF-8, the data was still being passed to my PHP script as latin1 – which is why I got a lot of ????? question marks.

The secret/workaround is to place this piece of PHP code before your MySQL query to retrieve data from the database:

$db_charset = mysql_query( "SHOW VARIABLES LIKE 'character_set_database'" );
$charset_row = mysql_fetch_assoc( $db_charset );
mysql_query( "SET NAMES '" . $charset_row['Value'] . "'" );
unset( $db_charset, $charset_row );

This piece of code grabs the character set from the database and ensures that when the data is passed from the database to PHP, it remains in the same collation – in my case, UTF-8!

Victory!