[development] Converting UTF8 strings to ASCII in Drupal

[Khalid Baheyeldin]

On Mon, Nov 16, 2009 at 10:42 AM, Brian Vuyk <brian at brianvuyk.com> wrote:

> Hi all.
>
> I am having a bit of a unique problem here, and I am hoping someone can
> help me out.
>
> In short, we are outputting strings from a custom module we wrote to an
> external service that only accepts the ASCII character set. So, I am trying
> to run all the strings through iconv() to convert them.
>
> Here's the weird part. Take the script below:
>
> $string = "Stéphanie,D Hérouville";
> $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string);
> print $output;
>
> If I run this from the command line, I get the proper output - 'Stephanie D
> Herouville'. However, if I run it in my Drupal installation, whether as part
> of a module or from the Devel 'Execute PHP' box, I get different output:
> 'St?phanie D H?rouville'. That is, instead of replacing the character with
> it's expected ASCII counterpart, it replaces it with a ?.
>
> Has anyone encountered this, or have any idea why this is happening?
>
> Brian
>


I confirm what you are seeing:

If you have a file called ascii.php with this in it:

<?php
$string = "Stéphanie,D Hérouville";
$output = iconv("UTF-8", 'ASCII//TRANSLIT', $string);
print $output . "\n";
?>

And run it as:

$ php ascii.php

The output is without the accented characters.

However, if you boot Drupal first, like so:

<?php
require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$string = "Stéphanie,D Hérouville";
$output = iconv("UTF-8", 'ASCII//TRANSLIT', $string);
print $output . "\n";
?>

The output is with question marks, as you said.

I suspect something in includes/unicode.inc is causing this.
-- 
Khalid M. Baheyeldin
2bits.com, Inc.
http://2bits.com
Drupal optimization, development, customization and consulting.
Simplicity is prerequisite for reliability. --  Edsger W.Dijkstra
Simplicity is the ultimate sophistication. --   Leonardo da Vinci
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.drupal.org/pipermail/development/attachments/20091116/c44f2c58/attachment.html