Hi all. I am having a bit of a unique problem here, and I am hoping someone can help me out. In short, we are outputting strings from a custom module we wrote to an external service that only accepts the ASCII character set. So, I am trying to run all the strings through iconv() to convert them. Here's the weird part. Take the script below: $string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output; If I run this from the command line, I get the proper output - 'Stephanie D Herouville'. However, if I run it in my Drupal installation, whether as part of a module or from the Devel 'Execute PHP' box, I get different output: 'St?phanie D H?rouville'. That is, instead of replacing the character with it's expected ASCII counterpart, it replaces it with a ?. Has anyone encountered this, or have any idea why this is happening? Brian
Brian, Might want to take a look at http://www.php.net/manual/en/function.iconv.php#86077 Dave Reid dave@davereid.net On Mon, Nov 16, 2009 at 9:42 AM, Brian Vuyk <brian@brianvuyk.com> wrote:
Hi all.
I am having a bit of a unique problem here, and I am hoping someone can help me out.
In short, we are outputting strings from a custom module we wrote to an external service that only accepts the ASCII character set. So, I am trying to run all the strings through iconv() to convert them.
Here's the weird part. Take the script below:
$string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output;
If I run this from the command line, I get the proper output - 'Stephanie D Herouville'. However, if I run it in my Drupal installation, whether as part of a module or from the Devel 'Execute PHP' box, I get different output: 'St?phanie D H?rouville'. That is, instead of replacing the character with it's expected ASCII counterpart, it replaces it with a ?.
Has anyone encountered this, or have any idea why this is happening?
Brian
On Mon, Nov 16, 2009 at 10:42 AM, Brian Vuyk <brian@brianvuyk.com> wrote:
Hi all.
I am having a bit of a unique problem here, and I am hoping someone can help me out.
In short, we are outputting strings from a custom module we wrote to an external service that only accepts the ASCII character set. So, I am trying to run all the strings through iconv() to convert them.
Here's the weird part. Take the script below:
$string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output;
If I run this from the command line, I get the proper output - 'Stephanie D Herouville'. However, if I run it in my Drupal installation, whether as part of a module or from the Devel 'Execute PHP' box, I get different output: 'St?phanie D H?rouville'. That is, instead of replacing the character with it's expected ASCII counterpart, it replaces it with a ?.
Has anyone encountered this, or have any idea why this is happening?
Brian
I confirm what you are seeing: If you have a file called ascii.php with this in it: <?php $string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output . "\n"; ?> And run it as: $ php ascii.php The output is without the accented characters. However, if you boot Drupal first, like so: <?php require_once './includes/bootstrap.inc'; drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL); $string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output . "\n"; ?> The output is with question marks, as you said. I suspect something in includes/unicode.inc is causing this. -- Khalid M. Baheyeldin 2bits.com, Inc. http://2bits.com Drupal optimization, development, customization and consulting. Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra Simplicity is the ultimate sophistication. -- Leonardo da Vinci
It's because of the call to setlocale(LC_CTYPE, 'C'); in _unicode_check() in includes/unicode.inc.
From http://www.php.net/manual/en/function.iconv.php#86077 I tried doing the following and it worked for me (very important to note setting back the original value to setlocale().
setlocale(LC_CTYPE, 'fr_FR.utf8'); $string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output; setlocale(LC_CTYPE, 'C'); I'm not sure which locale string you'd use in the first call to setlocale (I used French in this case) but you can get all the values on your system by running locale -a from the command line. Dave Reid dave@davereid.net
On Mon, 16 Nov 2009 11:02:12 -0500 Brian Vuyk <brian@brianvuyk.com> wrote:
<?php require_once './includes/bootstrap.inc'; drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
$string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output . "\n"; ?>
even after bootstrapping: var_export output the correct result. echo... report the correct result... -- Ivan Sergio Borgonovo http://www.webthatworks.it
On Mon, 16 Nov 2009 17:15:24 +0100 Ivan Sergio Borgonovo <mail@webthatworks.it> wrote:
On Mon, 16 Nov 2009 11:02:12 -0500 Brian Vuyk <brian@brianvuyk.com> wrote:
<?php require_once './includes/bootstrap.inc'; drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
$string = "Stéphanie,D Hérouville"; $output = iconv("UTF-8", 'ASCII//TRANSLIT', $string); print $output . "\n"; ?>
even after bootstrapping:
var_export output the correct result. echo... report the correct result...
forget about it. It was an artefact of the setup of the box I had under my finger at the moment. Dave Reid was right. -- Ivan Sergio Borgonovo http://www.webthatworks.it
participants (4)
-
Brian Vuyk -
Dave Reid -
Ivan Sergio Borgonovo -
Khalid Baheyeldin