Php mb convert encoding windows 1251

function w1250_to_utf8($text) {
// map based on:
// http://konfiguracja.c0.pl/iso02vscp1250en.html
// http://konfiguracja.c0.pl/webpl/index_en.html#examp
// http://www.htmlentities.com/html/entities/
$map = array(
chr(0x8A) => chr(0xA9),
chr(0x8C) => chr(0xA6),
chr(0x8D) => chr(0xAB),
chr(0x8E) => chr(0xAE),
chr(0x8F) => chr(0xAC),
chr(0x9C) => chr(0xB6),
chr(0x9D) => chr(0xBB),
chr(0xA1) => chr(0xB7),
chr(0xA5) => chr(0xA1),
chr(0xBC) => chr(0xA5),
chr(0x9F) => chr(0xBC),
chr(0xB9) => chr(0xB1),
chr(0x9A) => chr(0xB9),
chr(0xBE) => chr(0xB5),
chr(0x9E) => chr(0xBE),
chr(0x80) => '€',
chr(0x82) => '‚',
chr(0x84) => '„',
chr(0x85) => '…',
chr(0x86) => '†',
chr(0x87) => '‡',
chr(0x89) => '‰',
chr(0x8B) => '‹',
chr(0x91) => '‘',
chr(0x92) => '’',
chr(0x93) => '“',
chr(0x94) => '”',
chr(0x95) => '•',
chr(0x96) => '–',
chr(0x97) => '—',
chr(0x99) => '™',
chr(0x9B) => '’',
chr(0xA6) => '¦',
chr(0xA9) => '©',
chr(0xAB) => '«',
chr(0xAE) => '®',
chr(0xB1) => '±',
chr(0xB5) => 'µ',
chr(0xB6) => '¶',
chr(0xB7) => '·',
chr(0xBB) => '»',
);
return html_entity_decode(mb_convert_encoding(strtr($text, $map), 'UTF-8', 'ISO-8859-2'), ENT_QUOTES, 'UTF-8');
}

<?php
$cleanText = mb_convert_encoding($text, 'UTF-8', 'Windows-1252');
?>

The annoying part is that the auto detection (ie: the mb_detect_encoding function) will often think Windows-1252 is ISO-8859-1. Close, but no cigar. This is critical if you're then trying to do unserialize on the resulting text, because the byte count of the string needs to be perfect.

<?php

$text = "A strange string to pass, maybe with some ø, æ, å characters.";

foreach(

mb_list_encodings() as $chr){

echo mb_convert_encoding($text, 'UTF-8', $chr)." : ".$chr."<br>";

}

?>



The line that looks good, gives you the encoding it was written in.

Hope can help someone

public function encodeToUtf8($string) {
return mb_convert_encoding($string, "UTF-8", mb_detect_encoding($string, "UTF-8, ISO-8859-1, ISO-8859-15", true));
}

public function encodeToIso($string) {
return mb_convert_encoding($string, "ISO-8859-1", mb_detect_encoding($string, "UTF-8, ISO-8859-1, ISO-8859-15", true));
}

For me these functions are working fine. Give it a try

mbstring.substitute_character = "none"

in your php.ini. Be sure to include the quotes around none. Or at run-time with

<?php
ini_set('mbstring.substitute_character', "none");
?>

So [insert korean characters here] turned into ?????.

I found myself needing to pass by reference (which of course is deprecated/nonexistent in recent versions of PHP)

so instead of

<?php

mb_convert_encode(&$s,'HTML-ENTITIES','UTF-8');

?>

which worked perfectly until I upgraded, so I had to use

<?php

call_user_func_array('mb_convert_encoding', array(&$s,'HTML-ENTITIES','UTF-8'));

?>



Hope it helps someone else out

Again, this is a quick and dirty solution to stop mb_convert_encoding from filling your string with question marks whenever it encounters an illegal character for the target encoding.

<?php
function convert_to ( $source, $target_encoding )
{
// detect the character encoding of the incoming file
$encoding = mb_detect_encoding( $source, "auto" );// escape all of the question marks so we can remove artifacts from
// the unicode conversion process
$target = str_replace( "?", "[question_mark]", $source );// convert the string to the target encoding
$target = mb_convert_encoding( $target, $target_encoding, $encoding);// remove any question marks that have been introduced because of illegal characters
$target = str_replace( "?", "", $target );// replace the token string "[question_mark]" with the symbol "?"
$target = str_replace( "[question_mark]", "?", $target );

return

$target;
}
?>

Hope this helps someone! (Admins should feel free to delete my previous, incorrect, post for clarity)
-A

To convert all non-ASCII characters into entities (to produce pure 7-bit HTML output), I was using:

<?php
echo mb_convert_encoding( htmlspecialchars( $text, ENT_QUOTES, 'UTF-8' ), 'HTML-ENTITIES', 'UTF-8' );
?>

I can get the identical result with:

<?php
echo mb_encode_numericentity( htmlentities( $text, ENT_QUOTES, 'UTF-8' ), [0x80, 0x10FFFF, 0, ~0], 'UTF-8' );
?>

The output contains well-known named entities for some often used characters and numeric entities for the rest.

function htmltoflash($htmlstr)
{
return str_replace("&lt;br /&gt;","\n",
str_replace("<","&lt;",
str_replace(">","&gt;",
mb_convert_encoding(html_entity_decode($htmlstr),
"UTF-8","ISO-8859-1"))));
}

<?php

//Separado OK, chars MAL

iconv('MACINTOSH', 'UTF8', $CSV);

//Separado MAL, chars OK

chr(255).chr(254).mb_convert_encoding( $CSV, 'UCS-2LE', 'UTF-8');

?>



But the first one didn't show extended chars correctly, and the second one, did't separe fields correctly

mb_substitute_character("none");

private array $mapChrChr = [
0x8A => 0xA9,
0x8C => 0xA6,
0x8D => 0xAB,
0x8E => 0xAE,
0x8F => 0xAC,
0x9C => 0xB6,
0x9D => 0xBB,
0xA1 => 0xB7,
0xA5 => 0xA1,
0xBC => 0xA5,
0x9F => 0xBC,
0xB9 => 0xB1,
0x9A => 0xB9,
0xBE => 0xB5,
0x9E => 0xBE
];

private array $mapChrString = [
0x80 => '&euro;',
0x82 => '&sbquo;',
0x84 => '&bdquo;',
0x85 => '&hellip;',
0x86 => '&dagger;',
0x87 => '&Dagger;',
0x89 => '&permil;',
0x8B => '&lsaquo;',
0x91 => '&lsquo;',
0x92 => '&rsquo;',
0x93 => '&ldquo;',
0x94 => '&rdquo;',
0x95 => '&bull;',
0x96 => '&ndash;',
0x97 => '&mdash;',
0x99 => '&trade;',
0x9B => '&rsquo;',
0xA6 => '&brvbar;',
0xA9 => '&copy;',
0xAB => '&laquo;',
0xAE => '&reg;',
0xB1 => '&plusmn;',
0xB5 => '&micro;',
0xB6 => '&para;',
0xB7 => '&middot;',
0xBB => '&raquo;'
];

/**
* @param $text
* @return string
*/
public function execute($text): string
{
$map = $this->prepareMap();

return html_entity_decode(
mb_convert_encoding(strtr($text, $map), self::ENCODING_TO, self::ENCODING_FROM),
ENT_QUOTES,
self::ENCODING_TO
);
}

/**
* @return array
*/
private function prepareMap(): array
{
$maps[] = $this->arrayMapAssoc(function ($k, $v) {
return [chr($k), chr($v)];
}, $this->mapChrChr);

$maps[] = $this->arrayMapAssoc(function ($k, $v) {
return [chr($k), $v];
}, $this->mapChrString);

return array_merge([], ...$maps);
}

/**
* @param callable $function
* @param array $array
* @return array
*/
private function arrayMapAssoc(callable $function, array $array): array
{
return array_column(
array_map(
$function,
array_keys($array),
$array
),
1,
0
);
}
}

mb_detect_order(array('UTF-8', 'ISO-8859-1'));

It is necessary to force a specific search order for the conversion to work

$fileName = 'Test:!"$%&/()=ÖÄÜöäü<<';
echo strtr(mb_convert_encoding($fileName,'ASCII'),
' ,;:?*#!§$%&/(){}<>=`´|\\\'"',
'____________________________');

HTML-ENTITIES

Example:

$text = mb_convert_encoding($text, 'HTML-ENTITIES', "UTF-8");

<?php

$string

= iconv('UTF-8', 'macintosh', $string);?>

('macintosh' is the IANA name for the MacRoman character set.)

// have fun

$dos = mb_convert_encoding($utf8_text, "CP850", mb_detect_encoding($utf8_text, "UTF-8, CP850, ISO-8859-15", true));

<?php
// 0 - win
// 1 - koi
function detect_encoding($str) {
$win = 0;
$koi = 0;

for(

$i=0; $i<strlen($str); $i++) {
if( ord($str[$i]) >224 && ord($str[$i]) < 255) $win++;
if( ord($str[$i]) >192 && ord($str[$i]) < 223) $koi++;
}

if(

$win < $koi ) {
return 1;
} else return 0;

}

// recodes koi to win
function koi_to_win($string) {$kw = array(128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 254, 224, 225, 246, 228, 229, 244, 227, 245, 232, 233, 234, 235, 236, 237, 238, 239, 255, 240, 241, 242, 243, 230, 226, 252, 251, 231, 248, 253, 249, 247, 250, 222, 192, 193, 214, 196, 197, 212, 195, 213, 200, 201, 202, 203, 204, 205, 206, 207, 223, 208, 209, 210, 211, 198, 194, 220, 219, 199, 216, 221, 217, 215, 218);
$wk = array(128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 225, 226, 247, 231, 228, 229, 246, 250, 233, 234, 235, 236, 237, 238, 239, 240, 242, 243, 244, 245, 230, 232, 227, 254, 251, 253, 255, 249, 248, 252, 224, 241, 193, 194, 215, 199, 196, 197, 214, 218, 201, 202, 203, 204, 205, 206, 207, 208, 210, 211, 212, 213, 198, 200, 195, 222, 219, 221, 223, 217, 216, 220, 192, 209);$end = strlen($string);
$pos = 0;
do {
$c = ord($string[$pos]);
if ($c>128) {
$string[$pos] = chr($kw[$c-128]);
}

} while (++

$pos < $end);

return

$string;
}

function

recode($str) {$enc = detect_encoding($str);
if ($enc==1) {
$str = koi_to_win($str);
}

return

$str;
}
?>

<?php
$text = iconv($from_enc, 'US-ASCII//TRANSLIT', $text);
?>

The only disadvantage is that it does not convert "ä" to "ae", but it handles punctuation and other special characters better.
--
David

even if you explicitly specify the character encoding of a page as iso-8859-1(via headers and strict xml defs), windows 2000 will ignore that and interpret it as whatever character set it has natively installed.

for example, i wrote char #128 into a page, with char encoding iso-8859-1, and it displayed in internet explorer (& mozilla) as a euro symbol.

it should have displayed a box, denoting that char #128 is undefined in iso-8859-1. The problem was it was displaying in "Windows: western europe" (my native character set).

this led to confusion when i tried to convert this euro to UTF-8 via mb_convert_encoding()

IE displays UTF-8 correctly- and because PHP correctly converted #128 into a box in UTF-8, IE would show a box.

so all i saw was mb_convert_encoding() converting a euro symbol into a box. It took me a long time to figure out what was going on.

Here's a simple snippet of code that worked great for me, getting php to show special Danish characters, from a Flash email form:

<?php
// Name Escape
$escName = mb_convert_encoding($_POST["Name"], "ISO-8859-1", "UTF-8");// message escape
$escMessage = mb_convert_encoding($_POST["Message"], "ISO-8859-1", "UTF-8");// Headers.. and so on...
?>

I had this problem converting to japanese charset:

$txt=mb_convert_encoding($txt,'SJIS',$this->encode);

And I could fix it by using this:

$txt = iconv('UTF-8', 'SJIS', $txt);

Maybe it´s helpfull for someone else! ;)

- &agrave;,&ccedil;,&eacute;,&icirc;,... to a,c,e,i,...

- &szlig; to ss

- &auml;,&Auml;,... to ae,Ae,...

- &euml;,... to e,...

(mb_convert \&quot;7bit\&quot; would simply delete any offending characters).

I might have missed on your country\&#039;s typographic

conventions--correct me then.

&lt;?php

/**

* @args string $text line of encoded text

* string $from_enc (encoding type of $text, e.g. UTF-8, ISO-8859-1)

*

* @returns 7bit representation

*/

function to7bit($text,$from_enc) {

$text = mb_convert_encoding($text,\&#039;HTML-ENTITIES\&#039;,$from_enc);

$text = preg_replace(

array(\&#039;/&szlig;/\&#039;,\&#039;/&amp;(..)lig;/\&#039;,

\&#039;/&amp;([aouAOU])uml;/\&#039;,\&#039;/&amp;(.)[^;]*;/\&#039;),

array(\&#039;ss\&#039;,\&quot;$1\&quot;,\&quot;$1\&quot;.\&#039;e\&#039;,\&quot;$1\&quot;),

$text);

return $text;

}

?&gt;

Enjoy :-)

Johannes

==

[EDIT BY danbrown AT php DOT net: Author provided the following update on 27-FEB-2012.]

==

An addendum to my &quot;to7bit&quot; function referenced below in the notes.

The function is supposed to solve the problem that some languages require a different 7bit rendering of special (umlauted) characters for sorting or other applications. For example, the German &szlig; ligature is usually written &quot;ss&quot; in 7bit context. Dutch &yuml; is typically rendered &quot;ij&quot; (not &quot;y&quot;).

The original function works well with word (alphabet) character entities and I&#039;ve seen it used in many places. But non-word entities cause funny results:

E.g., &quot;&copy;&quot; is rendered as &quot;c&quot;, &quot;&shy;&quot; as &quot;s&quot; and &quot;&amp;rquo;&quot; as &quot;r&quot;.

The following version fixes this by converting non-alphanumeric characters (also chains thereof) to &#039;_&#039;.

&lt;?php

/**

* @args string $text line of encoded text

* string $from_enc (encoding type of $text, e.g. UTF-8, ISO-8859-1)

*

* @returns 7bit representation

*/

function to7bit($text,$from_enc) {

$text = preg_replace(/W+/,&#039;_&#039;,$text);

$text = mb_convert_encoding($text,&#039;HTML-ENTITIES&#039;,$from_enc);

$text = preg_replace(

array(&#039;/&szlig;/&#039;,&#039;/&amp;(..)lig;/&#039;,

&#039;/&amp;([aouAOU])uml;/&#039;,&#039;/&yuml;/&#039;,&#039;/&amp;(.)[^;]*;/&#039;),

array(&#039;ss&#039;,&quot;$1&quot;,&quot;$1&quot;.&#039;e&#039;,&#039;ij&#039;,&quot;$1&quot;),

$text);

return $text;

}

?&gt;

Enjoy again,

Johannes

echo

'<br>';
$s=mb_convert_encoding($s,'UCS-2','utf8');
mb_internal_encoding('UCS-2');
$t1=microtime(true);
$i=0;
while($i<mb_strlen($s)){
$a=mb_substr($s,$i,2);
$i+=2;
if($i==10)echo mb_convert_encoding($a,'utf8','ucs2').'. ';
//echo$a.'. ';
}
echo$i.'. ';
echo(microtime(true)-$t1);

echo

Flash submits its HTML special characters in UTF-8, so you can use the following function to convert those into HTML entity characters:

function utf8html($utf8str)
{
return htmlentities(mb_convert_encoding($utf8str,"ISO-8859-1","UTF-8"));
}

I use it as a fail-safe, in case mb_convert_encoding is not installed. Download it from http://mikolajj.republika.pl/

This is not my own library, so technically it's not spamming, right? ;)

Hope this helps.

<?php

iconv

("windows-874","UTF-8",$input);?>

If you don't specify a source encoding, then it assumes the internal (default) encoding. ñ is a multi-byte character whose bytes in your configuration default (often iso-8859-1) would actually mean ñ. mb_convert_encoding() is upgrading those characters to their multi-byte equivalents within UTF-8.

Try this instead:
<?php
print mb_convert_encoding( "ñ", "UTF-8", "UTF-8" );
?>
Of course this function does no work (for the most part - it can actually be used to strip characters which are not valid for UTF-8).