Forum    Search    FAQ

Board index » Chat Forums » Suggestions and Comments




Post new topic  Reply to topic  [ 12 posts ] 
 
Author Message
 Post Posted: Fri Mar 13, 2009 1:14 pm 
User avatar
Offline
Joined: Fri Jan 02, 2009 4:20 am
Posts: 883
Website: http://insolenz.echoing.org
Location: The Dark Pyramid
Pérháps Í óúght tó máké thís póínt própérly.

These boards don't deal well with characters not found on american keyboards. They splice all kinds of crap in their place which renders the word illegible.
That was bad enough, but I found the boards can't even handle the Pound Sterling sign: £ ... that being standard fare for us British folk.

Can't anything be done to fix this?

Top 
   
 Post Posted: Fri Mar 13, 2009 1:35 pm 
User avatar
Offline
Joined: Mon Mar 31, 2003 12:00 am
Posts: 2523
Website: http://www.axelfendersson.co.uk/blog/
WLM: [email protected]
AOL: AxelFendersson
Location: Darkest Oxfordshire
That's odd. I'm sure that non-ASCII characters used to work fine.

Top 
   
 Post Posted: Fri Mar 13, 2009 1:46 pm 
Member of the Fraternal Order of the Emergency Pants
User avatar
Offline
Joined: Fri Sep 08, 2006 6:26 pm
Posts: 2811
Location: This account has been suspended
Well,

The page source for this page (and presumably the entire board) says the character set is UTF-8. So there's no reason, in theory, that all the characters shouldn't be supported.

Heck, Cyrillic and Hebrew cahracters should be supported!

My guess is it's in the php engine and how the page gets rendered. I think full php support for unicode characters only starts in the newest version (6.0) and that's got no set release date...php 5 probably has extensions for that sort of support...but i don't know if they were included in the last update (or maybe got dropped by accident?).

I think this is a question for rit!

Top 
   
 Post Posted: Sat Mar 14, 2009 3:38 am 
Member of the Fraternal Order of the Emergency Pants
User avatar
Offline
Joined: Tue Feb 27, 2007 1:26 am
Posts: 1866
Location: Directly over the center of the Earth.
I've been wondering about this recently myself. I know that I've seen pound sterling signs, as well as e-with-dieresis, and trademark and suchlike signs. Normally I use UTF-8 for everything, and it has always Just Worked. Hmmm...

Pound Sterling: £, Copyright: ©, Yen: ¥, Trademark: ™, Zoe-with-a-dieresis: Zoë.

That worked, at least when I previewed it. For the record, I'm running Linux, with Firefox, using UTF-8 encoding.

Whoa... I just checked the topic review. SotiCoto's original post is comprehensible there. He just put standard accents over all the vowels. he Pound Sterling sign displayed correctly, too. This is definitely a problem for Rit.

EDIT: Whoa again... My post looked fine when I typed it, and when I reviewed it. But now that it's posted, it's corrupted. Something has changed the characters that I wrote. They're wrong even in the edit text box.

--tll

Top 
   
 Post Posted: Sat Mar 14, 2009 10:07 am 
Member of the Fraternal Order of the Emergency Pants
User avatar
Offline
Joined: Fri Sep 08, 2006 6:26 pm
Posts: 2811
Location: This account has been suspended
Well, yes. Writing and previewing take place in your browser.

Viewing the page and editing require that the page be loaded from the server.

The problem is the server's support for unicode...I'm fairly certain of it.

Top 
   
 Post Posted: Sat Mar 14, 2009 12:51 pm 
Member of the Fraternal Order of the Emergency Pants
User avatar
Offline
Joined: Tue Feb 27, 2007 1:26 am
Posts: 1866
Location: Directly over the center of the Earth.
Ah, I didn't realize that the previewing was a browser function. Makes sense, then.

Oh well, a job for rit.

--tll

Top 
   
 Post Posted: Sun Mar 15, 2009 10:15 am 
Offline
Joined: Thu Dec 04, 2008 9:48 am
Posts: 106
Location: Earth
£

Did that work?


No it didn't!

Cool :p

Top 
   
 Post Posted: Tue Mar 17, 2009 10:28 am 
Nifty Admin of Doom
Nifty Admin of Doom
User avatar
Offline
Joined: Thu Jun 13, 2002 12:00 am
Posts: 1699
Website: http://www.dabbledstudios.com
Location: Atlanta, GA
We will make sure rit hears about this.

Top 
   
 Post Posted: Tue Jul 14, 2009 7:12 pm 
User avatar
Offline
Joined: Thu Nov 29, 2007 12:25 am
Posts: 391
Sorry to necropost, but may I inquire as to whether any progress has been made on this front?

Since I'm a nosy person (who also happens to know a thing or two about forum software and encoding issues), I can tell you that this problem has its origins in the phpBB3 upgrade, and is actually not uncommon. The data seems to be correct to the byte as it's stored (i.e. as UTF-8), but when read, it seems to be interpreted as Latin-1 (or at least that's the way I see the problem; I don't happen to have spare phpBB installs just lying around to diagnose this sort of thing! :bert: ). I've found "solutions" to this problem online, but they seem a little asinine, to tell the truth (most of them involve converting everything in the database or having the phpBB2 install still around so you can reconvert, which doesn't seem like the right way to go about it when you can probably just change certain settings here and there).

Not to press the issue, of course; rushing headfirst into the problem will absolutely make this particular one worse (several levels of mojibake'ifying is a no-go). I'm just concerned because I have a vested interest in curvy punctuation ("''" -> “‘’”) and foreign languages (hai, sou desu -> はい、そうです).

May you and yours have a pleasant evening. Or morning or afternoon or night as the case may be.

Top 
   
 Post Posted: Thu Jul 23, 2009 1:35 pm 
Member of the Fraternal Order of the Emergency Pants
User avatar
Offline
Joined: Mon Dec 15, 2003 12:00 am
Posts: 2994
Not really necroing, considering that it's still an issue. It would be nice to have this issue resolved, particularly since I'm ramping up to run a Werewolves-style game in feudal Japan, and it would be nice to be able to write stuff like Hōjō or 鬼... but as you can see, I can't.

One possibility: The release notes for phpBB 3.0.5 include this:
phpBB wrote:
[Fix] Set connection encoding for MySQL versions 4.1.0 to 4.1.2. This may fix some conversion issues with special characters. (Bug #41805)

Don't know if you're running MySQL or what version, but it's worth a look. It may turn out that a simple board update is all that's needed.

EDIT: The corresponding bug ticket seems to indicate that this might be the problem. The characters store correctly into the database, but get mangled when it reads them out to display them. This is consistent with what we experience here: special characters look fine in the preview (since the post has not yet been stored in the database), but when you submit and then view the post, it looks mangled. The good news is that if this is the correct diagnosis, it means that the correct characters are in the database, and once the update is applied, old posts with mangled characters will start showing up correctly.

Top 
   
 Post Posted: Sat Jul 25, 2009 12:04 pm 
the root of all evil
the root of all evil
User avatar
Offline
Joined: Thu Jan 01, 1970 7:00 pm
Posts: 257
Website: http://www.evilmonkeylabs.com
AOL: rit+0042
Location: New York, NY
Ya we're not running MySQL 4 :)

I'm back on looking at a bunch of stuff, and will see if I can't figure this out.

I suspect the reason however is simpler... PHP has no proper support for unicode. Thus, typically there are hacks and such which, for interoperability with Sluggy's aging codebase may be disabled.
I am a big fan of making sure things "just work". I'll follow up.

Top 
   
 Post Posted: Wed Aug 05, 2009 11:10 am 
Member of the Fraternal Order of the Emergency Pants
User avatar
Offline
Joined: Mon Dec 15, 2003 12:00 am
Posts: 2994
Awesome! It's fixed now! Just for our general edification, what did you do, rit?

Top 
   
Display posts from previous:  Sort by  
 
Post new topic  Reply to topic  [ 12 posts ] 

Board index » Chat Forums » Suggestions and Comments


Who is online

Users browsing this forum: No registered users and 1 guest

 
 

 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: