![\"unicode_arial\"](\"https:\/\/www.kamremake.com\/devblog\/wp-content\/uploads\/2013\/08\/unicode_arial.png\")
<\/a><\/p>\nHowever we can not simply convert everything from ANSI to Unicode, because some 3-rd party libraries work only with ANSI and because ANSI should still be used for backwards compatibility. Also, being simpler, ANSI is faster to process. Small example, in ANSI each letter always takes exactly 1 byte, but in Unicode (UTF16) letters may take 2 or 4 bytes, hence knowing the text length is impossible without checking all the text letters first.<\/p>\n
So, to make it work right we need a plan that clearly separates ANSI and Unicode areas. After all we would not want to get into troubles when text is saved as ANSI and read as Unicode, that would produce garbage.<\/p>\n
This pic illustrates various text input locations and how they are planned to be dealt with:<\/p>\n
First and topmost comes generic string type which means that programmer does not care how exactly the string is stored, but since we deal with a serious stuff we need to be in control. Using generic string type is unsafe because it differs between different Delphi versions and Lazarus. Early Delphi used AnsiString, but later on switched to UTF16, where’s Lazarus uses UTF8. Simply said – these types stored differently in memory and need different handling.<\/p>\n Most of the text that gets into KaM Remake can be pretty easily split into Unicode and ANSI. Unicode means we get all benefits of internationalization and ANSI means we keep backwards compatibility, smaller size and simplicity. Simplicity is not a perfect word in the context, it means avoiding unnecessary trouble, such as: we want all multiplayer players to be able to input a password to enter the lobby, so we allow only Latin passwords (anyone can input Latin, right?)<\/p>\n For now, for backwards compatibility (this may be changed soon) locale texts are stored in ANSI with a codepage info derived from file extension. Later we will change them to Unicode. We already had number of occasions when locale texts were saved by translators in wrong codepages, which lead to some letters were garbaged.<\/p>\n Scripts are in ANSI because they don’t need any text to be stored in them (maybe except comments, but they are fine to be unreadable \ud83d\ude09<\/p>\n Filepaths should be Unicode since OS handles them that way, the game could be installed into a folder named in Cyrillic and sub-folder named in Czech, or just Chinese.<\/p>\n UI text and chat texts are obviously Unicode.<\/p>\n Now comes the tricky ones that actually made me to write all this, there are couple of edge cases that have both pros and cons in each area:<\/p>\n Player names<\/strong> If you read till this part you probably got a hold of the general idea and the uneasy choice between Unicode and ANSI presented above. Please share your opinion in comments or in this forum post:\u00c2\u00a0https:\/\/www.knightsandmerchants.net\/forum\/viewtopic.php?f=22&t=1746<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" This is an open-ended article about Unicode in KaM Remake. Easy at it sounds, Unicode is better to have than not. However there are few complications that need to be sorted out and YOUR opinion might help.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[4],"tags":[],"_links":{"self":[{"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/posts\/256"}],"collection":[{"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/comments?post=256"}],"version-history":[{"count":3,"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/posts\/256\/revisions"}],"predecessor-version":[{"id":261,"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/posts\/256\/revisions\/261"}],"wp:attachment":[{"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/media?parent=256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/categories?post=256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kamremake.com\/devblog\/wp-json\/wp\/v2\/tags?post=256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}<\/a><\/p>\n
\nUnicode: players like their names to be in native locale
\nANSI: \u00c2\u00a0other players may have troubles addressing players whose names they can’t type (e.g. Chinese)
\nLobby passwords<\/strong>
\nUnicode: no good reasons actually
\nANSI: no matter the spoken language all players need to be able to type the password. The single common locale is Latin, so we can just restrict passwords to 0..9..A..z
\nMission names<\/strong> – currently they are not localized, they are named after the folder they put into.
\nUnicode: do we need mission names in native locales, would not that cause confusion if we have Cyrillic, Chinese and Polish missions in one list?
\nANSI: simplicity<\/p>\n