Skip to content

Fix Windows locale problem#13107

Closed
ken2812221 wants to merge 7 commits into
bitcoin:masterfrom
ken2812221:win-enc
Closed

Fix Windows locale problem#13107
ken2812221 wants to merge 7 commits into
bitcoin:masterfrom
ken2812221:win-enc

Conversation

@ken2812221

@ken2812221 ken2812221 commented Apr 28, 2018 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes #13103
This PR make internal strings of Windows always be utf8-encoded

  • Add UTF8 <-> localized string conversion function
  • Use localized string on Berkeley DB
  • Use localized string on LevelDB
  • Use localized string on fstream::open
  • Use std::fstream instead of fs::fstream

Patched the boost library to convert fs::path::string() to be UTF8 encoded. Add two functions that convert between UTF8 and Local encodes.

To those who want to test this, adding -datadir=(some non-ASCII characters) option make every file that Bitcoin Core use contain non-ASCII characters

Defect: If the user's codepage does not support the filename, it will be encoded to ?
I don't known how to solve this. Leveldb and Berkeley DB don't support wide char.

laanwj
laanwj reviewed Apr 28, 2018
View reviewed changes
Comment thread src/util.cpp Outdated

@laanwj laanwj Apr 28, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe move to utilstrencodings?

@laanwj

laanwj commented Apr 28, 2018 •
edited
Loading

Copy link
Copy Markdown
Member

Thanks. Though I'm sad that this is needed.
To stay sane, and keep locales and encodings out of everything as much as possible:

  • Please keep all our internal state in UTF-8 everywhere. This was always the intent, so in interaction with JSONRPC we can assume all strings are UTF-8, and matches with Qt's view.
  • Only convert to local encoding when needed, so just before going to the file system layer, or when coming from the file system layer (or filenames, command-line arguments etc).

BTW, there are GUIUtil::qstringToBoostPath and GUIUtil::qstringToBoostPath to convert Qt strings from/to local paths but I don't think they are useful in this case as it doesn't happen on the interface from Qt to core code.

@ken2812221 ken2812221 changed the title Fix Windows locale problem [WIP] Fix Windows locale problem Apr 28, 2018
@ken2812221 ken2812221 changed the title [WIP] Fix Windows locale problem Fix Windows locale problem Apr 28, 2018
@ken2812221 ken2812221 closed this May 9, 2018
@ken2812221 ken2812221 mentioned this pull request May 9, 2018
Closed
@bitcoin bitcoin deleted a comment from mrthuan9xpro May 9, 2018
@ken2812221 ken2812221 reopened this May 9, 2018
@ken2812221

ken2812221 commented Jun 11, 2018

Copy link
Copy Markdown
Contributor Author

prefer #13426

@ken2812221 ken2812221 closed this Jun 11, 2018
@ken2812221 ken2812221 deleted the win-enc branch June 11, 2018 16:27
@ken2812221 ken2812221 mentioned this pull request Jun 13, 2018
Closed
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@laanwj laanwj laanwj left review comments

Assignees

No one assigned

Labels

Utils/log/libs UTXO Db and Indexes Wallet Windows

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Invalid wallet path with Chinese characters in windows

4 participants

@ken2812221 @laanwj @fanquake @maflcko