#4436

failed to print error code `:s` fail on windows due to encoding error

Opentrim21opened 1y ago
#include <exception>
#include <filesystem>
#include <iostream>

#include "fmt/ostream.h"
#include "fmt/std.h"

namespace fs = std::filesystem;

int main() {
    const fs::path from{"/none1/a"}, to{"/none2/b"};

    try {
        fs::copy_file(from, to);
    } catch (fs::filesystem_error const &ex) {
        std::cout << "what():  " << ex.what() << '\n'
                  << "path1(): " << ex.path1() << '\n'
                  << "path2(): " << ex.path2() << '\n'
                  << "code().value():    " << ex.code().value() << '\n'
                  << "code().message():  " << ex.code().message() << '\n'
                  << "code().category(): " << ex.code().category().name() << '\n';

        try {
            fmt::println("{:s}", ex.code());
        } catch (std::exception &e) {
            fmt::println("fmt error: {}", e.what());
        }
    }

    fmt::println("exit");
    return 0;
}

Activity

11
vitaut1y ago

What error do you get and what encodings do you use?

trim211y ago

it's error code 3 ERROR_PATH_NOT_FOUND https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-

the ex.code().message() is "系统找不到指定的路径。" in gbk, cfb5cdb3d5d2b2bbb5bdd6b8b6a8b5c4c2b7beb6a1a3 in hex

trim211y ago

I think we may need MultiByteToWideChar and WideCharToMultiByte from windows.h to convert locale encoding to utf8 when handling error code on windows ...

sunmy20191y ago

Can you elaborate more on the output and the error?

Also an error here:

    try {
      fmt::println("{:s}", ex.code());
      fmt::println("{}", ex.code().message());
-   } catch (std::exception e) {
+   } catch (std::exception& e) { 
      fmt::println("fmt error: {}", e.what());
    }
trim211y ago

OK, sorry, I didn't provide a clear description about this bug.

the root case of this bug is, the error message in ex.code().message() is not always utf-8 on windows, so when fmt try to print a error code, it cause a utf-8 encoding error. at fmt::println("fmt error: {}", e.what()); line it output "fmt error: invalid utf8"

base on my windows locale config, the error message may be gbk and not utf-8, which is "系统找不到指定的路径。" in gbk, cfb5cdb3d5d2b2bbb5bdd6b8b6a8b5c4c2b7beb6a1a3 in hex.

the fmt will throw a error "invalid utf8". And if I use fmt::format, it will give a string with mixed utf-8 and non utf-8 content.

sunmy20191y ago

The question could be rephrased as:

Formatter of std::error_code does not handle non-UTF-8 encoding with s spec.

Maybe we can add a wrapper to convert ec.message() to UTF-8.

trim211y ago

yeah, this should work?

#include <windows.h>

std::string ConvertToUTF8(const std::string &input_string) {
    if (input_string.empty()) {
        return std::string();
    }

    int wide_char_len = MultiByteToWideChar(CP_ACP, 0, input_string.c_str(), -1, nullptr, 0);
    if (wide_char_len == 0) {
        return input_string;
    }

    std::vector<wchar_t> wide_char_buffer(wide_char_len);
    if (MultiByteToWideChar(CP_ACP, 0, input_string.c_str(), -1, wide_char_buffer.data(),
                            wide_char_len) == 0) {
        return input_string;
    }

    int utf8_len =
        WideCharToMultiByte(CP_UTF8, 0, wide_char_buffer.data(), -1, nullptr, 0, nullptr, nullptr);
    if (utf8_len == 0) {
        return input_string;
    }

    std::vector<char> utf8_buffer(utf8_len);
    if (WideCharToMultiByte(CP_UTF8, 0, wide_char_buffer.data(), -1, utf8_buffer.data(), utf8_len,
                            nullptr, nullptr) == 0) {
        return input_string;
    }

    return std::string(utf8_buffer.data());
}
vitaut1y ago

It's a known bug in the standard (mostly affecting Windows): https://cplusplus.github.io/LWG/issue4156.

shreyas-s14037d ago

Hi! I’d like to work on this issue and would appreciate being assigned to it if possible. I’m happy to investigate the Windows encoding issue and prepare a fix with a PR if that’s acceptable.

If you’d rather not assign it, I can still look into the bug and submit a fix for review.

vitaut12h ago

@IRainman, please keep comments relevant to the technical discussion. Further off-topic comments may result in a ban.