-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
unicode issue #3
Comments
Hello, I am sorry, for the moment there is no Unicode support, just plain ASCII. Introducing Unicode should not too difficult, however, but at the moment I have no time to work on this project. Best regards, |
here is my test: unicode_test.cpp#include <sstream>
#include <tinyparser.hpp>
using namespace tipa;
using namespace std;
struct Record {
string name, unlock, message;
void clear() { name = ""; unlock = ""; message = ""; }
};
int main()
{
const token tk_ident = create_lib_token("^[^\\d\\W][\\w\\s\\.,!?:\\\\]*");
rule unlock = rule("unlock") >> rule(":") >> rule("\"") >> rule(tk_ident) >> rule("\"");
rule message = rule("message") >> rule(":") >> rule("\"") >> rule(tk_ident) >> rule("\"");
rule property = unlock | message;
rule button = rule("\"") >> rule(tk_ident) >> rule("\"") >> rule('{') >> *property >> rule('}');
rule root = *button;
vector<Record> records;
Record temp;
unlock.read_vars(temp.unlock);
message.read_vars(temp.message);
button.set_action([&temp, &records](parser_context &pc) {
vector<string> v;
pc.collect_tokens(1, back_inserter(v));
temp.name = v[0];
records.push_back(temp);
temp.clear();
});
stringstream s(R"(
"door ascii" {
unlock: "silver key"
message: "This place is unknown to you. Would you like to enter?"
}
"door ascii_2" {
unlock: "test key"
message: "test"
}
"door unicode" {
unlock: "silver key"
message: "Это место не изведано. Желаете войти?"
}
)");
parser_context pc;
pc.set_stream(s);
bool f = false;
try { f = parse_all(root, pc); }
catch(parse_exc exc) { cout << "exception caught" << endl; cout << exc.what() << endl; f = false; }
cout << "Parser status : " << boolalpha << f << endl;
if (!f) { cout << pc.get_formatted_err_msg() << endl; }
for (auto x: records)
cout << "> " << x.name << ", unlock: " << x.unlock << ", message: " << x.message << endl;
}
/*
g++ unicode_test.cpp tinyparser.a -Itipa/src && a.exe
*/
|
Thank you :) I should have some free time next week, let's see what I can do. |
Hi, i have an issue when ident contains non-latin characters.
for example, one character
Ы
will be parsed as two characters:Р«
The text was updated successfully, but these errors were encountered: