allow non ascii chars according to go spec

[mmlb]

See https://golang.org/ref/spec#Identifiers

[mmlb]

See following script for demo.

#!/usr/bin/env python3

import re

# pattern = r'(?:[a-zA-Z_]\w*|[\]\)])\.(?:[a-zA-Z_]\w*)?'
pattern = r'(?:[^\W\d]\w*|[\]\)])\.(?:[^\W\d]\w*)?'
prog = re.compile(pattern)
tests = {
    'a.': True,
    '_x9.': True,
    'ThisVariableIsExported.': True,
    'αβ.': True,
    '].a': True,
    ']._x9': True,
    '].ThisVariableIsExported': True,
    '].αβ': True,
    ').a': True,
    ')._x9': True,
    ').ThisVariableIsExported': True,
    ').αβ': True,
    '9.': False,
}

for k, v in tests.items():
    match = prog.match(k) is not None
    if v == match:
        print('"%s" expected: %s, got: %s' % (k, v, match))
    else:
        print()
        print('!!!! "%s" expected: %s, got: %s !!!!' % (k, v, match))
        print()

[zchee]

/cc @Shougo @svanharmelen

[svanharmelen]

Yeah... Looks good! 👍

@mmlb while your at it, could you maybe trow in a \b as well? As I just noticed that case while testing this:

6a. should not match, but it does. By adding a \b at the start we fix this as well. The regex then becomes:

(?:\b[^\W\d]\w*|[\]\)])\.(?:[^\W\d]\w*)?

[zchee]

and /cc @nhooyr

[mmlb]

@svanharmelen do you mean that you saw this while testing in nvim? I added the '6a.': False case to my little script there and it does not match.

[svanharmelen]

@mmlb are you sure? It should match without the \b. Check here: https://regex101.com/r/vB7rC7/1

[zchee]

@mmlb @svanharmelen Thanks for assistance :)

I do not quite understand the regexp, because I can not determine... 😅
so I entrust to @svanharmelen and @Shougo judgment. 😱

[Shougo]

LGTM

Edit: But yes, \b is needed.

[mmlb]

updated

[svanharmelen]

LGTM! Thanks @mmlb

[zchee]

@mmlb @Shougo @svanharmelen
Thanks!! LGTM.

[mmlb]

hopefully last of the regex PRs for a while ;)

[zchee]

@mmlb Thanks, I'm relieved to hear that :)