Allow Kinds to be registered by packages outside JuliaSyntax

[c42f]

Extensible kinds are quite tricky. We want

• To use a small number of bits for them
• To have the string representation in the source, but have the compiler able to fully inline the integer representation.
• Allow modules with different kinds to cooperate together on the same integer representation.
• Not trigger invalidation when new kinds are added
• Different Kind modules to not require cooperation

This is a very hard set of constraints to satisfy. The last one is already impossible in a single flat namespace so in this design we've given up on it and require cooperation between all kind extension modules, including module authors allocating non-colliding id's for their modules, in addition to non-colliding kind names.

Why do we need this? It's very convenient for JuliaLowering to extend the set of Kinds, rather than needing to clone/convert Kind just to add a few lowering-specific forms.

[codecov]

Codecov Report

Attention: Patch coverage is 74.07407% with 14 lines in your changes missing coverage. Please review.

Project coverage is 95.69%. Comparing base (ae7d6ac) to head (ebea012).
Report is 6 commits behind head on main.

Files	Patch %	Lines
src/kinds.jl	74.07%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #461      +/-   ##
==========================================
- Coverage   96.43%   95.69%   -0.74%     
==========================================
  Files          14       13       -1     
  Lines        4208     3973     -235     
==========================================
- Hits         4058     3802     -256     
- Misses        150      171      +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[c42f]

Ok codecov, you are partly right I forgot to cover some obscure error pathways for this obscure feature :-/
Unsure it's worth covering them but let's do it I guess.

[c42f]

Let's go with this. I don't love it. But I've thought a lot about a better solution and haven't found one so far.

[c42f]

Chatted to @JeffBezanson about this - I think his opinion was roughly

hmm don't do this, I don't think we want extensible kinds. Oh no, especially not with __init__()

Sorry Jeff 😂 For what it's worth, I tried other options and they also were bad!

To explain a few things about how this implementation is one of the less terrible options:

• "But just add the lowering kinds to JuliaSyntax.jl?". This is the easiest option and I tried this. But it's extremely inconvenient to have to edit JuliaSyntax to add new kinds for JuliaLowering: you're basically always on a JuliaSyntax.jl branch making CI hard downstream and introducing really tight coupling.
• The implementation in this PR plays nice with precompilation and macro expansion in the sense that K"function" (for example) can be safely macro expanded to Kind(0x001f), and this also works with kinds from "Kind extension modules"
• It's extremely convenient for JuliaSyntaxFormatter to have its own token kinds for whitespace. The alternatives aren't very satisfying and involve inventing an inefficient replacement for Kind for use in that package, such as Union{Kind,JuliaSyntaxFormatterKind}.

One thing which is currently rather bad about this implementation (in light of the very recent juliac work from JuliaCon) is that we do have significant code in __init__(). But I think we could remove nearly all of that and limit it to a single push! to a Vector{Any}. At least JuliaSyntax itself doesn't suffer any inefficiency and doesn't need to use __init__().

I'm not sure what the long term plan for this should be. For now I still think it's an improvement from the previous status quo and can be an internal API I guess.