Implement full Unicode 16.0.0 extended grapheme breaking.

[lrhn]

Includes rule GB9c (Indic Conjunt Break rule).

This change has a significant cost in size since the information needed per character no longer fits in 4 bits. The base table is therefore twice as big (one byte per entry rather than half of that).

The number of states in the state automatons have also increased slightly, but in comparison that's a negligible change.

Tests have been made more thorough, testing not only the Unicode Consortium provided tests, but also variants of those with representative characters for each category of character that either in or not-in the BMP, to test that surrogate pair decoding works correctly.

Test also check that the created automatons are minimal, in that no state is unreachable and no two states are indistinguishable.

[github-actions]

Package publishing

Package	Version	Status	Publish tag (post-merge)
package:characters	1.4.0	ready to publish	characters-v1.4.0
package:args	2.6.1-wip	WIP (no publish necessary)
package:async	2.12.0	already published at pub.dev
package:collection	1.19.1-wip	WIP (no publish necessary)
package:convert	3.1.2	already published at pub.dev
package:crypto	3.0.6	already published at pub.dev
package:fixnum	1.1.1	already published at pub.dev
package:logging	1.3.0	already published at pub.dev
package:os_detect	2.0.3-wip	WIP (no publish necessary)
package:path	1.9.1	already published at pub.dev
package:platform	3.1.6	already published at pub.dev
package:typed_data	1.4.0	already published at pub.dev

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

[github-actions]

PR Health

Breaking changes ✔️

Package	Change	Current Version	New Version	Needed Version	Looking good?
characters	None	1.4.0	1.4.0	1.4.0	✔️

Coverage ⚠️

File	Coverage
pkgs/characters/benchmark/benchmark.dart	💔 Not covered
pkgs/characters/lib/characters.dart	💔 Not covered
pkgs/characters/lib/src/characters.dart	💚 100 %
pkgs/characters/lib/src/characters_impl.dart	💚 89 %
pkgs/characters/lib/src/grapheme_clusters/breaks.dart	💚 97 %
pkgs/characters/lib/src/grapheme_clusters/constants.dart	💔 Not covered
pkgs/characters/lib/src/grapheme_clusters/table.dart	💚 100 %
pkgs/characters/tool/benchmark.dart	💔 Not covered
pkgs/characters/tool/bin/generate_tables.dart	💔 Not covered
pkgs/characters/tool/bin/generate_tests.dart	💔 Not covered
pkgs/characters/tool/generate.dart	💔 Not covered
pkgs/characters/tool/src/args.dart	💔 Not covered
pkgs/characters/tool/src/atsp.dart	💔 Not covered
pkgs/characters/tool/src/automaton_builder.dart	💔 Not covered
pkgs/characters/tool/src/data_files.dart	💔 Not covered
pkgs/characters/tool/src/debug_names.dart	💚 12 %
pkgs/characters/tool/src/graph.dart	💔 Not covered
pkgs/characters/tool/src/grapheme_category_loader.dart	💔 Not covered
pkgs/characters/tool/src/indirect_table.dart	💔 Not covered
pkgs/characters/tool/src/list_overlap.dart	💔 Not covered
pkgs/characters/tool/src/shared.dart	💔 Not covered
pkgs/characters/tool/src/string_literal_writer.dart	💔 Not covered
pkgs/characters/tool/src/table_builder.dart	💔 Not covered

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package	Leaked API symbols

License Headers ⚠️

// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

Files
pkgs/characters/lib/src/grapheme_clusters/breaks.dart

All source files should start with a license header.

This check can be disabled by tagging the PR with skip-license-check.

[lrhn]

Health check is wrong. The changelog is correct since the version wasn't changed, and the existing changelog didn't list missing part that is now implemented.

[natebosch]

Out of these comments my primary concern is the return statements in loops in tests.

Should some of those be continue instead of return?

[lrhn]

I think I broke isGraphemeClusterBoundary. Have to fix that too.
... and fixed. That was a silly bug.

[lrhn]

Out of these comments my primary concern is the return statements in loops in tests.

Should some of those be continue instead of return?

They could be break, and now they are.
(They should break to an expect that will not be able to fail, but that still means every test run goes through that expect, one way or another.)

[lrhn]

Think I'm done now. Much cleaned up (and nicer, IMO).