v60.1.0-alpha.436

Summary

This release focuses primarily on:

• Fixing issues with deploying satellite assemblies to the build and publish output
• Removing support for ICharSequence and changing to ReadOnlySpan<char>
• Removing support for StringBuilder as a character sequence, since it has documented performance problems. See: https://learn.microsoft.com/en-us/dotnet/api/system.text.stringbuilder.chars?view=net-8.0#remarks
• Reducing unnecessary heap allocations by using the stack and/or array pool
• Removing duplicated business logic from generated code
• Redesigning APIs to work with Span<char> and ReadOnlySpan<char>
• Removing support for old target frameworks (such as netstandard1.x and netcoreapp3.x)
• Adding support for net8.0

Note that APIs that accept or return a StringBuilder are being considered for removal in a future release.

What's Changed

• BUG: ICU4N.Reflection.AssemblyExtensions: Fixed bug that was causing GetCustomAttribute() not to function on net40 by @NightOwl888 in #47
• Added dependency on System.Memory and enabled Span support in net451 and netstandard2.0 by @NightOwl888 in #49
• ICU4N.Impl.ResourceKey: Removed unnecessary unsafe code from InternalSubString() method by @NightOwl888 in #50
• ICU4N.Text.ValueStringBuilder: Added AppendCodePoint() and Remove() methods + tests by @NightOwl888 in #48
• PERFORMANCE: ICU4N.Impl.LocaleIDParser: Converted to ref struct using ValueStringBuilder internally to reduce allocations by @NightOwl888 in #51
• PERFORMANCE: ICU4N.Impl.RuleCharacterIterator: Fixed bottleneck when calling Lookahead() method by @NightOwl888 in #52
• Fixed all build warnings by @NightOwl888 in #53
• BUG: ICU4N.Impl.Utility::ParseInteger(): Fixed logic to ignore case when searching for the prefix "0x" by @NightOwl888 in #58
• BUG: ICU4N.Text.ValueStringBuilder: Use FEATURE_STRING_IMPLCIT_TO_READONLYSPAN instead of NETCOREAPP by @NightOwl888 in #59
• PERFORMANCE: ICU4N.Transliterator: Converted lambda style "anonymous classes" into concrete implementations by @NightOwl888 in #60
• Converted Java array-style ref/out/return values into ref/out parameters in .NET (Fixes #57) by @NightOwl888 in #61
• Auto-generate T4 Templates (Closes #40) by @NightOwl888 in #63
• SWEEP: Changed naming convention suffix of T4 templates by @NightOwl888 in #64
• Added support for Span/ReadOnlySpan in net40 by @NightOwl888 in #68
• Dropped support for ICharSequence in favor of ReadOnlySpan/ReadOnlyMemory by @NightOwl888 in #71
• Removed .NET Standard 1.x and .NET Core 1.x build features (Fixes #72) by @NightOwl888 in #74
• Support building (some) satellite assemblies on macOS via Mono by @paulirwin in #75
• Added net8.0 target and tests (Fixes #73) by @NightOwl888 in #78
• ICU4N.Support.AnonymousComparer: Deleted because this is the same functionality as Comparer.Compare() by @NightOwl888 in #83
• ICU4N.Impl.CacheValue::SoftValue: Removed finalizer by @NightOwl888 in #84
• Rename numeric methods/types to conform with .NET conventions, #12 by @paulirwin in #76
• Normalize constants/field names in Normalizer2Impl, #11 by @paulirwin in #77
• ICU4N.Resources.csproj: Removed NETStandard.Library as a transitive dependency by @NightOwl888 in #86
• Fix for many issues with copying, naming, and loading satellite assemblies by @NightOwl888 in #88
• ICU4N.Resources: Include Newtonsoft.Json binaries so we don't conflict with consumers by @NightOwl888 in #89
• .build/dependencies.props: Added version constraint by @NightOwl888 in #90

New Contributors

• @paulirwin made their first contribution in #75

Full Changelog: v60.1.0-alpha.402...v60.1.0-alpha.436

v60.1.0-alpha.402

Known Issues

1. The ICU4N.FormatNumberRuleBased uses the .NET formatter and replaces the ASCII digit output with native digits of the current language. This formatter doesn't support all of the features of DecimalFormat. Once known issue with it is that it doesn't follow the "half even" or "round toward even" rounding as is in the IEEE 754 specfication. See TR35#Rounding. Instead it rounds "toward positive infinity". To our knowledge, this only affects behavior of floating point numbers when they are larger than the value of Int64.MaxValue (9,223,372,036,854,775,807; that is, hexadecimal 0x7FFFFFFFFFFFFFFF) or smaller than Int64.MinValue (negative 9,223,372,036,854,775,808; that is, hexadecimal 0x8000000000000000), however, there may be other cases where rounding is incorrect when plural formatting is part of the rule.

Change Log

Breaking Changes

1. ICU4N.Impl.ResourceKey::Substring(): Converted from start/end to startIndex/length to match .NET conventions. Optimized to use ValueStringBuilder, where Span<T> is supported.
2. ICU4N.Impl.PluralRulesLoader + ICU4N.Text.PluralRules: Renamed ForLocale() > GetInstance() to match .NET naming conventions.
3. ICU4N.Text.NumberPresentation: Moved NumberPresentation enum and NumberPresentationExtensions class from the ICU4N.Text to ICU4N.Globalization namespace. Changed the values to start with 0 instead of 1 so we have a reasonable default value in .NET.
4. ICU4N.Globalization.UCultureInfo: Removed ClearCachedData() method. This is in .NET only to allow it to fetch culture settings that have changed in the underlying OS.
5. ICU4N.Text (DisplayContext + DisplayContextType + DisplayContextExtensions): Removed from the public API since they had been previously refactored in the ICU4N.Globalization namespace as Capitalization, DialectHandling, DisplayLength, and SubstituteHandling enums and the DisplayContextOptions class.
6. ICU4N.Text (NumberFormat + DecimalFormat + RuleBasedNumberFormat): Removed from the public API and added an IncludeLegacyNumberFormat build property so they can be optionally compiled for those who need these features. The plan is to port the remaining frunctionality that isn't already covered by FormatNumberRuleBased into static APIs later.
7. ICU4N.Globalization.Capitalization: Added "For" prefix to the names as they were in ICU4J, since this adds clarity to their intended purpose. Also explicitly specified numeric values of Capitalization, DialectHandling, DisplayLength, and SubstituteHandling enums in ICU4N.Globalization to match ICU4J numeric values.

New Features

1.ICU4N.Impl.PatternProps: Added WhiteSpace static field with the same values that are matched with PatternProps.IsWhiteSpace() for use in Trim() method overloads.
8. Added new FormatNumberRuleBased static class to act as the new API for the ICU4J RuleBasedNumberFormat functionality. This API consists of extension methods for the numeric data types:

• System.Byte
• System.Int16
• System.Int32
• System.Int64
• System.Int128
• System.Numerics.BigInteger
• System.SByte
• System.IntPtr
• System.UInt16
• System.UInt32
• System.UInt64
• System.UInt128
• System.Half
• System.Single
• System.Double
• System.Decimal

9. ICU4N.Globalization: Added NumberFormatRules class to serve as the rules engine for the Rule-based Number Format functionality. This class is immutable and thread safe. Optimized Rule-based Number Format rule description string parsing to eliminate unnecessary substring allocations.
10. ICU4N.Text (SplitTokenizerEnumerator + MultiDelimiterSplitTokenizerEnumerator): Added features to control delimiter length to exclude from the token and trim behavior for start/end/both (or specify no trimChars for no trimming)
11. ICU4N.Globalization: Added UNumberFormatInfo class to cover the .NET NumberFormatInfo API. This class can be instantiated and used in the same way as the .NET API.
12. ICU4N.Globalization.UNumberFormatInfo: Added SpellOut, Ordinal, Duration, and NumberingSystem properties to expose the rules engines and DefaultRuleSetName and RuleSetNames properties publicly. These can be used to deterimine the ruleSetName to pass to the FormatNumberRuleBased APIs.
13. ICU4N.Globalization.UNumberFormatInfo: Added Capitalization property (DisplayContext enum in ICU4J).
14. ICU4N: Added IcuNumber internal class to serve as the business logic layer for formatting and parsing. Added formatters for long and double to output native digits + tests to confirm that they are the same as the DecimalFormat instance that backs RuleBasedNumberFormat. Also added FormatPlural() method + tests and FormatXXXRuleBased() overloads to prepare the parameters for the call into the rules engine (NumberFormatRules) that is passed.
15. ICU4N.Text.ValueStringBuilder: Added Insert() overload to handle ReadOnlySpan<char>
16. Added target for .NET 7.0. Fixed conflicts due to the new AsReadOnly() extension methods and build errors due the unsafe logic in PluralRules.

Improvements

1. PERFORMANCE: ICU4N.Globalization.UCultureInfo.Name: Cache the name locally so it doesn't have to be parsed on every request.
2. ICU4N.Globalization.UCultureInfo::Canonicalize(): Added null guard clause
3. ICU4N.Impl.PluralRulesLoader + ICU4N.Text.PluralRules: Added GetInstance() overload that accepts the culture name as a string, for use by internal UCultureData class.
4. ICU4N.Globalization.UNumberFormatInfoTest: Added tests to verify all cultures load decimal format settings correctly (we still have some way to go for currency data)
5. ICU4N.Globalization.NumberPresentationExtensions: Added IsDefined() method so we can quickly check for valid values in guard clauses.
6. ICU4N.Numerics.DecimalQuantity_AbstractBCD: Optimized number parsing using ReadOnlySpan<char> where appropriate.
7. Changed the name of FEATURE_READONLYDICTIONARY to FEATURE_IREADONLYCOLLECTIONS to include IReadOnlyList<T>.
8. ICU4N.Globalization.NumberPresentationExtensions: Added an internal extension method to quickly lookup the NumberingSystemRules instance for the current UNumberFormatInfo.
9. ICU4N.Globalization.UCultureInfo: Added ReadOnly() and Clone() implementations and added API documentation for NumberFormat, IsReadOnly, ReadOnly() and Clone() members.
10. ICU4N.Impl.ICUResourceBundle.WholeBundle: Lazy-load the UCultureInfo instance so we don't fill up the UCultureData cache with all cultures and non-culture resource names.

Bug Fixes

1. ICU4N.Util.ResourceBundle::SetRootType(): Fixed fallthrough to default for Missing case

v60.1.0-alpha.401

Known Issues

1. This package contains a preview of RuleBasedBreakIterator which has dependencies on DecimalFormat, DecimalFormatSymbols, NumberFormat, and Formatter. These will not be made public in the final release, so use these features at your own peril.

Change Log

1. Changed to use official buildTransitive NuGet feature instead of using a viral copy of the ICU4N.targets file. This allows more options for including subsets of resource data.
2. Updated README.md with a Managing Resources section.

v60.1.0-alpha.400

Known Issues

1. This package relies on an unofficial "viral" copy of the ICU4N.targets file during dotnet pack for the transitive dependency to be distributed to consumers. It is recommended to use v60.1.0-alpha.401 or higher, which uses the official buildTransitive NuGet support. This will prevent collisions for consumers that use their own build/<packageName>.targets file.
2. This package contains a preview of RuleBasedBreakIterator which has dependencies on DecimalFormat, DecimalFormatSymbols, NumberFormat, and Formatter. These features will not be made public in the final release, so use at your own peril.

Change Log

1. BREAKING: Restructured NuGet packaging and assemblies. There is now only 1 main assembly, ICU4N.dll (in the package ICU4N) and a separate NuGet package ICU4N.Resources that contains satellite assemblies. Resources are no longer included in the main assembly. This provides the following benefits:
   1. We can target additional frameworks without having to include additional copies of the resource data
   2. End users can choose to exclude resource data in distributions for languages that are not required. NOTE: The neutral language cultures contain shared resource data for specific cultures, so don't delete the neutral language file (such as fr/ICU4N.resources.dll unless all of the specific cultures (such as fr-CA/ICU4N.resources.dll) are also deleted.
2. Created ICU4J Resource Converter tool to transform resource files into satellite assemblies
3. Updated build to generate satellite assemblies incrementally and download ICU4J JAR files during the build.
4. Added tests to verify satellite assembly loading.
5. Added support for loading raw resource files from the local /data folder if it exists. Local resource files override satellite assembly data. The structure is the same as ICU4J, excluding the top-level /impl folder.
6. Updated Azure DevOps build script to run the tests in parallel background tasks.
7. Added RuleBasedNumberFormat and dependencies + tests. NOTE: This API matches ICU4J, but we plan to build a .NET-like API to supersede this one.
8. BUG: ICU4N.Support.Globalization.ResourceUtil: Ensure resource names are converted with both forward and backward slashes.
9. BUG: ICU4N.Text.MessagePattern: use J2N.Collections.Generic.List<T> because the Equals() method compares structural equality.
10. BUG: ICU4N.Text.MessageFormat: Use J2N.Collections.Generic.Dictionary<TKey, TValue> because this uses structural equality.
11. BUG: ICU4N.Text.PluralFormat: Corrected append length calculation.
12. BUG: ICU4N.Text.MessagePattern: Fixed 3 Append() calls to calculate the correct length
13. BUG: ICU4N.Text.SelectFormat: Corrected Append() method length calculations in 3 places
14. BUG: ICU4N.Text.MessageFormat: Corrected Append() method length calculations in 4 places
15. Added net6.0 target support.
16. ICU4N.Text.PluralRules: added TryParse() overloads and made use of ReadOnlySpan<char> to reduce allocations when loading rules. Added SplitTokenEnumerator and MultiCharSplitTokenEnumerator structs to tokenize rule strings without allocating substrings.
17. ICU4N.Impl.PluralRulesLoader: Refactored to use PluralRules.TryParseRule() directly instead of building a description string from the resource data only to parse the data back out again. Encapsulated HasExplicitBoundingInfo logic inside of RuleList.AddRule() so it doesn't have to be duplicated.
18. ICU4N.Text.PluralRules.RuleList: Refactored to use Dictionary instead of List as the underlying collection to take advantage of hashtable lookups and remove a large amount of looping code.
19. ICU4N.Text.MessagePatternPart: Eliminated static array allocation, since we can do a direct cast to the Enum symbol in .NET. Explicitly numbered MessagePatternArgType symbols.
20. Directory.Build.props: Upgraded LangVersion to 9.0
21. BUG: ICU4N.Text.CurrencyPluralInfo: Use J2N dictionary for structural equality
22. Added tests for net7.0.
23. Changed test target from net461 to net48.

v60.1.0-alpha.356

Change Log

1. Added Source Link Support
2. Added local dependency on Microsoft.NETFramework.ReferenceAssemblies to remove dependency on .NET Framework Targeting Pack during builds
3. Fixed XML documentation so it will be included in the NuGet package
4. Updated build with the latest features
   • Use latest SDKs for build and tests (6.0.101, 5.0.404, 3.1.416)
   • Added readme.md to NuGet package
   • Added target-framework-conditional support for dependencies.props
   • Added .gitattributes file
   • Added CLI build script for Windows, macOS, and Linux
   • Added documentation for building, testing, releasing
   • Renamed build directory to .build
   • Added PrintTargetFrameworks MSBuild target to read the supported target frameworks for a given test project
   • Added SkipGitVersioning MSBuild property to remove the Nerdbank.GitVersioning NuGet package reference when creating versions via scripts
   • Updated azure-pipelines.yml to allow overriding version info using parameters
5. Upgraded J2N package dependency to 2.0.0

v60.1.0-alpha.355

Change Log

1. BREAKING: ICU4N.Collation.Text.CollationElementIterator::Ingorable: Renamed Ignorable (fixes #35)
2. BUG: ICU4N.Util.VersionInfo.INVALID_VERSION_NUMBER_: Corrected invalid version number string (fixes #36)
3. Upgraded to J2N 2.0.0-beta-0017
4. ICU4N.Support.Text.ChoiceFormat: Fixed ambiguous reference caused by the new J2N.Numerics.Double class
5. Added tests for .NET 6.0 RC2
6. Changed build to use ubuntu-latest and macOS-latest during tests
7. BUG: ICU4N.Tests.Support.Globalization.UCultureInfoTest::TestDisplayKeyword(): Test using display culture. ICU was using culture of the current thread instead, which may be different than the display culture.
8. BUG: ICU4N.Support.Globalization.UCultureInfo::GetFullName(): Fixed cache initialization issue for creating a nameCache entry
9. ICU4N.Dev.Test.Collate.CollationServiceTest::TestRegisterFactory(): Ignore test failures on Linux until #37 can be resolved

v60.1.0-alpha.354

Change Log

1. BUG: Fixed infinite recursion issue when using the locales zh-CN, zh-HK, zh-MO, zh-SG, zh-TW (Fixes #29).
2. Added Lazy<T> to make GetOrAdd() method calls atomic (Fixes #31).
3. Added test targets for x86 on all target frameworks.
4. Upgraded to J2N 2.0.0-beta-0012
5. Upgraded to Nerdbank.GitVersioning 3.3.37
6. Added tests for .NET 5.0
7. Removed support for .NET Standard 1.3
8. Downgraded Microsoft.Extensions.Caching.Memory to 2.0.0

v60.1.0-alpha.353

Change Log

1. apache/lucenenet#343: Fixed configuration of NuGet package so the LICENSE.txt and icon files are in the root of the NuGet package, not in the content directory

v60.1.0-alpha.352

This release contains impactful performance improvements.

Change Log

1. ICU4N.Support.Collections: Factored out ListExtensions and moved CopyTo method to J2N
2. PERFORMANCE: Swapped in J2N's optimized ICollection<T>.ToArray() method where appropriate
3. PERFORMANCE: ICU4N.TestFramework: Fixed TestBoilerplate.GetSet() method so it doesn't lookup a value in every loop from the same dictionary it is iterating over
4. PERFORMANCE: ICU4N.Text.UTF16Extension::CharAt(): Fixed StringBuilder overload to copy out the characters it is analyzing to an array, since the indexer of the .NET StringBuilder is extremely slow.
5. PERFORMANCE: Replaced BitArray with BitSet in BreakIterator and Normalizer
6. PERFORMANCE: Fixed HashSet<T> capacity allocation with J2N's HashSet<T>, which has a capacity constructor overload on all target platforms
7. PERFORMANCE: ICU4N.Impl.ICUResourceBundle: Removed extra HashSet for tracking unique values, since the existing list can be utilized for that without the extra memory allocations and operations
8. PERFORMANCE: Swapped in J2N's HashSet<T> in other appropriate places
9. ICU4N.TestFramework: Added implicit operator support for Character, Integer, Long, and Double classes
10. PERFORMANCE: Factored out IDictionary<TKey, TValue>.Get(TKey) extension method in favor of IDictionary.TryGetValue(TKey, out TValue) where a performance advantage can be gained
11. PERFORMANCE: Added asserts overloads for generic collection types and UnicodeSet
12. PERFORMANCE: ICU4N.Tests.Transliterator: Optimized UnicodeMapTest by replacing Integer reference type with int
13. PERFORMANCE: ICU4N.Text.UTF16Extension.tt: Updated code generation to create a special case overload for StringBuilder, whose indexer is extremely slow in .NET
14. PERFORMANCE: Changed ICU4N.Text.ReplaceableString implementation to cache a copy of the string in the StringBuilder, since the Char32At() method needs to index into the characters and a string is much faster than StringBuilder. The cached string is updated whenever a change is made through the ReplaceableString API or if the StringBuilder it wraps changes in Length or GetHashCode() or when ReplaceableString.ToString() is called.
15. PERFORMANCE: ICU4N.Collation.Impl.Coll.CollationDataBuilder: Changed to use String.IndexOf() which performs far better than StringBuilderExtensions.IndexOf()
16. PERFORMANCE: ICU4N.Tests.Collation: Updated Counter class to use Interlocked rather than a lock
17. PERFORMANCE: ICU4N.Impl.ICUNotifier: Updated to use a Queue<T> rather than List<T>
18. PERFORMANCE: Use LazyInitializer throughout the project to load data rather than using a lock and volatile member
19. PERFORMANCE: Changed classes where appropriate to use a lock object instead of lock (this)
20. PERFORMANCE: ICU4N.Globalization.UCharacterNameIterator: Removed unnecessary redundant lock
21. PERFORMANCE: ICU4N.Util.VersionInfo: Use ConcurrentDictionary and remove locking
22. PERFORMANCE: ICU4N.Transliterator: Use lock object, LazyInitializer, and ConcurrentDictionary where appropriate to avoid lock (this)
23. Changed properties to use expression-style syntax, where appropriate
24. PERFORMANCE: Optimized empty collection creation
25. PERFORMANCE: Eliminated InvalidCastExceptions and created if-then blocks that test the type compatibility instead
26. PERFORMANCE: Implmentd GetHashCode() where it was "not designed" in the original implementation
27. BREAKING: ICU4N.Util.UResourceBundle: Changed GetLocaleID() and GetBaseName() to properties LocaleID and BaseName
28. ICU4N.Tests.Collation.Dev.Test.Util.ICUResourceBundleCollationTest: Enabled TestFunctionalEquivalent() test, as it now passes
29. ICU4N.Tests.Dev.Test.Rbbi.RBBITestExtended::TestExtended(): Enabled the test, as it now passes
30. ICU4N.Impl.ICUService: Fixed broken reference in XML doc comment
31. Ugraded to J2N 2.0.0-beta-0009

v60.1.0-alpha.351

Change Log

1. BREAKING: Moved LocaleIDParser, LocaleIDs, and LocaleUtility to ICU4N.Globalization namespace
2. ICU4N.Support.Collections: Deleted unneeded GenericComparer and swapped in the implementation from J2N.
3. BREAKING: ICU4N.Impl.ICULocaleService: Refactored Get() and GetKey() overloads to use out parameters rather than passing in single dimensional arrays (Java has no out parameters, so this was a workaround in ICU4J)
4. Upgraded to J2N 2.0.0-beta-0008
5. Upgraded to Nerdbank.GitVersioning 3.0.28