Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add collation from ICU4X #740

Merged
merged 58 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
0327cde
Add collation
mosuem Nov 7, 2023
f90d888
Remove unused import
mosuem Nov 7, 2023
44a8e36
Update to merged
mosuem Nov 15, 2023
a7e253c
Merge branch 'main' into addCollationICU4X
mosuem Dec 6, 2023
2d40719
Merge branch 'main' into addCollationICU4X
mosuem Jan 3, 2024
f36d1cf
Merge branch 'main' into addCollationICU4X
mosuem Jan 11, 2024
e57d200
Setup submodule
mosuem Jan 11, 2024
c204c09
Checkout submodules
mosuem Jan 11, 2024
bf61fbc
Add build.dart
mosuem Jan 12, 2024
78da54e
move bindings
mosuem Jan 12, 2024
79a21b8
Use path to build output dir
mosuem Jan 12, 2024
2842d2c
Fix build.dart
mosuem Jan 12, 2024
5bb56c7
Enable test
mosuem Jan 12, 2024
d48fd3f
Add help
mosuem Jan 12, 2024
eeddc31
Fix help
mosuem Jan 12, 2024
f7ec5d0
Make data default
mosuem Jan 12, 2024
09535cc
Move collation test to general testing
mosuem Jan 12, 2024
aafb7d1
Fix workflow
mosuem Jan 12, 2024
1e0e5e6
Run tests on dev
mosuem Jan 12, 2024
2987977
Fix imports
mosuem Jan 12, 2024
92c6626
Add changelog and rev version
mosuem Jan 12, 2024
ec50fab
take health from branch
mosuem Jan 12, 2024
2a39b17
Add todo
mosuem Jan 15, 2024
21beea9
Add top level analysis options
mosuem Jan 15, 2024
b41e661
Merge branch 'main' into addCollationICU4X
mosuem Jan 15, 2024
7a10af1
Fix collator options
mosuem Jan 16, 2024
2aad31a
Do checkout submodules
mosuem Jan 16, 2024
e8f3570
Use checkout submodules
mosuem Jan 16, 2024
c9beb3a
Make casefirst non null
mosuem Jan 16, 2024
cea2e3e
enable native assets experiment
mosuem Jan 16, 2024
6f27941
Rev SDK
mosuem Jan 16, 2024
8d8bd35
And env
mosuem Jan 16, 2024
480ab1b
set default mode
mosuem Jan 16, 2024
9761887
Add licenses
mosuem Jan 16, 2024
493f420
Adapt option matching
mosuem Jan 23, 2024
910bf87
Update icu4x
mosuem Jan 23, 2024
778ccb9
Update icu4x ref
mosuem Jan 29, 2024
de1383b
Add ignores
mosuem Jan 29, 2024
c14158f
Merge branch 'main' into addCollationICU4X
mosuem Jan 29, 2024
8aa9a69
Update submodule
mosuem Jan 29, 2024
0184312
Remove cached
mosuem Jan 29, 2024
021c27a
Remove branch
mosuem Jan 29, 2024
caf8477
remove submodule
mosuem Jan 29, 2024
7086061
Add submodule
mosuem Jan 29, 2024
60321b3
Remove submodules
mosuem Jan 29, 2024
3eb3e80
Add submodule
mosuem Jan 29, 2024
03f1127
Merge main
mosuem Jan 29, 2024
2a30df8
Test current_repo
mosuem Jan 29, 2024
f317d47
remove bracket
mosuem Jan 29, 2024
aafef74
Add ignores
mosuem Jan 29, 2024
fcc4afa
Add changelog
mosuem Jan 29, 2024
2290897
Add trailing newline
mosuem Jan 29, 2024
f369e56
Merge branch 'main' into addCollationICU4X
mosuem Feb 1, 2024
bf67394
Incorporate build_libs.dart
mosuem Feb 1, 2024
8c90889
Add simulator special casing
mosuem Feb 1, 2024
eed6b05
Update pkgs/intl4x/build.dart
mosuem Feb 1, 2024
9462035
Update pkgs/intl4x/build.dart
mosuem Feb 1, 2024
e1c5662
Reintroduce platform name for `ubuntu`
mosuem Feb 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 22 additions & 41 deletions pkgs/intl4x/lib/intl4x.dart
Original file line number Diff line number Diff line change
Expand Up @@ -41,45 +41,43 @@ typedef Icu4xKey = String;
/// ```
class Intl {
final EcmaPolicy ecmaPolicy;

// ignore: unused_field, prefer_final_fields
String _dyliblocation = 'path.dll'; //What about path.wasm? How to load this?
// ignore: unused_field, prefer_final_fields
String _datalocation = 'data.blob'; //What about additional data?

final Data data;
final List<Locale> supportedLocales;
final LocaleMatcher localeMatcher;

Collation collation([CollationOptions options = const CollationOptions()]) =>
Collation(
options,
CollationImpl.build(locale, localeMatcher, ecmaPolicy),
CollationImpl.build(locale, data, options, localeMatcher, ecmaPolicy),
);

NumberFormat numberFormat([NumberFormatOptions? options]) => NumberFormat(
options ?? NumberFormatOptions.custom(),
NumberFormatImpl.build(locale, localeMatcher, ecmaPolicy),
NumberFormatImpl.build(locale, data,
options ?? NumberFormatOptions.custom(), localeMatcher, ecmaPolicy),
);

ListFormat listFormat([ListFormatOptions? options]) => ListFormat(
options ?? const ListFormatOptions(),
ListFormatImpl.build(locale, localeMatcher, ecmaPolicy),
ListFormat listFormat(
[ListFormatOptions options = const ListFormatOptions()]) =>
ListFormat(
ListFormatImpl.build(locale, data, options, localeMatcher, ecmaPolicy),
);

DisplayNames displayNames([DisplayNamesOptions? options]) => DisplayNames(
options ?? const DisplayNamesOptions(),
DisplayNamesImpl.build(locale, localeMatcher, ecmaPolicy),
DisplayNames displayNames(
[DisplayNamesOptions options = const DisplayNamesOptions()]) =>
DisplayNames(
DisplayNamesImpl.build(
locale, data, options, localeMatcher, ecmaPolicy),
);

DateTimeFormat datetimeFormat([DateTimeFormatOptions? options]) =>
DateTimeFormat datetimeFormat(
[DateTimeFormatOptions options = const DateTimeFormatOptions()]) =>
DateTimeFormat(
options ?? const DateTimeFormatOptions(),
DateTimeFormatImpl.build(locale, localeMatcher, ecmaPolicy),
DateTimeFormatImpl.build(
locale, data, options, localeMatcher, ecmaPolicy),
);

PluralRules plural([PluralRulesOptions? options]) => PluralRules(
options ?? PluralRulesOptions(),
PluralRulesImpl.build(locale, localeMatcher, ecmaPolicy),
PluralRulesImpl.build(locale, data, options ?? PluralRulesOptions(),
localeMatcher, ecmaPolicy),
);

/// Construct an [Intl] instance providing the current [locale] and the
Expand All @@ -90,6 +88,7 @@ class Intl {
this.ecmaPolicy = defaultPolicy,
this.supportedLocales = allLocales,
this.localeMatcher = LocaleMatcher.lookup,
this.data = const NoData(),
}) : locale = locale ?? findSystemLocale();

Intl.includeLocales({
Expand Down Expand Up @@ -120,10 +119,12 @@ class Intl {
Locale? locale,
EcmaPolicy ecmaPolicy = defaultPolicy,
LocaleMatcher localeMatcher = LocaleMatcher.lookup,
Data? data,
}) : this._(
locale: locale,
ecmaPolicy: ecmaPolicy,
supportedLocales: allLocales,
data: data ?? const NoData(),
);

Locale locale;
Expand All @@ -135,23 +136,3 @@ class Intl {
return shouldUse && canUse;
}
}

/// ICU4X will be compiled into the application, so there is no need to
/// specify any data here. Users may want to add additional data at runtime,
/// which could be supported through this API.
///
/// TODO: Wire this through to the ICU4X formatters.
final Map<String, List<Icu4xKey>> additionalICU4XData = {};

void addIcu4XData(Data data) {
final callbackFromICUTellingMeWhatLocalesTheDataContained =
extractKeysFromData();
additionalICU4XData
.addAll(callbackFromICUTellingMeWhatLocalesTheDataContained);
throw UnimplementedError('Call to ICU4X here');
}

Map<String, List<Icu4xKey>> extractKeysFromData() {
//TODO: Add implementation
return {};
}
5 changes: 2 additions & 3 deletions pkgs/intl4x/lib/src/collation/collation.dart
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,9 @@ import 'collation_impl.dart';
import 'collation_options.dart';

class Collation {
final CollationOptions _options;
final CollationImpl _collationImpl;

const Collation(this._options, this._collationImpl);
const Collation(this._collationImpl);

/// Compare two strings in a locale-dependant manner.
///
Expand All @@ -25,7 +24,7 @@ class Collation {
if (isInTest) {
return a.compareTo(b);
} else {
return _collationImpl.compareImpl(a, b, _options);
return _collationImpl.compareImpl(a, b);
}
}
}
58 changes: 54 additions & 4 deletions pkgs/intl4x/lib/src/collation/collation_4x.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,67 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'package:icu/icu.dart' as icu;

import '../data.dart';
import '../data_4x.dart';
import '../locale/locale.dart';
import '../locale/locale_4x.dart';
import 'collation_impl.dart';
import 'collation_options.dart';

CollationImpl getCollator4X(Locale locale) => Collation4X(locale);
CollationImpl getCollator4X(
Locale locale, Data data, CollationOptions options) =>
Collation4X(locale, data, options);

class Collation4X extends CollationImpl {
Collation4X(super.locale);
final icu.ICU4XCollator _collator;

Collation4X(Locale locale, Data data, CollationOptions options)
: _collator = icu.ICU4XCollator.v1(
data.to4X(),
locale.to4X(),
options.toDartOptions(),
),
super(locale, options);

@override
int compareImpl(String a, String b, CollationOptions options) {
throw UnimplementedError('Insert diplomat bindings here');
int compareImpl(String a, String b) => _collator.compare(a, b).index;
}

extension on CollationOptions {
robertbastian marked this conversation as resolved.
Show resolved Hide resolved
icu.ICU4XCollatorOptionsV1 toDartOptions() {
final icu4xOptions = icu.ICU4XCollatorOptionsV1();

mosuem marked this conversation as resolved.
Show resolved Hide resolved
//Usage usage;
//TODO: find matching

//Sensitivity? sensitivity;
//TODO: find matching

//bool ignorePunctuation;
//TODO: find matching

//bool numeric;
//TODO: what about auto?
icu4xOptions.numeric =
numeric ? icu.ICU4XCollatorNumeric.on : icu.ICU4XCollatorNumeric.off;

//CaseFirst? caseFirst;
//TODO: what about localeDependent? What about icu.off and icu.auto?
final caseFirst4X = switch (caseFirst) {
CaseFirst.upper => icu.ICU4XCollatorCaseFirst.upperFirst,
CaseFirst.lower => icu.ICU4XCollatorCaseFirst.lowerFirst,
CaseFirst.localeDependent => throw UnsupportedError(''),
null => null,
};
if (caseFirst4X != null) {
icu4xOptions.caseFirst = caseFirst4X;
}

//String? collation;
//TODO: find matching

return icu4xOptions;
}
}
10 changes: 6 additions & 4 deletions pkgs/intl4x/lib/src/collation/collation_ecma.dart
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ import 'collation_options.dart';

CollationImpl? getCollatorECMA(
Locale locale,
CollationOptions options,
LocaleMatcher localeMatcher,
) =>
CollationECMA.tryToBuild(locale, localeMatcher);
CollationECMA.tryToBuild(locale, options, localeMatcher);

@JS('Intl.Collator')
class CollatorJS {
Expand All @@ -29,15 +30,16 @@ external List<String> supportedLocalesOfJS(
]);

class CollationECMA extends CollationImpl {
CollationECMA(super.locale);
CollationECMA(super.locale, super.options);

static CollationImpl? tryToBuild(
Locale locale,
CollationOptions options,
LocaleMatcher localeMatcher,
) {
final supportedLocales = supportedLocalesOf(localeMatcher, locale);
return supportedLocales.isNotEmpty
? CollationECMA(supportedLocales.first)
? CollationECMA(supportedLocales.first, options)
: null;
}

Expand All @@ -54,7 +56,7 @@ class CollationECMA extends CollationImpl {
}

@override
int compareImpl(String a, String b, CollationOptions options) {
int compareImpl(String a, String b) {
final collatorJS = CollatorJS(
[locale.toLanguageTag()],
options.toJsOptions(),
Expand Down
14 changes: 10 additions & 4 deletions pkgs/intl4x/lib/src/collation/collation_impl.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import '../data.dart';
import '../ecma/ecma_policy.dart';
import '../locale/locale.dart';
import '../options.dart';
Expand All @@ -15,24 +16,29 @@ import 'collation_stub.dart' if (dart.library.js) 'collation_ecma.dart';
abstract class CollationImpl {
/// The current locale, selected by the localematcher
final Locale locale;
final CollationOptions options;

CollationImpl(this.locale);
CollationImpl(this.locale, this.options);

/// Factory to get the correct implementation, either calling on ICU4X or the
/// in-built browser implementation.
factory CollationImpl.build(
Locale locales,
Locale locale,
Data data,
CollationOptions options,
LocaleMatcher localeMatcher,
EcmaPolicy ecmaPolicy,
) =>
buildFormatter(
locales,
locale,
data,
options,
localeMatcher,
ecmaPolicy,
getCollatorECMA,
getCollator4X,
);

/// Actual implementation of the [compare] method.
int compareImpl(String a, String b, CollationOptions options);
int compareImpl(String a, String b);
}
5 changes: 3 additions & 2 deletions pkgs/intl4x/lib/src/collation/collation_stub.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import '../../collation.dart';
import '../locale/locale.dart';
import '../options.dart';
import 'collation_impl.dart';

/// Stub for the conditional import
CollationImpl? getCollatorECMA(Locale locale, LocaleMatcher localeMatcher) =>
CollationImpl? getCollatorECMA(
Locale locale, CollationOptions options, LocaleMatcher localeMatcher) =>
throw UnimplementedError('Cannot use ECMA outside of web environments.');
23 changes: 9 additions & 14 deletions pkgs/intl4x/lib/src/data.dart
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
// Copyright (c) 2023, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:typed_data';

/// Placeholder for the data type of ICU4X - tbd!
abstract final class Data {}
sealed class Data {
const Data();
}

final class JsonData extends Data {
final String value;
final class AssetData extends Data {
final String key;

JsonData(this.value);
const AssetData(this.key);
}

final class BlobData extends Data {
final Uint8List value;
final class BundleData extends Data {}

BlobData(this.value);
final class NoData extends Data {
const NoData();
}
15 changes: 15 additions & 0 deletions pkgs/intl4x/lib/src/data_4x.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import 'dart:io';
import 'package:icu/icu.dart';
import 'data.dart';

extension DataProvider on Data {
ICU4XDataProvider to4X() {
final icu4xDataProvider = switch (this) {
AssetData() => ICU4XDataProvider.fromByteSlice(
File((this as AssetData).key).readAsBytesSync()),
BundleData() => ICU4XDataProvider.compiled(),
NoData() => ICU4XDataProvider.empty(),
};
return icu4xDataProvider;
}
}
6 changes: 2 additions & 4 deletions pkgs/intl4x/lib/src/datetime_format/datetime_format.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

import '../test_checker.dart';
import 'datetime_format_impl.dart';
import 'datetime_format_options.dart';

/// `DateTime` formatting, for example:
///
Expand All @@ -20,16 +19,15 @@ import 'datetime_format_options.dart';
/// .format(date); // Output: '4 mat.'
/// ```
class DateTimeFormat {
final DateTimeFormatOptions _options;
final DateTimeFormatImpl impl;

DateTimeFormat(this._options, this.impl);
DateTimeFormat(this.impl);

String format(DateTime datetime) {
if (isInTest) {
return '$datetime//${impl.locale}';
} else {
return impl.formatImpl(datetime, _options);
return impl.formatImpl(datetime);
}
}
}
13 changes: 9 additions & 4 deletions pkgs/intl4x/lib/src/datetime_format/datetime_format_4x.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,23 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import '../data.dart';
import '../locale/locale.dart';
import 'datetime_format_impl.dart';
import 'datetime_format_options.dart';

DateTimeFormatImpl getDateTimeFormatter4X(Locale locale) =>
DateTimeFormat4X(locale);
DateTimeFormatImpl getDateTimeFormatter4X(
Locale locale,
Data data,
DateTimeFormatOptions options,
) =>
DateTimeFormat4X(locale, data, options);

class DateTimeFormat4X extends DateTimeFormatImpl {
DateTimeFormat4X(super.locale);
DateTimeFormat4X(super.locale, Data data, super.options);

@override
String formatImpl(DateTime datetime, DateTimeFormatOptions options) {
String formatImpl(DateTime datetime) {
throw UnimplementedError('Insert diplomat bindings here');
}
}
Loading
Loading