Testing the Temporal proposal

[jugglinmike]

The Temporal proposal reached Stage 3 on 2021-03-10, and like all ECMA262 proposals, it needs coverage in Test262 to advance to Stage 4. In terms of normative text, it is easily the largest proposal we've seen. Testing it will definitely require significant time, effort, and coordination (even with the proposal's existing tests as a starting point). We'd like to document progress publicly for transparency and to encourage participation.

To get the ball rolling, I'm suggesting that we track progress on an per-API basis using this GitHub.com issue. Compared to alternatives like collaborative document editing services and an in-repository text file, this seems like it balances ease of hosting, maintainability, and proximity to the tests. This approach worked well for ES6's well-known Symbols--the most similar (though much smaller) effort that comes to mind.

Working mode

Anyone can comment here when they'd like to begin writing tests for a specific API. A maintainer will update the list to reflect the ongoing and completed work. When you submit a pull request, please mention this issue (just write "gh-3002") so GitHub displays the status of your work in this thread.

Tests for multiple APIs can be submitted in a single pull request, but patches should be small enough for another contributor to review them in a single session. For instance, it probably makes sense to submit tests for all of the @@toStringTag methods in one patch. There's some subjectivity here, but when in doubt, we should err on the side of smaller patches (and remain open to reviewers' requests to break up pull requests).

Status

• Temporal.now.timeZone (reviewed @Ms2ger 2022-04-13)
• Temporal.now.instant (reviewed @Ms2ger 2022-04-13)
• Temporal.Now.plainDateTime
• Temporal.Now.plainDateTimeISO
• Temporal.Now.zonedDateTime
• Temporal.Now.zonedDateTimeISO
• Temporal.Now.plainDate
• Temporal.Now.plainDateISO
• Temporal.Now.plainTimeISO
• Temporal.PlainDate (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.from (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.compare (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.constructor (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainDate.prototype.calendar (reviewed @Ms2ger 2022-05-25)
• Temporal.PlainDate.prototype.year (reviewed @Ms2ger 2022-05-25)
• Temporal.PlainDate.prototype.month (reviewed @Ms2ger 2022-05-25)
• Temporal.PlainDate.prototype.monthCode (reviewed @Ms2ger 2022-05-25)
• Temporal.PlainDate.prototype.day (reviewed @Ms2ger 2022-05-25)
• Temporal.PlainDate.prototype.dayOfWeek (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.dayOfYear (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.weekOfYear (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.daysInWeek (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.daysInMonth (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.daysInYear (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.monthsInYear (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.inLeapYear (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.toPlainYearMonth (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.toPlainMonthDay (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.getISOFields (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.add (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.subtract (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.with (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.withCalendar (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.until (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.since (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.equals (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.toPlainDateTime (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.toZonedDateTime (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.toString (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.toLocaleString
• Temporal.PlainDate.prototype.toJSON (reviewed @Ms2ger 2022-06-20)
• Temporal.PlainDate.prototype.valueOf (reviewed @Ms2ger 2022-05-25)
• Temporal.PlainTime
• Temporal.PlainTime.from
• Temporal.PlainTime.compare
• Temporal.PlainTime.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.PlainTime.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.calendar (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.hour (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.minute (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.second (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.millisecond (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.microsecond (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.nanosecond (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.add
• Temporal.PlainTime.prototype.subtract
• Temporal.PlainTime.prototype.with
• Temporal.PlainTime.prototype.until
• Temporal.PlainTime.prototype.since
• Temporal.PlainTime.prototype.round
• Temporal.PlainTime.prototype.equals
• Temporal.PlainTime.prototype.toPlainDateTime
• Temporal.PlainTime.prototype.toZonedDateTime
• Temporal.PlainTime.prototype.getISOFields (reviewed @ptomato 2022-08-03)
• Temporal.PlainTime.prototype.toString
• Temporal.PlainTime.prototype.toLocaleString
• Temporal.PlainTime.prototype.toJSON (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainTime.prototype.valueOf (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainDateTime
• Temporal.PlainDateTime.from
• Temporal.PlainDateTime.compare
• Temporal.PlainDateTime.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.PlainDateTime.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainDateTime.prototype.calendar
• Temporal.PlainDateTime.prototype.year
• Temporal.PlainDateTime.prototype.month
• Temporal.PlainDateTime.prototype.monthCode
• Temporal.PlainDateTime.prototype.day
• Temporal.PlainDateTime.prototype.hour
• Temporal.PlainDateTime.prototype.minute
• Temporal.PlainDateTime.prototype.second
• Temporal.PlainDateTime.prototype.millisecond
• Temporal.PlainDateTime.prototype.microsecond
• Temporal.PlainDateTime.prototype.nanosecond
• Temporal.PlainDateTime.prototype.dayOfWeek
• Temporal.PlainDateTime.prototype.dayOfYear
• Temporal.PlainDateTime.prototype.weekOfYear
• Temporal.PlainDateTime.prototype.daysInWeek
• Temporal.PlainDateTime.prototype.daysInMonth
• Temporal.PlainDateTime.prototype.daysInYear
• Temporal.PlainDateTime.prototype.monthsInYear
• Temporal.PlainDateTime.prototype.inLeapYear
• Temporal.PlainDateTime.prototype.with
• Temporal.PlainDateTime.prototype.withPlainTime
• Temporal.PlainDateTime.prototype.withPlainDate
• Temporal.PlainDateTime.prototype.withCalendar
• Temporal.PlainDateTime.prototype.add
• Temporal.PlainDateTime.prototype.subtract
• Temporal.PlainDateTime.prototype.until
• Temporal.PlainDateTime.prototype.since
• Temporal.PlainDateTime.prototype.round
• Temporal.PlainDateTime.prototype.equals
• Temporal.PlainDateTime.prototype.toString
• Temporal.PlainDateTime.prototype.toLocaleString
• Temporal.PlainDateTime.prototype.toJSON
• Temporal.PlainDateTime.prototype.valueOf
• Temporal.PlainDateTime.prototype.toZonedDateTime
• Temporal.PlainDateTime.prototype.toPlainDate
• Temporal.PlainDateTime.prototype.toPlainYearMonth
• Temporal.PlainDateTime.prototype.toPlainMonthDay
• Temporal.PlainDateTime.prototype.toPlainTime
• Temporal.PlainDateTime.prototype.getISOFields (reviewed @ptomato 2022-08-03)
• Temporal.ZonedDateTime
• Temporal.ZonedDateTime.from
• Temporal.ZonedDateTime.compare
• Temporal.ZonedDateTime.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.ZonedDateTime.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.ZonedDateTime.prototype.calendar
• Temporal.ZonedDateTime.prototype.timeZone
• Temporal.ZonedDateTime.prototype.year
• Temporal.ZonedDateTime.prototype.month
• Temporal.ZonedDateTime.prototype.monthCode
• Temporal.ZonedDateTime.prototype.day
• Temporal.ZonedDateTime.prototype.hour
• Temporal.ZonedDateTime.prototype.minute
• Temporal.ZonedDateTime.prototype.second
• Temporal.ZonedDateTime.prototype.millisecond
• Temporal.ZonedDateTime.prototype.microsecond
• Temporal.ZonedDateTime.prototype.nanosecond
• Temporal.ZonedDateTime.prototype.epochSeconds
• Temporal.ZonedDateTime.prototype.epochMilliseconds
• Temporal.ZonedDateTime.prototype.epochMicroseconds
• Temporal.ZonedDateTime.prototype.epochNanoseconds
• Temporal.ZonedDateTime.prototype.dayOfWeek
• Temporal.ZonedDateTime.prototype.dayOfYear
• Temporal.ZonedDateTime.prototype.weekOfYear
• Temporal.ZonedDateTime.prototype.hoursInDay
• Temporal.ZonedDateTime.prototype.daysInWeek
• Temporal.ZonedDateTime.prototype.daysInMonth
• Temporal.ZonedDateTime.prototype.daysInYear
• Temporal.ZonedDateTime.prototype.monthsInYear
• Temporal.ZonedDateTime.prototype.inLeapYear
• Temporal.ZonedDateTime.prototype.offsetNanoseconds
• Temporal.ZonedDateTime.prototype.offset
• Temporal.ZonedDateTime.prototype.with
• Temporal.ZonedDateTime.prototype.withPlainTime
• Temporal.ZonedDateTime.prototype.withPlainDate
• Temporal.ZonedDateTime.prototype.withTimeZone
• Temporal.ZonedDateTime.prototype.withCalendar
• Temporal.ZonedDateTime.prototype.add
• Temporal.ZonedDateTime.prototype.subtract
• Temporal.ZonedDateTime.prototype.until
• Temporal.ZonedDateTime.prototype.since
• Temporal.ZonedDateTime.prototype.round
• Temporal.ZonedDateTime.prototype.equals
• Temporal.ZonedDateTime.prototype.toString
• Temporal.ZonedDateTime.prototype.toLocaleString
• Temporal.ZonedDateTime.prototype.toJSON
• Temporal.ZonedDateTime.prototype.valueOf
• Temporal.ZonedDateTime.prototype.startOfDay
• Temporal.ZonedDateTime.prototype.toInstant
• Temporal.ZonedDateTime.prototype.toPlainDate
• Temporal.ZonedDateTime.prototype.toPlainTime
• Temporal.ZonedDateTime.prototype.toPlainDateTime
• Temporal.ZonedDateTime.prototype.toPlainYearMonth
• Temporal.ZonedDateTime.prototype.toPlainMonthDay
• Temporal.ZonedDateTime.prototype.getISOFields
• Temporal.Duration
• Temporal.Duration.from
• Temporal.Duration.compare
• Temporal.Duration.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.Duration.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.Duration.prototype.years
• Temporal.Duration.prototype.months
• Temporal.Duration.prototype.weeks
• Temporal.Duration.prototype.days
• Temporal.Duration.prototype.hours
• Temporal.Duration.prototype.minutes
• Temporal.Duration.prototype.seconds
• Temporal.Duration.prototype.milliseconds
• Temporal.Duration.prototype.microseconds
• Temporal.Duration.prototype.nanoseconds
• Temporal.Duration.prototype.sign
• Temporal.Duration.prototype.blank
• Temporal.Duration.prototype.with
• Temporal.Duration.prototype.negated
• Temporal.Duration.prototype.abs
• Temporal.Duration.prototype.add
• Temporal.Duration.prototype.subtract
• Temporal.Duration.prototype.round
• Temporal.Duration.prototype.total
• Temporal.Duration.prototype.toString
• Temporal.Duration.prototype.toJSON
• Temporal.Duration.prototype.toLocaleString
• Temporal.Duration.prototype.valueOf
• Temporal.Instant
• Temporal.Instant.from
• Temporal.Instant.fromEpochSeconds
• Temporal.Instant.fromEpochMilliseconds
• Temporal.Instant.fromEpochMicroseconds
• Temporal.Instant.fromEpochNanoseconds
• Temporal.Instant.compare
• Temporal.Instant.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.Instant.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.Instant.prototype.epochSeconds
• Temporal.Instant.prototype.epochMilliseconds
• Temporal.Instant.prototype.epochMicroseconds
• Temporal.Instant.prototype.epochNanoseconds
• Temporal.Instant.prototype.add
• Temporal.Instant.prototype.subtract
• Temporal.Instant.prototype.until
• Temporal.Instant.prototype.since
• Temporal.Instant.prototype.round
• Temporal.Instant.prototype.equals
• Temporal.Instant.prototype.toString
• Temporal.Instant.prototype.toLocaleString
• Temporal.Instant.prototype.toJSON
• Temporal.Instant.prototype.valueOf
• Temporal.Instant.prototype.toZonedDateTime
• Temporal.Instant.prototype.toZonedDateTimeISO
• Temporal.PlainYearMonth
• Temporal.PlainYearMonth.from
• Temporal.PlainYearMonth.compare
• Temporal.PlainYearMonth.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.PlainYearMonth.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainYearMonth.prototype.calendar
• Temporal.PlainYearMonth.prototype.year
• Temporal.PlainYearMonth.prototype.month
• Temporal.PlainYearMonth.prototype.monthCode
• Temporal.PlainYearMonth.prototype.daysInYear
• Temporal.PlainYearMonth.prototype.daysInMonth
• Temporal.PlainYearMonth.prototype.monthsInYear
• Temporal.PlainYearMonth.prototype.inLeapYear
• Temporal.PlainYearMonth.prototype.with
• Temporal.PlainYearMonth.prototype.add
• Temporal.PlainYearMonth.prototype.subtract
• Temporal.PlainYearMonth.prototype.until
• Temporal.PlainYearMonth.prototype.since
• Temporal.PlainYearMonth.prototype.equals
• Temporal.PlainYearMonth.prototype.toString
• Temporal.PlainYearMonth.prototype.toLocaleString
• Temporal.PlainYearMonth.prototype.toJSON
• Temporal.PlainYearMonth.prototype.valueOf
• Temporal.PlainYearMonth.prototype.toPlainDate
• Temporal.PlainYearMonth.prototype.getISOFields (reviewed @ptomato 2022-08-03)
• Temporal.PlainMonthDay
• Temporal.PlainMonthDay.from
• Temporal.PlainMonthDay.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.PlainMonthDay.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.PlainMonthDay.prototype.calendar
• Temporal.PlainMonthDay.prototype.monthCode
• Temporal.PlainMonthDay.prototype.day
• Temporal.PlainMonthDay.prototype.with
• Temporal.PlainMonthDay.prototype.equals
• Temporal.PlainMonthDay.prototype.toString
• Temporal.PlainMonthDay.prototype.toLocaleString
• Temporal.PlainMonthDay.prototype.toJSON
• Temporal.PlainMonthDay.prototype.valueOf
• Temporal.PlainMonthDay.prototype.toPlainDate
• Temporal.PlainMonthDay.prototype.getISOFields (reviewed @ptomato 2022-08-03)
• Temporal.TimeZone
• Temporal.TimeZone.from
• Temporal.TimeZone.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.TimeZone.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.TimeZone.prototype.id
• Temporal.TimeZone.prototype.getOffsetNanosecondsFor
• Temporal.TimeZone.prototype.getOffsetStringFor
• Temporal.TimeZone.prototype.getPlainDateTimeFor
• Temporal.TimeZone.prototype.getInstantFor
• Temporal.TimeZone.prototype.getPossibleInstantsFor
• Temporal.TimeZone.prototype.getNextTransition
• Temporal.TimeZone.prototype.getPreviousTransition
• Temporal.TimeZone.prototype.toString
• Temporal.TimeZone.prototype.toJSON
• Temporal.Calendar
• Temporal.Calendar.from
• Temporal.Calendar.prototype.constructor (reviewed @ptomato 2022-08-03)
• Temporal.Calendar.prototype[ @@toStringTag ] (reviewed @Ms2ger 2022-05-19)
• Temporal.Calendar.prototype.id
• Temporal.Calendar.prototype.dateFromFields
• Temporal.Calendar.prototype.yearMonthFromFields
• Temporal.Calendar.prototype.monthDayFromFields
• Temporal.Calendar.prototype.dateAdd
• Temporal.Calendar.prototype.dateUntil
• Temporal.Calendar.prototype.year
• Temporal.Calendar.prototype.month
• Temporal.Calendar.prototype.monthCode
• Temporal.Calendar.prototype.day
• Temporal.Calendar.prototype.dayOfWeek
• Temporal.Calendar.prototype.dayOfYear
• Temporal.Calendar.prototype.weekOfYear
• Temporal.Calendar.prototype.daysInWeek
• Temporal.Calendar.prototype.daysInMonth
• Temporal.Calendar.prototype.daysInYear
• Temporal.Calendar.prototype.monthsInYear
• Temporal.Calendar.prototype.inLeapYear
• Temporal.Calendar.prototype.fields
• Temporal.Calendar.prototype.mergeFields
• Temporal.Calendar.prototype.toString
• Temporal.Calendar.prototype.toJSON
• Temporal.Calendar.prototype.era
• Temporal.Calendar.prototype.eraYear
• Temporal.PlainDate.prototype.era
• Temporal.PlainDate.prototype.eraYear
• Temporal.PlainDateTime.prototype.era
• Temporal.PlainDateTime.prototype.eraYear
• Temporal.PlainYearMonth.prototype.era
• Temporal.PlainYearMonth.prototype.eraYear
• Temporal.ZonedDateTime.prototype.era
• Temporal.ZonedDateTime.prototype.eraYear"

(generated via the following script)

Array.from(document.querySelectorAll('[id^="sec-temporal."], [id^="sec-get-temporal."]'))
  .map((el) => {
    return el
      .querySelector('h1')
      .innerText
      .replace(/^[0-9\. ]*(?:get )?([^ ([]+)(\[[^\]]+])?.*/, '- [ ] $1$2');
  })
  .join('\n');

I executed that in the browser's Developer Console on the proposal's specification text.

Notes about specific things that should have test coverage:

• Behaviour of different rounding modes at a 0.5 boundary, for positive and negative values (from Test coverage missing for roundingMode: 'halfExpand' in since for negative numbers with 0.5 remainders proposal-temporal#1111)
• Reversibility of ZonedDateTime.p.since/until with DST changes involved (from Add test coverage for Temporal.ZonedDateTime.since/until proposal-temporal#1140). Given diffUntil = earlier.until(later, { largestUnit: 'days' }) and diffSince = later.since(earlier, { largestUnit: 'days' }), we need to test cases where:
  • diffUntil not equal to diffSince
  • earlier.add(diffUntil) equals later
  • later.subtract(diffUntil) does not equal earlier
  • later.subtract(diffSince) equals earlier
  • earlier.add(diffSince) does not equal later
  • Some of these tests may exist already in Temporal.ZonedDateTime.prototype.since/until, and in staging.
• Add more gregory calendar coverage, possibly via codegen, to check it does the same arithmetic as iso8601 for year > 1 (from Add more calendar: 'gregory' coverage to Test262 tests and/or demitasse tests proposal-temporal#1776)
• From Various bugfixes and making things consistent proposal-temporal#1393: Does Temporal check that retrieved methods from user-supplied objects are callable?
• From Various bugfixes and making things consistent proposal-temporal#1393: timeZone.getInstantFor throw for non-Temporal.Instant values
• What if calendar.fields() returns "calendar" as one of the fields? proposal-temporal#1047: calendar.fields return includes 'calendar'
• What if calendar.fields() returns "calendar" as one of the fields? proposal-temporal#1047: calendar.fields return includes non-calendar fields, e.g. 'nanoseconds'
• Temporal.now missing tests for user-supplied time zones proposal-temporal#1134: Time zones for Temporal.now()
• From Remove some internal TimeZone method calls proposal-temporal#1397: Observable lookups & calls for timeZone objects from other temporal methods
• Delegate review: Specify in which order properties are accessed when passing object to implementation-defined operation proposal-temporal#1388: Calendar order of operations (with and)? without Intl
• From Correct + make runnable documentation code samples. proposal-temporal#1396: PlainMonthDay.prototype.toPlainDate correctly handle [['year', undefined]] for input fields
• From Consume iterables where possible proposal-temporal#1489: Test CalendarFields calls, including with fields added, and getPossibleInstantsFor type checks
• Check fast-paths from delegate review: why are "Temporal-like" objects supported at all? proposal-temporal#1428:
  • PlainDateTime -> PlainTime
  • ZonedDateTime -> TimeZone
  • PlainTime -> Calendar
  • PlainDate -> relativeTo object
  • Callers of GetOptionalTemporalCalendar and ToTemporalMonthDay, with a not-previously-covered Temporal object (e.g. ToTemporalDate with some object like Object.assign(new Temporal.MonthDay(1, 1), { year: 2000 }))
  • ZonedDateTime -> PlainDate, PlainDateTime, PlainTime
• Check that with() methods don't call getters on Temporal objects
• Check all remaining callers of GetPossibleInstantsFor with checkTimeZonePossibleInstantsIterable
• Check that things of the wrong type in an iterable (calendar.fields return, calendar.fields input, timeZone.getPossibleInstantsFor return) throw before exhausting the iterator
• Check that calendar.fields is always called with the correct list of fields
• Check that Gets occur for every extra field returned by calendar.fields
• Check that toString is not called in CalendarEquals, TimeZoneEquals, ConsolidateCalendars if the two calendars are the same object (Some Calendar questions proposal-temporal#1425 point 2)
• Check that any calendar/timezone method that receives an internally-created options object, doesn't cause lookups on Object.prototype (Editor review: Kevin Gibbons proposal-temporal#1424)
• Check that property accesses on null-prototype objects created internally in Temporal don't cause lookups on Object.prototype (https://github.com/tc39/proposal-temporal/pull/1875/files#r748524136)
• Add tests for ParseTemporalDurationString overflowing one of the fields to infinity (e.g., PT99999....99999Y)
• Add tests for BalanceDuration causing a Duration field to overflow to infinity and throwing:
  • AddDuration (3x instances)
  • PlainTime.until()/since()
  • PlainDateTime.until()/since()
• Add tests for CreateDurationRecord / CreateTemporalDuration combining mixed signs or overflowing and throwing:
  • AddDateTime
  • new Temporal.Duration()
  • Duration.from()
  • Duration.with()
  • RoundDuration (3rd instance)
• Verify that there is no possible test case that would cause CreateDurationRecord / CreateTemporalDuration to combine values from different sources having different signs:
  • AdjustRoundedDurationDays (last instance)
  • AddDuration (2nd instance)
  • ZonedDateTime.until()/since()
• Verify that there is no possible test case that would cause CreateDurationRecord / CreateTemporalDuration to overflow to infinity:
  • DifferenceZonedDateTime (last instance)
  • RoundDuration
• Check that all (optionName)-undefined.js tests also test plain function objects
• Figure out a sequence of custom calendar and time zone return values that exercise the checks in Normative: Validation of NanosecondsToDays and getOffsetNanoseconds proposal-temporal#2387
• Run some DST tests against every time zone (from DST testing across many timezones? proposal-temporal#556)
• Check that a calendar annotation with an ISO date (11111111 or 1111-11-11) as the calendar name throws, and doesn't default to the ISO calendar (from Missing calendar validation in ToRelativeTemporalObject and ToTemporalZonedDateTime proposal-temporal#2546 (comment))
• Check that methods are looked up on calendar and time zone instances even if they are pristine instances of builtin calendars (this can't be tested directly but can be observed e.g. with array iteration on Calendarfields)
• Check when GetOptionsObject is performed relative to the other user-observable operations (currently this is not included in the order-of-operations.js tests) - see AddDurationToOrSubtractDurationFromPlainYearMonth: Move GetOptionsObject closer to the top proposal-temporal#2721

[jugglinmike]

@ptomato @Ms2ger I know this work is near and dear to your hearts :)

[ptomato]

Thanks for getting this started! FWIW, I've been finding it easier to write tests by topic, rather than by API entrypoint — for example, here are a series of largely identical tests for everything that uses the GetOption abstract operation: https://github.com/tc39/proposal-temporal/pull/1531/files

One thing I'm planning to do is convert the Mocha-style tests of the research polyfill in that repo to test262-style tests, and delete the old-style tests as I go. So anyone picking one of these items can use those old-style tests as a starting point.

[Ms2ger]

Thanks, @jugglinmike! I think that for the moment it would be easiest to keep developing the tests in the proposal repo, so we can run them against the polyfill as a sanity check. Does that make sense to you?

[ljharb]

Hopefully the polyfill will be finished and deprecated soon tho, so the advantage to keeping the tests there shouldn’t last long?

[jugglinmike]

@Ms2ger I've had luck using Node.js and the test262-harness project to run Test262 against the polyfill:

$ test262-harness \
    --hostType node \
    --hostPath node \
    --prelude inject-temporal.js \
    'test/built-ins/Temporal/**/*'

Where inject-temporal.js looks like this:

const { Temporal } = require('./node_modules/proposal-temporal');

I'm using the version published to npm, but it ought to work for the proposal's repository, too. Does that help at all?

[ptomato]

@jugglinmike For one thing, we're planning to deprecate that NPM module... An advantage to keeping it in the proposal repository is that we have code coverage metrics for the test262 tests there, and it doesn't require synchronization between git repositories.

(And I can't remember the details off the top of my head, but using require() in the prelude breaks the code coverage metrics, so we generate an IIFE bundle and include that as the prelude.)

[FrankYFTang]

#3048

[ptomato]

I left one comment in #3049 but it could simplify things in all of these pull requests:

Consider porting over the Temporal asserts from the proposal repo to temporalHelpers.js and using those instead of assert.sameValue("...", ....toJSON()). They should give better error messages and be more specific.

[linusg]

@jugglinmike I noticed that all the getters are missing from this list - here's a change to the code snipped you posted to include them in the list:

-Array.from(document.querySelectorAll('[id^="sec-temporal."]'))
+Array.from(document.querySelectorAll('[id^="sec-temporal."], [id^="sec-get-temporal."]'))
   .map((el) => {
     return el
       .querySelector('h1')
       .innerText
-      .replace(/^[0-9\. ]*([^ ([]+)(\[[^\]]+])?.*/, '- [ ] $1$2');
+      .replace(/^[0-9\. ]*(?:get )?([^ ([]+)(\[[^\]]+])?.*/, '- [ ] $1$2');
   })
   .join('\n');

You also might want to update now => Now :)

[Ms2ger]

It's clear that there's no full coverage yet, but it's not clear to me how it's helpful to do the work to review the coverage outside this repo. On the other hand, having them in this repo would be very helpful for the people who are implementing the proposal as we speak.

[rwaldron]

On the other hand, having them in this repo would be very helpful for the people who are implementing the proposal as we speak.

This is actually really compelling. Which engines are currently implementing?

[ptomato]

Hi @rwaldron! I think my concern is actually the opposite — I don't want two sets of tests, only one! We would like to delete the tests from tc39/proposal-temporal as soon as they are present in tc39/test262, and pretty much have a setup prepared just like the one you outlined.

I'm just concerned about the two sets of tests getting out of sync during review, if the PR would (understandably, because large) take a long time to merge, because that would cause a lot of extra work for me. That's why I asked if it could be merged swiftly and stylistic points cleaned up later.

I get your point about coverage, but maybe that could be mitigated using the checklist in the OP of this issue? As we check off each item in the checklist, we could consider the coverage for that item audited. The difference would be that someone working on a checklist item would start from incomplete coverage instead of from zero, which has disadvantages as well as advantages.

The proposal is currently being implemented by V8, SpiderMonkey, and JSC. One other implementation that I'm aware of is in SerenityOS's engine.

[jugglinmike]

Hey there, @ptomato. Let's do it!

@rwaldron and I talked this through today. Although we'd prefer to be more familiar with the test material before merging it, we think the opportunity to assist in-progress implementations, coupled with the correctness (if not completeness) of the tests merged so far, warrant expedition.

[ptomato]

Thanks, that's great news! I am on vacation tomorrow, so will make this pull request as soon as possible on Monday. If you would like to plan a Matrix chat session or video call next week to go through the test material so I can help you get that familiarity, I would certainly be available for that.

[jugglinmike]

Thanks, Philip, that's very kind!

Regarding coordination: I personally won't be available for reviews until Thursday of next week (that is, September 30). @rwaldron should be around on Monday, but we'll collectively have more bandwidth at the end of the week. You can send a pull request whenever you like, of course--I just know you're interested in limiting the review period.

[ptomato]

OK, thanks! In that case I won't prioritize the pull request today, but I'll make sure to send it by Wednesday. Does that sound good to you?

[jugglinmike]

Works for me!

[justingrant]

Are the checkboxes in the OP of this thread still accurate? Or have some additional methods been fully covered by Temporal's 3000+ tests?

[Ms2ger]

I don't think anyone has reviewed the coverage we gained by landing the polyfill tests - I'll see if someone on our side can do it

[HKalbasi]

Will there be tests for each different calendar? Calendars are not symmetric like timezones and it's easy to mistake in only one of them.

[Ms2ger]

We are currently focusing on the ISO calendar, so deep testing of the actual calendar operations is likely a while off. I imagine we would take tests if someone else had time to write them, though.

[ptomato]

I'm wondering what the policy on these should be, as well. Tests for time zones can be difficult because time zones can be adjusted by civil authorities. If that happened, tests could start failing whereas that would not indicate any standards noncompliance, just outdated time zone data.

Calendars are somewhat less subject to change, but still, I do know of a few cases that could cause tests to be invalidated. The Hijri calendar may get correction days depending on astronomical observations (see heading "Difficult to Predict" of this document) and any calendar that uses eras may decide to create a new era in the future.

[HKalbasi]

I feel lacking tests for time zones is fine since time zones are pure data, and implementation can be auto generated from IANA database. but calendars need some code as well, so at least there should be some tests for different classes of calendars. But with this logic, something like era doesn't need test since it is data-ish. Anyway this is not Temporal specific and DateTimeFormat.formatToParts also doesn't test for different calendars, so maybe not having tests for calendars is intended? The other case I can imagine is that formatToParts is implementation defined and can not be tested here (I'm not familiar with formatToParts spec, just know that it is the current way to get calendar data from browser)

I imagine we would take tests if someone else had time to write them, though.

I'm volunteer to write some tests if policy becomes that we need tests. Also I need specification of calendars for writing tests which I didn't find them in ecma-402. Where does it exists?

[ptomato]

@HKalbasi Sorry, I missed your comment back in February. There isn't any specification of calendar arithmetic because it's expected that ECMA-402 delegates it to a library like ICU4C or ICU4X as they do with locale-specific formatting. So, if you write tests for calendar arithmetic I'd recommend doing it on a best-effort / representative-sample basis.

[FrankYFTang]

Anyway this is not Temporal specific and DateTimeFormat.formatToParts also doesn't test for different calendars, so maybe not having tests for calendars is intended? The other case I can imagine is that formatToParts is implementation defined and can not be tested here (I'm not familiar with formatToParts spec, just know that it is the current way to get calendar data from browser)

There are a lot of error throwing behavior need to be tested in DateTimeFormat.formatToParts with the Temporal proposal.

for example step 6-a of
15.3.13 HandleDateTimeTemporalDate ( dateTimeFormat, temporalDate )
step 4-a of
15.3.14 HandleDateTimeTemporalYearMonth ( dateTimeFormat, temporalYearMonth )
and
15.3.15 HandleDateTimeTemporalMonthDay ( dateTimeFormat, temporalMonthDay )
15.3.17 HandleDateTimeTemporalDateTime ( dateTimeFormat, dateTime )
15.3.19 HandleDateTimeTemporalZonedDateTime ( dateTimeFormat, zonedDateTime )

and step 6-a of
15.3.19 HandleDateTimeTemporalZonedDateTime ( dateTimeFormat, zonedDateTime )

also Step 5-a of
15.3.7 PartitionDateTimeRangePattern ( dateTimeFormat, x, y )
but that is for formatRange and formatRangeToParts