Forthcoming changes in TOML v1.1.0 will cause TOMLDecodeError. tomllib.loads doesn't support optional seconds in times.

[JamesParrott]

[Edit] I erroneously assumed the "minimal configuration file format", that I personally find to be a pleasant alternative to JSON, was done and dusted. But TOML is still very much under active development. On its main branch.

Python 3.12 is compliant with TOML v1.0.0 as far as I'm aware. But optional seconds, and other issues will arise if TOML v1.1.0 files are attempted to be loaded with tomllib. This is absolutely not a request to support TOML v1.1.0.

This issue is an informational advanced warning only, of changes coming down the line. Feel free to decide not to support them all (TOML v1.1 is completely unnecessary for my projects, and I won't be supporting it). But if so, it's still good to make a decision, and record that somewhere. If nothing else so it is clear what should be included in test data.

Background: In summer last year (2022) the TOML language's latest version relaxed the requirement for seconds to be specified in a time: toml-lang/toml@7e8a748

A test for optional seconds was also added to toml-test's development version: https://github.com/BurntSushi/toml-test/blob/master/tests/valid/datetime/no-seconds.toml

Implications: In Python 3.12, tomllib requires seconds to be present. Currently in the absence of seconds, the regex _TIME_RE_STR does not match. Otherwise both local variables called sec_str could not be defined. So if others decide to support TOML v1.1, this will affects all platforms.

Regex:

cpython/Lib/tomllib/_re.py

Line 17 in fbf3596

_TIME_RE_STR = r"([01][0-9]|2[0-3]):([0-5][0-9]):([0-5][0-9])(?:\.([0-9]{1,6})[0-9]*)?"

sec_str:

cpython/Lib/tomllib/_re.py

Line 74 in fbf3596

hour, minute, sec = int(hour_str), int(minute_str), int(sec_str)

sec_str:

cpython/Lib/tomllib/_re.py

Line 101 in fbf3596

return time(int(hour_str), int(minute_str), int(sec_str), micros)

Bug report

Python 3.12.0a7 (tags/v3.12.0a7:b861ba4, Apr  4 2023, 16:33:41) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tomllib
>>> tomllib.loads('t=12:00:00\n')
{'t': datetime.time(12, 0)}
>>> tomllib.loads('t=12:00\n')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python312\Lib\tomllib\_parser.py", line 127, in loads
    raise suffixed_err(
tomllib.TOMLDecodeError: Expected newline or end of document after a statement (at line 1, column 5)
>>>

[AlexWaygood]

Cc. @hauntsaninja / @hukkin for tomllib

[JamesParrott]

It's mentioned on the TOML github: https://github.com/toml-lang/toml/blob/main/toml.md#user-content-local-time
but not currently on the spec on the toml website: https://toml.io/en/v1.0.0#local-time

It looks to me like the decision to allow optional seconds was taken, but could have been better communicated.

[hauntsaninja]

Thanks for bringing this up!

hukkin has done an excellent job staying on top of changes and has a PR for this specific change ready to go: https://github.com/hukkin/tomli/pull/203/files

PEP 680 specifies we can treat minor revisions to TOML as bug fixes, so when v1.1 is released we should have the option to backport it if we think that's wise: https://peps.python.org/pep-0680/#stability-of-toml Currently defaulting to avoid shipping changes until v1.1 is released: toml-lang/toml#928

cc @pradyunsg in case you have opinions for how the release cycle of TOML and CPython should intersect :-)

[JamesParrott]

You're welcome. Great job hukkin!

I was just very surprised earlier today to see a Python core library 'fail' a toml-test. But I'd got my wires crossed, and didn't realise I was using cutting edge tests, for features ahead of Toml 1.0.0

[pradyunsg]

cc @pradyunsg in case you have opinions for how the release cycle of TOML and CPython should intersect :-)

We should be able to pull it in the next minor version of Python, once "upstream" (AKA me) cuts TOML v1.1.0.

I don't think we should try backporting to an older minor version in a patch release, but anything stated in the PEP takes precedence over my opinions. :)