-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Datetimes. The draft ISO 8601-201x and EDTF. #656
Comments
The following are my recommendations about:
I don't necessarily feel strongly about all of my recommendations. That is, I'd be very much open to be persuaded differently. Especially since creative solutions might provide a way out of problems. So I provide the recommendations as something, at least, to push against. Biblatex and EdtfIso201xLevelsRecommendation:
Biblatex formerly supported EdtfLoc level 1 and level 0 (with a few extras). Given the rough correspondence between the levels between EdtfLoc and EdtfIso201x, and that both specs seem only to support rare, or very rare, edge cases at level 2: probably coding time could be saved in the biblatex context by not bothering with EdtfIso201x level 2. It is pleasing that the EdtfIso201x profile should attempt an exhaustive (and novel) handling of possibilities at level 2. If need to support the relevant rare edges cases arose, then EdtfIso201x looks well built to provide the relevant stipulation to implementers. EdtfIso201xUnknown and open rangesRecommendation:
The designators used for unknown and open in EdtfLoc, the strings "unknown" and "open", have an obvious problem: they are English words. That's not a good thing for internationalization reasons. So it is right that EdtfIso201x should address this by having, instead, characters (counting blank as a character). In EdtfIso201x the double dot ".." for open seems OK. Although it risks looking unintentional at the end of a sentence 2006/... Given, that is, that the end of a sentence might be marked by ellipses ... So for those reasons I think a double dot ".." should be rejected. Blank for unknown seems like an odd choice conceptually. Biblatex historically uses blank not for unknown but for open. And that seems right conceptually. Blank simply seems better suited to indicate what open means: "some date that shouldn't be fixed". So I recommend blank for open dates. In making this recommendation I give no weight to the convenience of preserving biblatex backward compatibility. An argument against this recommendation, or the use of blank in any date context, is that blank might be confused for a value that has not yet been entered. That is, that the author of "/1572" has not yet asserted anything about the start date (neither that it is open nor unknown). If that argument has purchase then it speaks in favour have having explicit (non blank) characters for any kind of value (an explicit symbol for both open and unknown dates). But assuming blank should be for open what, then, what should be the symbol for one of the dates in a range that is unknown? It can't be "?" on its own, as in "2006/?", for "?" is already used to designate the that whole term is "uncertain". I take it that in this bibliographic context "uncertain" and "unknown" are conceptually distinct. "Uncertain", expressed as the question mark in "1587?", meaning "date whose source is considered dubious" (See "3 Terms and definitions", ISO DIS 8601-201x Part 2, p6). But an "unknown" date seems fitted to identify some date you wouldn't hazard a guess at, the "unknown" in "1783/unknown". So for those reasons the question mark "?" should be preserved for uncertain and not be used for unknown (whether alone or in combination with other characters). What about, for unknown, the asterisk "*", as formerly supported by biblatex? For me the connotations of "*" come from SQL where it means: all possible values. That's a little bit jarring because the asterisk "*" in "2006/*" doesn't mean "all years" but "all years greater than 2006". However, that's no large objection. One could come to understand that, in an EDTF context, the asterisk "*" takes on a slightly different meaning. However I think I'd rather another character that didn't carry those connotations. Perhaps the caret "^": e.g. "^/1982"; "2006/^". I think that lends itself more readily to a stipulated connotation of being a pointer to a series dates either above or bellow the date that is specified. So, for the sake of presenting alternatives, that's my recommendation. CenturyRecommendation:
EdtfIso201x's provision of a two digit number to represent a century ...
... is superfluous if an implementation (like biblatex) supports EdtfIso201x to level 1. For at EdtfIso201x level 1 the unspecified "A year with one or two (rightmost) unspecified digits." is supported ("4.3 Unspecified", ISO DIS 8601-201x Part 2, p10). That is, "19XX" could be used to represent to "1900/1999". Even more significantly, even if an implementation only supported EdtfIso201x to level 0, then a century can be represented by the EdtfIso201x level 0 supported time interval "1900/1999". Any spec should be made simpler if it can. So I think removing support for centuries as two digits ought be done to make the spec simpler, given that centuries can be expressed at EdtfIso201x:
Reduced time precisionRecommendation:
Times with minute precision are common; and forcing such a time to go from "2017-10-28T04:47" to "2017-10-28T04:47:00" misrepresents the time. It remains possible that the Edtfloc writers didn't intend to forbid reduced precision times. If their intention was to forbid they could have, and should have, written in the spec something like "omitting seconds in a time is forbidden". And so it remains possible that EdtfIso201x has simply inherited this oversight. Iso201xMove EDTF to its own part 3?Recommendation:
As mentioned in the post above: this would make referring to sections in part 2 that aren't part of EDTF, conceptually easier. BookmarksRecommendation:
... for the love of easy navigation. IsoOrgGithubRecommendation:
The github issue tracker is a good one. And, moreover, provides an easy route to participation. No costRecommendation:
It is an absurd state of affairs that any standard, being a thing the we want to be adopted as widely as possible, should have a barrier to adoption like cost. Edit: "supports" to "supported"; added commas to fix grammar. Edit: 2017-12-28 Grouped some recommendations under the "IsoOrg" term. |
I have no strong opinions on the matter other than that:
|
Note to @inukshuk (the EDTF.js author). |
I have just released biblatex/biber 3.9/2.9 to correct some bugs but these do not have the ISO changes. I did put in a deprecation notice in the changes file to warn people what is coming in terms of syntax changes, which are the only things users really need to care about. |
@plk thanks, that better gives us breathing room given the state of flux/uncertainty. @retorquere all your reflections seem reasonable. On ...
... while I think we can be confident about some of the contexts where EDTF is likely to be applied, I don't we can't be confident all of the contexts where these sort of standards might end up being applied. And the EDTF standard really ought be built to be context robust, as far as we can anticipate. EDTF datetimes might end up contexts where they are parsed or just presented. They might end up in urls, epub documents; and future protocols and tools not yet conceived. In the specific context of presented prose it seems entirely plausible (although extremely rare given open date ranges are extremely rare) someone might write ...
And so I think we need to test our candidate symbols by imagining as many plausible contexts as we can, and ask "Will that symbol break in that context?". Would a caret "^", or a double dot ".." cause parsing difficulties when using regex to parse? And, does a double dot ".." look ok in prose? etc,. I don't think any of the above is likely to be a novel thought for you (nor anyone else likely to read this far in the thread): it's just what leaks out of my head in response to your question. :) At all: I've made an email request via my national standards body to join the relevant ISO committee (i.e. to get access to the relevant online forum). |
To join up to the relevant ISO committee I emailed the committee secretary directly. I have received no response from them. But there has been some progress in my joining up via my national standards body (which seems to be the prescribed method). But I'll be waiting on a further response from that body not before 2017-12-07. Edit: "2017-11-07" to ""2017-12-07" (whoops). |
So I see from #665 you've (@plk and @moewew) decided to release changes to conform to EdtfIso201x fairly soon, on the basis that:
All that seems reasonable. Perhaps @plk you could let us know in this thread when EdtfIso201x features are merged to the master (and released). Edit: "ratification"; "[to]". |
We're hoping to release the next version of |
Thanks @moewew. Great. I'm still progressing through the ISO committee joining up process. |
@JohnLukeBentley Any news here? The LOC has removed its link to the draft standard at the request of the ISO (https://www.loc.gov/standards/datetime/). And apparently the ISO 8601 draft is near the end of the enquiry phase, judging from how long things took the last time it will take another six months until the norm is published. And then of course we need to lay our hands on the standard. My university library does not have a subscription for standards any more. |
@moewew (and all) I thought I replied to this minutes after your post: it probably got lost in a browser mishap at my end ... Read my post first at inukshuk/edtf.js#12 (comment). That thread in general also demonstrates: that others in the ecosystem are depending on the standard; and it demonstrates an issue that requires a change in the standard (the specific case is to make it clear whether seasons in intervals are permitted ... if not to accept seasons in intervals). I haven't heard back from my national standards body since that application I made 10 days ago (but I took some weeks to send in the application; and the national standards body was quite welcoming towards my making the application). As noted in that other post of mine one of the first things I've intended to do, on the assumption that I'd be accepted, is suggest that standards process be opened up. E.g., By putting the standard on github (or an equivalent open platform). But ISO ...
... has made an obnoxious move. Evidently EDTF was created as an open standard, then apparently given to ISO to be the shepherds of. ISO has turned around to the EDTF community and denied them (and therefore all) access to the evolved standard. It's like giving your cousin land by the creek only for your cousin to kick you off as a trespasser when you wander down for a swim. Given that obnoxious move I'm tempted to suggest that we, all parties in the relevant ecosystem who have an interest in it, create an open EDTF datetime standard (which would probably have to march under a new name). Against that suggestion is:
So the issue is: as weighty as those reasons are, might they nevertheless be overridden by the odiousness of a closed standards development process and that ISO standards are not made available made at zero cost, once published? I mean, by comparison, I found something (trivially) wrong with the W3C HTML 5.2 standard. I posted the error on github: w3c/html#1119 (comment). Three days later it was all fixed. There was no need to submit detailed applications to a standards body (detailing relevancy to stakeholders, importance for the nation, the industries that will benefit from the standard, etc.) simply to be allowed to express the wrong. And of, course, at zero cost we can read the W3C HTML 5.2 standard via a simple link https://www.w3.org/TR/html52. (A similar real world example happened when I found a trivial error with the WHATWG standard: I'm not, here, favouring one side in the W3C V WHATWG split). Edit: Added "(and all)" ... as all posts on github are ... the half-baked suggestion is very much to all. Edit: Note to @njbart, @inukshuk, @retorquere, Edit: "who's" to "whose". |
“Obnoxious move” quite nails it – what the **** were these people thinking? Evading scrutiny by hiding a document most of us have downloaded long ago anyway? Still, I don’t think creating our own, independent standard is a realistic project. But what we could do is to create and maintain an ISO8601 profile. Quoting from ISO/DIS 8601-2:2016(e) (2016-10-26), p. 27 (my emph.):
According to p. 26, a profile may specify the following:
Unless we feel that cases such as clarifying the validity of season intervals are already covered by 3., we should of course reserve our right to amend the profile accordingly. What would be needed is a name – possibly “ISO 8601 – biblatex profile” would do – and a clear idea of which features to permit – possibly simply the date/time expressions currently valid in biblatex. Ideally, functions for dealing with ISO 8601 profiles could be added to parsers such as edtf.js, so they would not only be able to parse specific ISO 8601 levels (“1”, “2”, or the [experimental] “3” in the case of edtf.js), but also specific ISO 8601 profiles. |
That lateral and clever suggestion is very helpful. It probably avoids two problems:
So, from the parts you quoted, it seems perfectly consistent with ISO 8601:201X for the profile to be open. I'm a bit unclear on what you mean by ...
Of course season intervals was a mere example to make your point. But, on that example, I do think season intervals is ambiguous in ISO/DIS 8601-2:2016(e) (2016-10-26) and therefore, in a profile, be clarified under 3. But by "we should of course reserve our right to amend the profile accordingly" I suppose that could mean, whatever your intentions, either:
On my reading of ISO/DIS 8601-2:2016(e) (2016-10-26), essentially from the parts you quoted, an ISO 8601 profile can clarify and define a subset of ISO 8601. So, for example, we could clarify that seasons are permitted in intervals; and drop support for two digit centuries. But, on my reading, an ISO 8601 profile can't extend ISO 8601 (Part 2 is already an extension to which a profile could refer): for that would make it cease to be an ISO 8601 profile ... and become an independent standard. For example, ISO/DIS 8601:2016(e) (2016-10-26), part1 or 2, doesn't appear to support quarters, e.g. Also it doesn't seem that an ISO 8601 profile could contradict ISO 8601. E.g. By changing the "uncertain and approximate" symbol from Let's assume that's the case, that under an ISO 8601 profile extensions and contradictions of that sort are not permitted. Does that necessarily cause us problems? Let's test it against the suggestions on the table. We have, from you:
From @plk:
From me, there was originally only three relevant suggestions:
And from me lately:
Taking these in order from easiest to hardest:
In short it seems we could get away with creating an ISO profile, if that requires merely clarifying or removing features in the larger ISO 8601. Alternatively we could create an "ISO 8601 profile-but". That is, an ISO profile but for a few extensions (e.g. quarters) or contradictions (e.g. a chosen symbol for "uncertain and approximate"). In other words, bite the bullet and create what is, in effect, an independent standard - although for the most part it looks like an ISO 8601 profile. On a name for either kind of profile I think we'd want it to be broadly useful, not just for biblatex. From ISO/DIS 8601-2:2016(e) (2016-10-26), p28 ...
Our profile ought have the same usefulness, I'd suggest: to the bibliographic community and any other community that would find it useful. For what's being suggested here is a minor tweaking to EDTF (in either its old or new form). We'd also want to invite the EDTF crowd from http://www.loc.gov/standards/datetime/pre-submission.html. I hold off any suggesting any specific name until we know what kind of profile we'd want. Of course, there remains the third alternative: just suffer under the limits and ambiguities of EdtfIso201x. By the way looking at https://www.iso.org/obp/ui#iso:std:iso:8601:-2:dis:ed-1:v1:en (see "Preview our standards" on https://www.iso.org/iso-8601-date-and-time-format.html) we can see that the "Reference number" remains as ISO/DIS 8601:2016(e) although the document is laid out differently to ISO/DIS 8601:2016(e) (2016-10-26). Presumably this latest ISO/DIS 8601:2016(e) is substantially identical to ISO/DIS 8601:2016(e) (2016-10-26). |
My point is, even if ISO unambiguously stated they consider season/season intervals, or minute level precision to be invalid according to the norm, I’d still want them to be considered valid according to the biblatex (or whatever) profile. Technically, this would be a contradiction, but pragmatically I’d just do what @inukshuk did and declare that “Seasons in intervals are supported at the experimental/non-standard level 3” (see https://github.com/inukshuk/edtf.js). (Worst case is, we’d have to call it “The profile formely known as ISO 8601/biblatex” …) As to quarters, ISO/DIS 8601-2:2016(e) (2016-10-26), p. 14, “4.7 Divisions of a year”, lists “33-36 = Quarter 1, Quarter 2, Quarter 3, Quarter 4 (3 months each)” as a level 2 feature – apart from your |
On the profile scope it does seems you want what I described as the second alternative ...
I do like it that this is what you are going into bat for. You are probably right that we don't want to abandon the right to extend or contradict ISO 8601 - that's probably unnecessarily tying our own hands. Presumably if we extend or contradict ISO 8601 then ISO can't complain about copyright violation. Because to the extent it is a profile this is allowed to be openly published and to the extent that we express something new, a contradiction or extension, ... well you don't violate the copyright of an author for expressions they didn't make.
... yes, or something like: "IS0 8601 Bibliographic [or whatever] Datetime Profile With Extensions And Contradictions". That might make it an "independent standard" after all. However, adjudicating between competing abstract natures, "independent standard" or "ISO 8601 Profile-Like-Thing", might be a bit arbitrary and for that reason unnecessary. It's probably more important to get clear on whether we'd want it be ISO 8601 profile conformant (we don't necessarily) or largely 8601 profile conformant with room for extensions and contradictions (the current position is that we (you and I at least) do want this). On the incidental matter of quarters, thanks for pointing to it in ISO/DIS 8601-2:2016(e) (2016-10-26). I simply missed it. Tony Benedetti has a very worthy suggestion with regard to divisions of a year DATETIME@LISTSERV.LOC.GOV > "Division of Year codes", Thu, 4 Jan 2018 20:01:04 -0500. A mnemonic scheme that, even though it would reject the traditional This is worth mentioning here as evidence of continuing ideas about the EDTF standard (whichever one we have in mind). Moreover ideas that contradict the existing EDTF standard (whichever one we have in mind). It does seem right that if we created an open standard we'd want to do so to allowed that standard to openly evolve, to be open to evaluate plausibly good ideas like Tony's, in a way that could very well extend or contradict ISO 8601. |
@plk (biblatex datetime implementer) and @inukshuk (edtf.js implementer upon which zotero-better-bibtex depends) what say you ... Would you, in principle, be interested in supporting an "Bibliographic [or some more general identifier] Datetimes: An IS0 8601 Profile with Extensions And Contradictions"? That is:
Those examples, of course, are mere examples. I don't mean to open debate on them here. To save you reading the above thread, the ISO process just seems too odiously closed. So the motivation for creating our own standard, which nevertheless attempts to track parts of ISO 8601-201x as closely as possible, is allow for an open development and publication. Namely develop the standard on Github, thereby enabling anybody to raise issues about it, and make suggestions for its improvement. And allow anybody to read the standard, without cost. For the moment I think we ought be, at least I am, agnostic on:
If you both approved I'd then make invitations to the EDTF folk about what they think of such a standard. Approval from both of you I think would be necessary, but sufficient, for we who are interested to proceed. Note also to @njbart, @moewew (thanks for the stack overflow answers in recent days), @retorquere. Any further thoughts from you fine folk would be most welcome. |
I think that this is a good idea and I support it. |
@plk: awesome. |
I'll be happy to pitch in where I can, but you guys know a lot more than I do about this. I'm just a consumer of the standard, and I usually lean on @njbart to assess whether what I'm doing with it makes much sense. |
@retorquere, great. I'll note you'll still be busy with the start of your PhD, among other things in life. So I'm essentially wanting to keep you in the loop rather than place further time burdens on you. And am I right to understand you essentially rely (at the moment) on whatever EDTF.js permits? That is, with respect to all those test datetime formats that I threw at you, that essentially entailed checking that zotero-better-bibtex would delegate to EDTF.js properly? |
I'd be interested in joining an effort to develop a public open-source profile of ISO 8601. For a v1.0 release, we'd obviously need to wait for ISO to finally release the two parts of its date and time notation standard. Meanwhile, work could be done under the umbrella of https://standards.github.io. Full disclosure: I've been exploring the possibilities of extensions to ISO 8601 notation for years, resulting in the International Calendar. Much of this would be beyond the scope of a proper profile of ISO 8601, but it may be in scope of a community standard for date and time notation. |
@Crissov, great! To be clear a "ISO 8601 profile with Extension and Contradictions" would not be a "ISO 8601 profile" as such. For to be an ISO 8601 profile, the nature of such a profile is defined ISO 8601:201x, you couldn't have extensions or contradictions of ISO 8601:201x. That confusion might bear on the name of our standard. So that you have been interested in extensions "beyond the scope of a proper profile of ISO 8601" means your aims are shared here. Naturally there would be a future discussion around the scope of our standard: whether we want to keep to small, but large enough to cope with bibliographic, or other, needs V be permissive about what extensions are permitted. At the moment I'd be on the small side of the fence, in keeping with the heritage from EDTF.
For a v1.0 release, yes. But the ISO draft already in the wild looks to be near completion. I think it unlikely any substantive changes will be made before release. And if there are changes, whether substantive or minor, on release they could be readily incorporated. So, as you suggest, I don't think we need to wait for ISO release to begin developing our standard. Thanks very much for that! I was unaware of its existence. It looks like a new thing. Very timely. At the moment it essentially looks like a meta standard (there is https://standards.github.io/explore/meta/) for using github for standards. I'll have to give that a proper look later! Good to learn there's another standard around which a great deal of work and thought has been already done. |
@JohnLukeBentley thanks for understanding. I'm stretched pretty thin these days. BBT uses EDTF.js as one of the ways to detect and parse dates, but it's not the only thing I do. BBT gets offered quite a bit of input that is easily recognizable by humans as dates but are pretty gnarly to parse. To parse dates:
step 2 (with EDTF.js) is important because it picks up the bulk of the parsing work, but the other 3 do substantial work nonetheless. It's not perfect, and there are known problems (such as that you can't specify the EDTF range 11/2010, just 0011/2010), but it gets the job done most of the time. |
EDTF.js was written primarily for its use in Tropy and (hopefully / eventually) Zotero; it was picked mostly because CSL / citeproc was leaning towards adopting EDTF some time ago. While the parser is mostly complete, we'll be improving locale-aware formatting and want to design a dedicated input widget (again, those needs arise mostly from its use in Tropy, but would extend to Zotero as well). Having said all that, I'm happy to support the format which will be most relevant for users of Tropy, Zotero and CSL-aware tools in general. As I'm not too keen on standards proliferation, I still hope that the upcoming ISO standard will manage to incorporate all your requirements, but if that's not possible I'm happy to make the necessary adjustments to EDTF.js. BibLaTeX, Tropy, Zotero, CSL – this is pretty much the target audience of the EDTF.js parser. |
Assuming tropy is an electron app, the widget would become relevant to zotero after their port to electron. It's currently Firefox (xul) based. That, and users entering date information manually isn't the primary way (in my experience) that date information gets into zotero; much is scraped from websites, and that date info is usually not EDTF. They have a hacky heuristic parser just like mine. |
Obligatory XKCD: https://xkcd.com/927/ Setting up a new standard means a lot of work. At least that is what I imagine - and it takes a lot of time if done properly. If that is done, having a standard that no one adopts it is hardly of any use. Even if I'm also worried about feature creep, but that might just be me. As soon as someone insists on being able to input their time zone not as UTC offset, but using their favourite zone abbreviation all hell will break loose. I assume that the ISO people have a good think about what would be 'too much'. If the new standard is not supposed to be an ISO profile it would probably need to distance itself from ISO to avoid infringing upon their rights (whatever that would mean in practice). |
If this community standard was a small superset of a big subset of ISO 8601, the goal would probably be that it could and would become a proper profile of the international standard after its next revision in a couple of years. For instance, ISO probably won’t add established quarter notation |
(for the same reasons as I outlined above, locale-aware won't help in many use-cases; in western philosophy eg it is quite common to import French or German (textual) dates even if my locale is set to English. One of the major bonuses of using a tool like Zotero is that it automates a lot of crap-cleaning when scraping references, but it doesn't touch dates, probably to avoid unintentional data loss on misparse. Date and name parsing... Oy) |
I've created a new thread "Should we create an Open Datetime Format?" in a new repository, open-datetime-standard-bootstrap. I'll link to that thread below. To segue there ... @inukshuk wrote:
Terrific @inukshuk, thanks! @inukshuk wrote:
At this stage, that's unlikely to happen. I've had no response back from my national standards body, Standards Australia, about my application to join the relevant ISO technical committee. Standards Australia have been most welcoming and encouraging during the application process. So I'm making the assumption that my email ended up in a spam trap, was missed in the inbox, or some such innocent explanation. However, as mentioned earlier in this thread (which I'm not expecting you to have read) ISO pulled the pdf copy of their draft standard from the EDTF Library of Congress site (http://www.loc.gov/standards/datetime/). That move by ISO doesn't express a move away from developing standards under a closed regime. So that's become enough for me to not chase up my application to join ISO. So, as far as I'm aware, ISO has not seen my recommendations. Therefore my recommendations are unlikely to be incorporated. Moreover, I didn't necessarily want my recommendation to be incorporated. Rather I'd want them discussed because there may well be good reasons not to incorporate them.
Well then, I'll need to tar you with the "implementer" brush. It will be therefore necessary to get your provisional endorsement, as has been given by @plk and @inukshuk, to proceed. Provisional on further details of the proposed standard continuing to be feasibly a good idea. But I'd invite you to do that in the new thread, which provides some further details.
Yes, that helps us here. We don't need to convince the Zotero devs of anything, although I do intend to keep them in the loop, if we go ahead with the standard.
Yes, that's current intention. The "subset" (it's hard to say whether it is "big" or not) of ISO 8601:201x is the ISO 8601:201x Profile - Annex C The Extended Date/Time Format.
I address this sort of idea in the new thread, proposing a version of it. I've become much clearer about a possible relationship between our standard and ISO 8601 profiles. @inukshuk and @moewew have raised the standards proliferation concern. @moewew further raised concerns about: the work needed to be done; copyright; that our standard would languish as incidental; and feature creep. Apart from feature creep I've mentioned those concerns earlier in the thread. And (apart from feature creep) it's good for these issue to be raised again, for that has prompted me to flesh them out in the new thread. Feature creep is also a legitimate concern. I'd agree that just because someone proposes a datetime format, it doesn't follow it should be admitted. I repeat this concern in the new thread. So given the further details in the new thread, in that new thread, from @plk (biblatex datetime (and may other aspects of biblatex) implementer); @inukshuk (EDTF.js implementer); and @retorquere (zotero-better-bibtex implementer) - I'll invite you to make a further provisional endorsement of the proposal. A "Go:let's-create-it"/"no-go: let's not create it" decision. Of course in that thread from all, @plk, @inukshuk, @retorquere, @njbart, @moewew, @Crissov, and any other folk who are interested, you should feel free to: press objections; mention creative solutions; explore ideas, etc. Continues at "Should we create an Open Datetime Format?" JohnLukeBentley/open-datetime-standard-bootstrap#1 |
Hi folks. Checking in here from Schema.org, where we are hoping to find an answer to the longstanding question of how to express open-ended data ranges (e.g. for datasets that are currently being generated/extended live). Currently we've been hoping an updated ISO 8601 will address this. Is that naive? Do you have any better suggestions? |
@njbart, @plk, @inukshuk, and @moewew. Have you seen JohnLukeBentley/open-datetime-standard-bootstrap To save you having to read the whole thread. I'm essentially wanting to know if BibliographicDatetimeFormat.pdf (direct download) looks worth developing. It would be better to reply in that thread. |
Intro
Biblatex formerly (between v3.5 and v3.8, inclusive) supported an "Extended Date/Time Format (EDTF)" specification (to "level 1") located at http://www.loc.gov/standards/datetime/pre-submission.html. A new International Standards Organisation (ISO) draft specification has emerged that incorporates, with minor changes, that specification and "supersedes" it (thanks @moewew for flagging the change).
I've read the recent draft ISO and would like to clarify the differences between the old EDTF spec, former and current biblatex support, and the new draft ISO.
@plk indulge me addressing you in the third person even though you'll be the chief audience member in virtue of being the person who has implemented all this datetime stuff ...
@plk, with laudable enthusiasm, has gone ahead and started implementing the draft ISO for v3.9. @plk's partial implementation of the draft ISO spec has established that backwards compatibility is impossible (at least, and probably at most, with respect to open and unknown date ranges). And it seems that @plk has broken backwards compatibility in other ways on the basis that ....
I think that's reasonable. However, I'd would have suggested holding off for now, or at least, for now, having biblatex not break the old EDTF spec. Given that the new draft ISO warns "it is subject to change without notice" and that, indeed, it is we who might very well want to lobby ISO to make such changes.
Therefore, for biblatex, given the option between:
... I suppose we are going to roll with the latter, having started down that path. That need not be too onerous: the ISO revisions are probably going to be slow and few. And I also suppose that I'm happy that @plk has been prepared to break backwards compatibility in all sorts of ways. For that better allows us to evaluate the new draft ISO as a general matter freed, to a greater extent, from parochial worries about biblatex.
Anyway I suggest the purposes of this thread be:
And by "we" I mean to invite anyone who reads this post to participate. But I have in mind specific individuals who may find this of interest: @plk, @moewew, @retorquere, @njbart. ... but, as always, feel free not to engage this thread.
In this first post I largely confine myself to describing the issues. In the post to follow I'll make recommendations about those issues.
Semantics
To speak about this I'll use the following terms for the different standards/specifications:
As mentioned in a former version of biblatex.pdf ("2.3.8 Date and Time Specifications", p36) biblatex supported EdtfLoc levels 0 and 1. EdtfIso201x also uses the concept of levels but, naturally enough, what falls under those levels are (slightly) different with respect to EdtfLoc. A further subtle difference occurs between EdtfIso201x levels and Iso201x (the larger document) levels:
All this is largely to make the point that we need to be specific when referencing levels. For example EdtfLoc level 1, EdtfIso201x level 1, and Iso201x level 1 are (slightly) different.
Note also that generally, unless exceptions are mentioned, in all of those standards or profiles support for a higher level entails support for a lower level. E.g. To claim that an application, like biblatex, supports "EdtfLoc level 1" would be to say that it supports "EdtfLoc level 0 and level 1" (because this is what the specs oblige of an implementer). I have no objection, however, to anyone explicitly enumerating level support (i.e. speaking of support for "EdtfLoc level 0 and level 1" rather than "EdtfLoc level 1"). Indeed it is probably clearer to be explicit.
The differences
By and large EdtfIso201x (and the overall Iso201x) looks well thought through (as was EdtfLoc) and feature complete. But the overarching issue of interest is: What are the relevant differences between between EdtfLoc and EdtfIso201x?
Levels
The first difference is on the matter of levels. And the relevant issue is what EdtfIso201x level should biblatex, in the future, support?
Both EdtfLoc and EdtfIso201x categorize datetime formats into levels: 0, 1, 2. As mentioned above the levels in each spec are slightly different in terms of what they reference. However, there is a rough correspondence.
Biblatex formerly supported EdtfLoc level 1 (with a few extras). EdtfLoc level 2 seemed to target rare edge cases.
EdtfIso201x level 2 seems to target highly rare edge cases. For example,
So both EdtfLoc level 2 and EdtfIso201x level 2 seem to target rare, or highly rare, edge cases; although there's a slight variation in what those edge cases are.
Unknown and open ranges
EdtfLoc supports "unknown" and "open" ranges at EdtfLoc Level 1 as follows:
(See http://www.loc.gov/standards/datetime/pre-submission.html#extendedintervall1)
EdtfIso201x supports "unknown" and "open" ranges (at EdtfIso201x Level 0) as follows:
(See "4.4 Enhanced time interval", ISO DIS 8601-201x Part 2, p11).
Biblatex formerly supported "unknown" and "open" ranges as follows:
In summary ...
That's a problem. If biblatex is to transition to support EdtfIso201x (as currently specified) the meaning of blank, in the biblatex context, has to change from open to unknown.
Approximate and uncertain dates
EdtfIso201x preserves, from EdtfLoc, the representation of approximate (e.g. "1985-04-12~") and uncertain (e.g. "1985-04-12?") dates.
However, the representation of dates that are both approximate and uncertain has changed:
Century
EdtfIso201x supports a mere two digits as representing a century ...
EdtfLoc doesn't support this. But under EdtfLoc a century could be represented: with the unspecified range as in "19uu"; in addition to an explicit range like "1900/1999".
Other issues
There are other issues that arise that are not due to differences between Edtfloc and EdtfIso201x. Rather, they arise because of absolute reasons about the current form of EdtfIso201x.
Reduced time precision
When a time is present, shouldn't EdtfIso201x support minute level precision?
Edtfloc doesn't appear to support precisions less granular than a second, when a time is present at all. That is, something like "09:30:01" appears to be permitted but minute level precision, "09:30", appears to be forbidden.
Edtfloc stipulates:
And the Edtfloc BNF stipulates
So, as written, a reduce precision datetime, like "2017-10-28T04:47" appears to be forbidden under Edtfloc.
Biblatex currently (and formerly) reflects this with a message like:
... although biblatex permits output where the seconds are dispensed with.
EdtfIso201x essentially repeats the specification on this issue (See "C.4.2 Date and Time", ISO DIS 8601-201x Part 2, p29). That is, EdtfIso201x also appears to forbid minute level precision, where a time is expressed.
Move EDTF to its own Part 3?
Should the EDTF section of EdtfIso201x be an annex in part 2, or be moved to its own part 3?
Take the (currently existing) sentence like ...
("Annex C (informative) The Extended Date/Time Format - A Profile of ISO 8601 (Parts 1 and 2)", ISO DIS 8601-201x Part 2, p28)
... I think that (quoted) sentence would be a easier to follow if the EDTF profile were in its own part 3.
Participating in the ISO forum?
Participating in an ISO forum seems rather labyrinth. However, the relevant clues seem to be ...
From How we develop standards there is ...
https://www.iso.org/get-involved.html
From ISO/DIS 8601-1 the relevant technical committee is: ISO/TC 154 Processes, data elements and documents in commerce, industry and administration
... perhaps some of you are already participating and have a shortcut route to join?
In any case this very thread might serve as a calling card for those of us that want to join the ISO process.
The ISO 8601 landing page
Just to have it to hand:
Edit: 2017-12-28 Added "IsoOrg" term.
The text was updated successfully, but these errors were encountered: