Improve CFF import/export and craft a round-trip test

[jeanprbt]

This PR improves the actual CFF importer/exporter by many ways. The exporter is no longer based on a *.layout file, but is now code-based in a brand new CffExporter.java file, so as to handle all the logic related to the CFF format.
Key improvements are the following.

• Author's names are now parsed to use CFF fields family-names, given-names, name-particle and name-suffix
• Fields references and preferred-citation are now parsed.
  • The importer now creates separate entries for the topmost software or dataset and for each of the preferred-citation or references cited. Importer adds a cites relationship between topmost software and preferred-citation, whereas it adds a related relationship between topmost software and each of the references.
  • The exporter now parses the software and dataset types correctly without adding a useless preferred-citation field in these cases. For software or dataset entries with JabRef cites or related fields, exporter now exports the corresponding CFF fields.
• Multiple exports at a time are now supported to comply with the CFF format.
  • If multiple "non-software-nor-dataset" entries are selected, then exporter uses the references field with a dummy topmost software element.
  • If several entries including a software or dataset one are selected, then exporter uses this one as topmost element and the others as references, adding a preferred-citation for the cites element.
  • If several entries including several software ones are selected, then exporter uses a dummy topmost element, and selected entries are exported as references. The cites or related won't be exported in this case.
• Exporter now supports UnknownField and exports them if they correspond to existing CFF fields
• Importer and Exporter now support keywords fields
• Importer and Exporter support all fields from the CFF specification
  Closes Craft round-trip test for CFF importer/exporter #10993.

This PR also features a round-trip test, which imports JabRef's citation file. This creates two entries, that are then exported to one file.

Here are some screenshots to show the updated behavior. These screenshots use the original CITATION.cff file of the JabRef repository.

Here is the original CFF file

Here is what is imported

The article has a citation key, used by the software

The software has a cites relationship towards the article

When exporting both entries, like this:

Here is the resulting output

It complies with the CFF format

Sub-tasks

• Parse author's names to use CFF fields family-names, etc.
• Support JabRef's UnknownField in the exporter and check if field complies to CFF format
• Support keywords field in both importer and exporter
• Support all fields from CFF specification
• Avoid using useless preferred-citation field when exporting software or dataset
• Import several entries in the case of a preferred-citation or references field
• Add a cites or related relationship between imported entries in relevant cases in the importer
• Parse JabRef cites and related fields correctly in the exporter
• If a software and its cites entry are both exported, then exporter discards the cites entry since it has already parsed it as a preferred-citation of the software entry
• Document changes in the MADR format at https://github.com/JabRef/jabref/tree/main/docs/decisions

Mandatory checks

• Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
• Tests created for changes (if applicable)
• Manually tested changed features in running JabRef (always required)
• Screenshots added in PR description (for UI changes)
• Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
• Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

[koppor]

The roundtrip tests should include all entries from the CFF format specification

• https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md
• https://github.com/citation-file-format/citation-file-format/tree/main/examples/1.2.0/pass
  • especially https://github.com/citation-file-format/citation-file-format/blob/main/examples/1.2.0/pass/reference-conference-paper/CITATION.cff
• https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-citation-files

---

Do a consistency check. How does the official converter convert? https://github.com/citation-file-format/cffconvert - does it also use cites and related?

[jeanprbt]

The following is written on the cffconvert README.

cffconvert does not support converting items from references or preferred-citation keys at the moment.

Consistency check will be difficult to perform without these two fields.
But I'll try to include all entries from the CFF format specification.

[koppor]

Consistency check will be difficult to perform without these two fields.

You mean: consistency check of your work with the result of the tool? - It is OK to be better than other tools :p.

[jeanprbt]

Yes that's what I meant, since it is the official tool :)
But let's try to be better than it !

[jeanprbt]

I'm done with all the features I stated above in the PR. The exporter now parses the cites and related relationships correctly, and the round-trip test is fully working : importing CITATION.cff creates two entries, and exporting these two entries only creates one file which is consistent with the original one. I completely modified the behavior of CffExporter to achieve this, I'll let you check it out but in a few words it works like this.

• Firstly, it checks how many software or dataset entries are contained in the list entries to export.
  • If there is only one, it is used as the main (i.e. topmost) entry
  • If there is none or several, a dummy main entry is created
• Secondly, the method parseEntry is generic and can manage not only the main but also the preferred-citation and references entries (few differences between main and other entries are handled by the boolean parameter main)
• This method is called on main entry to have the root data HashMap which will be dumped into the output yaml file.
• Then, if the main contains a StandardField.CITES and/or a StandardField.RELATED field(s), the corresponding entries are retrieved using the databaseContext and then passed as parameters to parseEntry => they are added either as a preferred-citation or as references to the final HashMap data.
• Finally, the remaining entries of the original list are passed as parameters to parseEntry to be added to the List<Map<String, Object>> related, which will be put in the main data HashMap as references.

I also fixed the changes you requested. Now, I will try to focus on my proposal for the GSoC project you mentioned earlier in this PR. I would really like to conduct this project and we are in the contributors' proposal phase, that is why I need to take some time for it. Do you think I have the required skills and commitment to conduct such a project, and that I have my chances to be selected ?

Thanks a lot for your responsiveness, your hints and your continuous feedback on this one, hopefully it will be merged in the official JabRef project ! Users will be happy with such a CFF importer/exporter.

[koppor]

@jeanprbt Your text reads as if should be put as JavaDoc. Otherwise, it will get lost. I know only one contributor who used "git blame" to recover a commit and the deception. Usually, no one checks the PR and all (!) comments of the PR to learn about the reasons behind the code. JavaDoc was intended to capture those... (At least partially). - Regarding GSoC, your skills improved. Will be a tough competition - and you also have chances!

[koppor]

After more than 80 conversations, this is finally going to an end. 🥂 Hope, the users will like this feature.

If also should be available at the CLI. Thus, we (you @jeanprbt) could craft a blog post with examples and advertise this feature ^^.

Glad that you added test cases! That offered the possibility to improve the code without worrying.

• Use JabRef's org.jabref.model.entry.Date instead of manual date parsing
• Use JabRef's org.jabref.model.entry.BibEntry#getEntryLinkList instead of manual link parsing
• Use more stream magic