Journal abbreviations in UTF-8 not recognized

[jorgman1]

When running an integrity check, Jabref complains about non-ASCII characters in the journal title. However, changing UTF-8 to ASCII generates the error: "Journal not found in abbreviation list".

Since the journal abbreviation lists are encoded in UTF-8, it would be nice if Jabref recognizes both formats and treats them equally.

[bernhard-kleine]

this is most probably related to #5562

[Hollyqqqqq]

Hi, I am interested in this issue, but I can not reproduce this bug.

@jorgman1 Can you give the example of a Journal name containing UTF-8 characters and briefly describe how you change UTF-8 to ASCII?

Thanks!

[jorgman1]

E.g. a lot of German journals: Zeitschrift für Meteorologie. If I use the Convert Unicode to LaTeX tool, it gets converted to f{\"{u}}r. I've noticed that many such journals are in the built-in abbreviation list with fur, but that is wrong. The German convention is to put fuer.

[Siedlerchr]

@jorgman1 Have you tested the latest development version?
There have been recently some major changes to the storage/and import of journals and their abbreviation
https://builds.jabref.org/master/

[jorgman1]

In the newest version (JabRef 5.1--2020-05-04--b5599c9; Windows 7 6.1 amd64; Java 14.0.1) still see in the quality check either "non-ASCII error" or "journal not in abbreviation list error".

How should I put the journal name?

[github-actions]

This issue will be closed in 7 days due to inactivity 💤 Please provide the requested information if the problem persists.

[jorgman1]

This behaviour still persist in the latest version (JabRef 5.1--2020-05-04--b5599c9; Windows 7 6.1 amd64; Java 14.0.1).

How should I put the journal name?

[github-actions]

This issue will be closed in 7 days due to inactivity 💤 Please provide the requested information if the problem persists.

[bernhard-kleine]

This has not been solved. Why do we get this status: stale messages. It is very annoying.

[MrGhabi]

Hi, I'm interested in this issue, and I think I have reproduced this bug.

The main reason for this bug is the check-tools Check integrity only accept the charset ASCII. It works well in English citations, but jabref has users worldwide and has different charsets. Here are Steps to reproduce:

1. New library

2. New article

3. BibeTx source adds the following:

   @article{杨芙清2005软件工程技术发展思索,
     title={软件工程技术发展思索},
     author={杨芙清},
     journal={软件学报},
     volume={16},
     number={1},
     year={2005},
     publisher={Citeseer}
   }
   

4. click "check integrity."

I want to fix this issue, and our goal is to check integrity only by warning the charset, which is not UTF-8. To meet this need, maybe I need an external jar to detect which charset does this string use. I'm trying to fix this issue by this method right now.

[Siedlerchr]

@MrGhabi Thanks for your interest. JabRef fetches the journals from the https://github.com/JabRef/abbrv.jabref.org repo which are then assembled to a MV database
https://github.com/JabRef/jabref/blob/main/buildSrc/src/main/groovy/org/jabref/build/JournalAbbreviationConverter.groovy

[MrGhabi]

@MrGhabi Thanks for your interest. JabRef fetches the journals from the https://github.com/JabRef/abbrv.jabref.org repo which are then assembled to a MV database
https://github.com/JabRef/jabref/blob/main/buildSrc/src/main/groovy/org/jabref/build/JournalAbbreviationConverter.groovy

Thank you~

[jorgman1]

Thanks for the fix!