Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix latex2unicode and unicode2latex #6155

Closed
wants to merge 6 commits into from
Closed

Fix latex2unicode and unicode2latex #6155

wants to merge 6 commits into from

Conversation

koppor
Copy link
Member

@koppor koppor commented Mar 22, 2020 •
edited
Loading

Begin to fix #5547

Together with @calixtus

Still work in progess. Early draft - just to save our work if my machine crashes.

tobiasdiez
tobiasdiez reviewed Mar 22, 2020
View reviewed changes
Copy link
Member

@tobiasdiez tobiasdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uiii, replacing the latex2unicode library may turn out to be a big project (especially to get the more advanced parsing to work). Anyway, please also add the test from https://github.com/tomtung/latex2unicode/blob/master/src/test/scala/com/github/tomtung/latex2unicode/Latex2UnicodeTest.scala as our tests are not as extensive.

@Siedlerchr
Copy link
Member

Why not port the Scala stuff to plain Java?

@koppor
Copy link
Member Author

koppor commented Mar 22, 2020

This is a very early PR and we are in the middle of discussing. We did not finish it. We (especially me) thought, it would be a good idea to push the changes here in the case of data loss.

We are currently in the process of understanding @oscargus's code. We aim for latex -> unicode and unicode -> latex working. They do not cover the same set of functionality. In the case of plain symbols, the unicode -> latex converter seems to cover more.

We really look into the latex2unicode code, but had not the time to port everything. We're just at the beginning.

Sorry for not explaining in the description. The time was up and I wanted to persist the work we achieved so far.

@koppor koppor changed the title [WIP] Fix latex2unicode and unicode2latex Fix latex2unicode and unicode2latex Apr 23, 2020
@Siedlerchr Siedlerchr mentioned this pull request May 15, 2020
Merged
10 tasks
@calixtus calixtus mentioned this pull request May 26, 2020
Open
1 task
@koppor
Copy link
Member Author

koppor commented May 26, 2020

For the ADR:

@calixtus calixtus closed this Jul 21, 2020
@calixtus
Copy link
Member

Since this PR was not moving forward and we didn't ha a good idea yet how to move on, were closing this for now.

@koppor koppor mentioned this pull request Aug 2, 2020
Draft
@koppor koppor deleted the fix-5547 branch August 2, 2020 19:16
@koppor
Copy link
Member Author

koppor commented Jan 4, 2021

This could maybe solve #7291.

@koppor
Copy link
Member Author

koppor commented Jan 4, 2021 •
edited
Loading

Tests should also include #2500 (comment) and #2498 and #2458

@koppor koppor added the status: freeze Issues posponed to a (much) later future label Jan 4, 2021
@koppor
Copy link
Member Author

koppor commented Feb 1, 2021

This could have fixed #3644.

@koppor koppor mentioned this pull request Feb 1, 2021
Closed
@calixtus calixtus mentioned this pull request Feb 15, 2021
Closed
@koppor koppor mentioned this pull request Sep 22, 2021
Open
@calixtus
Copy link
Member

Possible solution strategy: create a simple parser for simple latex text formatting and use a hashmap (sadly we were not able to see a clear systematic in the unicode char tables)

@koppor koppor mentioned this pull request Sep 22, 2021
Closed
@koppor
Copy link
Member Author

koppor commented Mar 21, 2022

Missing \mkbibquote{}, \enquote{}, \mkbibemph{}, and \emph{}. See https://discourse.jabref.org/t/recognize-quotation-commands/3296/3 for a discussion.

@calixtus
Copy link
Member

koppor#354

https://github.com/plk/biber/blob/dev/lib/Biber/LaTeX/recode_data.xml

@koppor
Copy link
Member Author

koppor commented Jun 25, 2022

See also https://tex.stackexchange.com/a/9793/9075

koppor pushed a commit that referenced this pull request Aug 1, 2022
Squashed 'buildres/csl/csl-styles/' changes from 3d3573c..c750b6e
871c8d7 
c750b6e APA: Put conditional event-title logic in a macro (#6161)
a87414f Remove month from association-for-compuational-linguistics.csl (#6158)
6153db0 Remove issue numbers from BJOC style (#6155)
e231ea3 Bug fix for `event` regression (#6154)
0dab651 Add event-title to other APA styles (#6153)
698cf1c APA: `event-title` and conditional `event` (#6152)
58d3f8f Update vancouver-author-date.csl (#6148)
f1638a9 add substitute to Vancouver author date (#6147)
39fede5 Update associacao-brasileira-de-normas-tecnicas.csl (#6138)
fde7695 Include chapter title (#6140)
1e3d8b4 Update n.d. abbreivation for DGP style (#6136)
ebb728b suffix '.' after first group; changed e-mail (#6135)
eed4f07 Update and rename sciences-po-ecole-doctorale-note-french.csl to scie… (#6127)
f194647 Delete TU Dresden Medizin as requested by library (#6131)
d8423d8 Create entomological-review.csl (#6120)
064a394 Create australasian-journal-of-philosophy.csl (#6063)
a998ded Add composer.json (#5668)
37083c9 Update copernicus-publications.csl (#6062)
694c97b Create chaucer review (#6061)
625a424 Create haffner-style-manual.csl (#6054)
8b7224b make annals-of-allergy-asthma-and-immunology independent (#6041)
710748c Create university-of-pretoria-harvard-theology-religion.csl (#6106)
d16dffd Create health-physics.csl (#6040)
ca9e184 Update style-manual-australian-government.csl (#6119)
e412277 Create chemical-engineering-technology.csl (#6039)
bebdb48 Create bibliothek-forschung-und-praxis.csl (#6038)
29e49cd Update nature.csl (#6117)
891897d fix short title for SBL (#6118)

git-subtree-dir: buildres/csl/csl-styles
git-subtree-split: c750b6e
@koppor
Copy link
Member Author

koppor commented Sep 1, 2022

Related work: https://www.ctan.org/pkg/texaccents

@koppor
Copy link
Member Author

koppor commented Dec 5, 2022

@koppor koppor mentioned this pull request Mar 18, 2023
Closed
@koppor koppor restored the fix-5547 branch May 12, 2023 22:03
@koppor
Copy link
Member Author

koppor commented May 15, 2023

@koppor
Copy link
Member Author

koppor commented Oct 12, 2023

Readings on Unicode https://tonsky.me/blog/unicode/ (via @Siedlerchr)

@koppor koppor mentioned this pull request Nov 21, 2023
Closed
@koppor
Copy link
Member Author

koppor commented Nov 21, 2023

Related work: https://github.com/zepinglee/citeproc-lua/blob/ab3ce712cc92073f12be26ff0b22b30eb906092d/citeproc/citeproc-latex-data.lua#L517

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tobiasdiez tobiasdiez tobiasdiez left review comments

Assignees

@koppor koppor

@calixtus calixtus

Labels
status: freeze Issues posponed to a (much) later future
Projects
Candidates for University Projects
Status: Free to take
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

LaTeX to Unicode and vice versa has stopped working
4 participants
@koppor @Siedlerchr @calixtus @tobiasdiez