issues with system-wide tinytex setup & on-the-fly style file installation

[cboettig]

Hi @yihui ,

Sorry to bother you with this but I'm not sure where things are going wrong.

In the system-wide tinytex installation we have on the Rocker images, tinytex::tlmgr_install() seems to insist on creating a symlink to /root/bin, which I believe it should not be doing.

I can get tlmgr_install() to work by giving the user permission to write to /root/bin (which clearly isn't the right solution), i.e. mkdir /root/bin && chown -R root:staff /root/bin && chmod g+rwx /root/bin. But even when tlmgr_install() is happy:

> tinytex::tlmgr_install("bbm")
tlmgr install bbm
tlmgr: package repository http://mirrors.concertpass.com/tex-archive/systems/texlive/tlnet (not verified: gpg unavailable)
tlmgr install: package already present: bbm
tlmgr path add

Something is going wrong with the paths, because I still cannot knit an example doc like

---
title: "Untitled"
header-includes:
   - \usepackage{bbm}
output: pdf_document
---

same error:

tlmgr search --file --global '/bbm.sty'
Trying to automatically install missing LaTeX packages...
tlmgr install bbm-macros
tlmgr: package repository http://mirrors.concertpass.com/tex-archive/systems/texlive/tlnet (not verified: gpg unavailable)
tlmgr install: package already present: bbm-macros
tlmgr path add
tlmgr search --file --global '/bbm.sty'
! LaTeX Error: File `bbm.sty' not found.

! Emergency stop.

I suspect these are all path issues related to the fact that tinytex is somehow assuming it should still be installed at the user level and not the system level, but we really would like it to work at the system-level so it can be shared across a multi-user environment if necessary. You should be able to reproduce this behavior with a vanilla rocker/verse image:
rocker-org/rocker-versioned#104

[cboettig]

Following up on this a bit: it's not clear to me why install-unx.sh hardwires directory to ~/.TinyTeX. I'd rather be able to set directly. At least if it used an environmental variable I could temporarily override that,
https://github.com/yihui/tinytex/blob/dc88ff892f28fc3eb58a6df91cff191d7f04af48/tools/install-unx.sh#L9

Meanwhile, just installing to root dir and then mving everything over to /opt seems to be fine, so I guess I should stick with that!

[yihui]

Okay. Glad you figured it out: rocker-org/rocker-versioned@a70608c (indeed tinytex::install_tinytex() is primarily for personal users, and for admins, you have to install via the shell script and move the dir).

[cboettig]

Sadly after going back to following the FAQ for the system install (silly me, that's what I had before switching to install_tinytex) this problem persists. It seems to be an issue with the automatic /on-the-fly installation of style files.

Why is LaTeX now attempting to automatically install style files automatically? I mean, that's 💯 amazing and all, but it also seems to be the source of this problem. Note that contrary to the error message LaTeX is showing above, running tlmgr search --file --global '/bbm.sty' finds that the package has installed no problem. Running tlmgr_search() and tlmgr_install() from R also run as expected.

So the above problem seems to be isolated to LaTeX's attempt to automatically install the style packages, which I think historically at least it did not do. Is this something tinytex's installation or RStudio has configured? Can it be turned off?

[yihui]

The feature of auto-installing missing LaTeX packages existed in the tinytex package from day one (December last year), and rmarkdown 1.9 started to use tinytex (March this year).

It can be turned off (I rarely force features even if they are nice), but I'd like to figure out the root cause of the mysterious issue first.

[kberry]

Hi Yihui - if the copy of $tmppath to $path fails (fix #77, line 630 of the upstream TLPDB.pm), why is not correct to die? seems to me you would be left with an incorrect texlive.tlpdb if that copy fails. unless you made other changes to the code. Just trying to understand ... --karl (from TeX Live)

[yihui]

@kberry Hi Karl, we actually thought about reporting this issue to TeX Live developers but didn't really do it because it would involve Docker, and we were not sure if you would like to try to reproduce the issue. I guess @cboettig might still be able help you set up a test environment on the cloud like he did for me last year so you don't have to run Docker locally to test it.

Anyway, below is a short version of my investigation last year. Basically I found that it was not possible to install or remove LaTeX packages, e.g.

$ tlmgr remove bbm-macros
[1/0, ??:??/??:??] remove: bbm-macros
copy /opt/TinyTeX/tlpkg/texlive.tlpdb.tmp to
/opt/TinyTeX/tlpkg/texlive.tlpdb failed: Operation not permitted at
/opt/TinyTeX/tlpkg/TeXLive/TLPDB.pm line 628.

Eventually I found that copy() signaled a false alarm:

TeXLive::TLUtils::copy ("-f", $tmppath, $path) or die ("copy $tmppath to $path failed: $!");

It actually succeeded in copying $tmppath to $path, and I had no idea why the exit status was non-zero, so I stripped off die() there.

[cboettig]

@kberry thanks for getting in touch with us!

As @yihui says, you should be able to replicate this with the Docker image as linked above (rocker/verse).

My possibly-mistaken recollection is that this had something to do with user permissions. We are copying the file to a directory which is owned by root but is in the group staff. The error would only occur when a non-root user, but also in the group staff, would run the script. It seems that the copy would succeed, since indeed group membership was granting permission to do so (as it should), but that, as @yihui observes, the TeXLive::TLUtils::copy was still throwing a non-zero exit status. I recall the problem would not occur if a root user did the copy, or otherwise if we changed the ownership of the target directory to match the user. (Thus this issue only arises when we try this multi-user setup using groups!).

To me, this seems like a bug in the exit status of copy() related to the handling of group permissions, but I'm way over my depth here. any insight?

[kberry]

you are essentially right; i'll spare you the details :). The upshot is that I just removed the "or die" in upstream TLPDB.pm. Thanks. And don't hesitate to report other upstream bugs.

And, congratulations on tinytex; nice initiative! Would you like to write an article for TUGboat about it? Please email me (karl at freefriends dot org) if you're interested.

[norbusan]

@kberry but shouldn't we do better? Now that any return value is ignored, we might loose stuff. Maybe we need to improve the copy routine?

[kberry]

i thought about that, but when I looked at the copy() fn in TLUtils.pm, it already dies in case of any real error, seemed to me. And that is good, because copy() is called in a half dozen other places which never had an "or die". Um, well, I just added some more die-on-failure cases in copy(). Take a look.

By the way, I surmise the cause of the original error was that utime() will fail when only group write is available, and the error from that fell through as the return value ...

[norbusan]

Indeed, it was the utime call that failed and produced the return code. Thanks for the changes, look fine.