strict=true & enable linkcheck by default

[mortenpi]

And any other checks we might have. This is to encourage people to fix the errors they have in their docs, and in particular to catch any new ones that pop up on CI. It's quite fair to say that no-one checks the warnings on CI.

This well be very breaking for existing workflows though.

X-ref: https://julialang.slack.com/archives/C67TK21LJ/p1676982324363569

[odow]

It's a reasonable thing to include in Documenter 0.28 since that is already a breaking release?

[mortenpi]

1.0 = 0.28 if that's what you mean 🙂

[ArnoStrouwen]

Activating strict links in the SciML packages, I find that there are quite a few stochastic false positives with link checking, see #2040. If some of the biggest false positives could be fixed before the breaking release, it will lessen the upgrade pains.
ScienceDirect links are a big offender that occur commonly in various Julia packages.

[mortenpi]

Like the linked issue, those are often due to the server not behaving well, so not much we can do. Those should just be ignored with linkcheck_ignore.

That said, linkchecking can be flaky and unreliable sometimes, even just due to random connectivity problems. So that one might have to be opt-in, rather than opt-out. E.g. it would be quite frustrating to have deployment of a tag fail due to an random, temporary linkcheck failure.

In Documenter, we have dedicated workflows for running linkcheck so that it wouldn't interrupt the main docs workflow.

[mortenpi]

A couple of follow-up thoughts here:

• We should probably deprecate (or even just remove) strict, and replace it with something like warnonly (list of categories to warn on, or warnonly=true to just warn on everything; bikeshed, please!). With strict, you would have to list all the categories you want still to error (there is Documenter.except, but that also requires extra typing).

• I think we can also enable linkchecking by default if we make it a bit easier to recover a failing deployment.

  What we could add is an environment variable to override any explicit or implicit makedocs arguments (DOCUMENTER_DISABLE_LINKCHECK?). This could be used when the CI for a package tag deployment is failing due to some spurious linkcheck failure -- you set the env. variable in repo settings, re-run the CI, and it should deploy.

  Furthermore, we should recommend in the docs to explicitly set linkcheck to conceptually belinkcheck = isdeployment() ? :warn : true. Maybe for like standard GitHub Actions deployments we can even cook up a re-usable isdeployment function. But I do think that linkcheck should not have any automagic behaviour by default, but it should always be on by default.