[archlinux] adjust pkgbuild + add files

[shibumi]

We are going to ship the default caddy welcome page on Arch Linux
and we've enhanced the caddy service file with additional security
options.

Note: Arch Linux uses the 'http' user/group for web servers on default.
Therefore we've changed the caddy user/group to 'http' in our service
file.

[mholt]

At a quick glance, this is looking good to me so far. Any particular thing you want reviewed?

[shibumi]

@anthraxx said you might want to talk about the hardening options, that we've choosed. I would also like to propose, that all distributions use the same service file to reduce confusion by end users.

The fedora maintainer mentioned that there is still a problem with systemctl restart and systemctl reload, right?

[shibumi]

@mholt you can merge this, if you like.

[mholt]

Thanks! Just a couple questions before merging.

[mholt]

Suggested change

	# https://github.com/caddyserver/caddy/wiki/v2:-Documentation
	# https://caddyserver.com/docs/caddyfile

PS. I forget if we talked about this already, but do we need a copy of the Caddyfile? I think this exact same Caddyfile is already in this repo. I'd like to avoid making copies of it, so we only have to update one file.

[shibumi]

Right now I only fetch the caddy welcome page from this repository. This is why I ship a dedicated caddy file. I would actually prefer if we could use just one source in the PKGBUILD (we could achieve this via moving the welcome page and a default caddy file to the standard caddy repo under github.com/caddyserver/caddy )

[mholt]

I really don't want to mix distribution resources with the actual code itself, though. Can you just fetch both the Caddyfile and the welcome page from this repo instead? (If you're already fetching one, why not another?)

[shibumi]

It's tedious, if we get the hardening options in this repository as well, I will end up with four sources instead of one. It's considered bad practice to fetch from so many sources in Arch Linux. Not sure how other maintainers would react.

I don't think this files are 'distribution' specific. I would actually consider them es necessary to build caddy and run it from source. At least it's the first time that I see somebody having a separate distribution repository. I will think about it.

[mholt]

So is each file considered a different source? Why is it bad practice? If the installation needs it, why not get it?

I don't think this files are 'distribution' specific. I would actually consider them es necessary to build caddy and run it from source.

They definitely are distribution-specific, i.e. not needed to compile or run. They're absolutely non-essential, and I'd rather people write their own configs, tbh. I really don't want extra resources cluttering up the main repository.

I learned from making Papa Parse that distribution can quickly double or triple the size of the actual repository. Distributing a program is a totally different (and way too difficult, IMO) task than just writing the program and compiling it, so it's in a different repo.

[mholt]

Dunno if it matters, but we had to make the tag v2.0.0-beta.13 in order to make Go modules happy...

[shibumi]

Cool. Then I can just use pkgver in the PKGBUILD and don't need to track this twice. Thanks a lot.

[shibumi]

Ok, sorry. Looks like I misunderstood you here :'D. It doesn't matter for us here, because I just pull your code by commit (for checking your signature on this commit). I am not sure if pacman (the Arch Linux package manager can handle versions like v2.0.0-beta.13... we can try, but I can't promise. Or is it okay to leave it as v2.0.0-beta13?

[mholt]

That's okay. I wasn't sure if the version had to be the same as in the Caddy repo.

[mholt]

Similarly, this file is also already in our repo: https://github.com/caddyserver/dist/blob/4d5728e7a4452d31030336c8e3ad9a006e58af18/init/caddy.service -- but this one is quite different. Is this one that we should use universally?

[shibumi]

This is the hardened service file. It has more hardening options. I have no problems with it so far, but other distribution maintainers were afraid it could have bad side effects. We should maybe have a discussion around this in the dist slack channel before shipping it universally.

[anthraxx]

Well it's somewhat unnatural to not have a homogeneous source when a distribution of the software is expected to contain unit files, start pages and other things. It's hard to track which commit is considered how stable or matching to certain changes and functionality in the original software. This is lot cleaner and self contained when having it in the same repo. Same applies to nginx and others, start pages and crucial elements belong to a self contained place where a release includes sets of changes that may be reflected by such. It's an unnecessary artificial split of things that belong together when packaging making it unclear and cumbersome to track another repo and decide when to bring stuff in

[mholt]

@anthraxx Thanks for the perspective. I wouldn't expect our situation is so unique though. Caddy is very extensible -- like a lot of other software -- so its code doesn't all live in one mega repository. Surely we can organize our code the best way we see fit and I don't see why that should affect distribution of binaries?

This is lot cleaner and self contained when having it in the same repo.

That's not the case here, though. The point of Caddy is that it's extensible, and putting everything into one repo would make it unwieldy and bloated. The benefit is that it's not self-contained, in that sense.

We have a build server that provides signed binaries of custom builds with a single HTTPS request. (I'll eventually upgrade it to support Caddy v2.) Tbh, this whole problem would be a lot easier if any package manager could simply make an HTTPS request, verify the checksum/signature, and plop the downloaded binary somewhere in the user's PATH. That's really all that needs to happen.

(To clarify, my comments are more about packaging and distribution in general -- not really related to this PR. I can still merge this if you're ready for me to do so.)

[shibumi]

Tbh, this whole problem would be a lot easier if any package manager could simply make an HTTPS request, verify the checksum/signature, and plop the downloaded binary somewhere in the user's PATH. That's really all that needs to happen.

We build all packages from source in respect to reproducible builds. All other distributions do it the same way.

The point of Caddy is that it's extensible, and putting everything into one repo would make it unwieldy and bloated.

We don't want that you put every plugin into that repository. Just all necessary files for basic usage should be in one repository and I don't see how a dist directory in the main repository would add clutter. You even had a dist directory for caddy v1. Did you make bad experience with it? :)

EDIT: The PR is fine for me. You are free to merge it.

[mholt]

We build all packages from source in respect to reproducible builds. All other distributions do it the same way.

Our build server builds from source too -- doesn't matter who or what does the compiling, as long as you can verify the result.

The point of Caddy is that builds aren't the same -- they are customized based on the features the user needs. So all the packaging systems that require the same build are working against their users here.

We don't want that you put every plugin into that repository. Just all necessary files for basic usage should be in one repository and I don't see how a dist directory in the main repository would add clutter. You even had a dist directory for caddy v1. Did you make bad experience with it? :)

Yeah, it was a mistake. Possible distribution platforms are almost as numerous as possible plugins, so to avoid growing pains, I want to separate them out. Remember that the Caddy repo is used as a library -- not a main command -- by all its plugins and other developers, so putting all the distribution resources, which may grow nearly unbounded, in the main repo is not a good idea.

EDIT: The PR is fine for me. You are free to merge it.

K, will do

[anthraxx]

frankly i personally disagree here. software should be closed for modification and open for extension which means you should be able to easily extend the application with plugins without the need to fundamentally rebuild the whole thing.

For a distro it matters, because distros build from source themselves and in average do not want to simply repackage software. this also gives finer grade of handling security related issues for instance when the go runtime gets another CVE and things need to be rebuilt (just a single out of lots of examples).

Here is not asked to put everything and the kitchen sink into the main repo, but the systemd service files and the startup pages which every basic distribution of caddy shall have.

[mholt]

frankly i personally disagree here. software should be closed for modification and open for extension which means you should be able to easily extend the application with plugins without the need to fundamentally rebuild the whole thing.

Well, sorry, but take that up with the Go project, that's not something I can solve. Like it or not, Caddy is customized per-build.

For a distro it matters, because distros build from source themselves

That's a choice they've made which limits their flexibility and adoption and makes things more difficult.

Here is not asked to put everything and the kitchen sink into the main repo, but the systemd service files and the startup pages which every basic distribution of caddy shall have.

From the perspective of a Linux distro maintainer, this of course makes sense, but remember that the world of distribution is far and wide, and extends beyond just systemd. For example, just the other day someone was asking about an official Ansible module: https://caddy.community/t/ansible-module-for-caddy-v2/6974 -- and there will be many more distributions to support as the project grows.

[anthraxx]

This is simply not true but a choice of caddy itself. Go supports modular plugins as shared loadables. On top there are many ways to achieve modular plugins by exposing functionality to a scripting interface like Lua just to pick a random example or via callables that do IPC. Please don't blame go exclusively for a design decision.

[mholt]

This is simply not true but a choice of caddy itself. Go supports modular plugins as shared loadables.

You obviously don't know much about this. :)

Currently plugins are only supported on Linux and macOS. (pkg/plugin docs)

This alone immediately disqualifies them, since Caddy is a cross-platform server. I know you only care about Linux, but we cannot.

Go plugins bite more than they help. Go plugins require the same version of the Go runtime.. Go plugins cannot be unloaded, which is essential in Caddy where configuration is so dynamic. Go plugins don't play well with vendoring. The list goes on, as well as experiences which are less than pleasant. It's a difficult problem, but the answer to the distribution problem is not "use Go plugins" -- it's for distribution systems to get their act together and support the users that want their required software to be distributed on its platform.

On top there are many ways to achieve modular plugins by exposing functionality to a scripting interface like Lua just to pick a random example or via callables that do IPC.

Oh please, as if we haven't looked into embedded scripting languages. Lua has a high VM overhead -- we tested many implementations -- and Go interpreters aren't flexible enough for what plugins require. This is performance-critical software, mind you -- not a toy. Caddy 2 will eventually support Starlark (so far, anyway), a non-Turing-complete scripting language that has about half the overhead of NGINX+Lua once optimized, but because it's not Turing complete it's not a satisfactory replacement to native Go packages.

Please don't blame go exclusively for a design decision.

Oh don't worry, I'm not. I'm blaming the packaging systems and distributions as well.

Please, do more research, and have more experience with Go, before you try to brush this aside as someone else's problem, or armchairing non-solutions that you think are easy.