nixos/mautrix-telegram: add module

[pacien]

Motivation for this change

This PR adds a module for mautrix-telegram.
The module has been tested and is confirmed to work.

A patch has been submitted and merged upstream (mautrix/telegram#332) to allow API tokens to be supplied securely through environment variables instead of being written to the world-readable nix store.

The module is based on #60151.

Blockers

• Safe Telegram API token handling: mautrix-telegram: 0.5.2 -> 0.6.0 #63947
• Removal of app service registration (to protect the AS' token)
• Serialisation of the config attribute set which contains mkOverride values (nixos/mautrix-telegram: add module #63589 (comment)) Will be solved later by migrating to the types and writers defined in RFC42.
• Rename services.mautrix-telegram.config to services.mautrix-telegram.settings?
• Set logging.loggers.aiohttp.level = "WARNING" instead of "INFO" which leaks the AS token in the logs by default
• No config rewrite: Catch exception when overwriting config mautrix/python#11

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Assured whether relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

CC: @Vskilet, @Ma27

[Vskilet]

Tested and approved :-)

[pacien]

Here's a configuration snippet that might be useful to reviewers:

let
  telegramSnapshot = builtins.fetchTarball {
    url = https://github.com/pacien/nixpkgs/archive/fa7701925048ee78af792b274f4619d426c56561.tar.gz;
  };

in {
  imports = [
    "${telegramSnapshot}/nixos/modules/services/misc/mautrix-telegram.nix"
  ];

  services.matrix-synapse = {
    enable = true;
    app_service_config_files = [
      # The registration file is automatically generated after starting the appservice for the first time.
      # cp /var/lib/mautrix-telegram/telegram-registration.yaml /var/lib/matrix-synapse/
      # chown matrix-synapse:matrix-synapse /var/lib/matrix-synapse/telegram-registration.yaml
      "/var/lib/matrix-synapse/telegram-registration.yaml"
    ];
    # ...
  };

  services.mautrix-telegram = {
    enable = true;
    environmentFile = /etc/secrets/mautrix-telegram.env; # file containing the appservice and telegram tokens
    settings = {
      homeserver = {
        address = "http://localhost:8008";
        domain = "domain.tld";
      };
      appservice = {
        provisioning.enabled = false;
        id = "telegram";
        public = {
          enabled = true;
          prefix = "/public";
          external = "http://domain.tld:8080/public";
        };
      };
      bridge = {
        relaybot.authless_portals = false;
        permissions = {
          "@someadmin:domain.tld" = "admin";
        };
      };
    };
  };
}

[infinisil]

No need to put this in apply, you can instead set the defaults by doing

{
  config.services.mautrix-telegram.config = {
    appservice = {
      database = "sqlite:///${dataDir}/mautrix-telegram.db";
      hostname = mkDefault "0.0.0.0";
      # ...
    };
  };
}

This makes merging values work correctly and allows a lot more control over what should be defaulted and how. The downside is that such defaults aren't visible in the docs, but this should be fixed in the module system eventually.

See also this section which talks about when to use mkDefault or not, from my RFC about such NixOS options. That section also recommends making the default as minimal as possible to get a working service, which I recommend you do here too (if you haven't already).

[pacien]

I changed the module accordingly.

It seems however that mkDefault doesn't play well with toJSON:

config.services.mautrix-telegram.config.logging = {
        version = 1;

        formatters.precise.format = "[%(levelname)s@%(name)s] %(message)s";

        handlers.console = {
          class = "logging.StreamHandler";
          formatter = "precise";
        };

        loggers = {
          mau.level = mkDefault "INFO";
          telethon.level = mkDefault "INFO";
          aiohttp.level = mkDefault "INFO";
        };

        root = {
          level = mkDefault "INFO";
          handlers = [ "console" ];
        };
};

toJSON generates:

logging: {formatters: {precise: {format: '[%(levelname)s@%(name)s] %(message)s'}},
    handlers: {console: {class: logging.StreamHandler, formatter: precise}}, loggers: {
        aiohttp: {level: {_type: override, content: INFO, priority: 1000}}, mau: {
            level: {_type: override, content: INFO, priority: 1000}}, telethon: {
            level: {_type: override, content: INFO, priority: 1000}}}, root: {handlers: [
            console], level: {_type: override, content: INFO, priority: 1000}}, version: 1}

(note the unwanted _type, content and priority fields).

[pacien]

It also appear that the attribute set isn't recursively merged.
Nested definitions outside of the module override the whole parent attribute set.

For instance, setting services.mautrix-telegram.logging.root.level = "DEBUG" causes all the default attributes defined in config.services.mautrix-telegram.config.logging inside the module to be forgotten.

[pacien]

types.attrs doesn't merge recursively:

• "Telegraf extraConfig is not recursively merged": Telegraf extraConfig is not recursively merged #32217

• nixpkgs/lib/types.nix

  Line 226 in 60e8fcf

  merge = loc: foldl' (res: def: mergeAttrs res def.value) {};

• nixpkgs/lib/trivial.nix

  Line 79 in 6cf583c

  /* Merge two attribute sets shallowly, right side trumps left

[infinisil]

Yeah so you'd need to use a fully specified type as described in the RFC for this to work properly, forgot about that. So for now with types.attrs just setting all values without a mkDefault might be best. Unfortunately when users set certain values, all the defaults under that attr get cleared, but that's been the case for all options that use types.attrs previously too, so I'd say we just use that anyways for now.

I should note in the RFC that when it's merged we should go over this module to specify the type properly and set the mkDefault's.

[nyanloutre]

mautrix-telegram 0.6.0 has been merged in master

[pacien]

This should be ready for another review.
I'm currently using this module and it works fine with mautrix-telegram 0.6.1.

[pacien]

Rebased and amended to remove a no-longer necessary workaround (mautrix/python#11).

[f4814]

Hi, would love to use this. What is the status of this?

[pacien]

It's usable as is (with the snippet provided in #63589 (comment)) but we're waiting on RFC42 to be fully implemented to have the right JSON attribute set type.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/matrix-use-appservice-registration-files/9613/1

[eyJhb]

I am currently trying to rework this, to have a central services.mautrix.* module, as there are ~7 Mautrix services atm. 6 of them written in Python using the exact same interface. And it seems redundant to have a module for each mautrix service.

I have done a initial PoC over here #110404 , where it requires less user interaction and can run entirely just by the config without any copy-pasting around :)