matrix-appservice-discord: init at 0.5.2

[pacien]

Motivation for this change

This PR adds a package and a module for the matrix-appservice-discord service.

Blockers / Things to get done

• Broken node packages due to bad tarballs: Broken node packages due to bad tarballs #62632
• Compilation fails with latest Typescript: Compilation fails with latest Typescript matrix-org/matrix-appservice-discord#487
• Override config with environment variables: Override config with environment variables matrix-org/matrix-appservice-discord#485
• Wait for upstream to tag and release 0.5.1 (containing the patches above).
• Removal of app service registration (to protect the AS' token)
• Serialisation of the config attribute set which contains mkOverride values (nixos/mautrix-telegram: add module #63589 (comment)) Will be solved later by migrating to the types and writers defined in RFC42.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Assured whether relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

[Ma27]

Thanks a lot for doing this! This was one of the matrix bridges I wanted to package, but I didn't have time to :)

[Ma27]

Thanks a lot for fixing the stuff I've addressed in my review! I'd suggest we wait for 0.5.1 tobe tagged, I'll hopefully be able to test the bridge during next week, after that this PR should be mergable 👍

[pacien]

@GrahamcOfBorg build matrix-appservice-discord

EDIT: wrong package

[pacien]

Amended to include admin tools for the appservice.

[pacien]

Here's a configuration snippet that might be useful for reviewers:

let
  discordSnapshot = builtins.fetchTarball {
    url = https://github.com/pacien/nixpkgs/archive/35d6ee8bf5f2197729bbc35509ffa9bf40b32103.tar.gz;
  };
  discordPkgs = import discordSnapshot { };

in {
  imports = [
    "${discordSnapshot}/nixos/modules/services/misc/matrix-appservice-discord.nix"
  ];

  nixpkgs.config.packageOverrides = super: {
    inherit (discordPkgs) matrix-appservice-discord;
  };

  services.matrix-synapse = {
    enable = true;
    app_service_config_files = [
      # The registration file is automatically generated after starting the appservice for the first time.
      # cp /var/lib/matrix-appservice-discord/discord-registration.yaml /var/lib/matrix-synapse/
      # chown matrix-synapse:matrix-synapse /var/lib/matrix-synapse/discord-registration.yaml
      "/var/lib/matrix-synapse/discord-registration.yaml"
    ];
    # ...
  };

  services.matrix-appservice-discord = {
    enable = true;
    environmentFile = /etc/keyring/matrix-appservice-discord/tokens.env;
    settings = {
      bridge = {
        domain = "test.tld";
        homeserverUrl = "https://public.endpoint.test.tld";
      };
    };
  };
}

[pacien]

This should be ready for review.
I'm currently using this module and package and everything seems to work fine.

[Ma27]

Just deployed this to my matrix homeserver, created a discord bot and connected it to a discord guild where I'm admin. Unfortunately I didn't manage to bridge messages, I experienced the following issues by following (https://github.com/Half-Shot/matrix-appservice-discord#setting-up-discord):

• Getting the authorization URL (using the addbot script) doesn't work as it tries to fetch the discord client_id from the config (the value is empty by default here) and doesn't care about environment variables. Also it expects a config.yaml in PWD, so I had to manually copy config.json to /tmp and modify it there. While the config.yaml thing is IMHO an upstream bug (however we should report/fix this), we should probably add some sort of systemd unit or wrapper script that ensures that you get that URL without fiddling around too much on the remote system.

• After allowing the bot to access the discord channel (with a rather high set of permissions), I was unable to join any discord servers, additionally the following things appeared in the log:

  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]: Oct-26 12:32:42.464 [UserSync] error: Failed to update user's profile Error: Error parsing server response: SyntaxError: Unexpected token < in JSON at position 0
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at /nix/store/jjdyzmplk1mfbp46kw9dn2a7nvlbv5f3-node_matrix-appservice-discord-0.5.2/lib/node_modules/matrix-appservice-discord/node_modules/matrix-js-sdk/src/http-api.js:834:23
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at handlerFn (/nix/store/jjdyzmplk1mfbp46kw9dn2a7nvlbv5f3-node_matrix-appservice-discord-0.5.2/lib/node_modules/matrix-appservice-discord/node_modules/matrix-js-sdk/src/http-api.js:775:21)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at Request._callback (/nix/store/jjdyzmplk1mfbp46kw9dn2a7nvlbv5f3-node_matrix-appservice-discord-0.5.2/lib/node_modules/matrix-appservice-discord/node_modules/matrix-appservice-bridge/lib/components/client-fact>
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at Request.self.callback (/nix/store/jjdyzmplk1mfbp46kw9dn2a7nvlbv5f3-node_matrix-appservice-discord-0.5.2/lib/node_modules/matrix-appservice-discord/node_modules/request/request.js:185:22)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at Request.emit (events.js:198:13)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at Request.<anonymous> (/nix/store/jjdyzmplk1mfbp46kw9dn2a7nvlbv5f3-node_matrix-appservice-discord-0.5.2/lib/node_modules/matrix-appservice-discord/node_modules/request/request.js:1161:10)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at Request.emit (events.js:198:13)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at IncomingMessage.<anonymous> (/nix/store/jjdyzmplk1mfbp46kw9dn2a7nvlbv5f3-node_matrix-appservice-discord-0.5.2/lib/node_modules/matrix-appservice-discord/node_modules/request/request.js:1083:12)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at Object.onceWrapper (events.js:286:20)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at IncomingMessage.emit (events.js:203:15)
  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]:     at endReadableNT (_stream_readable.js:1129:12)
  

• Also, we should probably drop the additional SQLite databases:

  Oct 26 13:33:02 matrix matrix-appservice-discord[13863]: Oct-26 13:33:02.540 [DiscordAS] warn: [DEPRECATED] The room store is now part of the SQL database.The config option roomStorePath no longer has any use.
  

[pacien]

• Getting the authorization URL (using the addbot script) doesn't work as it tries to fetch the discord client_id from the config (the value is empty by default here) and doesn't care about environment variables.

All this script does is a bitwise or and a string concatenation to generate a link which can be obtained on Discord's website. Though it's indeed quite inconvenient that the scripts ignore the environment variables. I opened an issue for this: matrix-org/matrix-appservice-discord#567

Also it expects a config.yaml in PWD, so I had to manually copy config.json to /tmp and modify it there. While the config.yaml thing is IMHO an upstream bug (however we should report/fix this),

That's matrix-org/matrix-appservice-discord#507. This should be fixed (matrix-org/matrix-appservice-discord#508) for the next main release of the bridge.

we should probably add some sort of systemd unit or wrapper script that ensures that you get that URL without fiddling around too much on the remote system.

All script tools are already wrapped. The configuration loading issues need to be fixed upstream though.

• After allowing the bot to access the discord channel (with a rather high set of permissions), I was unable to join any discord servers, additionally the following things appeared in the log:

  Oct 26 12:32:42 matrix matrix-appservice-discord[11993]: Oct-26 12:32:42.464 [UserSync] error: Failed to update user's profile Error: Error parsing server response: SyntaxError: Unexpected token < in JSON at position 0
  [SNIP]
  

This probably comes from a configuration error. This could also be a new upstream bug :/

• Also, we should probably drop the additional SQLite databases:

  Oct 26 13:33:02 matrix matrix-appservice-discord[13863]: Oct-26 13:33:02.540 [DiscordAS] warn: [DEPRECATED] The room store is now part of the SQL database.The config option roomStorePath no longer has any use.
  

The two additional databases are deprecated since 0.5.0 but the parameters are still required. The bridge refuses to start without a value for those due to an upstream bug: matrix-org/matrix-appservice-discord#490

[pacien]

Rebased.