feat: Add support for webhook listeners

[come-nc]

Summary

Add support for registering webhooks as listeners of events.

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[ChristophWurst]

Code looks good! How will this be used? Can we assume that apps know their webhook listeners when the app is just registered and the system isn't fully booted yet?

[come-nc]

Code looks good! How will this be used?

Plan is to use that for integration of an external workflow engine, but architecture is not final yet.
One problem is we’d prefer if the webhook gets called only on certains conditions, but then we end up with apps/workflowengine complexity.

Can we assume that apps know their webhook listeners when the app is just registered and the system isn't fully booted yet?

Hum, what is not accessible at that point?
If I want to have an app which search in its database and user settings to know which webhooks to register, will that work?

[ChristophWurst]

Can we assume that apps know their webhook listeners when the app is just registered and the system isn't fully booted yet?

Hum, what is not accessible at that point? If I want to have an app which search in its database and user settings to know which webhooks to register, will that work?

That works. Anything that is provided by other apps might be missing. Say information from a user backend.

If you always want to wire a specific event to webhook listeners it's fine. But the lack of this information could be critical when you need to conditionally register webhooks. I haven't yet understood the full picture of the new automations so this might not be necessary.

https://docs.nextcloud.com/server/latest/developer_manual/app_development/bootstrap.html#nextcloud-20-and-later for my original ideas and thoughts about the two step register/boot process.

[come-nc]

Webhooks are moved to an application with registrations in DB instead. They will also need to support filtering of events triggerring the webhook.

• Skeleton app
• API to register webhooks
• occ command to list them
• Actually call the webhooks, through background jobs
• Serialize the event to the webhook
• Add a filtering system like https://github.com/Akkroo/PHPMongoQuery/blob/master/src/Akkroo/PHPMongoQuery.php
• Switch to SPDX headers
• Add auth methods
• Encrypt auth info
• Remove core webhook registration
• Validate data passed to the API for webhook registration

[come-nc]

• Store app id from appapi header if available

[come-nc]

To test current version:
1 - register a webhook, for instance:

<?php
function callNextcloudApi(string $endpoint, array $postfields): void {
		$uri = 'http://nextcloud.local/';

		$ch = curl_init();

		curl_setopt($ch, CURLOPT_URL, "$uri/$endpoint");
		curl_setopt($ch, CURLOPT_POST, 1);
		curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($postfields));
		curl_setopt($ch, CURLOPT_HTTPHEADER,
			[
				'Accept:application/json',
				'Content-Type:application/json',
				'OCS-APIRequest: true',
				'Authorization: Basic '. base64_encode('admin:admin')
			]);
		curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);

		// Receive server response ...
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

		$server_output = curl_exec($ch);
		curl_close($ch);

		var_dump($server_output);
}

callNextcloudApi(
	'/ocs/v2.php/apps/webhooks/api/v1/webhooks',
	[
        'httpMethod' => 'POST',
        'uri' => 'http://127.0.0.1/hook.php',
        'event' => 'OCP\\Files\\Events\\Node\\NodeWrittenEvent',
        'eventFilter' => [
          'user.uid' => 'admin',
        ],
        'headers'		=> ['Header1' => 'value1'],
        'authMethod'	=> 'header',
        'authData'		=> ['Header2' => 'value2'],
	]
);

2 - react to webhook calls in your hook.php:

<?php

$filename = '/tmp/record.txt';

function callNextcloudApi($fp, string $endpoint, array $postfields): void {
	try {
		$uri = 'http://nextcloud.local/index.php/';

		$ch = curl_init();

		curl_setopt($ch, CURLOPT_URL, "$uri/$endpoint");
		curl_setopt($ch, CURLOPT_POST, 1);
		curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($postfields));
		curl_setopt($ch, CURLOPT_HTTPHEADER, ['Authorization: Basic '. base64_encode('admin:admin')]);
		curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);

		// Receive server response ...
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

		$server_output = curl_exec($ch);
		curl_close($ch);

		var_dump($server_output);

		if (fwrite($fp, $server_output."\n") === false) {
			die("Cannot write in ($filename)");
		}
	} catch (\Throwable $t) {
		if (fwrite($fp, $t."\n") === false) {
			die("Cannot write in ($filename)");
		}
	}
}

if (is_writable($filename)) {
	if (!$fp = fopen($filename, 'a')) {
		die("Cannot open ($filename)");
	}

	$event = json_decode(file_get_contents('php://input'), associative:true);

	if (fwrite($fp, file_get_contents('php://input').' - '.json_encode($_POST)."\n") === false) {
		die("Cannot write in ($filename)");
	}

	if (isset($event['event']['node']['id']) && isset($event['event']['node']['path']) && isset($event['user']['uid'])) {
		callNextcloudApi(
			$fp,
			'/apps/tables/api/1/tables/2/rows',
			[
				'data' => json_encode(
					[
						5 => $event['event']['node']['id'],
						6 => $event['event']['node']['path'],
						7 => $event['user']['uid'].' - '.($_SERVER['HTTP_HEADER1'] ?? '').' - '.($_SERVER['HTTP_HEADER2'] ?? ''),
						8 => time(),
					]
				)
			]
		);
	}

	fclose($fp);
} else {
	die("Cannot open ($filename) - no permission");
}

(this one needs a /tmp/records.txt writable file, and tables application with a table with id 2 and columns with id 5/6/7/8, adapt to your needs)
3 - You can check registered webhooks with occ webhooks:list

$ occ webhooks:list
+----+-------+--------+------------+---------------------------+----------------------------------------+----------------------+----------------------+------------+----------+
| id | appId | userId | httpMethod | uri                       | event                                  | eventFilter          | headers              | authMethod | authData |
+----+-------+--------+------------+---------------------------+----------------------------------------+----------------------+----------------------+------------+----------+
| 1  |       | admin  | POST       | http://127.0.0.1/hook.php | OCP\Files\Events\Node\NodeWrittenEvent | {"user.uid":"admin"} |                      | none       |          |
| 2  |       | admin  | POST       | http://127.0.0.1/hook.php | OCP\Files\Events\Node\NodeWrittenEvent | {"user.uid":"admin"} | {"Header1":"value1"} | header     |          |
+----+-------+--------+------------+---------------------------+----------------------------------------+----------------------+----------------------+------------+----------+

4 - You can check scheduled jobs with occ background-job:list --class="OCA\Webhooks\BackgroundJobs\WebhookCall"

$ occ background-job:list --class="OCA\Webhooks\BackgroundJobs\WebhookCall"
+-----+-----------------------------------------+---------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id  | class                                   | last_run                  | argument                                                                                                                                                                                        |
+-----+-----------------------------------------+---------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 104 | OCA\Webhooks\BackgroundJobs\WebhookCall | 1970-01-01T00:00:00+00:00 | [{"event":{"class":"OCP\\Files\\Events\\Node\\NodeWrittenEvent","node":{"id":185,"path":"\/admin\/files\/New text file.md"}},"user":{"uid":"admin","displayName":"admin"},"time":1717425671},1] |
+-----+-----------------------------------------+---------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

[come-nc]

Follow-up:

• Extract user filter from eventFilter so that the listener is not even registered depending on logged in user
• Same for group membership

[julien-nc]

Works great! A few minor comments and questions.

[provokateurin]

I'm unsubscribing from this PR for now, please ping me if you need a review of the (Open)API.

[nickvergessen]

@come-nc Please document in https://docs.nextcloud.com/server/latest/developer_manual/app_publishing_maintenance/app_upgrade_guide/upgrade_to_30.html#id1

(all changes in lib/public/)