Allow plugin authors to register housekeeping tasks

[jsenecal]

NetBox version

v4.0.7

Feature type

New functionality

Proposed functionality

NetBox includes a housekeeping management command that usually runs nightly. This command handles:

• Clearing expired authentication sessions from the database
• Deleting older changelog records
• Deleting older job result records
• Checking for new NetBox releases

It would be great if plugin authors could register additional tasks to run by the housekeeping process without having to write their own management commands and setup mechanisms to run those periodically.

Use case

I often have to write periodic tasks to alter model instances in my plugins, reacting to dates, sending emails etc. and having to setup the whole management commands and related mechanisms to run those periodically is not really convenient, even more so considering that NetBox already has mechanisms built in to do so.

Database changes

None whatsoever

External dependencies

None

[alehaa]

The background jobs framework of #15692 will include the ability to schedule system jobs, i.e. housekeeping or other tasks that run periodically in the background. While the plugin still requires a special management command for interactive execution (optional), periodic execution is covered by this framework.

[jeremystretch]

I think ultimately we probably want to do away with the separate housekeeping task and integrate everything using the job's framework, particularly in light of @alehaa's recent work.

[alehaa]

The JobRunner framework now supports the implementation of any kind of task, including housekeeping. However, the automatic registration of jobs on plugin load did not make it into the PR because the implementation is not stable enough for production. I suggest this FR to focus on this remaining issue, so that even housekeeping tasks can be set up automatically and solve this problem.

Here are the latest findings:

• Calling JobRunner.enqueue_once() directly from the ready() method of AppConfig logs a warning because the database is accessed during plugin setup. The Django documentation explicitly warns against this use case, as the ready() method is called every time a plugin is loaded, even if it is only for the nbshell management command.
• Using the connection_created signal solves the console warning. However, this triggers the JobRunner setup even more often, e.g. because the database is connected before each job execution.

So the problem for now is to find a way to automatically trigger the setup method outside of the request-response cycle that is called only once or say every few hours.

[alehaa]

How about adding the auto-registration to the beginning of the rqworker management command? It would only be called once for each worker started - not for the frontend and not for each database connection. After updates, restarting the rqworker would trigger auto-registration again.

I'd implement this using a new method in AppConfig (e.g. enqueue_job_runner()) being called by rqworker.handle() for each plugin. An alternative would be using a signal, which avoids a new method but could obfuscate the logic.

[jsenecal]

How about adding the auto-registration to the beginning of the rqworker management command? It would only be called once for each worker started - not for the frontend and not for each database connection. After updates, restarting the rqworker would trigger auto-registration again.

I'd implement this using a new method in AppConfig (e.g. enqueue_job_runner()) being called by rqworker.handle() for each plugin. An alternative would be using a signal, which avoids a new method but could obfuscate the logic.

This feels like a decent approach.
Netbox also has existing mechanisms working with "registries"; we could have a "Housekeeping tasks" registry which would be handled in a similar fashion by the worker and would provide a standardized way to register periodic tasks.

[alehaa]

I'm not sure if this is possible. Usually the registry handles static variables like a list of middleware to load. To register a JobRunner, its enqueue_once() method needs to be called and its arguments can vary depending on the plugin's implementation. Therefore I think some kind of method needs to be called for each plugin, but of course it can reside in the AppConfig class along with other registry-related variables. In this method, plugins could implement the logic to setup the JobRunner, i.e. by loading configuration or scheduling task depending on other NetBox objects, etc.

[alehaa]

@jeremystretch can you assign this to me so I can work on a PR? Maybe we can still make it into 4.1.

[alehaa]

@jeremystretch did you have any chance to review this? This would be the last missing piece to add auto-scheduling background tasks, which didn't make it into the JobRunner framework.

[DanSheps]

@alehaa I am not so sure about the approach, I think something more agnostic that can be called after initialization of all apps are done but not specifically tied jobs or RQ might be a better approach.

That said, I don't know if there is a viable solution for this as there is no "apps_ready" signal emitted by django. Your solution might be the most viable right now.