Add view and job table enhancements for improved observability

[maxschridde1494]

Changes include:

Additional que_jobs.first_run_at column
The first_run_at column will store the timestamp with time zone (timestamptz) of the initial scheduled execution time for jobs. The column default is set to now, but realistically this will always default to the initial run_at value. This enhancement helps re-trace the execution of jobs when reviewing failures.

NEW que_jobs_ext view
This view extends the functionality of the job management system by providing an enriched view of jobs. It combines data from the que_jobs table and the pg_locks table to present a comprehensive overview of jobs, including their status and locking details. Here is an example SQL command highlighting usage of the new view:

SELECT
  count(*) FILTER(where status='scheduled') as scheduled
  , count(*) FILTER(where status='queued') as queued
  , count(*) FILTER(where status='running') as running
  , count(*) FILTER(where status='failed') as failed
  , count(*) FILTER(where status='errored') as errored
  , count(*) FILTER(where status='completed') as completed
FROM que_jobs_ext

We have used these new enhancements to power some of our admin dashboards. Below is a video of one such example.

[ZimbiX]

Thanks for the contribution!

Interesting idea. This does sound useful, but I'm not entirely sure the view needs to be in Que itself. I'll have a think, but I'm leaning towards adding it.

Could you squash the commits please?

[hlascelles]

@maxschridde1494 We love to look of that dashboard. Can it be released as a gem or similar?

[hlascelles]

Linking bit about "first run at" extra data: #383 (comment)

[maxschridde1494]

@ZimbiX Thanks for the feedback. I went ahead and committed the change requests and squashed the commits. Re race conditions in the specs, I removed the sleep call in the run method and refactored the specs. I believe this removed the race conditions in most (if not all) the specs. For the two specs that test the locker info getting pulled in and the running status getting correctly set when lock ids are present, I referenced the locker_spec.rb and implemented the pattern used there. Please let me know if you have any feedback for this implementation. Thanks!

[maxschridde1494]

@maxschridde1494 We love to look of that dashboard. Can it be released as a gem or similar?

Thanks for the positive feedback! We are currently working on packaging the new dashboard and will let you know as soon as it is available.

[maxschridde1494]

@maxschridde1494 We love to look of that dashboard. Can it be released as a gem or similar?

Thanks for the positive feedback! We are currently working on packaging the new dashboard and will let you know as soon as it is available.

@hlascelles We are happy to announce the release of create-groovestack.

create-groovestack is a starter kit utility that generates a complete Groovestack application that you can run immediately. Groovestack is a fullstack application framework consisting of PostgreSQL, Rails, GraphQL and React-Admin. Here at Talysto, we have used Groovestack for many of our scalable applications across various industries.

create-groovestack includes several Groovestack CORE modules, including CORE Jobs, the module used in the demo above.