Feature: Dashboard page

[felixmosh]

There are several requests for a single place that will summarize queues status, number of fails, etc...
Lets make a list of requirements from this page.

Requirements:

1. Numbers of jobs per each queue

Design inspirations

Related to #252 #178 #163

[Deadly0]

@felixmosh exprected to see dashboard looks like it was in version 0.x where were list of all queues and task statuses in one page.

[PengWeibin-Mars]

@felixmosh Is it possible to use Chart.js (https://www.chartjs.org/docs/latest/samples/bar/vertical.html) as the UI of the dashboard

[felixmosh]

@felixmosh Is it possible to use Chart.js (https://www.chartjs.org/docs/latest/samples/bar/vertical.html) as the UI of the dashboard

Thank you for the suggestion, looks like a cool chart lib :]

[PengWeibin-Mars]

Hey @felixmosh ,When is this feature expected to go live

[felixmosh]

I don't have an estimation time, this is an open source project, You can contribute to it :]

Any PR regarding this feature is welcomed

[Jivings]

I would prefer proper bar charts rather then wheels to show things like response/execution times and throughput during a day.

I might have some time to work on this as I will definitely make use of it!

[nojacko]

The original dashboard was useful. Could we bring that back with each category linking to appropriate queue's tab?

[felixmosh]

Dashboard page should be split into 2 areas, a global stats & per queue stats.

The main purpose of this page should be a quick view of the entire queues. (maybe suitable to show on monitors).

[wootwoot1234]

Are there any updates on this? I would love to see some graphs of my jobs :)

[felixmosh]

@wootwoot1234 actually I didn't started it yet... there are some other features that are in line.

Can you make a simple PR, and we will evolve it from there?

[wootwoot1234]

@felixmosh sure, I would be happy to play with it a little. Can you help me get started by creating a branch with a new dashboard/metrics tab for each queue? Then I'll add the graphs.

You want to use chart.js? I don't personally have a favorite library for charts.

[mrfrase3]

Just thought I might throw in my two cents, I like the way drone.io presents build stats:

each run is a column, with the run time as the height, and the status as the color.

I suppose my gripe with other graph insights is that they spread duration and status over a time graph, which can become useless because of the time scale different tasks can run at (looking at you aws lambda). The example above answers the main question, are they mostly currently running?

[wootwoot1234]

@mrfrase3 I think that design is great.

I agree, I want an easy and quick way to tell if everything is running smoothly.

I would be great to also have a way to see the failures on all jobs. A way to at a glance, see if any queues have failing jobs and how many.

[felixmosh]

@mrfrase3 thank for sharing this screenshot 🙏🏼
We need to think how to split it between queues since runs=jobs in our case.
If we will be able to make such a design I'm free to implement it.

[mrfrase3]

@felixmosh I'd imagine it doesn't need to be as big, Google cloud implements a similar one that they often put inline where it's relevant:

[felixmosh]

I will publish some basic dashboard page which will solve most requested feature, we will be able to refactor & improve it over time...

WDYT?

[wootwoot1234]

That's great, I'll check it out when you publish it and give you my feedback. Thanks!

…

On Sun, Apr 23, 2023 at 1:37 PM Felix Mosheev ***@***.***> wrote: I will publish some basic dashboard page which will solve most requested feature, we will be able to refactor & improve it over time... WDYT? [image: image] <https://user-images.githubusercontent.com/9304194/233858480-69a230cf-7fa5-4c43-a6e4-be2d5d1044c2.png> — Reply to this email directly, view it on GitHub <#258 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADGOGM5OYQEGWGNQN7VSWDXCVZFJANCNFSM43NKCWTA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[felixmosh]

It is available in v5.1.1

[jaroslaw-weber]

it's really good, thanks

[alko89]

Thinking of migrating from Taskforce dashboard. One thing worth mentioning - the metrics from the issue description have to be enabled per Queue Worker.

https://docs.bullmq.io/guide/metrics

Peeking into Redis these metrics seem to be available in bull:<QUEUE_NAME>:metrics

[felixmosh]

Hi @alko89, can you open a separate issue for this.
Please provide the usage of these metrics in Taskforce dashboard.

[alko89]

@felixmosh sure thing. Thought this issue includes this. Basically the entire bottom part of the screenshot in the issue description seems to get the data from the metrics endpoint (Response time, Process time and Throughput), these dials are empty if metrics are not enabled. I'm also seeing this graph which also gets the data from metrics.