Tool for monitoring Arc GPU VRAM usage on Ubuntu?

[kyang-06]

Hi, thank you for excellent efforts on supporting LLM running on Arc GPU.
Do you know if there exists any tool for this feature? Sadly I only found their answer is no.

[rnwang04]

You can use xpu-smi tool to monitor your Arc GPU VRAM usage.
Install it by sudo apt-get install xpu-smi
Then monitor your Arc GPU VRAM usage by sudo xpu-smi stats -d 0, you will see something like:

+-----------------------------+--------------------------------------------------------------------+
| Device ID                   | 0                                                                  |
+-----------------------------+--------------------------------------------------------------------+
| GPU Utilization (%)         | 0                                                                  |
| EU Array Active (%)         |                                                                    |
| EU Array Stall (%)          |                                                                    |
| EU Array Idle (%)           |                                                                    |
|                             |                                                                    |
| Compute Engine Util (%)     | 0; Engine 0: 0, Engine 1: 0, Engine 2: 0, Engine 3: 0              |
| Render Engine Util (%)      | 0;                                                                 |
| Media Engine Util (%)       | 0                                                                  |
| Decoder Engine Util (%)     |                                                                    |
| Encoder Engine Util (%)     |                                                                    |
| Copy Engine Util (%)        | 0;                                                                 |
| Media EM Engine Util (%)    |                                                                    |
| 3D Engine Util (%)          |                                                                    |
+-----------------------------+--------------------------------------------------------------------+
| Reset                       |                                                                    |
| Programming Errors          |                                                                    |
| Driver Errors               |                                                                    |
| Cache Errors Correctable    |                                                                    |
| Cache Errors Uncorrectable  |                                                                    |
| Mem Errors Correctable      |                                                                    |
| Mem Errors Uncorrectable    |                                                                    |
+-----------------------------+--------------------------------------------------------------------+
| GPU Power (W)               | 45                                                                 |
| GPU Frequency (MHz)         | 1500                                                               |
| GPU Core Temperature (C)    | 52                                                                 |
| GPU Memory Temperature (C)  |                                                                    |
| GPU Memory Read (kB/s)      |                                                                    |
| GPU Memory Write (kB/s)     |                                                                    |
| GPU Memory Bandwidth (%)    |                                                                    |
| GPU Memory Used (MiB)       | 26                                                                 |
| GPU Memory Util (%)         | 0                                                                  |
| Xe Link Throughput (kB/s)   |                                                                    |
+-----------------------------+--------------------------------------------------------------------+

[kyang-06]

Thank you for soon reply!
But watch -n 1 xpu-smi -d 0 is always showing GPU memory used = 0.
I also tried pti-gpu/sysmon, showing running process unknown, as same as this

[kyang-06]

Sorry to bother, solved by using root privilege for xpu-smi. even though sysmon is still unavailable.
Thank you for the help!

[hkvision]

Closing this issue. Feel free to tell us if you have further issues :)