Failed to collect some domain info while migrating vm

[laushunyu]

I use openstack 23.2, and deployed prometheus-libvirt-exporter by apt install prometheus-libvirt-exporter (version 1.6.0).

I found some journal err log while live-migrate vm:

Nov 15 07:49:48 xfusion-3 prometheus-libvirt-exporter[4037]: ts=2024-11-15T07:49:48.773Z caller=prometheus-libvirt-exporter.go:461 level=warn warn="failed to get DomainBlockStats" domain=instance-00001045 msg="Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePrepare3Params)"
Nov 15 07:49:48 xfusion-3 prometheus-libvirt-exporter[4037]: ts=2024-11-15T07:49:48.773Z caller=prometheus-libvirt-exporter.go:445 level=warn warn="failed to collect some domain info" domain=instance-00001045 msg="Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePrepare3Params)"
Nov 15 07:49:48 xfusion-3 prometheus-libvirt-exporter[4037]: ts=2024-11-15T07:49:48.805Z caller=prometheus-libvirt-exporter.go:575 level=warn warn="failed to get DomainMemoryStats" domain=instance-00001045 msg="Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePrepare3Params)"
Nov 15 07:49:48 xfusion-3 prometheus-libvirt-exporter[4037]: ts=2024-11-15T07:49:48.805Z caller=prometheus-libvirt-exporter.go:445 level=warn warn="failed to collect some domain info" domain=instance-00001045 msg="Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePrepare3Params)"

Prometheus show there are some failed metric points:

[frittentheke]

This is somewhat of a known issue which I started a discussion about on the ML a while back: https://lists.libvirt.org/archives/list/users@lists.libvirt.org/thread/PXKLIU7DPF6MDGEHWQYX3KLGCDNRAIW3/

[swenske]

https://lists.libvirt.org/archives/list/users@lists.libvirt.org/message/USIPK3CHB2O35767UDR3CQMBM24HOUKM/

We shall be looking to using this to improve the exporters robustness.
I'll drop an update once I got to it

👋 @frittentheke

Did you find any leads?

Have a nice day and thank you!