Skip to content
This repository has been archived by the owner on Sep 8, 2020. It is now read-only.

ZFS ARC stats for Linux, FreeBSD, Darwin, & Solaris #920

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

ZFS ARC stats for Linux, FreeBSD, Darwin, & Solaris #920

wants to merge 9 commits into from

Conversation

overhacked
Copy link

@overhacked overhacked commented Jul 6, 2019

Added a new Meter type that displays ZFS ARC stats like FreeBSD's top(1).
Supported on FreeBSD, Darwin/OS X/macOS, and Linux. FreeBSD & Darwin are nearly identical. Doesn't break or affect other OSes, as it is only included in Platform_meterTypes for supported OSes. There is also a potential buffer overflow bug fix in the first commit.

Could have resulted in a buffer overflow if the
FreeBSD kernel returned more bytes than expected.
@overhacked overhacked changed the title ZFS ARC stats (currently FreeBSD; plan to extend to Linux et al) WIP: ZFS ARC stats (currently FreeBSD; plan to extend to Linux et al) Jul 6, 2019
@overhacked overhacked changed the title WIP: ZFS ARC stats (currently FreeBSD; plan to extend to Linux et al) WIP: ZFS ARC stats for FreeBSD & Linux Jul 7, 2019
@overhacked overhacked changed the title WIP: ZFS ARC stats for FreeBSD & Linux ZFS ARC stats for FreeBSD, Darwin, & Linux Jul 7, 2019
@overhacked overhacked changed the title ZFS ARC stats for FreeBSD, Darwin, & Linux ZFS ARC stats for Linux, FreeBSD, Darwin, & Solaris Jul 8, 2019
New meter displays same ARC stats as FreeBSD top(1).
Can be extended to other platforms that support ZFS.

Pulling kstat.zfs.misc.arcstats.c_max as the meter
total, so the meter has a meaningful value to work
up to.

The Text meter displays, first, the maximum
ARC size (Meter.total), then second, the total
ARC used, using the difference between Meter.maxItems
and Meter.curItems to "hide" the used value from the
Bar and Graph drawing functions by using an index
in Meter.values[] that is beyond curItems - 1, but
less than maxItems - 1.
If no pools are imported (ARC size == 0) or the
ZFS module is not in the kernel (/proc/spl/kstat/zfs/arcstats
does not exist), then the Meter reports "Unavailable".
Darwin and FreeBSD export zfs kstats through the
same APIs, so moving functions into a common file.
@ghost
Copy link

ghost commented Jul 19, 2019

...I love it when my to-do list gets shorter on its own. Thank you!

@Mno-hime
Copy link

Works great on OpenIndiana 2019.04 in global and non-global zones. We will ship it via sources from @ninefathom soon. Thanks, @overhacked!

@Mno-hime
Copy link

FYI: #937.

@overhacked
Copy link
Author

Should be reasonably simple. I think visually it would need to be a different meter type. @ninefathom, will you still merge if I add another "ZFS ARC Compression" meter type?

@ghost
Copy link

ghost commented Aug 23, 2019 via email

openzfs_sysctl_init() now returns void instead of int.
The ZfsArcStats->enabled flag is set inside the init function
now, instead of having to be set from its return value.
Preparation for more flag setting in Compressed ARC commit.

ZfsArcMeter_readStats() added and all Meter->values[] setting
moved to it, eliminating duplicated code in
{darwin,freebsd,linux,solaris}/Platform.c.
@overhacked
Copy link
Author

Resolving CI issues…

@overhacked
Copy link
Author

Tests out on FreeBSD, Linux, OmniOS, and Darwin.

@ghost
Copy link

ghost commented Sep 3, 2019

@Mno-hime release with the compressed ARC stats in the usual place, as 220-sunos_11-p3

@overhacked
Copy link
Author

Closes #937

@despair86
Copy link

despair86 commented Feb 14, 2020

does this separate the ZFS ARC size as disk cache from the rest of resident core set? As of now default settings allow the ARC to fill all unused core and this condition is impossible to tell apart in current htop:
image
#976 only fixes ZoL proper. this is ZoLo[SI]

@overhacked
Copy link
Author

@despair86, I didn't mess with the code that calculates other types of memory usage in this PR, but I agree that it is more useful to have htop report ARC as cache usage. That code already exists for the FreeBSD platform but not for any other platforms.

I could add the tweaking of reported usage for Linux and Solaris/Illumos as a part of this PR. @ninefathom, any feedback? Would you rather see those changes in a separate PR, maybe an expansion of #976?

@Harvie
Copy link

Harvie commented Aug 6, 2020

This is needed so badly. There are several PRs addressing this issue, We should decide which one is most useful:

#153 #920 #976 #937

@natoscott
Copy link
Collaborator

Merged here: htop-dev/htop@9a55efc

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants