Warning for long-term load is incorrect

[Pingger]

Versions

• Pi-hole: 5.9
• AdminLTE: 5.11
• FTL: 5.14

Platform

• OS and version: Ubuntu 20.04 Server Core

• Platform: lxd

Expected behavior

"Warnings" being

1. Correct
2. permanently hideable
3. Warnings not being shown without being logged in!

Actual behavior / bug

1. Getting

"Long-term load (15min avg) larger than number of processors: 14.9 > 8
This may slow down DNS resolution and can cause bottlenecks."

Despite
- having 16 Processor-Cores on the Host and only a single one assigned to the container.
- This Warning being completely irrelevant!
- The warning annimation annoying the heck out of me.
2. Displaying the Wrong amount of processor cores
3. Warnings are visible

Steps to reproduce

Steps to reproduce the behavior:

1. Install PI-Hole
2. Open the Admin UI

Debug Token

• URL: https://tricorder.pi-hole.net/efMfHbXk/

Screenshots

Additional context

I thought this would be fixed, but after >3 months this issue still persists.

[yubiuser]

Disable the check
https://docs.pi-hole.net/ftldns/configfile/#check_load

[Pingger]

Does not solve the issue, that warnings are displayed without login!
Also does not resolve wrong information being displayed.

But apparently bugs are still resolved by putting your head into the ground.

[PromoFaux]

Does not solve the issue, that warnings are displayed without login!

This bit may have been overlooked - however I have just tried this myself.. I cannot access the messages page unless it is logged in. There is a hint that there are warnings to review, but one still needs to actually log in to see the warnings.

Also does not resolve wrong information being displayed.

I can't help on this one, @DL6ER, if you have moment free at some point - does the output reported by the OP look right/wrong to you? And is there any kind of useful information OP could provide that might actually help us help them (and/or help them to help themselves)

I thought this would be fixed, but after >3 months this issue still persists.

Have you reported this before?

And please, improve your tone.

[PromoFaux]

"Long-term load (15min avg) larger than number of processors: 14.9 > 8

Despite

• having 16 Processor-Cores

What CPU is in your machine? Is it 16 actual cores, or 8 cores and 16 threads? This is pertinent information that would help in working out if this is indeed a bug or not

[Pingger]

It is a rented Server having an AMD EPYC™ 7702. And yes the contract actually contains 16Cores 16Threads. And htop correctly reports this:

The commented, that "this hadn'd been fixed" was aimed at the issue, that warnings are visible without login, which is just a plain a bad idea, since it would give a potential attacker the information, that and how many warnings were generated. I considered the issue so glaringly obvious, that I indeed did not report on it.

Regarding my attitude: Sorry, you unfortunately received the tail end of a discussion with the nextcloud support team and a previous experience with the pihole community.

[dschaper]

What does lxc info --resources show for the CPU limits?

[dschaper]

It is a rented Server having an AMD EPYC™ 7702. And yes the contract actually contains 16Cores 16Threads. And htop correctly reports this:

Can you confirm this? It seems really odd that the vendor would do 16c/16t, that's a contradiction in itself. I'm pretty sure that they are providing you 8c/16t. HTOP shows threads, not cores.

Can you show where you are seeing the actual warnings and not just the icon when you are not logged in?

[Pingger]

lxc info --resources

CPU (x86_64):
  Vendor: AuthenticAMD
  Name: AMD EPYC 7702P 64-Core Processor
  Caches:
    - Level 1 (type: Data): 64KiB
    - Level 1 (type: Instruction): 64KiB
    - Level 2 (type: Unified): 512KiB
    - Level 3 (type: Unified): 16MiB
  Cores:
    - Core 0
      Frequency: 0Mhz
      Threads:
        - 0 (id: 0, online: true, NUMA node: 0)
    - Core 1
      Frequency: 0Mhz
      Threads:
        - 0 (id: 1, online: true, NUMA node: 0)
    - Core 2
      Frequency: 0Mhz
      Threads:
        - 0 (id: 2, online: true, NUMA node: 0)
    - Core 3
      Frequency: 0Mhz
      Threads:
        - 0 (id: 3, online: true, NUMA node: 0)
    - Core 4
      Frequency: 0Mhz
      Threads:
        - 0 (id: 4, online: true, NUMA node: 0)
    - Core 5
      Frequency: 0Mhz
      Threads:
        - 0 (id: 5, online: true, NUMA node: 0)
    - Core 6
      Frequency: 0Mhz
      Threads:
        - 0 (id: 6, online: true, NUMA node: 0)
    - Core 7
      Frequency: 0Mhz
      Threads:
        - 0 (id: 7, online: true, NUMA node: 0)
    - Core 8
      Frequency: 0Mhz
      Threads:
        - 0 (id: 8, online: true, NUMA node: 0)
    - Core 9
      Frequency: 0Mhz
      Threads:
        - 0 (id: 9, online: true, NUMA node: 0)
    - Core 10
      Frequency: 0Mhz
      Threads:
        - 0 (id: 10, online: true, NUMA node: 0)
    - Core 11
      Frequency: 0Mhz
      Threads:
        - 0 (id: 11, online: true, NUMA node: 0)
    - Core 12
      Frequency: 0Mhz
      Threads:
        - 0 (id: 12, online: true, NUMA node: 0)
    - Core 13
      Frequency: 0Mhz
      Threads:
        - 0 (id: 13, online: true, NUMA node: 0)
    - Core 14
      Frequency: 0Mhz
      Threads:
        - 0 (id: 14, online: true, NUMA node: 0)
    - Core 15
      Frequency: 0Mhz
      Threads:
        - 0 (id: 15, online: true, NUMA node: 0)

(No idea, why lxc reports all Frequencies as 0 MHz, htop correctly reports 1996Mhz, when enabled)

lxc info --resources pihole

Name: pihole
Location: none
Remote: unix://
Architecture: x86_64
Created: 2022/02/12 18:31 UTC
Status: Running
Type: container
Profiles: default
Pid: 4818
Ips:
  eth0: inet    10.1.x.x      vethxxxxxxxb
  eth0: inet6   fe80::xxxx:xxxx:xxxx:xxxx        vethxxxxxxxb
  lo:   inet    127.0.0.1
  lo:   inet6   ::1
Resources:
  Processes: 28
  CPU usage:
    CPU usage (in seconds): 2679
  Memory usage:
    Memory (current): 433.61MB
    Memory (peak): 1.18GB
  Network usage:
    eth0:
      Bytes received: 179.39MB
      Bytes sent: 102.65MB
      Packets received: 592133
      Packets sent: 653153
    lo:
      Bytes received: 18.15MB
      Bytes sent: 18.15MB
      Packets received: 257322
      Packets sent: 257322

Yes, I can confirm, that the Core count is correct.

Warning icon, with number of warnings
At no point did I say, that I can see the message content of the warnings without being logged in. But I can see how my initial Message could be interpreted as such.

[...] , that and how many warnings were generated. [...]

[dschaper]

Yes, I can confirm, that the Core count is correct.

It's not, you're getting 16 threads masked as 16 cores. You can not have 16 cores and 16 threads on that CPU. Are they selling it as 16vCPUs? But that's outside the scope.

Warning icon, with number of warnings

I don't see anything there that a potential attacker can utilize. Warnings could be anything from packet size too big to misconfiguration. I do see your side and I can appreciate that but we need a way to let the admin know there are issues and most of our users would never know there were warnings. I can see making the display of that on non-authenticated pages a configuration boolean option, and we'll talk about that with the team.

[dschaper]

Can you do a lscpu?

[Pingger]

Welp ... ok small idiocy has been resolved by me, because I'm stupid:
lxc distinguishes between core limit and core allowance ...
limits.cpu -> Core count allowed (only this many cores are visible in the container)
limits.cpu.allowance -> Maximum core usage (all cores are visible, but under contention only the given value can be used by the container)

Stupid me has set it to in pihole
limits.cpu.allowance = 1
And in the parent template
limits.cpu = 8

Result: load is visible from host, core count is visible as 8 and what I expected is a value of 1 or 16 ...

I guess this explains how this very weird edge case output occured.

[Pingger]

lscpu

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   40 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-15
Thread(s) per core:              1
Core(s) per socket:              16
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC 7702P 64-Core Processor
Stepping:                        0
CPU MHz:                         1996.250
BogoMIPS:                        3992.50
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        4 MiB
L3 cache:                        16 MiB
NUMA node0 CPU(s):               0-15
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, IBPB conditional, IBRS_FW, STIBP disabled, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulq
                                 dq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw perfctr_core ssbd ibrs ib
                                 pb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat umip rdpid arch_capabilities

[dschaper]

Okay, still seems that a 15 min load average of 14 is high, but I'm thinking that is looking at the load on the entire host and not just on the Pi-hole container.

[dschaper]

Sorry, fat fingers.

I'll still ask about making the visibility a toggle.

[Pingger]

and yes lscpu actually shows 8 CPU(s) with weird 16 Cores
The host is running an ffmpeg encoding job with nice -n 19, so it IS expected, that the load is not close to 0

[dschaper]

That lscpu is what I thought, it's 8c/16t with that KVM configuration. May want to chat with the vendor about that.

[Pingger]

But the weird output was actually generated by "bad" configuration of lxc, because of inheritance of the container settings and forgetting about them.

[PromoFaux]

I considered the issue so glaringly obvious,

Just remember, not everyone sees everything exactly the same way. What's obvious for some is not so obvious for all. We're all of us human (at least, I think so) , with each of us having different experiences that have got us to where we are today

Regarding my attitude: Sorry, you unfortunately received the tail end of a discussion with the nextcloud support team and a previous experience with the pihole community.

Accepted - but as a hint for the future, try to compartmentalise and approach each conversation on fresh, even ground - it helps people to not get rubbed up the wrong way.

Have a good weekend✌️

[DL6ER]

I'll still ask about making the visibility a toggle.

CHECK_LOAD=false in /etc/pihole/pihole-FTL.conf disables the warning.