schedulers: add some metrics for hotreigon

[nolouch]

Signed-off-by: nolouch nolouch@gmail.com

What problem does this PR solve?

• add some metrics and log to observe hot region scheduling.
• Reduce the frequency of hotspot scheduling.

What is changed and how it works?

add metrics.

Release note

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Code changes

• Has configuration change
• Has HTTP API interfaces change (Don't forget to update API document)
• Has persistent data change

Side effects

• Possible performance regression
• Increased code complexity
• Breaking backward compatibility

Related changes

• PR to update pingcap/docs/pingcap/docs-cn:
• PR to update pingcap/tidb-ansible:
• Need to cherry-pick to the release branch

[codecov-io]

Codecov Report

Merging #2295 into master will increase coverage by 0.63%.
The diff coverage is 84.57%.

@@            Coverage Diff             @@
##           master    #2295      +/-   ##
==========================================
+ Coverage   76.25%   76.88%   +0.63%     
==========================================
  Files         199      201       +2     
  Lines       21591    21773     +182     
==========================================
+ Hits        16464    16740     +276     
+ Misses       3891     3795      -96     
- Partials     1236     1238       +2     

Impacted Files	Coverage Δ
server/server.go	81.25% <33.33%> (+0.79%)	⬆️
server/cluster/cluster.go	84.52% <50.00%> (+3.64%)	⬆️
server/cluster/coordinator.go	75.20% <50.00%> (-3.19%)	⬇️
server/grpc_service.go	57.26% <50.00%> (-1.29%)	⬇️
tools/pd-ctl/pdctl/command/component_command.go	63.54% <58.33%> (-0.51%)	⬇️
pkg/dashboard/servers.go	87.50% <80.00%> (+23.86%)	⬆️
pkg/dashboard/manager.go	82.07% <82.07%> (ø)
pkg/dashboard/apiserver.go	73.91% <100.00%> (ø)
pkg/dashboard/redirector.go	100.00% <100.00%> (ø)
server/api/router.go	91.23% <100.00%> (+0.04%)	⬆️
... and 32 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c709cf...fd3ff71. Read the comment docs.

[nolouch]

/rebuild

[nolouch]

/merge

[sre-bot]

/run-all-tests

[sre-bot]

@nolouch merge failed.