Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log: requesting log desensitization #2852

Closed
Yisaer opened this issue Aug 27, 2020 · 7 comments · Fixed by #3011
Closed

log: requesting log desensitization #2852

Yisaer opened this issue Aug 27, 2020 · 7 comments · Fixed by #3011
Labels
component/security Security logic. type/enhancement The issue or PR belongs to an enhancement.

Comments

@Yisaer
Copy link
Contributor

Yisaer commented Aug 27, 2020

Development Task

To reinforce the security in the PD, one thing we need to do is to do the log desensitization.
Here are some examples we need to hide from the logs in my view:

  1. region key in logs like following:
log.Error("wrong range keys",zap.String("start-key", string(HexRegionKey(startKey))),zap.String("end-key", string(HexRegionKey(endKey))))
  1. etcd key and value in logs like following:
log.Warn("kv gets too slow", zap.String("request-key", key), zap.Duration("cost", cost), zap.Error(err))
  1. store label key and value in logs like following:
log.Warn("not found the key match with the store label",zap.Stringer("store", s.GetMeta()),zap.String("label-key", key))
  1. placement rule key and value in logs like following:
log.Error("rule is in bad format", zap.String("rule-key", k), zap.String("rule-value", v), zap.Error(errs.ErrLoadRule.FastGenByArgs()), zap.NamedError("cause", err))

We will add a new configuration like "enable-log-desensitization"(default false). If this configuration is enabled, the sensitive information won't appear on the previous log.

@Yisaer Yisaer added the type/enhancement The issue or PR belongs to an enhancement. label Aug 27, 2020
@Yisaer
Copy link
Contributor Author

Yisaer commented Aug 27, 2020

@nolouch @disksing @rleungx @HunDunDM @lhy1024 WDYT? or any other advices?

@lhy1024
Copy link
Contributor

lhy1024 commented Aug 27, 2020

LGTM. In addition to the key, what else do we need to hide, such as topology?

@disksing
Copy link
Contributor

Can we log sha1 instead? At least we can know if the key is changed.

@Yisaer
Copy link
Contributor Author

Yisaer commented Aug 27, 2020

@disksing It's ok to me. And from pingcap/tidb#19409, I found that tidb directly replace the key by '?'. Maybe we should unify the action.

@rleungx
Copy link
Member

rleungx commented Aug 28, 2020

IMO, ? is no meaning for PD.

@disksing
Copy link
Contributor

I think use hash instead of ? can provide more info without leaking user information. But we need to inquire about the compliance.

@Yisaer
Copy link
Contributor Author

Yisaer commented Sep 11, 2020

After discussion with tikv/tidb group, currently we will omit the region key information if log-redact is enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/security Security logic. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants