Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

html: reduce entity memory usage and init load #26727

Closed
bradfitz opened this issue Jul 31, 2018 · 2 comments
Closed

html: reduce entity memory usage and init load #26727

bradfitz opened this issue Jul 31, 2018 · 2 comments
Labels
FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done. Performance
Milestone

Comments

@bradfitz
Copy link
Contributor

The html package has two large maps that are populated during an implicit init():

https://github.com/golang/go/blob/master/src/html/entity.go

Because of that, the memory is always in the heap and the linker can't discard it, even if the user never uses it. (via https://golang.org/pkg/html/#UnescapeString)

On linux/amd64 at tip, it adds 110 KB of heap just to import _ "html". A more realistic use case is code that only uses html.EscapeString. Such code should never need to run init() or retain that code in memory.

It seems this is a perfect use case of perfect hashing.

@cespare, it looks like your https://github.com/cespare/mph is nice & simple and suitably licensed.

It seems like we could pull entity.go into a gen-entity.go file and go generate an ugly zentity.go file that is just one big string literal, suitably packed.

Then there'd be no init work to construct the map, and the linker could discard it.

/cc @josharian

@bradfitz bradfitz added Performance help wanted NeedsFix The path to resolution is known, but the work has not been done. labels Jul 31, 2018
@bradfitz bradfitz added this to the Go1.12 milestone Jul 31, 2018
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/127075 mentions this issue: html: lazily populate Unescape tables

@bradfitz
Copy link
Contributor Author

Actually, https://go-review.googlesource.com/c/go/+/127075 ended up working too and is probably simpler.

@bradfitz bradfitz modified the milestones: Go1.12, Go1.11 Jul 31, 2018
@golang golang locked and limited conversation to collaborators Jul 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done. Performance
Projects
None yet
Development

No branches or pull requests

2 participants