speed up prefix record loading #1800

Job	Run time
test (ubuntu-latest)	15m 42s
Build x86_64-unknown-linux-musl	7m 46s
test (macos-latest)	13m 21s
Build aarch64-unknown-linux-musl	8m 9s
test (windows-latest)	9m 36s
Build powerpc64le-unknown-linux-gnu	8m 36s
Build x86_64-apple-darwin	19m 44s
Build aarch64-apple-darwin	10m 44s
Build x86_64-pc-windows-gnu	16m 12s
Build x86_64-pc-windows-msvc	15m 26s
Post-process	4s
	2h 5m 20s

Provide feedback