Add blake2b hashing algorithm #5926

andrewkroh · 2017-12-19T23:01:37Z

Allow BLAKE2b as a file hashing algorithm (https://blake2.net/). In my benchmarks
it is quite fast.

Hash Algorithm	Time per 100 MiB File (ns)	MiB / sec
BenchmarkHashFile/blake2b_256-8	127116193	786.6818353
BenchmarkHashFile/blake2b_512-8	127239979	785.9165082
BenchmarkHashFile/blake2b_384-8	129671956	771.1767686
BenchmarkHashFile/sha1-8	131347484	761.3392884
BenchmarkHashFile/md5-8	170146968	587.7271936
BenchmarkHashFile/sha512-8	200803749	497.9986703
BenchmarkHashFile/sha384-8	201153073	497.1338419
BenchmarkHashFile/sha512_224-8	201987854	495.0792734
BenchmarkHashFile/sha512_256-8	202126889	494.7387282
BenchmarkHashFile/sha256-8	297884549	335.7005267
BenchmarkHashFile/sha224-8	299384125	334.0190466
BenchmarkHashFile/sha3_224-8	335496603	298.0656111
BenchmarkHashFile/sha3_256-8	352318502	283.834086
BenchmarkHashFile/sha3_384-8	461460154	216.7034339
BenchmarkHashFile/sha3_512-8	651817080	153.4172747

houndci-bot · 2017-12-19T23:01:42Z

auditbeat/module/file_integrity/config.go

+	SHA3_512    HashType = "sha3_512"
+	SHA512      HashType = "sha512"
+	SHA512_224  HashType = "sha512_224"
+	SHA512_256  HashType = "sha512_256"


don't use ALL_CAPS in Go names; use CamelCase

houndci-bot · 2017-12-19T23:01:42Z

auditbeat/module/file_integrity/config.go

+	SHA3_384    HashType = "sha3_384"
+	SHA3_512    HashType = "sha3_512"
+	SHA512      HashType = "sha512"
+	SHA512_224  HashType = "sha512_224"


don't use ALL_CAPS in Go names; use CamelCase

houndci-bot · 2017-12-19T23:01:42Z

auditbeat/module/file_integrity/config.go

+	SHA3_224    HashType = "sha3_224"
+	SHA3_256    HashType = "sha3_256"
+	SHA3_384    HashType = "sha3_384"
+	SHA3_512    HashType = "sha3_512"


don't use ALL_CAPS in Go names; use CamelCase

houndci-bot · 2017-12-19T23:01:42Z

auditbeat/module/file_integrity/config.go

+	SHA384      HashType = "sha384"
+	SHA3_224    HashType = "sha3_224"
+	SHA3_256    HashType = "sha3_256"
+	SHA3_384    HashType = "sha3_384"


don't use ALL_CAPS in Go names; use CamelCase

houndci-bot · 2017-12-19T23:01:42Z

auditbeat/module/file_integrity/config.go

+	SHA256      HashType = "sha256"
+	SHA384      HashType = "sha384"
+	SHA3_224    HashType = "sha3_224"
+	SHA3_256    HashType = "sha3_256"


don't use ALL_CAPS in Go names; use CamelCase

houndci-bot · 2017-12-19T23:01:43Z

auditbeat/module/file_integrity/schema/Hash.go

+func HashAddBlake2b384(builder *flatbuffers.Builder, blake2b384 flatbuffers.UOffsetT) {
+	builder.PrependUOffsetTSlot(13, flatbuffers.UOffsetT(blake2b384), 0)
+}
+func HashStartBlake2b384Vector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT {


exported function HashStartBlake2b384Vector should have comment or be unexported

houndci-bot · 2017-12-19T23:01:43Z

auditbeat/module/file_integrity/schema/Hash.go

+func HashStartBlake2b256Vector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT {
+	return builder.StartVector(1, numElems, 1)
+}
+func HashAddBlake2b384(builder *flatbuffers.Builder, blake2b384 flatbuffers.UOffsetT) {


exported function HashAddBlake2b384 should have comment or be unexported

houndci-bot · 2017-12-19T23:01:43Z

auditbeat/module/file_integrity/schema/Hash.go

+func HashAddBlake2b256(builder *flatbuffers.Builder, blake2b256 flatbuffers.UOffsetT) {
+	builder.PrependUOffsetTSlot(12, flatbuffers.UOffsetT(blake2b256), 0)
+}
+func HashStartBlake2b256Vector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT {


exported function HashStartBlake2b256Vector should have comment or be unexported

houndci-bot · 2017-12-19T23:01:44Z

auditbeat/module/file_integrity/schema/Hash.go

@@ -305,6 +356,24 @@ func HashAddSha3512(builder *flatbuffers.Builder, sha3512 flatbuffers.UOffsetT)
 func HashStartSha3512Vector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT {
 	return builder.StartVector(1, numElems, 1)
 }
+func HashAddBlake2b256(builder *flatbuffers.Builder, blake2b256 flatbuffers.UOffsetT) {


exported function HashAddBlake2b256 should have comment or be unexported

houndci-bot · 2017-12-19T23:01:44Z

auditbeat/module/file_integrity/schema/Hash.go

+	return 0
+}
+
+func (rcv *Hash) Blake2b512Length() int {


exported method Hash.Blake2b512Length should have comment or be unexported

andrewkroh · 2017-12-19T23:02:24Z

All of the Hound comments are on generated code.

andrewkroh · 2017-12-19T23:05:07Z

auditbeat/module/file_integrity/event_test.go

-			SHA3_256:   mustDecodeHex("3cb5385a2987ca45888d7877fbcf92b4854f7155ae19c96cecc7ea1300c6f5a4"),
-			SHA3_384:   mustDecodeHex("f19539818b4f29fa0ee599db4113fd81b77cd1119682e6d799a052849d2e40ef0dad84bc947ba2dee742d9731f1b9e9b"),
-			SHA3_512:   mustDecodeHex("f0a2c0f9090c1fd6dedf211192e36a6668d2b3c7f57a35419acb1c4fc7dfffc267bbcd90f5f38676caddcab652f6aacd1ed4e0ad0a8e1e4b98f890b62b6c7c5c"),
+			BLAKE2B_256: mustDecodeHex("0f0cc1f0ea4ef962d6a150ae0b77bc320b57ed24e1609b933fa2274484f59145"),


These new values were computed with pyblake2.

Allow BLAKE2b as a file hashing algorithm (https://blake2.net/). In my benchmarks it is quite fast. | Hash Algorithm | Time per 100 MiB File (ns) | MiB / sec | |---------------------------------|----------------------------|-------------| | BenchmarkHashFile/blake2b_256-8 | 127116193 | 786.6818353 | | BenchmarkHashFile/blake2b_512-8 | 127239979 | 785.9165082 | | BenchmarkHashFile/blake2b_384-8 | 129671956 | 771.1767686 | | BenchmarkHashFile/sha1-8 | 131347484 | 761.3392884 | | BenchmarkHashFile/md5-8 | 170146968 | 587.7271936 | | BenchmarkHashFile/sha512-8 | 200803749 | 497.9986703 | | BenchmarkHashFile/sha384-8 | 201153073 | 497.1338419 | | BenchmarkHashFile/sha512_224-8 | 201987854 | 495.0792734 | | BenchmarkHashFile/sha512_256-8 | 202126889 | 494.7387282 | | BenchmarkHashFile/sha256-8 | 297884549 | 335.7005267 | | BenchmarkHashFile/sha224-8 | 299384125 | 334.0190466 | | BenchmarkHashFile/sha3_224-8 | 335496603 | 298.0656111 | | BenchmarkHashFile/sha3_256-8 | 352318502 | 283.834086 | | BenchmarkHashFile/sha3_384-8 | 461460154 | 216.7034339 | | BenchmarkHashFile/sha3_512-8 | 651817080 | 153.4172747 |

houndci-bot · 2017-12-19T23:15:57Z

auditbeat/module/file_integrity/schema/Hash.go

+	return 0
+}
+
+func (rcv *Hash) Blake2b512(j int) int8 {


exported method Hash.Blake2b512 should have comment or be unexported

houndci-bot · 2017-12-19T23:15:58Z

auditbeat/module/file_integrity/schema/Hash.go

+	return 0
+}
+
+func (rcv *Hash) Blake2b384Length() int {


exported method Hash.Blake2b384Length should have comment or be unexported

houndci-bot · 2017-12-19T23:15:58Z

auditbeat/module/file_integrity/schema/Hash.go

+	return 0
+}
+
+func (rcv *Hash) Blake2b384(j int) int8 {


exported method Hash.Blake2b384 should have comment or be unexported

houndci-bot · 2017-12-19T23:15:58Z

auditbeat/module/file_integrity/schema/Hash.go

+	return 0
+}
+
+func (rcv *Hash) Blake2b256Length() int {


exported method Hash.Blake2b256Length should have comment or be unexported

houndci-bot · 2017-12-19T23:15:58Z

auditbeat/module/file_integrity/schema/Hash.go

@@ -230,8 +230,59 @@ func (rcv *Hash) Sha3512Length() int {
 	return 0
 }

+func (rcv *Hash) Blake2b256(j int) int8 {


exported method Hash.Blake2b256 should have comment or be unexported

adriansr

LGTM

Add golang.org/x/crypto/blake2b to vendor

64f429b

andrewkroh added Auditbeat enhancement review labels Dec 19, 2017

houndci-bot reviewed Dec 19, 2017

View reviewed changes

andrewkroh commented Dec 19, 2017

View reviewed changes

andrewkroh force-pushed the feature/auditbeat/blake2 branch from f53f205 to 98915b9 Compare December 19, 2017 23:10

andrewkroh force-pushed the feature/auditbeat/blake2 branch from 98915b9 to 7abe013 Compare December 19, 2017 23:15

houndci-bot reviewed Dec 19, 2017

View reviewed changes

adriansr approved these changes Dec 21, 2017

View reviewed changes

adriansr merged commit 24b3b8e into elastic:master Dec 21, 2017

andrewkroh deleted the feature/auditbeat/blake2 branch January 17, 2018 16:44

andrewkroh mentioned this pull request Mar 22, 2018

[Auditbeat] Add xxhash as a hash option for faster file integrity checks on large files #6626

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add blake2b hashing algorithm #5926

Add blake2b hashing algorithm #5926

andrewkroh commented Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

andrewkroh commented Dec 19, 2017

andrewkroh Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

houndci-bot Dec 19, 2017

adriansr left a comment

Add blake2b hashing algorithm #5926

Add blake2b hashing algorithm #5926

Conversation

andrewkroh commented Dec 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewkroh commented Dec 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adriansr left a comment

Choose a reason for hiding this comment