-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Heartbeat TLS metadata #7545
Heartbeat TLS metadata #7545
Conversation
libbeat/outputs/transport/tls.go
Outdated
return TestTLSMetaDialer(d, forward, config, timeout, nil) | ||
} | ||
|
||
func TestTLSMetaDialer( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported function TestTLSMetaDialer should have comment or be unexported
libbeat/outputs/transport/tls.go
Outdated
return TLSMetaDialer(forward, config, timeout, nil) | ||
} | ||
|
||
func TLSMetaDialer(forward Dialer, config *TLSConfig, timeout time.Duration, meta *DialerMeta) (Dialer, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported function TLSMetaDialer should have comment or be unexported
} | ||
} | ||
|
||
func (dm *DialerMeta) SetTls(cs func() tls.ConnectionState) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported method DialerMeta.SetTls should have comment or be unexported
method SetTls should be SetTLS
import "crypto/tls" | ||
|
||
type DialerMeta struct { | ||
Tls struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
struct field Tls should be TLS
|
||
import "crypto/tls" | ||
|
||
type DialerMeta struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported type DialerMeta should have comment or be unexported
for chainIndex, chain := range connState.VerifiedChains { | ||
for certIndex, cert := range chain { | ||
j := common.MapStr{ | ||
"chain_index": chainIndex, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is only a subset of the fields for now, but I would expand this to the full field list.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like it would be a good opportunity to reuse the code from Packetbeat for turning this data into an event. This would help ensure that we are consistent with field names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also take into account what @ph contributed to ECS: https://github.com/elastic/ecs#tls Perhaps we can extend this further.
502de7f
to
165c51c
Compare
packetbeat/protos/tls/parse.go
Outdated
@@ -543,7 +543,7 @@ func getKeySize(key interface{}) int { | |||
return 0 | |||
} | |||
|
|||
func certToMap(cert *x509.Certificate, includeRaw bool) common.MapStr { | |||
func CertToMap(cert *x509.Certificate, includeRaw bool) common.MapStr { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported function CertToMap should have comment or be unexported
heartbeat/hbtest/hbtestutil.go
Outdated
@@ -58,6 +61,18 @@ func ServerPort(server *httptest.Server) (uint16, error) { | |||
return uint16(p), nil | |||
} | |||
|
|||
func TLSChecks(chainIndex, certIndex int, certificate *x509.Certificate) mapval.Validator { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported function TLSChecks should have comment or be unexported
165c51c
to
4b09ec2
Compare
config.SetString("urls", 0, url) | ||
// testRequest tests the given request.certPath is optional, if given | ||
// an empty string no cert will be set. | ||
func testRequest(t *testing.T, testUrl string, certPath string) beat.Event { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func parameter testUrl should be testURL
4b09ec2
to
8e5f250
Compare
Introduces initial support for slices into mapval. There are lots of good refactor opportunities this opens up, but I want to draw the line somewhere close. I came across a case where I needed array support in elastic#7545 which I'll be rebasing on this PR. This required a lot of internals refactoring. The highlights are: While the code is more complex, there are fewer special cases, and the control flow is less complicated Making value validation less of a special case Redesigning the walk function to work correctly with slices. This required using reflect extensively. Validating a value involves returning a full mapval.Results map which gets merged into the main Validator results. This is a better internal design and allows for more flexibility. It does mean that custom IsDef validations have a slightly more complicated signature since they need to know the path of the thing they're validating, but that's a good thing in terms of power. Introducing a Path type instead of using simple strings for paths. Moved away from using the MapStr Get method to fetch subkeys. This was required to allow users to specify slice subscript. Tests was added for comparing slices in various ways. As literals, with nested matchers, etc.
3476b6c
to
1869ce7
Compare
libbeat/outputs/transport/tls.go
Outdated
return TestTLSMetaDialer(d, forward, config, timeout, nil) | ||
} | ||
|
||
// TestTLSDialer tests dialing a TCP connection over TLS recording TLS metadata. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment on exported function TestTLSMetaDialer should be of the form "TestTLSMetaDialer ..."
libbeat/outputs/transport/tls.go
Outdated
@@ -49,15 +49,33 @@ const ( | |||
VerifyNone = tlscommon.VerifyNone | |||
) | |||
|
|||
// TLSMetaDialer is a dialer variant that does not carry along TLS metadata. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment on exported function TLSDialer should be of the form "TLSDialer ..."
// DialerMeta contains extra metadata about dialed connections. | ||
// If you have extra info about the connection you'd like to bubble up, it goes here. | ||
type DialerMeta struct { | ||
Tls struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
struct field Tls should be TLS
339ce2f
to
d51d435
Compare
This is much improved, but now needs some modifications to work with ECS. We have much richer data than the few fields ECS asks for, but I'll examine this in more detail tomorrow. |
newMap := common.MapStr{} | ||
rv := reflect.ValueOf(o) | ||
|
||
for _, key := range rv.MapKeys() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This panics if rv is no map. You can add a guard by checking rv.Kind() == reflect.Map
.
|
||
func sliceToSliceOfInterfaces(o interface{}) []interface{} { | ||
rv := reflect.ValueOf(o) | ||
converted := make([]interface{}, rv.Len()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
panics if o
is no array or slice or map.
rv := reflect.ValueOf(o) | ||
converted := make([]interface{}, rv.Len()) | ||
for i := 0; i < rv.Len(); i++ { | ||
var indexV = rv.Index(i) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
panic if o
is no array or slice.
timer.stop() | ||
event.Put("tls.rtt.handshake", look.RTT(timer.duration())) | ||
|
||
connState := meta.TLS.ConnectionState() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We really need all the wiring up? There is quite some indirection here, I'd rather keep out if possible. The conn
connection object should be a tls connection (as is being produced by TLSDialer). We should read the connection state from the conn
parameter if possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm, what exactly are you thinking here? tls.Conn
in crypto/tls/conn.go is not a super-type (is that the go lingo?) of net.Conn
, so cannot be cast. It simply nests a net.Conn
as a property.
AFAICT this means we have to use this sort of indirection. Would love a simpler alternative though. Do I misunderstand?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tls.Conn wraps another connection, because TLS sits on top of another network layer. But tls.Conn implements net.Conn
itself. In go we do not have inheritance or subtype relationships. But tls.Conn
structurally satisfies the net.Conn
interface. When building layers of dialers, we have to fix the return values of dialers, due to lack of generics and sub-typing (no co-/contra-variance on methods). The common denominator for dialers is given by:
type Dialer interface {
Dial(network, address string) (net.Conn, error)
}
As net.Conn
is an interface, each layer will wrap it's input layer + use a concrete type which is finally promoted to the interface value being returned.
This is reflected in the DialerChain type:
type DialerChain struct {
Net NetDialer
Layers []Layer
}
The NetDialer produces the most low-level connection object. Each layer will wrap the current connection object, returning a new connection object.
As we want to do some measurements before and after a network layer becomes active, I introduced the beforeDial
and afterDial
wrappers, so to reduce common boilerplate and amount of wrapper functions needed. Again, lack of generics and sub-type relationships we do enforce the use of net.Conn
. But in general, once a dialer-chain is setup, we have this layout:
+----+-----------------------+----+ net.Conn
| After Layer II |
+-----------------------+ Layer II Conn type
| Dial Layer II |
+-----------------------+
| Before Layer II |
+---------------------------------+ net.Conn
| After Layer I |
+-----------------------+ Layer I Conn type
| Dial Layer I |
+-----------------------+
| Before Layer I |
+---------------------------------+ net.Conn
| Net Layer (e.g. TCP) |
+-----------------------+
A Layer must make no assumptions about the underlying layer. A layer must assume the connection object to wrap to be of type net.Conn
. But within a layer (dial + after steps), we know the concrete type returned by the dialer and to be consumed by the 'after'-handler. Due to limitations of the type-system, we are forced to cast the interface value to the concrete type, though.
In case of the TLS layer, we use transport.TLSDialer. The concrete type returned by (*TLSDialer).Dial
is *tls.Conn
. => we can cast the interface value to *tls.Conn
in the 'after'-handler. As we know the concrete type in the 'after'-handler, we don't need another behind-the-scenes type capturing the concrete type. Admittedly, it's more type safe, but it's somewhat more involved and somewhat over-engineered for something that should be simple (as the after-handler is the only consumer that requires to access the tls state).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the very detailed writeup. I'm realizing that my confusion was in typing conn.(tls.Conn)
instead of conn.(*tls.Conn)
. That was the root of my assumption that the interface wasn't implemented.
Thanks for the detailed diagram of the dialer stack, that's very helpful!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should move that diagram to the code docs somewhere.
d51d435
to
4154528
Compare
* Enhance mapval to handle arrays Introduces initial support for slices into mapval. There are lots of good refactor opportunities this opens up, but I want to draw the line somewhere close. I came across a case where I needed array support in #7545 which I'll be rebasing on this PR. This required a lot of internals refactoring. The highlights are: While the code is more complex, there are fewer special cases, and the control flow is less complicated Making value validation less of a special case Redesigning the walk function to work correctly with slices. This required using reflect extensively. Validating a value involves returning a full mapval.Results map which gets merged into the main Validator results. This is a better internal design and allows for more flexibility. It does mean that custom IsDef validations have a slightly more complicated signature since they need to know the path of the thing they're validating, but that's a good thing in terms of power. Introducing a Path type instead of using simple strings for paths. Moved away from using the MapStr Get method to fetch subkeys. This was required to allow users to specify slice subscript. Tests was added for comparing slices in various ways. As literals, with nested matchers, etc. * Validate Map objects when compiling schema. Rename Schema->Compile. We no longer need to return an error from actually executing a Validator as a result. This also fixes a bug where we would expand keynames in input maps to validators.
4154528
to
8327d7d
Compare
I'm pausing this a bit because we need better ECS schema definitions for TLS. I've opened up a GH issue to start the discussion here: elastic/ecs#64 |
certs = append(certs, j) | ||
} | ||
} | ||
event.Put("tls.certificates", certs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit concerned that certificates here is going to be a nested object in Elasticsearch it seems.
What are the expected queries the user will run against these fields?
Added a bunch of specs for I added a little to this file, so it's fair to add more coverage in this PR IMHO. |
OK, should be ready for final review. |
@@ -21,9 +21,14 @@ import ( | |||
"net" | |||
"time" | |||
|
|||
cryptoTLS "crypto/tls" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as usual goimports messes with you :-)
"github.com/elastic/beats/heartbeat/look" | ||
"github.com/elastic/beats/libbeat/common" | ||
"github.com/elastic/beats/libbeat/outputs/transport" | ||
"github.com/elastic/beats/packetbeat/protos/tls" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit concerned about this import. Beats should not have dependencies with each other. Share code like this be moved into libbeat.
// TODO: extract TLS connection parameters from connection object. | ||
tlsConn, ok := conn.(*cryptoTLS.Conn) | ||
if !ok { | ||
panic(fmt.Sprintf("TLS afterDial received a non-tls connection %t. This should never happen", conn)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really want to panic here or rather return / log a critical error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets keep the panic here. If this fails it's definitely a programming error. Given we have TLS tests, this panic should only occur during testing.
If we hide this behind an error message being logged, we might also have test failures (or a bug at worst), but then we will have to search for the root cause. Being a little more offensive can help.
// Pointers because we need a nil value | ||
var chainNotValidBefore *time.Time | ||
var chainNotValidAfter *time.Time | ||
for _, chain := range tlsConn.ConnectionState().VerifiedChains { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be nice to have a code comment here that states we check all certs and take the oldest / newest timestamp. for before and after and not sure from the last certificate in the chain.
heartbeat/hbtest/hbtestutil.go
Outdated
|
||
"github.com/elastic/beats/libbeat/common/mapval" | ||
"github.com/elastic/beats/packetbeat/protos/tls" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comment below about packetbeat dependency.
"net/http/httptest" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/require" | ||
"crypto/x509" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
goimports messing with you.
libbeat/common/mapval/path.go
Outdated
} else if pct == PCSliceIdx { | ||
return "slice" | ||
} else { | ||
panic(fmt.Sprintf("Unknown path component type! %d", int(pct))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use an error here instead of a panic. What if someone uses this for production code and is not aware this can apnice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, this practically cannot panic. The only way that would happen is if someone was messing deep in the guts of mapval and added a new type without overriding this.
I would say then, that if this did panic, it'd be a bug in mapval itself, not in user-facing code. Checking invariants like that seems like the right place for panic no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm realizing this is similar to the other panic described earlier. Any case where another type is encountered would definitely be a programming error. You would need to create a new PathComponentType
with a different (manual) value.
@urso I see you +1'd this change, but opposed removing the other panic. Maybe you can help me understand the difference?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming mapval package could be used in other places in beats, the mapval package might be used to validate any kind of external input. We might even consider users to configure validation (based on mapval) in the heartbeat config file. When working with external input, mapval should not panic.
If the panic can only indicate programming errors, I'm fine with the panic. If the panic can be triggered by external (yet unchecked) input then we should go with an error. mapval has to check input, so we must assume any kind of invalid input.
It's common in String methods to not create a panic, but return an 'Unknown'-string.
Btw. consider using stringer like:
//go:generate stringer -type=PathComponentType -linecomment=true
const (
PCMapKey PathComponentType = 1 + iota // map
// PCSliceIdx is the Type for slice indices.
PCSliceIdx // slice
)
Notice how I set PCMapKey
to 1. That is, the zero-value is invalid. Oftentimes one defines an 'invalid'-const to be 0 like:
//go:generate stringer -type=PathComponentType -linecomment=true
const (
PCMapInvalid PathComponentType = iota // invalid
PCMapKey // map
PCSliceIdx // slice
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the feedback, I'll take your advice and return an unknown string. I think that's a great compromise.
Also, thanks for the tip on the 1+, makes a lot of sense.
packetbeat/protos/tls/parse.go
Outdated
func certToMap(cert *x509.Certificate, includeRaw bool) common.MapStr { | ||
// CertToMap takes an x509 cert and converts it into a map. If includeRaw is set | ||
// to true a PEM encoded copy of the cert is encoded into the map as well. | ||
func CertToMap(cert *x509.Certificate, includeRaw bool) common.MapStr { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Better move this libbeat.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool, will move that whole package.
libbeat/common/mapval/is_defs.go
Outdated
) | ||
}) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you are already a heavy user of reflection, how about this:
var equalChecks = map[reflect.Type]reflect.Value
// accepts and register equal function for function signatures:
// func(a, b T) bool
func RegisterEqual(fn interface{}) {
v := reflect.ValueOf(fn)
t := v.Type()
if t.Kind() != reflect.Function { ... }
if t.NumIn() != 2 { ...}
if t.NumOut() != 1 { ... }
if t.In(0) != t.In(1) { ... }
if t.Out(0) != <bool> { ... }
equalChecks[t.In(0)] = v
}
func IsEqual(to interface{}) IsDef {
vTo := reflect.ValueOf(to)
return Is("equals", func(path Path, in interface{}) *Results {
vIn := reflect.ValueOf(in)
if vTo.Type() != vIn.Type() { ... }
// we always query, in case someone did add a comparison function after
checker := equalChecks(vTo.Type())
if checker == nil {
// use DeepEqual
}
equal := checker.Call([]reflect.Value{vTo, VIn}).Bool()
if !equal {
...
}
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is pretty nifty, out of curiosity I wonder if using reflect is faster than a type switch? I assume they must be pretty similar perf-wise under the covers?
I am however worried about YAGNI. How many special equals cases are there? reflect.DeepEquals
works pretty well, this is the first case it hasn't.
If there are only a few, I personally prefer the switch because its very concise, however, if we can foresee adding 10 of these cases a map sounds better.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reflection is somewhat smaller.
In go an interface{}
value has the internal representation:
type ifcType struct {
typ *typeDescr
value *unsafe.Pointer
}
The reflection package defines Value to be:
type Value struct {
typ *typeDescr
value *unsafe.Pointer
flags flags
}
when using a type-switch, the generated code checks if the 'typ' matches and interprets the value pointer as your actual type. If your target type is also a pointer, its a comparison + copy pointer into new variable.
When using reflection, each call to any Value method checks flags, types and so on, checking if your call is actually valid. The Value.Call
method is super involved. Reflection is definitely slower, but gives you some flexibility here.
reflect.DeepEquals has it's limits. It compares each field (including private fields) in a struct. Private fields might be different, while 2 objects might still compare equal. E.g. when comparing regular expressions. DeepEquals compares public and private fields. But the regex package introduced an object pool pointer in tip. Each regexp.Regex type holds a unique pointer => comparison fails, if one does not compare the same pointer, even if two regular expression are generated from the same string.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool, thanks for the detailed walkthrough. I think you're right, we'll probably add more of these, I'll add the registry.
"github.com/elastic/beats/libbeat/common" | ||
) | ||
|
||
func TestPathComponentType_String(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do not use underscores for variable names or function names. Event tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, that was generated by gotests
, so I assumed it was idiomatic. Will change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After a discussion on slack, we decided this was fine since the linter is fine. https://go-review.googlesource.com/c/go/+/86001 seems to have added support for this.
d72d43d
to
796472d
Compare
libbeat/common/mapval/is_defs.go
Outdated
}) | ||
} | ||
|
||
func IsDeepEqual(to interface{}) IsDef { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported function IsDeepEqual should have comment or be unexported
libbeat/common/mapval/is_defs.go
Outdated
}) | ||
} | ||
|
||
func IsDeepEqual(to interface{}) IsDef { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exported function IsDeepEqual should have comment or be unexported
@urso I've incorporated both of your suggestions, for using reflection in equality in c274e0f and for path panic-ing in 5a0a42e |
c5feeac
to
4777be7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@andrewvc Could you update the PR message to be the commit message or squash the commits into 1 so it can be merged?
* We now write certificate PEM data, and chain validation times to events * Enhanced test helpers (including mapval IsDefs) to handle new testing scenarios * Improved mapval handling of type mismatches to help with the tests here * Move CertToPEMString to libbeat to share between hb and pb * Incorporate @urso 's PR feedback using registration for equality types
4777be7
to
4baf903
Compare
Squashed |
* Enhance mapval to handle arrays Introduces initial support for slices into mapval. There are lots of good refactor opportunities this opens up, but I want to draw the line somewhere close. I came across a case where I needed array support in elastic#7545 which I'll be rebasing on this PR. This required a lot of internals refactoring. The highlights are: While the code is more complex, there are fewer special cases, and the control flow is less complicated Making value validation less of a special case Redesigning the walk function to work correctly with slices. This required using reflect extensively. Validating a value involves returning a full mapval.Results map which gets merged into the main Validator results. This is a better internal design and allows for more flexibility. It does mean that custom IsDef validations have a slightly more complicated signature since they need to know the path of the thing they're validating, but that's a good thing in terms of power. Introducing a Path type instead of using simple strings for paths. Moved away from using the MapStr Get method to fetch subkeys. This was required to allow users to specify slice subscript. Tests was added for comparing slices in various ways. As literals, with nested matchers, etc. * Validate Map objects when compiling schema. Rename Schema->Compile. We no longer need to return an error from actually executing a Validator as a result. This also fixes a bug where we would expand keynames in input maps to validators.
…7545) * We now write certificate PEM data, and chain validation times to events * Enhanced test helpers (including mapval IsDefs) to handle new testing scenarios * Improved mapval handling of type mismatches to help with the tests here * Move CertToPEMString to libbeat to share between hb and pb * Incorporate @urso 's PR feedback using registration for equality types This backport updates heartbeat/include/fields.go which had a conflict via make update
* Enhance mapval to handle arrays Introduces initial support for slices into mapval. There are lots of good refactor opportunities this opens up, but I want to draw the line somewhere close. I came across a case where I needed array support in #7545 which I'll be rebasing on this PR. This required a lot of internals refactoring. The highlights are: While the code is more complex, there are fewer special cases, and the control flow is less complicated Making value validation less of a special case Redesigning the walk function to work correctly with slices. This required using reflect extensively. Validating a value involves returning a full mapval.Results map which gets merged into the main Validator results. This is a better internal design and allows for more flexibility. It does mean that custom IsDef validations have a slightly more complicated signature since they need to know the path of the thing they're validating, but that's a good thing in terms of power. Introducing a Path type instead of using simple strings for paths. Moved away from using the MapStr Get method to fetch subkeys. This was required to allow users to specify slice subscript. Tests was added for comparing slices in various ways. As literals, with nested matchers, etc. * Validate Map objects when compiling schema. Rename Schema->Compile. We no longer need to return an error from actually executing a Validator as a result. This also fixes a bug where we would expand keynames in input maps to validators.
* We now write certificate PEM data, and chain validation times to events * Enhanced test helpers (including mapval IsDefs) to handle new testing scenarios * Improved mapval handling of type mismatches to help with the tests here * Move CertToPEMString to libbeat to share between hb and pb * Incorporate @urso 's PR feedback using registration for equality types This backport updates heartbeat/include/fields.go which had a conflict via make update
WIP Work toward adding extra TLS metadata. Would resolve #6300 and #3741 once complete.
The PR is based off of #7784 , hence the large diff. We should wait for that to merge before reviewing this further.
I'm submitting this PR early to get feedback from @urso on this approach to injecting the TLS metadata into the dialchain. The code is very rough, but I wanted to get feedback early before I start adding tests and rounding this out.
Another goal of this PR would be to add tests to the namespaces and types this PR touches. Heartbeat needs more tests and the strategy here is to add a feature, and retroactively add tests alongside it.
One thing we might also want to consider is that there is shared code for the dialchain in
libbeat/outputs/transport
that probably should just move up a level tolibbeat/transport
.