-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use match.Matcher for checking heartbeat response bodies #5981 #6539
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the contribution especially for adding docs and tests. Left a minor comment. LGTM.
@urso Do you also want to have a look?
- type: http | ||
schedule: '@every 5s' | ||
urls: ["https://myhost:80"] | ||
check.request: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the indentation is off here. It should be on the same level as urls
for example.
Same in all the examples below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch! The current example also has the wrong indentation too (which I copy pasted from)
. So I'll fix up the original too.
if !bytes.Equal(body, content) { | ||
return errBodyMismatch | ||
for _, m := range body { | ||
if m.MatchString(string(content)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use m.Match(content)
, so to reduce the chance of allocating and copying all contents, just for matching.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
} | ||
|
||
for _, test := range matchTests { | ||
t.Run(test.description, func(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
|
||
for _, test := range matchTests { | ||
t.Run(test.description, func(t *testing.T) { | ||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Requiring an http server is somewhat overkill. Makes me wonder if we can 'simplify' the validators interface/function, so to not depend on a http response.
Especially the fact the body
validator consuming the complete body is not useful if we want to be able to combine (and/or) more complicated filtering in the future.
Using interfaces one could define:
type response struct {
Status() uint
Body() ([]byte, error)
...
}
The adapter for use with http.Response could lazily read and cache the body then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I totally agree 👍
Makes me wonder if we can 'simplify' the validators interface/function, so to not depend on a http response.
The wording you used here makes this comment sound like "something we would like to improve in the future". Can you confirm that I interpreted this correctly? Otherwise I'd be happy to have a go refactoring in your suggestion but I would definitely need a bit of help since this goes beyond my basic go knowledge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, it's something we can do in the future. Rather have not 'polute' this PR with too many cleanups.
var matchTests = []struct { | ||
description string | ||
body string | ||
patterns []match.Matcher |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as these are patterns, how about []string
, so to make the table a little more readable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it isn't obvious I'm still very new to writing go :) Does this look a bit more readable now? f63257b Or is there maybe an even cleaner way to do it?
Switching to For the future I was considering more enhanced body matching, that can also support parsing json/xml, so to check a document field it's value. This would require the full contents:
using matches:
json:
... |
@urso I would be fine with the breaking change here as we are still in beta and can document it. WDYT? In both cases the change to the above propose structure will be breaking I think. |
@urso Thanks for the review! I'm still very new to writing Go so the feedback really taught me a lot!
Ahh yes you are right! It would be easy enough to modify the regex to remain backwards compatible by requiring it to be an exact match. I don't feel strongly either way but I personally would vote for the default to be "match this string anywhere in the contents". In my experience checking the body is normally done as a "site is up and has real content" sanity check. Instead of only checking for a HTTP 200 which a lot of "website is on fire" status pages return. However as a user I'm trying to think of a realistic scenario where this change in behaviour would actually cause issues. I can think of one example where this could bite someone but I would really hope nobody has a healthcheck endpoint like this.
I like all of these suggestions. In case you missed it I made some similar proposals in #5981 . The main one I am interested in is having a proper way to check JSON output. The main use case would be monitoring the health of our Elasticsearch clusters that are running in cloud.
Which could of course be done with a regex too
|
To move forward on this I suggest we go with the breaking change but document it properly in the changelog. From there we can further discuss about the above proposal to make it even more powerful. |
@Crazybus I see you put this PR 'On Hold'. Personally I'm fine with the breaking change, as it's still a massive improvement over the current situation. Given there are no critical blockers in this PR, how about merging it as is and do further (general) improvements on validation in future PRs. |
Sorry for the confusion, my "On Hold" status is also used when I'm waiting to hear back from someone (in this case you).
Sounds good to me 👍Is it ok for me to merge this in myself? I do have the rights (as a GitHub admin) however I'm not part of the beats team so I just wanted to check first. |
Normally the PR dev doesn't merge the PR. Can you rebase on top of master and add an changelog entry? I will merge the PR once CI is green. Thanks! |
f63257b
to
e71f6df
Compare
@urso Added a changelog entry, rebased and CI is green! |
Merged. Thank you for improving Heartbeat! |
The current behaviour of
check.response.body
is to match the entire response body. This works great for simple examples however doesn't work so great when you want to find a small string in a large body.As discussed in #5981 I have modified the
check.response.body
parameter to expect a regex pattern (match.Matcher
) instead of the entire body string to match.The awesome thing is this remained fully backwards compatible with the old body check method without any config changes even when using the json example in the docs :)
Another nice enhancement that I added was the use of subtests. I originally copied some tests from another beat and was disappointed to find out that it would stop as soon as a test case failed and not run all of them. Subtests allows you to run all tests using the table driven pattern with the output and behaviour of writing multiple tests.