Skip to content

Commit

Permalink
Add explicit username and password option for Mongodb
Browse files Browse the repository at this point in the history
We previously only had the option of specifying a user/pass in the
URL string, which is problematic because it results in the password
being indexed in Elasticsearch (#2888).

This adds the option to specify a username/password at the module
configuration. To make this happen, I had to copy some unexported
functions from the mgo driver.
  • Loading branch information
Tudor Golubenco committed Oct 31, 2016
1 parent b80e26e commit c03da67
Show file tree
Hide file tree
Showing 9 changed files with 287 additions and 9 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ https://github.com/elastic/beats/compare/v5.0.0...master[Check the HEAD diff]
- Add experimental filebeat metricset in the beats module. {pull}2297[2297]
- Add experimental libbeat metricset in the beats module. {pull}2339[2339]
- Add experimental docker module. Provided by Ingensi and @douaejeouit based on dockbeat.
- Add username and password config options to the MongoDB module. {pull}2889[2889]

*Packetbeat*
- Define `client_geoip.location` as geo_point in the mappings to be used by the GeoIP processor in the Ingest Node pipeline.
Expand Down
28 changes: 26 additions & 2 deletions metricbeat/docs/modules/mongodb.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ servers.
When configuring the `hosts` option, you must use MongoDB URLs of the following format:

-----------------------------------
[mongodb://][user:pass@]host[:port]
[mongodb://host[:port][?options]
-----------------------------------

The URL can be as simple as:
Expand All @@ -33,6 +33,21 @@ Or more complex like:
hosts: ["mongodb://myuser:mypass@localhost:40001", "otherhost:40001"]
----------------------------------------------------------------------

WARNING: In case you use username and password in the hosts array, this
information will be sent with each event as part of the `metricset.host` field.
To prevent sending username and password the config options `username` and
`password` can be used.

[source,yaml]
----
- module: mongodb
metricsets: ["status"]
hosts: ["localhost:27017"]
username: root
password: test
----



[float]
=== Compatibility
Expand All @@ -55,9 +70,18 @@ metricbeat.modules:
#period: 10s
# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]
# Username to use when connecting to MongoDB. Empty by default.
#username: user
# Password to use when connecting to MongoDB. Empty by default.
#password: pass
----

[float]
Expand Down
11 changes: 10 additions & 1 deletion metricbeat/etc/beat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass


#-------------------------------- MySQL Module -------------------------------
#- module: mysql
Expand Down
11 changes: 10 additions & 1 deletion metricbeat/metricbeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,18 @@ metricbeat.modules:
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass


#-------------------------------- MySQL Module -------------------------------
#- module: mysql
Expand Down
11 changes: 10 additions & 1 deletion metricbeat/module/mongodb/_meta/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@
#period: 10s

# The hosts must be passed as MongoDB URLs in the format:
# [mongodb://][user:pass@]host[:port]
# [mongodb://][user:pass@]host[:port].
# Warning: specifying the user/password in the hosts array is possible
# but it will result in the password being present in the output documents.
# We recommend using the username and password options instead.
#hosts: ["localhost:27017"]

# Username to use when connecting to MongoDB. Empty by default.
#username: user

# Password to use when connecting to MongoDB. Empty by default.
#password: pass

17 changes: 16 additions & 1 deletion metricbeat/module/mongodb/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ servers.
When configuring the `hosts` option, you must use MongoDB URLs of the following format:

-----------------------------------
[mongodb://][user:pass@]host[:port]
[mongodb://host[:port][?options]
-----------------------------------

The URL can be as simple as:
Expand All @@ -28,6 +28,21 @@ Or more complex like:
hosts: ["mongodb://myuser:mypass@localhost:40001", "otherhost:40001"]
----------------------------------------------------------------------

WARNING: In case you use username and password in the hosts array, this
information will be sent with each event as part of the `metricset.host` field.
To prevent sending username and password the config options `username` and
`password` can be used.

[source,yaml]
----
- module: mongodb
metricsets: ["status"]
hosts: ["localhost:27017"]
username: root
password: test
----



[float]
=== Compatibility
Expand Down
123 changes: 123 additions & 0 deletions metricbeat/module/mongodb/parseurl.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
package mongodb

import (
"errors"
"fmt"
"net/url"
"strings"

mgo "gopkg.in/mgo.v2"
)

/*
* Functions copied from the mgo driver to help with parsing the URL.
*
* http://bazaar.launchpad.net/+branch/mgo/v2/view/head:/session.go#L382
*/

type urlInfo struct {
addrs []string
user string
pass string
db string
options map[string]string
}

func parseURL(s string) (*urlInfo, error) {
if strings.HasPrefix(s, "mongodb://") {
s = s[10:]
}
info := &urlInfo{options: make(map[string]string)}
if c := strings.Index(s, "?"); c != -1 {
for _, pair := range strings.FieldsFunc(s[c+1:], isOptSep) {
l := strings.SplitN(pair, "=", 2)
if len(l) != 2 || l[0] == "" || l[1] == "" {
return nil, errors.New("connection option must be key=value: " + pair)
}
info.options[l[0]] = l[1]
}
s = s[:c]
}
if c := strings.Index(s, "@"); c != -1 {
pair := strings.SplitN(s[:c], ":", 2)
if len(pair) > 2 || pair[0] == "" {
return nil, errors.New("credentials must be provided as user:pass@host")
}
var err error
info.user, err = url.QueryUnescape(pair[0])
if err != nil {
return nil, fmt.Errorf("cannot unescape username in URL: %q", pair[0])
}
if len(pair) > 1 {
info.pass, err = url.QueryUnescape(pair[1])
if err != nil {
return nil, fmt.Errorf("cannot unescape password in URL")
}
}
s = s[c+1:]
}
if c := strings.Index(s, "/"); c != -1 {
info.db = s[c+1:]
s = s[:c]
}
info.addrs = strings.Split(s, ",")
return info, nil
}

func isOptSep(c rune) bool {
return c == ';' || c == '&'
}

// ParseURL parses the given URL and returns a DialInfo structure ready
// to be passed to DialWithInfo
func ParseURL(host, username, pass string) (*mgo.DialInfo, error) {
uinfo, err := parseURL(host)
if err != nil {
return nil, err
}
direct := false
mechanism := ""
service := ""
source := ""
for k, v := range uinfo.options {
switch k {
case "authSource":
source = v
case "authMechanism":
mechanism = v
case "gssapiServiceName":
service = v
case "connect":
if v == "direct" {
direct = true
break
}
if v == "replicaSet" {
break
}
fallthrough
default:
return nil, errors.New("unsupported connection URL option: " + k + "=" + v)
}
}

info := &mgo.DialInfo{
Addrs: uinfo.addrs,
Direct: direct,
Database: uinfo.db,
Username: uinfo.user,
Password: uinfo.pass,
Mechanism: mechanism,
Service: service,
Source: source,
}

if len(username) > 0 {
info.Username = username
}
if len(pass) > 0 {
info.Password = pass
}

return info, nil
}
78 changes: 78 additions & 0 deletions metricbeat/module/mongodb/parseurl_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package mongodb

import (
"testing"

"github.com/stretchr/testify/assert"
)

func TestParseURL(t *testing.T) {
tests := []struct {
Name string
URL string
Username string
Password string
ExpectedAddr string
ExpectedUsername string
ExpectedPassword string
}{
{
Name: "basic test",
URL: "localhost:40001",
Username: "user",
Password: "secret",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "with schema",
URL: "mongodb://localhost:40001",
Username: "user",
Password: "secret",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "user password in url",
URL: "mongodb://user:secret@localhost:40001",
Username: "",
Password: "",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "user",
ExpectedPassword: "secret",
},
{
Name: "user password overwride",
URL: "mongodb://user:secret@localhost:40001",
Username: "anotheruser",
Password: "anotherpass",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "anotheruser",
ExpectedPassword: "anotherpass",
},
{
Name: "with options",
URL: "mongodb://localhost:40001?connect=direct&authSource=me",
Username: "anotheruser",
Password: "anotherpass",

ExpectedAddr: "localhost:40001",
ExpectedUsername: "anotheruser",
ExpectedPassword: "anotherpass",
},
}

for _, test := range tests {
info, err := ParseURL(test.URL, test.Username, test.Password)
assert.NoError(t, err, test.Name)
assert.Equal(t, info.Addrs[0], test.ExpectedAddr, test.Name)
assert.Equal(t, info.Username, test.ExpectedUsername, test.Name)
assert.Equal(t, info.Password, test.ExpectedPassword, test.Name)
}
}
Loading

0 comments on commit c03da67

Please sign in to comment.