Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow disabling of elastic user. #7723

Merged
merged 11 commits into from
Apr 25, 2024
Merged

Allow disabling of elastic user. #7723

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
1e72d5c
Allow disabling of elastic user.
naemono Apr 17, 2024
b499997
Add helper to test func.
naemono Apr 18, 2024
e2040f7
Add documentation for disabling the elastic user.
naemono Apr 22, 2024
4028ed0
Add `disableElasticUser` to the api docs.
naemono Apr 22, 2024
da83a81
Fix unit tests
naemono Apr 22, 2024
ba8ce8d
Merge branch 'main' into remove-elastic-user
naemono Apr 22, 2024
dc15ca1
Fix e2e smoke tests.
naemono Apr 22, 2024
7f69257
Wording update.
naemono Apr 22, 2024
cbb3d8a
remove conditional for adding file realm users
naemono Apr 23, 2024
7b81d54
Try a minimal set of privileges.
naemono Apr 23, 2024
b26c6db
revert fo fleet.all
naemono Apr 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions cmd/manager/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ import (
"strings"
"time"

logstashv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/logstash/v1alpha1"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/logstash"

"github.com/go-logr/logr"
"github.com/spf13/cobra"
"github.com/spf13/viper"
Expand Down Expand Up @@ -55,6 +52,7 @@ import (
entv1beta1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/enterprisesearch/v1beta1"
kbv1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/kibana/v1"
kbv1beta1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/kibana/v1beta1"
logstashv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/logstash/v1alpha1"
emsv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/maps/v1alpha1"
policyv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/stackconfigpolicy/v1alpha1"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/agent"
Expand Down Expand Up @@ -82,6 +80,7 @@ import (
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/kibana"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/license"
licensetrial "github.com/elastic/cloud-on-k8s/v2/pkg/controller/license/trial"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/logstash"
lsvalidation "github.com/elastic/cloud-on-k8s/v2/pkg/controller/logstash/validation"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/maps"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/remoteca"
Expand Down
4 changes: 4 additions & 0 deletions config/crds/v1/all-crds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3826,6 +3826,10 @@ spec:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
disableElasticUser:
description: DisableElasticUser disables the default elastic user
that is created by ECK.
type: boolean
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
Expand Down
4 changes: 4 additions & 0 deletions config/crds/v1/bases/elasticsearch.k8s.elastic.co_elasticsearches.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ spec:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
disableElasticUser:
description: DisableElasticUser disables the default elastic user
that is created by ECK.
type: boolean
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
Expand Down
4 changes: 4 additions & 0 deletions deploy/eck-operator/charts/eck-operator-crds/templates/all-crds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3862,6 +3862,10 @@ spec:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
disableElasticUser:
description: DisableElasticUser disables the default elastic user
that is created by ECK.
type: boolean
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
Expand Down
19 changes: 19 additions & 0 deletions docs/orchestrating-elastic-stack-applications/security/users-and-roles.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,25 @@ kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic |

To rotate this password, refer to: <<{p}-rotate-credentials>>.

=== Disabling the default `elastic` user

If your prefer to manage all users via SSO, for example using <<{p}-saml-authentication>> or OpenID Connect, you can disable the default `elastic` superuser by setting the `auth.disableElasticUser` field in the Elasticsearch resource to `true`:

[source,yaml,subs="attributes"]
----
apiVersion: elasticsearch.k8s.elastic.co/{eck_crd_version}
kind: Elasticsearch
metadata:
name: elasticsearch-sample
spec:
version: {version}
auth:
disableElasticUser: true
nodeSets:
- name: default
count: 1
----

== Creating custom users

WARNING: Do not run the `elasticsearch-service-tokens` command inside an Elasticsearch Pod managed by the operator. This would overwrite the service account tokens used internally to authenticate the Elastic stack applications.
Expand Down
4 changes: 3 additions & 1 deletion docs/quickstart.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,13 +162,15 @@ quickstart-es-http ClusterIP 10.15.251.145 <none> 9200/TCP 34m

. Get the credentials.
+
A default user named `elastic` is automatically created with the password stored in a Kubernetes secret:
A default user named `elastic` is created by default with the password stored in a Kubernetes secret:
+
[source,sh]
----
PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
----

NOTE: The `elastic` user creation can be disabled if desired. Check <<{p}-users-and-roles>> for more information.

. Request the Elasticsearch endpoint.
+
From inside the Kubernetes cluster:
Expand Down
1 change: 1 addition & 0 deletions docs/reference/api-docs.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1200,6 +1200,7 @@ Auth contains user authentication and authorization security settings for Elasti
| Field | Description
| *`roles`* __xref:{anchor_prefix}-github-com-elastic-cloud-on-k8s-v2-pkg-apis-elasticsearch-v1-rolesource[$$RoleSource$$] array__ | Roles to propagate to the Elasticsearch cluster.
| *`fileRealm`* __xref:{anchor_prefix}-github-com-elastic-cloud-on-k8s-v2-pkg-apis-elasticsearch-v1-filerealmsource[$$FileRealmSource$$] array__ | FileRealm to propagate to the Elasticsearch cluster.
| *`disableElasticUser`* __boolean__ | DisableElasticUser disables the default elastic user that is created by ECK.
|===


Expand Down
2 changes: 2 additions & 0 deletions pkg/apis/elasticsearch/v1/elasticsearch_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ type Auth struct {
Roles []RoleSource `json:"roles,omitempty"`
// FileRealm to propagate to the Elasticsearch cluster.
FileRealm []FileRealmSource `json:"fileRealm,omitempty"`
// DisableElasticUser disables the default elastic user that is created by ECK.
DisableElasticUser bool `json:"disableElasticUser,omitempty"`
}

// RoleSource references roles to create in the Elasticsearch cluster.
Expand Down
7 changes: 6 additions & 1 deletion pkg/controller/elasticsearch/user/predefined.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ import (
const (
// ElasticUserName is the public-facing user.
ElasticUserName = "elastic"

// ControllerUserName is the controller user to interact with ES.
ControllerUserName = "elastic-internal"
// MonitoringUserName is used for the Elasticsearch monitoring.
Expand All @@ -35,6 +34,8 @@ const (
PreStopUserName = "elastic-internal-pre-stop"
// ProbeUserName is used for the Elasticsearch readiness probe.
ProbeUserName = "elastic-internal-probe"
// DiagnosticsUserName is used for the ECK diagnostics.
DiagnosticsUserName = "elastic-internal-diagnostics"
)

// reconcileElasticUser reconciles a single secret holding the "elastic" user password.
Expand All @@ -46,6 +47,9 @@ func reconcileElasticUser(
userProvidedFileRealm filerealm.Realm,
passwordHasher cryptutil.PasswordHasher,
) (users, error) {
if es.Spec.Auth.DisableElasticUser {
return nil, nil
}
secretName := esv1.ElasticUserSecret(es.Name)
// if user has set up the elastic user via the file realm do not create the operator managed secret to avoid confusion
if userProvidedFileRealm.PasswordHashForUser(ElasticUserName) != nil {
Expand Down Expand Up @@ -89,6 +93,7 @@ func reconcileInternalUsers(
{Name: PreStopUserName, Roles: []string{ClusterManageRole}},
{Name: ProbeUserName, Roles: []string{ProbeUserRole}},
{Name: MonitoringUserName, Roles: []string{RemoteMonitoringCollectorBuiltinRole}},
{Name: DiagnosticsUserName, Roles: []string{DiagnosticsUserRole}},
},
esv1.InternalUsersSecret(es.Name),
true,
Expand Down
2 changes: 1 addition & 1 deletion pkg/controller/elasticsearch/user/predefined_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ func Test_reconcileInternalUsers(t *testing.T) {
got, err := reconcileInternalUsers(context.Background(), c, es, tt.existingFileRealm, testPasswordHasher)
require.NoError(t, err)
// check returned users
require.Len(t, got, 4)
require.Len(t, got, 5)
controllerUser := got[0]
probeUser := got[2]
// names and roles are always the same
Expand Down
12 changes: 6 additions & 6 deletions pkg/controller/elasticsearch/user/reconcile.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,12 +122,12 @@ func aggregateFileRealm(
}

// merge all file realms together, the last one having precedence
fileRealm := filerealm.MergedFrom(
internalUsers.fileRealm(),
elasticUser.fileRealm(),
associatedUsers.fileRealm(),
userProvidedFileRealm,
)
fileRealmUsers := []filerealm.Realm{internalUsers.fileRealm(), associatedUsers.fileRealm(), userProvidedFileRealm}
if !es.Spec.Auth.DisableElasticUser {
// not using append here as order matters.
fileRealmUsers = []filerealm.Realm{internalUsers.fileRealm(), elasticUser.fileRealm(), associatedUsers.fileRealm(), userProvidedFileRealm}
}
fileRealm := filerealm.MergedFrom(fileRealmUsers...)
naemono marked this conversation as resolved.
Show resolved Hide resolved

// grab the controller user credentials for later use
controllerCreds, err := internalUsers.credentialsFor(ControllerUserName)
Expand Down
47 changes: 40 additions & 7 deletions pkg/controller/elasticsearch/user/reconcile_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (

"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/tools/record"
Expand Down Expand Up @@ -84,18 +85,50 @@ func Test_ReconcileRolesFileRealmSecret(t *testing.T) {
}

func Test_aggregateFileRealm(t *testing.T) {
c := k8s.NewFakeClient(sampleUserProvidedFileRealmSecrets...)
fileRealm, controllerUser, err := aggregateFileRealm(context.Background(), c, sampleEsWithAuth, initDynamicWatches(), record.NewFakeRecorder(10), testPasswordHasher)
require.NoError(t, err)
require.NotEmpty(t, controllerUser.Password)
actualUsers := fileRealm.UserNames()
require.ElementsMatch(t, []string{"elastic", "elastic-internal", "elastic-internal-pre-stop", "elastic-internal-probe", "elastic-internal-monitoring", "user1", "user2", "user3"}, actualUsers)
sampleEsWithAuthAndElasticUserDisabled := sampleEsWithAuth.DeepCopy()
sampleEsWithAuthAndElasticUserDisabled.Spec.Auth.DisableElasticUser = true
tests := []struct {
name string
es esv1.Elasticsearch
expected []string
assertions func(t *testing.T, c k8s.Client, es esv1.Elasticsearch)
}{
{
name: "file realm users with elastic user enabled",
es: sampleEsWithAuth,
expected: []string{"elastic", "elastic-internal", "elastic-internal-pre-stop", "elastic-internal-probe", "elastic-internal-diagnostics", "elastic-internal-monitoring", "user1", "user2", "user3"},
},
{
name: "file realm users with elastic user disabled",
es: *sampleEsWithAuthAndElasticUserDisabled,
expected: []string{"elastic-internal", "elastic-internal-pre-stop", "elastic-internal-probe", "elastic-internal-diagnostics", "elastic-internal-monitoring", "user1", "user2", "user3"},
assertions: func(t *testing.T, c k8s.Client, es esv1.Elasticsearch) {
t.Helper()
var secret corev1.Secret
err := c.Get(context.Background(), types.NamespacedName{Namespace: es.Namespace, Name: esv1.ElasticUserSecret(es.Name)}, &secret)
require.True(t, apierrors.IsNotFound(err))
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
c := k8s.NewFakeClient(sampleUserProvidedFileRealmSecrets...)
fileRealm, controllerUser, err := aggregateFileRealm(context.Background(), c, tt.es, initDynamicWatches(), record.NewFakeRecorder(10), testPasswordHasher)
require.NoError(t, err)
require.NotEmpty(t, controllerUser.Password)
actualUsers := fileRealm.UserNames()
require.ElementsMatch(t, tt.expected, actualUsers)
if tt.assertions != nil {
tt.assertions(t, c, tt.es)
}
})
}
}

func Test_aggregateRoles(t *testing.T) {
c := k8s.NewFakeClient(sampleUserProvidedRolesSecret...)
roles, err := aggregateRoles(context.Background(), c, sampleEsWithAuth, initDynamicWatches(), record.NewFakeRecorder(10))
require.NoError(t, err)
require.Len(t, roles, 55)
require.Len(t, roles, 56)
require.Contains(t, roles, ProbeUserRole, ClusterManageRole, "role1", "role2")
}
48 changes: 48 additions & 0 deletions pkg/controller/elasticsearch/user/roles.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"fmt"

"gopkg.in/yaml.v3"
"k8s.io/utils/ptr"

beatv1beta1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/beat/v1beta1"
esclient "github.com/elastic/cloud-on-k8s/v2/pkg/controller/elasticsearch/client"
Expand All @@ -25,6 +26,8 @@ const (
ProbeUserRole = "elastic_internal_probe_user"
// RemoteMonitoringCollectorBuiltinRole is the name of the built-in remote_monitoring_collector role.
RemoteMonitoringCollectorBuiltinRole = "remote_monitoring_collector"
// DiagnosticsUserRole is the name of the built-in role for ECK diagnostics use.
DiagnosticsUserRole = "elastic_internal_diagnostics"

// ApmUserRoleV6 is the name of the role used by 6.8.x APMServer instances to connect to Elasticsearch.
ApmUserRoleV6 = "eck_apm_user_role_v6"
Expand Down Expand Up @@ -66,6 +69,51 @@ var (
PredefinedRoles = RolesFileContent{
ProbeUserRole: esclient.Role{Cluster: []string{"monitor"}},
ClusterManageRole: esclient.Role{Cluster: []string{"manage"}},
DiagnosticsUserRole: esclient.Role{
Cluster: []string{"monitor", "monitor_snapshot", "manage", "read_ilm", "read_security"},
Indices: []esclient.IndexRole{
{
Names: []string{"*"},
Privileges: []string{"monitor", "read", "view_index_metadata"},
AllowRestrictedIndices: ptr.To[bool](true),
},
},
Applications: []esclient.ApplicationRole{
{
Application: "kibana-.kibana",
Resources: []string{"*"},
// Unfortunately the following privileges are not sufficient access all Kibana APIs
// that are required for the ECK diagnostics. In the future we should try again to
// generate a more fine-grained role and not use "*".
Privileges: []string{
"*",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you still investigating this or is this the least privileges we can give?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm going to take another shot at this, as one of the items referenced in the readme (/api/exception_lists/items/_find?list_id=endpoint_host_isolation_exceptions&namespace_type=agnostic) is still under investigation. The rest of the pieces here are ready for review. This could change, and I'll note it here if it does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pebrc update: I've found that some of these endpoints require platinum licenses, which is why I'm getting unauthorized errors when attempting to call them, even with privileges that should allow me to call them, as I'm running a dev version of ECK make go-run which doesn't support enterprise features.

I'm going to try and test this a bit differently and update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok @pebrc with the current roles, which are as limited as I can get them, I can successfully run eck-diagnostics with no issues. This should be ready for final reviews. An upcoming PR to eck-diagnostics will be upcoming once this is merged.

// "feature_ml.all",
// "feature_siem.all",
// "feature_siem.read_alerts",
// "feature_siem.crud_alerts",
// "feature_siem.policy_management_all",
// "feature_siem.endpoint_list_all",
// "feature_siem.trusted_applications_all",
// "feature_siem.event_filters_all",
// "feature_siem.host_isolation_exceptions_all",
// "feature_siem.blocklist_all",
// "feature_siem.actions_log_management_read",
// "feature_securitySolutionCases.all",
// "feature_securitySolutionAssistant.all",
// "feature_actions.all",
// "feature_builtInAlerts.all",
// "feature_fleet.all",
// "feature_fleetv2.all",
// "feature_osquery.all",
// "feature_indexPatterns.all",
// "feature_discover.all",
// "feature_dashboard.all",
// "feature_maps.all",
// "feature_visualize.all",
},
},
},
},
ApmUserRoleV6: esclient.Role{
Cluster: []string{"monitor", "manage_index_templates"},
Indices: []esclient.IndexRole{
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/test/elasticsearch/checks_k8s.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ func CheckSecrets(b Builder, k *test.K8sClient) test.Step {
},
{
Name: esName + "-es-internal-users",
Keys: []string{"elastic-internal", "elastic-internal-monitoring", "elastic-internal-pre-stop", "elastic-internal-probe"},
Keys: []string{"elastic-internal", "elastic-internal-monitoring", "elastic-internal-diagnostics", "elastic-internal-pre-stop", "elastic-internal-probe"},
Labels: map[string]string{
"common.k8s.elastic.co/type": "elasticsearch",
"eck.k8s.elastic.co/credentials": "true",
Expand Down