Skip to content

Commit

Permalink
Allow disabling of elastic user. (#7723)
Browse files Browse the repository at this point in the history
* Allow disabling of elastic user.
* Create dedicated diagnostics user
* Minify diagnostics user's role.
---------

Signed-off-by: Michael Montgomery <mmontg1@gmail.com>
  • Loading branch information
naemono authored Apr 25, 2024
1 parent da474f8 commit 8420c9c
Show file tree
Hide file tree
Showing 13 changed files with 130 additions and 14 deletions.
5 changes: 2 additions & 3 deletions cmd/manager/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ import (
"strings"
"time"

logstashv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/logstash/v1alpha1"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/logstash"

"github.com/go-logr/logr"
"github.com/spf13/cobra"
"github.com/spf13/viper"
Expand Down Expand Up @@ -55,6 +52,7 @@ import (
entv1beta1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/enterprisesearch/v1beta1"
kbv1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/kibana/v1"
kbv1beta1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/kibana/v1beta1"
logstashv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/logstash/v1alpha1"
emsv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/maps/v1alpha1"
policyv1alpha1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/stackconfigpolicy/v1alpha1"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/agent"
Expand Down Expand Up @@ -82,6 +80,7 @@ import (
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/kibana"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/license"
licensetrial "github.com/elastic/cloud-on-k8s/v2/pkg/controller/license/trial"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/logstash"
lsvalidation "github.com/elastic/cloud-on-k8s/v2/pkg/controller/logstash/validation"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/maps"
"github.com/elastic/cloud-on-k8s/v2/pkg/controller/remoteca"
Expand Down
4 changes: 4 additions & 0 deletions config/crds/v1/all-crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3826,6 +3826,10 @@ spec:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
disableElasticUser:
description: DisableElasticUser disables the default elastic user
that is created by ECK.
type: boolean
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ spec:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
disableElasticUser:
description: DisableElasticUser disables the default elastic user
that is created by ECK.
type: boolean
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3862,6 +3862,10 @@ spec:
description: Auth contains user authentication and authorization security
settings for Elasticsearch.
properties:
disableElasticUser:
description: DisableElasticUser disables the default elastic user
that is created by ECK.
type: boolean
fileRealm:
description: FileRealm to propagate to the Elasticsearch cluster.
items:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,25 @@ kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic |

To rotate this password, refer to: <<{p}-rotate-credentials>>.

=== Disabling the default `elastic` user

If your prefer to manage all users via SSO, for example using <<{p}-saml-authentication>> or OpenID Connect, you can disable the default `elastic` superuser by setting the `auth.disableElasticUser` field in the Elasticsearch resource to `true`:

[source,yaml,subs="attributes"]
----
apiVersion: elasticsearch.k8s.elastic.co/{eck_crd_version}
kind: Elasticsearch
metadata:
name: elasticsearch-sample
spec:
version: {version}
auth:
disableElasticUser: true
nodeSets:
- name: default
count: 1
----

== Creating custom users

WARNING: Do not run the `elasticsearch-service-tokens` command inside an Elasticsearch Pod managed by the operator. This would overwrite the service account tokens used internally to authenticate the Elastic stack applications.
Expand Down
4 changes: 3 additions & 1 deletion docs/quickstart.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -162,13 +162,15 @@ quickstart-es-http ClusterIP 10.15.251.145 <none> 9200/TCP 34m

. Get the credentials.
+
A default user named `elastic` is automatically created with the password stored in a Kubernetes secret:
A default user named `elastic` is created by default with the password stored in a Kubernetes secret:
+
[source,sh]
----
PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
----

NOTE: The `elastic` user creation can be disabled if desired. Check <<{p}-users-and-roles>> for more information.

. Request the Elasticsearch endpoint.
+
From inside the Kubernetes cluster:
Expand Down
1 change: 1 addition & 0 deletions docs/reference/api-docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -1200,6 +1200,7 @@ Auth contains user authentication and authorization security settings for Elasti
| Field | Description
| *`roles`* __xref:{anchor_prefix}-github-com-elastic-cloud-on-k8s-v2-pkg-apis-elasticsearch-v1-rolesource[$$RoleSource$$] array__ | Roles to propagate to the Elasticsearch cluster.
| *`fileRealm`* __xref:{anchor_prefix}-github-com-elastic-cloud-on-k8s-v2-pkg-apis-elasticsearch-v1-filerealmsource[$$FileRealmSource$$] array__ | FileRealm to propagate to the Elasticsearch cluster.
| *`disableElasticUser`* __boolean__ | DisableElasticUser disables the default elastic user that is created by ECK.
|===


Expand Down
2 changes: 2 additions & 0 deletions pkg/apis/elasticsearch/v1/elasticsearch_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ type Auth struct {
Roles []RoleSource `json:"roles,omitempty"`
// FileRealm to propagate to the Elasticsearch cluster.
FileRealm []FileRealmSource `json:"fileRealm,omitempty"`
// DisableElasticUser disables the default elastic user that is created by ECK.
DisableElasticUser bool `json:"disableElasticUser,omitempty"`
}

// RoleSource references roles to create in the Elasticsearch cluster.
Expand Down
7 changes: 6 additions & 1 deletion pkg/controller/elasticsearch/user/predefined.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ import (
const (
// ElasticUserName is the public-facing user.
ElasticUserName = "elastic"

// ControllerUserName is the controller user to interact with ES.
ControllerUserName = "elastic-internal"
// MonitoringUserName is used for the Elasticsearch monitoring.
Expand All @@ -35,6 +34,8 @@ const (
PreStopUserName = "elastic-internal-pre-stop"
// ProbeUserName is used for the Elasticsearch readiness probe.
ProbeUserName = "elastic-internal-probe"
// DiagnosticsUserName is used for the ECK diagnostics.
DiagnosticsUserName = "elastic-internal-diagnostics"
)

// reconcileElasticUser reconciles a single secret holding the "elastic" user password.
Expand All @@ -46,6 +47,9 @@ func reconcileElasticUser(
userProvidedFileRealm filerealm.Realm,
passwordHasher cryptutil.PasswordHasher,
) (users, error) {
if es.Spec.Auth.DisableElasticUser {
return nil, nil
}
secretName := esv1.ElasticUserSecret(es.Name)
// if user has set up the elastic user via the file realm do not create the operator managed secret to avoid confusion
if userProvidedFileRealm.PasswordHashForUser(ElasticUserName) != nil {
Expand Down Expand Up @@ -89,6 +93,7 @@ func reconcileInternalUsers(
{Name: PreStopUserName, Roles: []string{ClusterManageRole}},
{Name: ProbeUserName, Roles: []string{ProbeUserRole}},
{Name: MonitoringUserName, Roles: []string{RemoteMonitoringCollectorBuiltinRole}},
{Name: DiagnosticsUserName, Roles: []string{DiagnosticsUserRole}},
},
esv1.InternalUsersSecret(es.Name),
true,
Expand Down
2 changes: 1 addition & 1 deletion pkg/controller/elasticsearch/user/predefined_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ func Test_reconcileInternalUsers(t *testing.T) {
got, err := reconcileInternalUsers(context.Background(), c, es, tt.existingFileRealm, testPasswordHasher)
require.NoError(t, err)
// check returned users
require.Len(t, got, 4)
require.Len(t, got, 5)
controllerUser := got[0]
probeUser := got[2]
// names and roles are always the same
Expand Down
47 changes: 40 additions & 7 deletions pkg/controller/elasticsearch/user/reconcile_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (

"github.com/stretchr/testify/require"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/tools/record"
Expand Down Expand Up @@ -84,18 +85,50 @@ func Test_ReconcileRolesFileRealmSecret(t *testing.T) {
}

func Test_aggregateFileRealm(t *testing.T) {
c := k8s.NewFakeClient(sampleUserProvidedFileRealmSecrets...)
fileRealm, controllerUser, err := aggregateFileRealm(context.Background(), c, sampleEsWithAuth, initDynamicWatches(), record.NewFakeRecorder(10), testPasswordHasher)
require.NoError(t, err)
require.NotEmpty(t, controllerUser.Password)
actualUsers := fileRealm.UserNames()
require.ElementsMatch(t, []string{"elastic", "elastic-internal", "elastic-internal-pre-stop", "elastic-internal-probe", "elastic-internal-monitoring", "user1", "user2", "user3"}, actualUsers)
sampleEsWithAuthAndElasticUserDisabled := sampleEsWithAuth.DeepCopy()
sampleEsWithAuthAndElasticUserDisabled.Spec.Auth.DisableElasticUser = true
tests := []struct {
name string
es esv1.Elasticsearch
expected []string
assertions func(t *testing.T, c k8s.Client, es esv1.Elasticsearch)
}{
{
name: "file realm users with elastic user enabled",
es: sampleEsWithAuth,
expected: []string{"elastic", "elastic-internal", "elastic-internal-pre-stop", "elastic-internal-probe", "elastic-internal-diagnostics", "elastic-internal-monitoring", "user1", "user2", "user3"},
},
{
name: "file realm users with elastic user disabled",
es: *sampleEsWithAuthAndElasticUserDisabled,
expected: []string{"elastic-internal", "elastic-internal-pre-stop", "elastic-internal-probe", "elastic-internal-diagnostics", "elastic-internal-monitoring", "user1", "user2", "user3"},
assertions: func(t *testing.T, c k8s.Client, es esv1.Elasticsearch) {
t.Helper()
var secret corev1.Secret
err := c.Get(context.Background(), types.NamespacedName{Namespace: es.Namespace, Name: esv1.ElasticUserSecret(es.Name)}, &secret)
require.True(t, apierrors.IsNotFound(err))
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
c := k8s.NewFakeClient(sampleUserProvidedFileRealmSecrets...)
fileRealm, controllerUser, err := aggregateFileRealm(context.Background(), c, tt.es, initDynamicWatches(), record.NewFakeRecorder(10), testPasswordHasher)
require.NoError(t, err)
require.NotEmpty(t, controllerUser.Password)
actualUsers := fileRealm.UserNames()
require.ElementsMatch(t, tt.expected, actualUsers)
if tt.assertions != nil {
tt.assertions(t, c, tt.es)
}
})
}
}

func Test_aggregateRoles(t *testing.T) {
c := k8s.NewFakeClient(sampleUserProvidedRolesSecret...)
roles, err := aggregateRoles(context.Background(), c, sampleEsWithAuth, initDynamicWatches(), record.NewFakeRecorder(10))
require.NoError(t, err)
require.Len(t, roles, 55)
require.Len(t, roles, 56)
require.Contains(t, roles, ProbeUserRole, ClusterManageRole, "role1", "role2")
}
43 changes: 43 additions & 0 deletions pkg/controller/elasticsearch/user/roles.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"fmt"

"gopkg.in/yaml.v3"
"k8s.io/utils/ptr"

beatv1beta1 "github.com/elastic/cloud-on-k8s/v2/pkg/apis/beat/v1beta1"
esclient "github.com/elastic/cloud-on-k8s/v2/pkg/controller/elasticsearch/client"
Expand All @@ -25,6 +26,8 @@ const (
ProbeUserRole = "elastic_internal_probe_user"
// RemoteMonitoringCollectorBuiltinRole is the name of the built-in remote_monitoring_collector role.
RemoteMonitoringCollectorBuiltinRole = "remote_monitoring_collector"
// DiagnosticsUserRole is the name of the built-in role for ECK diagnostics use.
DiagnosticsUserRole = "elastic_internal_diagnostics"

// ApmUserRoleV6 is the name of the role used by 6.8.x APMServer instances to connect to Elasticsearch.
ApmUserRoleV6 = "eck_apm_user_role_v6"
Expand Down Expand Up @@ -66,6 +69,46 @@ var (
PredefinedRoles = RolesFileContent{
ProbeUserRole: esclient.Role{Cluster: []string{"monitor"}},
ClusterManageRole: esclient.Role{Cluster: []string{"manage"}},
DiagnosticsUserRole: esclient.Role{
Cluster: []string{"monitor", "monitor_snapshot", "manage", "read_ilm", "read_security"},

This comment has been minimized.

Copy link
@barkbay

barkbay May 6, 2024

Contributor

I think the read_security role does not exist before 8.5.0, I get the following error in Kibana when I try to get a diagnostic on 8.1.3:

[2024-05-06T14:27:35.957+00:00][INFO ][plugins.security.authentication] Authentication attempt failed: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown cluster privilege [read_security]. a privilege must be either one of the predefined cluster privilege names [manage_own_api_key,none,cancel_task,delegate_pki,grant_api_key,manage_autoscaling,manage_enrich,manage_index_templates,manage_logstash_pipelines,manage_oidc,manage_saml,manage_service_account,manage_token,manage_user_profile,monitor_ml,monitor_rollup,monitor_snapshot,monitor_text_structure,monitor_watcher,read_ccr,read_ilm,read_pipeline,read_slm,transport_client,create_snapshot,manage_ccr,manage_ilm,manage_ml,manage_rollup,manage_slm,manage_watcher,monitor_data_frame_transforms,monitor_transform,manage_api_key,manage_ingest_pipelines,manage_pipeline,manage_data_frame_transforms,manage_transform,manage_security,monitor,manage,all] or a pattern over one of the available cluster actions"}],"type":"illegal_argument_exception","reason":"unknown cluster privilege [read_security]. a privilege must be either one of the predefined cluster privilege names [manage_own_api_key,none,cancel_task,delegate_pki,grant_api_key,manage_autoscaling,manage_enrich,manage_index_templates,manage_logstash_pipelines,manage_oidc,manage_saml,manage_service_account,manage_token,manage_user_profile,monitor_ml,monitor_rollup,monitor_snapshot,monitor_text_structure,monitor_watcher,read_ccr,read_ilm,read_pipeline,read_slm,transport_client,create_snapshot,manage_ccr,manage_ilm,manage_ml,manage_rollup,manage_slm,manage_watcher,monitor_data_frame_transforms,monitor_transform,manage_api_key,manage_ingest_pipelines,manage_pipeline,manage_data_frame_transforms,manage_transform,manage_security,monitor,manage,all] or a pattern over one of the available cluster actions"},"status":400}
14:27:35.635 [main] INFO  co.elastic.support.diagnostics.commands.CheckKibanaVersion - Getting Kibana Version.
14:27:36.027 [main] ERROR co.elastic.support.diagnostics.commands.CheckKibanaVersion - Unanticipated error:
co.elastic.support.diagnostics.DiagnosticException: Could not retrieve the Kibana version - unable to continue. Status: 400  Reason: Bad Request. Rejected
	at co.elastic.support.diagnostics.commands.CheckKibanaVersion.getKibanaVersion(CheckKibanaVersion.java:102) ~[diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.commands.CheckKibanaVersion.execute(CheckKibanaVersion.java:72) [diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.chain.DiagnosticChainExec.runDiagnostic(DiagnosticChainExec.java:126) [diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.DiagnosticService.exec(DiagnosticService.java:86) [diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.DiagnosticApp.main(DiagnosticApp.java:51) [diagnostics-8.5.0.jar:8.5.0]

This comment has been minimized.

Copy link
@barkbay

barkbay May 6, 2024

Contributor

Same for Elasticsearch:

co.elastic.support.diagnostics.DiagnosticException: Could not retrieve the Elasticsearch version - unable to continue. Status: 400  Reason: Bad Request. Rejected
	at co.elastic.support.diagnostics.commands.CheckElasticsearchVersion.getElasticsearchVersion(CheckElasticsearchVersion.java:82) ~[diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.commands.CheckElasticsearchVersion.execute(CheckElasticsearchVersion.java:67) [diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.chain.DiagnosticChainExec.runDiagnostic(DiagnosticChainExec.java:35) [diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.DiagnosticService.exec(DiagnosticService.java:86) [diagnostics-8.5.0.jar:8.5.0]
	at co.elastic.support.diagnostics.DiagnosticApp.main(DiagnosticApp.java:51) [diagnostics-8.5.0.jar:8.5.0]

I think https://github.com/elastic/eck-diagnostics does not collect ES or Kibana diagnostics for version < 8.5 where read_security has been introduced.

This comment has been minimized.

Copy link
@barkbay

barkbay May 7, 2024

Contributor

Created the following issue to track this: #7776

Indices: []esclient.IndexRole{
{
Names: []string{"*"},
Privileges: []string{"monitor", "read", "view_index_metadata"},
AllowRestrictedIndices: ptr.To[bool](true),
},
},
Applications: []esclient.ApplicationRole{
{
Application: "kibana-.kibana",
Resources: []string{"*"},
Privileges: []string{
"feature_ml.read",
"feature_siem.read",
"feature_siem.read_alerts",
"feature_siem.policy_management_read",
"feature_siem.endpoint_list_read",
"feature_siem.trusted_applications_read",
"feature_siem.event_filters_read",
"feature_siem.host_isolation_exceptions_read",
"feature_siem.blocklist_read",
"feature_siem.actions_log_management_read",
"feature_securitySolutionCases.read",
"feature_securitySolutionAssistant.read",
"feature_actions.read",
"feature_builtInAlerts.read",
"feature_fleet.all",
"feature_fleetv2.all",
"feature_osquery.read",
"feature_indexPatterns.read",
"feature_discover.read",
"feature_dashboard.read",
"feature_maps.read",
"feature_visualize.read",
},
},
},
},
ApmUserRoleV6: esclient.Role{
Cluster: []string{"monitor", "manage_index_templates"},
Indices: []esclient.IndexRole{
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/test/elasticsearch/checks_k8s.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ func CheckSecrets(b Builder, k *test.K8sClient) test.Step {
},
{
Name: esName + "-es-internal-users",
Keys: []string{"elastic-internal", "elastic-internal-monitoring", "elastic-internal-pre-stop", "elastic-internal-probe"},
Keys: []string{"elastic-internal", "elastic-internal-monitoring", "elastic-internal-diagnostics", "elastic-internal-pre-stop", "elastic-internal-probe"},
Labels: map[string]string{
"common.k8s.elastic.co/type": "elasticsearch",
"eck.k8s.elastic.co/credentials": "true",
Expand Down

0 comments on commit 8420c9c

Please sign in to comment.