Add Integration Test to confirm Core Change to Fix Search template request Auth #2921

derek-ho · 2023-06-29T21:34:12Z

Adds integration test to verify change in core, allowing proper authorization of search template request

related to: #1678

Description

[Describe what this change achieves]

Category (Enhancement, New feature, Bug fix, Test fix, Refactoring, Maintenance, Documentation)
Why these changes are required?
What is the old behavior before changes and new behavior after changes?

Issues Resolved

[List any issues this PR will resolve]

Is this a backport? If so, please add backport PR # and/or commits #

Testing

[Please provide details of testing done: unit testing, integration testing and manual testing]

Check List

New functionality includes testing
New functionality has been documented
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…ices you have access to Signed-off-by: Derek Ho <dxho@amazon.com>

codecov · 2023-06-29T21:40:10Z

Codecov Report

Merging #2921 (f3e5d46) into main (5e8f12c) will increase coverage by 0.03%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #2921      +/-   ##
============================================
+ Coverage     62.42%   62.45%   +0.03%     
- Complexity     3352     3354       +2     
============================================
  Files           254      254              
  Lines         19749    19749              
  Branches       3334     3334              
============================================
+ Hits          12329    12335       +6     
+ Misses         5788     5785       -3     
+ Partials       1632     1629       -3

see 5 files with indirect coverage changes

Signed-off-by: Derek Ho <dxho@amazon.com>

derek-ho · 2023-06-29T21:53:57Z

src/integrationTest/java/org/opensearch/security/privileges/PrivilegesEvaluatorTest.java

@@ -68,4 +73,11 @@ public void testRegexPattern() throws Exception {
        }

    }
+
+    @Test
+    public void testSearchTemplateRequest() {


Before adding search template into isClusterPerm function this would fail since it would come back with a 403 response code

Signed-off-by: Derek Ho <dxho@amazon.com>

cwperks · 2023-06-30T13:09:54Z

src/integrationTest/java/org/opensearch/security/privileges/PrivilegesEvaluatorTest.java

@@ -44,10 +45,15 @@ public class PrivilegesEvaluatorTest {
        new Role("negated_regex_role").indexPermissions("read").on("/^[a-z].*/").clusterPermissions("cluster_composite_ops")
    );

+    protected final static TestSecurityConfig.User SEARCH_TEMPLATE = new TestSecurityConfig.User("search_template_user").roles(
+        new Role("search_template_role").indexPermissions("read").on("/^[a-z].*/").clusterPermissions("indices:data/read/search/template")


Thank you @derek-ho ! This change looks good to me. I only have one concern about breaking existing roles that are unknowingly working with the bug in place, but since it is a bug fix I think its alright to provide documentation on how to update a role to make it so that SearchTemplateRequests are working properly. I also think it is worthwhile to modify the cluster_composite_ops_ro static action group to include search template and msearch template.

To be clear, the role below allows SearchTemplateRequests to work even with the bug in place:

search_all: index_permissions: - index_patterns: - "*" allowed_actions: - "read"

and after this change the role would need to be changed to:

search_all: cluster_permissions: - "indices:data/read/search/template" or "cluster_composite_ops_ro" <- action group containing the underlying cluster perm index_permissions: - index_patterns: - "*" allowed_actions: - "read"

With the bug in place the following role would not permit search template requests to work properly

logs_search: index_permissions: - index_patterns: - "logs-*" allowed_actions: - "read"

but that would be achievable now with the following role definition:

logs_search: cluster_permissions: - "indices:data/read/search/template" index_permissions: - index_patterns: - "logs-*" allowed_actions: - "read"

I don't think we can make the change for 2.x since it will be a breaking change. While it is an edge case, I don't think it matters if it is for a bug fix or not. Unless I am mistaken, semantic versioning would dictate we wait until 3.0 to make this available.

Can you remove the clusterPermissions section for this role?

cwperks · 2023-06-30T13:27:14Z

@shikharj05 FYI this PR is related to a bug ~~fix~~ you filed last year.

stephen-crawford

This looks like a good contribution @derek-ho. I don't believe we will be able to backport this to 2.x however since it will cause a breaking change and despite the niche case we should be strict in adherence to semantic versioning. Looks good to me after you are able to resolve Craig's comments.

stephen-crawford · 2023-07-03T20:41:51Z

src/integrationTest/java/org/opensearch/security/privileges/PrivilegesEvaluatorTest.java

@@ -44,10 +45,15 @@ public class PrivilegesEvaluatorTest {
        new Role("negated_regex_role").indexPermissions("read").on("/^[a-z].*/").clusterPermissions("cluster_composite_ops")
    );

+    protected final static TestSecurityConfig.User SEARCH_TEMPLATE = new TestSecurityConfig.User("search_template_user").roles(
+        new Role("search_template_role").indexPermissions("read").on("/^[a-z].*/").clusterPermissions("indices:data/read/search/template")


I don't think we can make the change for 2.x since it will be a breaking change. While it is an edge case, I don't think it matters if it is for a bug fix or not. Unless I am mistaken, semantic versioning would dictate we wait until 3.0 to make this available.

davidlago · 2023-07-17T12:50:56Z

Gentle bump on this one, @derek-ho. Are you still intending to merge this one?

derek-ho · 2023-07-19T15:46:42Z

Gentle bump on this one, @derek-ho. Are you still intending to merge this one?

Got caught up in some release work - will put it in draft, having some conversations with @cwperks about the right path forward on this one.

Signed-off-by: Derek Ho <dxho@amazon.com>

src/integrationTest/java/org/opensearch/security/privileges/PrivilegesEvaluatorTest.java

Signed-off-by: Derek Ho <dxho@amazon.com>

peternied

Going deep on these tests - last change before my approval, but you can still consider this nitpicking - can you make it clearer why the outcome of the test should be what it is?

I find the easiest format to follow is setup, action, and verification, what that could look like for testSearchTemplateRequestUnauthorizedAllIndices might follow:

final String searchForAllTemplates = "_search/template";
final TestRestClient client = cluster.getRestClient(ONLY_SERVICES_INDEX_ACCESS));

final Request searchForAllTemplatesResponse = client.getWithJsonBody(searchForAllTemplates, TEST_QUERY);

assertThat(getAllTemplatesResponse.getStatusCode(), equalTo(HttpStatus.SC_FORBIDDEN))

peternied · 2023-08-09T20:36:54Z

@derek-ho This are integration tests, could you change the name of the pull request and update the description?

Re-checking tests. How are the tests working without the index being created first?

…x-search-template

Signed-off-by: Derek Ho <dxho@amazon.com>

derek-ho · 2023-08-16T14:04:31Z

All tests pass on local after @cwperks's upstream change

cwperks

Looks good to me. Thank you @derek-ho! 🚢

derek-ho · 2023-08-16T15:58:13Z

@willyborankin can you add 2.x backport label? The Core change was backported to 2.x: opensearch-project/OpenSearch#9275

willyborankin · 2023-08-16T16:30:11Z

@derek-ho done

opensearch-trigger-bot · 2023-08-16T16:30:19Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/security/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/security/backport-2.x
# Create a new branch
git switch --create backport/backport-2921-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 bd084c89baff0312a1b11b961e8b850b68e83829
# Push it to GitHub
git push --set-upstream origin backport/backport-2921-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/security/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2921-to-2.x.

cwperks · 2023-08-16T17:24:22Z

The integration tests are not on 2.x so this does not need to be backported to 2.x.

fix search template auth to allow for search template requests on ind…

0122b60

…ices you have access to Signed-off-by: Derek Ho <dxho@amazon.com>

fix and add tests

24aa368

Signed-off-by: Derek Ho <dxho@amazon.com>

derek-ho commented Jun 29, 2023

View reviewed changes

spotless

b640386

Signed-off-by: Derek Ho <dxho@amazon.com>

derek-ho marked this pull request as ready for review June 29, 2023 21:56

derek-ho requested review from cliu123, cwperks, DarshitChanpura, davidlago, peternied, RyanL1997, stephen-crawford, reta and willyborankin as code owners June 29, 2023 21:56

cwperks reviewed Jun 30, 2023

View reviewed changes

Merge branch 'opensearch-project:main' into fix-search-template

445835d

stephen-crawford reviewed Jul 3, 2023

View reviewed changes

derek-ho marked this pull request as draft July 19, 2023 15:46

revert PR to only be tests

bbb1b86

Signed-off-by: Derek Ho <dxho@amazon.com>

derek-ho changed the title ~~fix search template auth to allow for search template requests on ind…~~ Add Unit Test to confirm Core Change to Fix Search template request Auth Aug 9, 2023

derek-ho marked this pull request as ready for review August 9, 2023 14:21

derek-ho added 2 commits August 9, 2023 14:48

add tests for other situations

e78def3

Signed-off-by: Derek Ho <dxho@amazon.com>

spotless

387f1a6

Signed-off-by: Derek Ho <dxho@amazon.com>

peternied reviewed Aug 9, 2023

View reviewed changes

src/integrationTest/java/org/opensearch/security/privileges/PrivilegesEvaluatorTest.java Outdated Show resolved Hide resolved

extract query to a constant

c4a3d5d

Signed-off-by: Derek Ho <dxho@amazon.com>

peternied reviewed Aug 9, 2023

View reviewed changes

cwperks mentioned this pull request Aug 9, 2023

Make SearchTemplateRequest implement IndicesRequest.Replaceable opensearch-project/OpenSearch#9122

Merged

6 tasks

cwperks previously approved these changes Aug 11, 2023

View reviewed changes

cwperks requested review from cwperks and removed request for cwperks August 11, 2023 20:33

derek-ho changed the title ~~Add Unit Test to confirm Core Change to Fix Search template request Auth~~ Add Integration Test to confirm Core Change to Fix Search template request Auth Aug 15, 2023

derek-ho added 2 commits August 15, 2023 15:58

Merge branch 'main' of github.com:opensearch-project/security into fi…

4e78c8d

…x-search-template

update integration tests

f3e5d46

Signed-off-by: Derek Ho <dxho@amazon.com>

cwperks approved these changes Aug 16, 2023

View reviewed changes

willyborankin approved these changes Aug 16, 2023

View reviewed changes

willyborankin merged commit bd084c8 into opensearch-project:main Aug 16, 2023
55 checks passed

willyborankin added the backport 2.x backport to 2.x branch label Aug 16, 2023

opensearch-trigger-bot bot added the backport-failed label Aug 16, 2023

peternied removed backport 2.x backport to 2.x branch backport-failed labels Aug 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Integration Test to confirm Core Change to Fix Search template request Auth #2921

Add Integration Test to confirm Core Change to Fix Search template request Auth #2921

derek-ho commented Jun 29, 2023 •

edited

Loading

codecov bot commented Jun 29, 2023 •

edited

Loading

derek-ho Jun 29, 2023

cwperks Jun 30, 2023

stephen-crawford Jul 3, 2023

cwperks Aug 9, 2023

cwperks commented Jun 30, 2023 •

edited

Loading

stephen-crawford left a comment

stephen-crawford Jul 3, 2023

davidlago commented Jul 17, 2023

derek-ho commented Jul 19, 2023

peternied left a comment

peternied commented Aug 9, 2023

derek-ho commented Aug 16, 2023

cwperks left a comment

derek-ho commented Aug 16, 2023 •

edited

Loading

willyborankin commented Aug 16, 2023

opensearch-trigger-bot bot commented Aug 16, 2023

cwperks commented Aug 16, 2023

Add Integration Test to confirm Core Change to Fix Search template request Auth #2921

Add Integration Test to confirm Core Change to Fix Search template request Auth #2921

Conversation

derek-ho commented Jun 29, 2023 • edited Loading

Description

Issues Resolved

Testing

Check List

codecov bot commented Jun 29, 2023 • edited Loading

Codecov Report

derek-ho Jun 29, 2023

Choose a reason for hiding this comment

cwperks Jun 30, 2023

Choose a reason for hiding this comment

stephen-crawford Jul 3, 2023

Choose a reason for hiding this comment

cwperks Aug 9, 2023

Choose a reason for hiding this comment

cwperks commented Jun 30, 2023 • edited Loading

stephen-crawford left a comment

Choose a reason for hiding this comment

stephen-crawford Jul 3, 2023

Choose a reason for hiding this comment

davidlago commented Jul 17, 2023

derek-ho commented Jul 19, 2023

peternied left a comment

Choose a reason for hiding this comment

peternied commented Aug 9, 2023

derek-ho commented Aug 16, 2023

cwperks left a comment

Choose a reason for hiding this comment

derek-ho commented Aug 16, 2023 • edited Loading

willyborankin commented Aug 16, 2023

opensearch-trigger-bot bot commented Aug 16, 2023

cwperks commented Aug 16, 2023

derek-ho commented Jun 29, 2023 •

edited

Loading

codecov bot commented Jun 29, 2023 •

edited

Loading

cwperks commented Jun 30, 2023 •

edited

Loading

derek-ho commented Aug 16, 2023 •

edited

Loading