[Alerting][Connectors] Refactor Jira: Generic Implementation (phase one)

[cnasikas]

Summary

This PR refactors the Jira connector to be generic. Specifically:

• Support of Jira alerts.
• Creation of two subactions: issueTypes and fieldsByIssueType

UI:

API

Push incident:

Pushes an incident to Jira

Endpoint: api/actions/action/<action_id>/_execute
Method: POST

Payload:

{
    "params": {
        "subAction": "pushToService",
        "subActionParams": {
            "savedObjectId": "66ce8cf1-2c0a-4d9a-9030-bff20d6c4690",
            "title": "A title",
            "description": "A description",
            "priority": "Low",
            "issueType": "10006",
            "labels": ["kibana", "alerts"],
            "comments": [
                {
                    "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                    "comment": "A comment"
                },
                {
                    "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                    "comment": "Another comment"
                }
            ]
        }
    }
}

Response

{
    "status": "ok",
    "data": {
        "title": "RJ2-8",
        "id": "10202",
        "pushedDate": "2020-09-01T10:58:55.157Z",
        "url": "https://siem-kibana.atlassian.net/browse/RJ2-8",
        "comments": [
            {
                "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                "pushedDate": "2020-09-01T10:58:56.453Z"
            },
            {
                "commentId": "b5b4c4d0-574e-11ea-9e2e-21b90f8a9631",
                "pushedDate": "2020-09-01T10:58:56.977Z"
            }
        ]
    },
    "actionId": "bda54773-aadf-4668-aead-69a72b5e83b4"
}

Get issue types:

Get issue types of a specific project (projectKey is configured at the creation of the connector).

Endpoint: api/actions/action/<action_id>/_execute
Method: POST

Payload:

{
    "params": {
        "subAction": "issueTypes",
        "subActionParams": {}
    }
}

Response

{
    "status": "ok",
    "data": [
        {
            "id": "10023",
            "name": "Bug"
        },
        {
            "id": "10024",
            "name": "Improvement"
        },
        {
            "id": "10017",
            "name": "Story"
        },
        {
            "id": "10008",
            "name": "My custom issue"
        }
    ],
    "actionId": "bda54773-aadf-4668-aead-69a72b5e83b4"
}

Get fields of issue type:

Get the fields of a specific issue type.

Endpoint: api/actions/action/<action_id>/_execute
Method: POST

Payload:

{
    "params": {
        "subAction": "fieldsByIssueType",
        "subActionParams": {
            "id": "10024"
        }
    }
}	

Response

{
    "status": "ok",
    "data": {
        "summary": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "issuetype": {
            "allowedValues": [
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/issuetype/10024",
                    "id": "10024",
                    "description": "An improvement or enhancement to an existing feature or task.",
                    "iconUrl": "https://siem-kibana.atlassian.net/secure/viewavatar?size=medium&avatarId=10310&avatarType=issuetype",
                    "name": "Improvement",
                    "subtask": false,
                    "avatarId": 10310
                }
            ],
            "defaultValue": {}
        },
        "attachment": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "duedate": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "description": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "project": {
            "allowedValues": [
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/project/10015",
                    "id": "10015",
                    "key": "RJ2",
                    "name": "RJ2",
                    "projectTypeKey": "business",
                    "simplified": false,
                    "avatarUrls": {
                        "48x48": "https://siem-kibana.atlassian.net/secure/projectavatar?pid=10015&avatarId=10412",
                        "24x24": "https://siem-kibana.atlassian.net/secure/projectavatar?size=small&s=small&pid=10015&avatarId=10412",
                        "16x16": "https://siem-kibana.atlassian.net/secure/projectavatar?size=xsmall&s=xsmall&pid=10015&avatarId=10412",
                        "32x32": "https://siem-kibana.atlassian.net/secure/projectavatar?size=medium&s=medium&pid=10015&avatarId=10412"
                    }
                }
            ],
            "defaultValue": {}
        },
        "assignee": {
            "allowedValues": [],
            "defaultValue": {}
        },
        "priority": {
            "allowedValues": [
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/1",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/highest.svg",
                    "name": "Highest",
                    "id": "1"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/2",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/high.svg",
                    "name": "High",
                    "id": "2"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/3",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/medium.svg",
                    "name": "Medium",
                    "id": "3"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/4",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/low.svg",
                    "name": "Low",
                    "id": "4"
                },
                {
                    "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/5",
                    "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/lowest.svg",
                    "name": "Lowest",
                    "id": "5"
                }
            ],
            "defaultValue": {
                "self": "https://siem-kibana.atlassian.net/rest/api/2/priority/3",
                "iconUrl": "https://siem-kibana.atlassian.net/images/icons/priorities/medium.svg",
                "name": "Medium",
                "id": "3"
            }
        },
        "labels": {
            "allowedValues": [],
            "defaultValue": {}
        }
    },
    "actionId": "bda54773-aadf-4668-aead-69a72b5e83b4"
}

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for keyboard-only and screenreader accessibility
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
• This was checked for cross-browser compatibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately

Resolves #56426

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[pmuellr]

Looking good! I've noted some things that I think will need to be changed, mainly regarding schema.maybe() usage and the rename of the config field.

Note, I haven't looked at the UI code yet, figured I'd post what I have so far before waiting to review that.

[pmuellr]

Seems like you are renaming an existing config value of casesConfiguration to incidentConfiguration. If you do that, any existing actions with casesConfiguration will be broken - they won't validate anymore.

I don't believe we had to deal with this yet, but I think the way to do it would be like the way we treat kibana.yml config values that we "rename". Basically, you need to deprecate the old one, and not delete it. In code that checks the value, you will potentially have to check both (use the new one unless it's not set, otherwise use the old one). @mikecote other thought?

I'm not sure if we would support some way for these to be migrated.

[mikecote]

Good catch @pmuellr, I saw the same rename got done in ServiceNow too. From what I see, the impact is that the connector would "lose" the field mappings because incidentConfiguration would be empty for existing ServiceNow connectors (possible bug in Cases app?) and the user would have to re-configure the mappings. I'm not sure how it would validate when updating 🤔

One way I see this being fixed since we're not GA yet is with a saved object migration in the actions plugin that moves the value from casesConfiguration to incidentConfiguration (including decrypt / encrypt process). In a post GA world, @pmuellr is correct and we'd probably have to think of a non-breaking way to support both until the next major where possible.

I'm indifferent what approach is done (migrate or revert to casesConfiguration or make users re-configure mappings). Since there's a phase 2 to remove this value from connectors, we could always skip it until the field gets removed.

[cnasikas]

@pmuellr @mikecote I followed ServiceNow PR example. As mapping for the moment is fixed to a default one (only thought the UI), creating or updating a case or validating the mapping will not be a problem. What Cases will loose is the appended text (Example: A title in Jira (created at 2020-04-22T18:18:03.896Z by Christos Nasikas)) in titles and descriptions. I totally agree with what you are saying but I think it's ok if some users lose temporarily that functionality as in phase 2 we gonna append this information beforehand in Cases. I think the easiest one is to revert back to casesConfiguration and lose that functionality only for ServiceNow. @XavierM What are your thoughts on that?

[XavierM]

As of today, this mapping is static meaning nobody can change it. So we can always default back but I think it will be better to do a migration. @cnasikas I can show you how to do it if you do not know, but this one seems pretty simple.

[pmuellr]

We could deal with a migration today, since the action is in the actions plugin, but in the future when we have actions NOT in the actions plugin, it's not clear to me if they can or should be doing migrations against the action SO's. Basically, in the future, we may need to have a per-actionType migration story that could be used outside of the actions plugin. Actually makes me wonder if every actionType (and alertType) should be their own saved object, or something, to make lifecycle management like this a little more straight-forward.

[pmuellr]

regarding the rename of casesConfiguration - since actions created with the old config value could already exist from the previous release, should probably have a test for that. Might be hard to test for. Especially hard since the field is part of AAD so you can't just overwrite the field with the generic saved object API. Probably a case for es-archiver, sorry to say. another one that @mikecote may have some input on

[mikecote]

++ es archiver is probably best in this scenario but there may be issues where it erases other data the test suite relies on (users / spaces). @gmmorris is going through the exact same issue in the RBAC work, maybe a change he's working on will help here?

[mikecote]

I think based on my note above: #73778 (comment). I'd be +1 on reverting it back to casesConfiguration since we know the field is going away soon and we don't have to worry about a migration or tests to cover this.

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 627a3f2

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
securitySolution	1967	-5	1972
triggers_actions_ui	241	+17	224
total		+12

async chunks size

id	value	diff	baseline
securitySolution	9.9MB	-31.8KB	10.0MB
triggers_actions_ui	894.5KB	+157.1KB	737.3KB
total		+125.3KB

miscellaneous assets size

id	value	diff	baseline
securitySolution	326.0KB	-27.7KB	353.7KB
triggers_actions_ui	27.7KB	+27.7KB	0.0B
total		-0.0B

page load bundle size

id	value	diff	baseline
securitySolution	810.0KB	-6.5KB	816.6KB
triggers_actions_ui	274.6KB	+10.5KB	264.1KB
total		+3.9KB

distributable file count

id	value	diff	baseline
default	45464	+5	45459

History

• 💚 Build #73340 succeeded 627a3f2
• 💚 Build #72898 succeeded 11f824b6ae2e8642f75e6aa3d91bd8693df691bf
• 💚 Build #72851 succeeded 6b1b32b78b3418b636e7764b2c00d9023a276e42
• 💚 Build #72796 succeeded 6a23270fc6b89488850954fb8c2991518e0b766c
• 💔 Build #72720 failed a3d60c9bd6763ea81326f7268139df61519264a5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[cnasikas]

@gchaps

API changes for creating a Jira connector:

casesConfiguration renamed to incidentConfiguration
Added optional attributeisCaseOwned.

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)