Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Cloud Support Slackbot #673

Merged
merged 52 commits into from
Aug 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
a10679b
Create README.md
TheLanceLord Aug 5, 2021
a6fb4f6
Add files via upload
TheLanceLord Aug 5, 2021
291f472
Create README.md
TheLanceLord Aug 5, 2021
067b809
Add files via upload
TheLanceLord Aug 5, 2021
92d4469
Update README.md
TheLanceLord Aug 5, 2021
1e53cf0
Delete gcloud_buddy_small.png
TheLanceLord Aug 5, 2021
64ca1e8
Add files via upload
TheLanceLord Aug 5, 2021
8498207
Update google_cloud_support_slackbot.py
TheLanceLord Aug 5, 2021
049ab32
Update README.md
TheLanceLord Aug 5, 2021
243d80e
Create .env
TheLanceLord Aug 5, 2021
606c8cb
Update README.md
TheLanceLord Aug 5, 2021
f45a5ea
Update README.md
TheLanceLord Aug 5, 2021
12afb90
Update README.md
TheLanceLord Aug 5, 2021
8bec817
Update README.md
TheLanceLord Aug 5, 2021
9d31016
Update README.md
TheLanceLord Aug 5, 2021
3f9a7b0
Update README.md
TheLanceLord Aug 5, 2021
cb0aebe
Update README.md
TheLanceLord Aug 5, 2021
98647e3
Updating main README.md
TheLanceLord Aug 5, 2021
a214e13
Update google_cloud_support_slackbot.py
TheLanceLord Aug 5, 2021
c623aec
Update README.md
TheLanceLord Aug 6, 2021
18e0328
Update google_cloud_support_slackbot.py
TheLanceLord Aug 6, 2021
7a75deb
Update README.md
TheLanceLord Aug 6, 2021
f6dd2b7
Update google_cloud_support_slackbot.py
TheLanceLord Aug 6, 2021
817e6a4
Update google_cloud_support_slackbot.py
TheLanceLord Aug 6, 2021
615f0d0
Update google_cloud_support_slackbot.py
TheLanceLord Aug 9, 2021
e1f3019
Update google_cloud_support_slackbot.py
TheLanceLord Aug 9, 2021
d0c4389
Update requirements.txt
TheLanceLord Aug 9, 2021
aef1720
Update google_cloud_support_slackbot.py
TheLanceLord Aug 9, 2021
cbb087f
Update README.md
TheLanceLord Aug 9, 2021
47375e2
Update README.md
TheLanceLord Aug 9, 2021
97e6fe9
Update google_cloud_support_slackbot.py
TheLanceLord Aug 9, 2021
338a311
Update google_cloud_support_slackbot.py
TheLanceLord Aug 10, 2021
8dd2a2b
Update google_cloud_support_slackbot.py
TheLanceLord Aug 10, 2021
4019215
Update google_cloud_support_slackbot.py
TheLanceLord Aug 10, 2021
fa91a62
Add files via upload
TheLanceLord Aug 11, 2021
bc386cc
Add files via upload
TheLanceLord Aug 11, 2021
a7f66dd
Delete google_cloud_support_slackbot_icon_big.png
TheLanceLord Aug 11, 2021
993884c
Delete google_cloud_support_slackbot_icon_small.png
TheLanceLord Aug 11, 2021
43732e0
Add files via upload
TheLanceLord Aug 11, 2021
f15b418
Delete google_cloud_support_buddy_small.png
TheLanceLord Aug 11, 2021
ad4b5e3
Delete google_cloud_support_buddy_big.png
TheLanceLord Aug 11, 2021
ba4ac10
Update README.md
TheLanceLord Aug 11, 2021
180f304
Add files via upload
TheLanceLord Aug 11, 2021
1526a4a
Delete google_cloud_support_slackbot_icon_big.svg
TheLanceLord Aug 11, 2021
3ef173e
Delete google_cloud_support_slackbot_icon_small.svg
TheLanceLord Aug 11, 2021
f531f72
Add files via upload
TheLanceLord Aug 11, 2021
7eec127
Update README.md
TheLanceLord Aug 11, 2021
847baf2
Update google_cloud_support_slackbot.py
TheLanceLord Aug 12, 2021
91f25cc
Update .env
TheLanceLord Aug 12, 2021
c479e71
Update README.md
TheLanceLord Aug 12, 2021
7394b59
Update requirements.txt
TheLanceLord Aug 13, 2021
cf0d99f
Merge branch 'main' into main
iht Aug 14, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ creates relationships between assets and outputs a format compatible with [graph
* [GCS Usage Recommender](tools/gcs-usage-recommender) - A tool that generates bucket-level intelligence and access patterns across all projects for a GCP project to generate recommended object lifecycle management.
* [GCS to BigQuery](tools/gcs2bq) - A tool fetches object metadata from all Google Cloud Storage buckets and exports it in a format that can be imported into BigQuery for further analysis.
* [GKE Billing Export](tools/gke-billing-export) - Google Kubernetes Engine fine grained billing export.
* [Google Cloud Support Slackbot](tools/google-cloud-support-slackbot) - Slack application that pulls Google Cloud support case information via the Cloud Support API and pushes the information to Slack
* [GSuite Exporter Cloud Function](tools/gsuite-exporter-cloud-function/) - A script that deploys a Cloud Function and Cloud Scheduler job that executes the GSuite Exporter tool automatically on a cadence.
* [GSuite Exporter](tools/gsuite-exporter/) - A Python package that automates syncing Admin SDK APIs activity reports to a GCP destination. The module takes entries from the chosen Admin SDK API, converts them into the appropriate format for the destination, and exports them to a destination (e.g: Stackdriver Logging).
* [Hive to BigQuery](tools/hive-bigquery/) - A Python framework to migrate Hive table to BigQuery using Cloud SQL to keep track of the migration progress.
Expand Down
19 changes: 19 additions & 0 deletions tools/google-cloud-support-slackbot/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


SLACK_TOKEN=<SLACK_TOKEN>
SIGNING_SECRET=<SLACK_SIGNING_SECRET>
ORG_ID=<ORG_ID>
API_KEY=<API_KEY>
163 changes: 163 additions & 0 deletions tools/google-cloud-support-slackbot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
# ![Google Cloud Support Slackbot](google_cloud_support_slackbot_icon.svg)
# Slack Integration for Google Cloud Support

Slack app for pulling Google Cloud Support case information via the Cloud Support API and pushing it to Slack. The goal is to help reduce the overall time to resolution for cases of all priorities by putting case updates in Slack where they will be more easily noticed by their devs and engineers.

The app currently supports the following commands:

* /google-cloud-support track-case [case_number] -- case updates will be posted to this channel
* /google-cloud-support add-comment [case_number] [comment] -- adds a comment to the case
* /google-cloud-support change-priority [case_number] [priority, e.g. P2] -- changes the priority of the case
* /google-cloud-support stop-tracking [case_number] -- case updates will no longer be posted to this channel
* /google-cloud-support list-tracked-cases -- lists all cases being tracked in this channel
* /google-cloud-support list-tracked-cases-all -- lists all cases being tracked in the workspace
* /google-cloud-support case-details [case_number] -- pull all of the case data as json
* /google-cloud-support sitrep -- report of all active cases in the org

# Setup Guide

**Before proceeding, you will need Premium Support to use the Cloud Support API and by association the slackbot**
Setting up your first Slack app can be a daunting task, which is why we are providing a step-by-step guide.

## Setup Part 1 - Allow list the Support API

To get access to the API, you will need to send your Techincal Account Manager the following:

1. The **org id** where you have Premium Support enabled
2. A **project id** where the API will be allow listed
3. The name of a **service account** in the project from step 2, with the service account having the following roles at the org level:
1. **Tech Support Editor**
1. **Org Viewer**
4. The **email addresses** of the people that will be enabling the API in the project

Your Techincal Account Manager will file a request with the Support API team to give you access. The team typically processes these requests within 24 hours

## Setup Part 2 - Google Cloud Phase 1

In the first phase of our Google Cloud setup, we will verify that our network is setup properly, create a lightweight VM to house our bot, and enable our Cloud Support API and create ourselves an API key. Go to [Google Cloud](https://cloud.google.com/console). **These steps need to be carried out in the project you specified in Part 1 of this setup guide.**

### Networking

From **VPC network > Firewall rules**, verify rules exist to **allow SSH and HTTP**.

1. **If your project doesn't have a VPC, you will need to create one from VPC networks**. Select **Automatic** for your Subnet creation mode, and **allow-ssh** from **Firewall rules**
2. If it doesn't exist, create the following firewall rule:
1. Name: `default-allow-http`
1. Priority: `1000`
1. Direction: `Ingress`
1. Action on match: `Allow`
1. Targets: `Specified target tags`
1. Target tags: `http-server`
1. Source filter: `IP ranges`
1. Source IP ranges: `0.0.0.0/0`
1. Protocols and Ports: `Specified protocols and ports`
1. tcp: `80`
3. If an SSH firewall rule doesn't exist, create the following firewall rule:
1. Name: `default-allow-ssh`
1. Priority: `65534`
1. Direction: `Ingress`
1. Action on match: `Allow`
1. Targets: `All instances in the network`
1. Source filter: `IP ranges`
1. Source IP ranges: `0.0.0.0/0`
1. Protocols and Ports: `Specified protocols and ports`
1. tcp: `22`

*Note that if you had to create the SSH firewall rule in Step 3, you will want to disable it after you complete the entire setup*

### VM

Go to **Compute Engine > VM instances** and perform the following:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use Google Cloud Run instead? I believe that dedicated virtual machine is pretty high maintance burden for so trival service. It might impact security, because that service require eg. security updates.

In case of Google Cloud Run maintenance of upgrades of operating system is out of the box. In addiction SSL is managed too.

I see that you have shared state on file system, but I believe you can easy save the same data to Google Cloud Storage.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative I can see that Google Cloud have tutorial how to write Slack bot on top of Google Cloud Function: https://cloud.google.com/functions/docs/tutorials/slack#functions_slack_search_tutorial-python
For shared storage you might use Google Cloud Storage or Google Firestore for reduce maintenance burden of storage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I modularize the code for the GA release of the Support API, I will also rewrite it so it can be used with containers by using Firestore or GCS for storage, which would also make it work with Cloud Functions as well. One of the first requests I got was to make this work with GKE, so a build that will let people deploy the app wherever they feel most comfortable housing it will be the best option here. I will also add this on the trello for this, and will share a link here once the board is ready

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make sure GKE users would not be satisfied with their needs better by Cloud Functions eg. ask them? I have the impression that the question about the GKE stems from both of the maintenance of the GCE instance. I have impression that GCF might allow you to run it for a few cents per month, which GKE does not, so it is also an economical and environmentally friendly solution thanks to the optimal use of resources.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can check with them, I agree that from a cost perspective that Cloud Functions is the way to go, the only issue that some people might have with it is code updates, because Cloud Functions doesn't let you edit your code. You would have to delete the function and then recreate it. I can put together a script to make it less painful though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can always deploy a new version of a Cloud Function (https://cloud.google.com/functions/docs/deploying#migrating_between_deployments), which, in a way, is an update of an existing function.


1. Click **+ Create Instance**
1. Under **Machine Configuration**, set the **Machine type** field to **e2-micro**. This should suffice for most implementations. If your team makes heavy use of the Cloud Support and the bot, you may need to upgrade the machine type
1. Under **Identity and API access > Service Account**, select your **service account** that was allow listed for the Cloud Support API
1. Under **Firewall**, select **Allow HTTP traffic**. If this option isn't available and you create the firewall rule in the Networking steps, then you will want to contact your Networking team about policies that may be preventing HTTP traffic
1. Click to expand **Management, security, disks, networking, sole tenancy**
1. Select the **Networking** tab
1. Under **Network interfaces**, click the network interface box
1. Set **Network** to the VPC where you have your firewall rules
1. Under **External IP**, select **Create IP address**. Choose whichever name and network service tier you prefer
1. Click **Create**
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All setup above can be performed in Terraform or Google Cloud Resource Manager. Is there any guidelines for that repository when Terraform should be used and when plain text tutorial? See example https://github.com/GoogleCloudPlatform/professional-services/tree/main/examples/kerberized_data_lake .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the answer is it depends on each team's maturity with Google Cloud. For an organization that is new to cloud, following the step-by-step guide will likely make the most sense. For more experienced teams though, they will likely have terraform already handy that they can use to build up vanilla projects with most of this implemented


### API Enablement and the API Key

From **APIs & Services > Library** ...

1. Search for and enable the **Cloud Logging API**
2. Search for and enable the **Cloud Support API**

From **APIs & Services > Credentials**

1. Click **+Create** and select **API key**
2. Copy your key and choose to **Restrict Key**
1. Under **Application restrictions**, you may select **IP addresses** to restrict usage the VM you created
1. Under **API restrictions**, select **Restrict Key** and from the **Select APIs** dropdown, click **Google Cloud Support API**

## Setup Part 3 - Slack App

Go to [Slack Apps](http://api.slack.com/apps) to do the following:

1. Click **Create New App** and select **From scratch**. Name your app `Google Cloud Support Bot` and select your workspace
2. Under **Settings > Basic Information**, scroll down to **Display Information** and upload the [google_cloud_support_buddy_big.png](google_cloud_support_buddy_big.png) or an icon of your choosing
3. Go to **Features > Slash Commands** and create the following command:
1. Command: `/google-cloud-support `
1. Request URL: `http://<your_vm_external_ip>/google-cloud-support`
1. Short description: `Track and manage your Google Cloud support cases in Slack. Use /google-cloud-support help for the list of commands`
1. Usage Hint: `[command] [parameter 1] [parameter 2] [parameter 3]`
4. Go to **Features > OAuth & Permissions**. Scroll down to **Scopes** and add the **chat:write** scope. Add the **commands** scope if it isn't listed already listed
5. At the top of the **Features > OAuth & Permissions** page, under **OAuth Tokens for Your Workspace**, click **Install to Workspace**. Copy the token. You may need Slack admin approval to install the app
6. Go to **Settings > Basic Information** and under **App Credentials** copy the `Signing Secret`

## Setup Part 4 - Google Cloud Phase 2

Return to [Google Cloud](https://cloud.google.com/console) and from **Compute Engine > VM instances**, perform the following:

1. SSH into the VM that you created in part 2 of this setup guide
2. Run the following commands:
1. `sudo apt-get update`
1. `sudo apt-get -y install subversion`
1. `sudo apt-get -y install python3-pip`
1. `sudo apt-get -y install nginx`
1. `cd /`
1. `sudo svn export https://github.com/GoogleCloudPlatform/professional-services/trunk/tools/google-cloud-support-slackbot`
1. `cd /google-cloud-support-slackbot`
1. Use sudo to open the `default` file with your editor of choice, and replace <STATIC_IP> with the external ip address of your VM. Then save and close the file
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this item supposed to be further on in the command list? It does not contain a command line command to be executed like the previous points.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional, the commands for this aren't listed out as I assume most of the people who will be setting this up will have a basic knowledge of vim or nano.

1. `sudo mv default /etc/nginx/sites-available/`
1. Use sudo to open the `.env` file with your editor of choice. Enter your API Key, Slack Token, and numeric org id in their respective locations. Then save and close the file
1. `sudo chmod +x google_cloud_support_slackbot.py`
3. Close the SSH session
4. From Compute Engine > VM instances, click your VM name to go to your VM instance details
5. Stop the VM
6. Once the VM is stopped, click the 'EDIT' button
7. Scroll down to the Custom metadata section and add the following key-value pair:
1. key: `startup-script`
1. value:
`cd /google-cloud-support-slackbot`
`pip3 install -r requirements.txt`
`/google-cloud-support-slackbot/google_cloud_support_slackbot.py`
8. Scroll to the bottom of the page and click 'Save'
9. Start your VM

## Testing

To verify that everything was setup correctly, do the following:
1. Go to your Slack workspace
2. Under **Channels** right-click the channel where you want to add the bot and select **Open channel details**
3. Select the **Integrations** tab
4. In the **Apps** section, click **Add apps** and then add the bot
5. Open the channel where you added the bot and run the `/google-cloud-support help` command. If it returns a list of available help commands then everything was setup correctly. If it returns an error that the dispatch failed, then you will need to debug. The most likely culprits are an issue with an org security policy, or a missed step somewhere

## Closing

With that you should be all setup! And as a reminder, if you had to create the SSH firewall rule, it is recommended that you go back and disable it. If you ever need to SSH into the machine you can always enable the rule again as needed.
TheLanceLord marked this conversation as resolved.
Show resolved Hide resolved

As the Cloud Support API continues to expand and we collect more feedback for requested features, we will release newer versions of the bot and move the previous version into the archive folder. To replace your current bot with the latest version you will only need to do the following:

1. SSH into your VM instance
2. Run the following commands:
1. `cd /google-cloud-support-slackbot`
1. `sudo svn export --force https://github.com/GoogleCloudPlatform/professional-services/trunk/tools/google-cloud-support-slackbot/google_cloud_support_slackbot.py`
1. `sudo chmod +x google_cloud_support_slackbot.py`
3. Close your SSH session
4. Stop and Start your VM
10 changes: 10 additions & 0 deletions tools/google-cloud-support-slackbot/archive/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Changelog
TheLanceLord marked this conversation as resolved.
Show resolved Hide resolved
## v0
/google-cloud-support track-case [case_number] -- case updates are posted to the channel where this command is run
/google-cloud-support add-comment [case_number] [comment] -- adds a comment to the case
/google-cloud-support change-priority [case_number] [priority, e.g. P2] -- changes the priority of the case
/google-cloud-support stop-tracking [case_number] -- case updates will no longer be posted to the channel where this command is run
/google-cloud-support list-tracked-cases -- lists all cases being tracked in the current channel
/google-cloud-support list-tracked-cases-all -- lists all cases being tracked in the workspace
/google-cloud-support case-details [case_number] -- pull all of the case data as json
/google-cloud-support sitrep -- report of all active cases in the org.
24 changes: 24 additions & 0 deletions tools/google-cloud-support-slackbot/default
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Copyright 2021 Google LLC

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

server {
listen 80;
listen [::]:80;

server_name <STATIC_IP>;

location /google-cloud-support {
proxy_pass http://localhost:5000/google-cloud-support;
}
}
Loading