Skip to content
This repository has been archived by the owner on Sep 21, 2023. It is now read-only.

docs(samples): add resources for interactive tutorials #145

Merged
merged 13 commits into from
Feb 8, 2022
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,31 @@
## Required environment variables

To successfully import the catalog data for tests, the following environment variables should be set:
- PROJECT_NUMBER
- GOOGLE_CLOUD_PROJECT_NUMBER
- GOOGLE_CLOUD_PROJECT_ID
- BUCKET_NAME
- EVENTS_BUCKET_NAME
These values are stored in the Secret Manager and will be submitted as
docker environment variables before the test run.

The Secret Manager name is set in .kokoro/presubmit/common.cfg file, SECRET_MANAGER_KEYS variable.

## Import catalog data

There is a JSON file with valid products prepared in the `product` directory:
`resources/products.json`.
There are a JSON files with valid products and user events prepared in `resources` directory:
parthea marked this conversation as resolved.
Show resolved Hide resolved
parthea marked this conversation as resolved.
Show resolved Hide resolved
`samples/resources/products.json` and `samples/resources/user_events.json` respectively.

Run the `create_test_resources.py` to perform the following actions:
- create the GCS bucket <BUCKET_NAME>,
- upload the product data from `resources/products.json` file,
- import products to the default branch of the Retail catalog.
- upload the product data from `resources/products.json` file to products bucket,
- import products to the default branch of the Retail catalog,
- create the GCS bucket <EVENTS_BUCKET_NAME>,
- upload the product data from `resources/user_events.json` file to events bucket,
- create a BigQuery dataset and table `products`,
- insert products from resources/products.json to the created products table,
- create a BigQuery dataset and table `events`,
- insert user events from resources/user_events.json to the created events table


```
$ python create_test_resources.py
Expand All @@ -33,6 +42,10 @@ Run the `remove_test_resources.py` to perform the following actions:
- remove all objects from the GCS bucket <BUCKET_NAME>,
- remove the <BUCKET_NAME> bucket,
- delete all products from the Retail catalog.
- remove all objects from the GCS bucket <EVENTS_BUCKET_NAME>,
- remove the <EVENTS_BUCKET_NAME> bucket,
- remove dataset `products` along with tables
- remove dataset `user_events` along with tables

```
$ python remove_test_resources.py
Expand Down
73 changes: 73 additions & 0 deletions samples/interactive-tutorials/resources/events_schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
[
{
"fields":[
{
"mode": "NULLABLE",
"name": "currencyCode",
"type": "STRING"
},
{
"mode": "NULLABLE",
"name": "revenue",
"type": "FLOAT"
}
],
"mode": "NULLABLE",
"name": "purchaseTransaction",
"type": "RECORD"
},
{
"fields":[
{
"mode": "NULLABLE",
"name": "quantity",
"type": "INTEGER"
},
{
"fields":[
{
"mode": "NULLABLE",
"name": "id",
"type": "STRING"
}
],
"mode": "NULLABLE",
"name": "product",
"type": "RECORD"
}
],
"mode": "REPEATED",
"name": "productDetails",
"type": "RECORD"
},
{
"mode": "REQUIRED",
"name": "eventTime",
"type": "STRING"
},
{
"mode": "REQUIRED",
"name": "visitorId",
"type": "STRING"
},
{
"mode": "REQUIRED",
"name": "eventType",
"type": "STRING"
},
{
"mode": "NULLABLE",
"name": "searchQuery",
"type": "STRING"
},
{
"mode": "NULLABLE",
"name": "cartId",
"type": "STRING"
},
{
"mode": "REPEATED",
"name": "pageCategories",
"type": "STRING"
}
]
Loading