Skip to content

Commit

Permalink
Reorganize howto guides into sections. Add links to relevant methods.
Browse files Browse the repository at this point in the history
  • Loading branch information
tswast committed Oct 18, 2018
1 parent 19b0159 commit 277aab1
Show file tree
Hide file tree
Showing 6 changed files with 153 additions and 97 deletions.
22 changes: 14 additions & 8 deletions bigquery/docs/howto/datasets.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Datasets
~~~~~~~~
Managing Datasets
~~~~~~~~~~~~~~~~~

A dataset represents a collection of tables, and applies several default
policies to tables as they are created:
Expand All @@ -17,47 +17,53 @@ See BigQuery documentation for more information on
Dataset operations
^^^^^^^^^^^^^^^^^^

List datasets for the client's project:
List datasets for a project with the
:func:`~google.cloud.bigquery.client.Client.list_datasets` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_list_datasets]
:end-before: [END bigquery_list_datasets]

Create a new dataset for the client's project:
Create a new dataset with the
:func:`~google.cloud.bigquery.client.Client.create_dataset` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_create_dataset]
:end-before: [END bigquery_create_dataset]

Refresh metadata for a dataset (to pick up changes made by another client):
Get a dataset resource (to pick up changes made by another client) with the
:func:`~google.cloud.bigquery.client.Client.get_dataset` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_get_dataset]
:end-before: [END bigquery_get_dataset]

Update a property in a dataset's metadata:
Update a property in a dataset's metadata with the
:func:`~google.cloud.bigquery.client.Client.update_dataset` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_update_dataset_description]
:end-before: [END bigquery_update_dataset_description]

Modify user permissions on a dataset:
Modify user permissions on a dataset with the
:func:`~google.cloud.bigquery.client.Client.update_dataset` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_update_dataset_access]
:end-before: [END bigquery_update_dataset_access]

Delete a dataset:
Delete a dataset with the
:func:`~google.cloud.bigquery.client.Client.delete_dataset` method:

.. literalinclude:: ../snippets.py
:language: python
Expand Down
52 changes: 52 additions & 0 deletions bigquery/docs/howto/encryption.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
Using Customer Managed Encryption Keys
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Table data is always encrypted at rest, but BigQuery also provides a way for
you to control what keys it uses to encrypt they data. See `Protecting data
with Cloud KMS keys
<https://cloud.google.com/bigquery/docs/customer-managed-encryption>`_
in the BigQuery documentation for more details.

Create a new table, using a customer-managed encryption key from
Cloud KMS to encrypt it.

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_create_table_cmek]
:end-before: [END bigquery_create_table_cmek]

Change the key used to encrypt a table.

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_update_table_cmek]
:end-before: [END bigquery_update_table_cmek]

Load a file from Cloud Storage, using a customer-managed encryption key from
Cloud KMS for the destination table.

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_load_table_gcs_json_cmek]
:end-before: [END bigquery_load_table_gcs_json_cmek]

Copy a table, using a customer-managed encryption key from Cloud KMS for the
destination table.

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_copy_table_cmek]
:end-before: [END bigquery_copy_table_cmek]

Write query results to a table, using a customer-managed encryption key from
Cloud KMS for the destination table.

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_query_destination_table_cmek]
:end-before: [END bigquery_query_destination_table_cmek]
4 changes: 2 additions & 2 deletions bigquery/docs/howto/jobs.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Jobs
~~~~
Managing Jobs
~~~~~~~~~~~~~

List jobs for a project
^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
4 changes: 2 additions & 2 deletions bigquery/docs/howto/queries.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Queries
~~~~~~~
Running Queries
~~~~~~~~~~~~~~~

Querying data
^^^^^^^^^^^^^
Expand Down
141 changes: 59 additions & 82 deletions bigquery/docs/howto/tables.rst
Original file line number Diff line number Diff line change
@@ -1,105 +1,75 @@
Tables
~~~~~~
Managing Tables
~~~~~~~~~~~~~~~

Tables exist within datasets. See BigQuery documentation for more information
on `Tables <https://cloud.google.com/bigquery/docs/tables>`_.

Table operations
^^^^^^^^^^^^^^^^
List tables for the dataset:
Listing Tables
^^^^^^^^^^^^^^

List the tables belonging to a dataset with the
:func:`~google.cloud.bigquery.client.Client.list_tables` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_list_tables]
:end-before: [END bigquery_list_tables]

Create a table:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_create_table]
:end-before: [END bigquery_create_table]
Getting a Table
^^^^^^^^^^^^^^^

Get a table:
Get a table resource with the
:func:`~google.cloud.bigquery.client.Client.get_table` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_get_table]
:end-before: [END bigquery_get_table]

Update a property in a table's metadata:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_update_table_description]
:end-before: [END bigquery_update_table_description]

Browse selected rows in a table:
Browse data rows in a table with the
:func:`~google.cloud.bigquery.client.Client.list_rows` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_browse_table]
:end-before: [END bigquery_browse_table]

Insert rows into a table's data:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_table_insert_rows]
:end-before: [END bigquery_table_insert_rows]

Copy a table:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_copy_table]
:end-before: [END bigquery_copy_table]

Extract a table to Google Cloud Storage:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_extract_table]
:end-before: [END bigquery_extract_table]
Creating a Table
^^^^^^^^^^^^^^^^

Delete a table:
Create an empty table with the
:func:`~google.cloud.bigquery.client.Client.create_table` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_delete_table]
:end-before: [END bigquery_delete_table]
:start-after: [START bigquery_create_table]
:end-before: [END bigquery_create_table]

Upload table data from a file:
Load table data from a file with the
:func:`~google.cloud.bigquery.client.Client.load_table_from_file` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_load_from_file]
:end-before: [END bigquery_load_from_file]

Load table data from Google Cloud Storage
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

See also: `Loading JSON data from Cloud Storage
<https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json>`_.

Load a CSV file from Cloud Storage:
Load a CSV file from Cloud Storage with the
:func:`~google.cloud.bigquery.client.Client.load_table_from_uri` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_load_table_gcs_csv]
:end-before: [END bigquery_load_table_gcs_csv]

See also: `Loading CSV data from Cloud Storage
<https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv>`_.

Load a JSON file from Cloud Storage:

.. literalinclude:: ../snippets.py
Expand All @@ -108,6 +78,9 @@ Load a JSON file from Cloud Storage:
:start-after: [START bigquery_load_table_gcs_json]
:end-before: [END bigquery_load_table_gcs_json]

See also: `Loading JSON data from Cloud Storage
<https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-json>`_.

Load a Parquet file from Cloud Storage:

.. literalinclude:: ../snippets.py
Expand All @@ -116,55 +89,59 @@ Load a Parquet file from Cloud Storage:
:start-after: [START bigquery_load_table_gcs_parquet]
:end-before: [END bigquery_load_table_gcs_parquet]

Customer Managed Encryption Keys
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
See also: `Loading Parquet data from Cloud Storage
<https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-parquet>`_.

Table data is always encrypted at rest, but BigQuery also provides a way for
you to control what keys it uses to encrypt they data. See `Protecting data
with Cloud KMS keys
<https://cloud-dot-devsite.googleplex.com/bigquery/docs/customer-managed-encryption>`_
in the BigQuery documentation for more details.
Updating a Table
^^^^^^^^^^^^^^^^

Create a new table, using a customer-managed encryption key from
Cloud KMS to encrypt it.
Update a property in a table's metadata with the
:func:`~google.cloud.bigquery.client.Client.update_table` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_create_table_cmek]
:end-before: [END bigquery_create_table_cmek]
:start-after: [START bigquery_update_table_description]
:end-before: [END bigquery_update_table_description]

Change the key used to encrypt a table.
Insert rows into a table's data with the
:func:`~google.cloud.bigquery.client.Client.insert_rows` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_update_table_cmek]
:end-before: [END bigquery_update_table_cmek]
:start-after: [START bigquery_table_insert_rows]
:end-before: [END bigquery_table_insert_rows]

Load a file from Cloud Storage, using a customer-managed encryption key from
Cloud KMS for the destination table.
Copying a Table
^^^^^^^^^^^^^^^

Copy a table with the
:func:`~google.cloud.bigquery.client.Client.copy_table` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_load_table_gcs_json_cmek]
:end-before: [END bigquery_load_table_gcs_json_cmek]
:start-after: [START bigquery_copy_table]
:end-before: [END bigquery_copy_table]

Copy a table, using a customer-managed encryption key from Cloud KMS for the
destination table.
Copy table data to Google Cloud Storage with the
:func:`~google.cloud.bigquery.client.Client.extract_table` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_copy_table_cmek]
:end-before: [END bigquery_copy_table_cmek]
:start-after: [START bigquery_extract_table]
:end-before: [END bigquery_extract_table]

Write query results to a table, using a customer-managed encryption key from
Cloud KMS for the destination table.
Deleting a Table
^^^^^^^^^^^^^^^^

Delete a table with the
:func:`~google.cloud.bigquery.client.Client.delete_table` method:

.. literalinclude:: ../snippets.py
:language: python
:dedent: 4
:start-after: [START bigquery_query_destination_table_cmek]
:end-before: [END bigquery_query_destination_table_cmek]
:start-after: [START bigquery_delete_table]
:end-before: [END bigquery_delete_table]
Loading

0 comments on commit 277aab1

Please sign in to comment.