Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate code from googleapis/python-bigquery-connection #10255

Merged
merged 47 commits into from
Jun 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
d1b575f
docs: add python quickstart sample (#141)
loferris Dec 1, 2021
9ceb033
chore(deps): update dependency google-cloud-testutils to v1.3.0 (#143)
renovate-bot Dec 3, 2021
d756eae
chore(deps): update dependency google-cloud-testutils to v1.3.1 (#144)
renovate-bot Jan 8, 2022
3cf1477
chore(samples): Add check for tests in directory (#152)
gcf-owl-bot[bot] Jan 11, 2022
757e145
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Jan 19, 2022
7428be9
chore(python): Noxfile recognizes that tests can live in a folder (#157)
gcf-owl-bot[bot] Jan 19, 2022
cf3202c
chore(deps): update dependency pytest to v7 (#164)
renovate-bot Feb 7, 2022
0b41a2d
chore(deps): update dependency pytest to v7.0.1 (#166)
renovate-bot Feb 14, 2022
ebb68e1
chore(deps): update all dependencies (#170)
renovate-bot Mar 1, 2022
bf9e5fe
chore: Adding support for pytest-xdist and pytest-parallel (#177)
gcf-owl-bot[bot] Mar 4, 2022
c30570b
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Mar 8, 2022
770e5ff
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Mar 8, 2022
ca249de
chore(deps): update dependency pytest to v7.1.0 (#185)
renovate-bot Mar 13, 2022
e269132
chore(deps): update dependency pytest to v7.1.1 (#186)
renovate-bot Mar 19, 2022
153d88f
chore(python): use black==22.3.0 (#189)
gcf-owl-bot[bot] Mar 28, 2022
b068ebd
docs(samples): create connection sample for MySQL instance (#147)
loferris Apr 15, 2022
091e8ec
chore(python): add nox session to sort python imports (#203)
gcf-owl-bot[bot] Apr 21, 2022
01546ab
fix: region tags in create_mysql_connection.py (#205)
loferris Apr 21, 2022
88f872c
chore(deps): update dependency pytest to v7.1.2 (#207)
renovate-bot Apr 25, 2022
ec5e62b
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot May 19, 2022
e3de096
chore(deps): update all dependencies (#223)
renovate-bot Jun 7, 2022
fd34793
fix: require python 3.7+ (#231)
gcf-owl-bot[bot] Jul 10, 2022
16db731
chore(deps): update all dependencies (#239)
renovate-bot Aug 2, 2022
87c8be0
chore(deps): update all dependencies (#241)
renovate-bot Aug 5, 2022
37b7653
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Aug 15, 2022
407dbc4
chore(deps): update dependency pytest to v7.1.3 (#256)
renovate-bot Sep 6, 2022
e3e3926
chore: detect samples tests in nested directories (#260)
gcf-owl-bot[bot] Sep 13, 2022
5a28647
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Oct 4, 2022
b3f0ada
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Oct 10, 2022
6a3233e
chore(deps): update dependency pytest to v7.2.0 (#269)
renovate-bot Oct 26, 2022
1f86bba
chore(python): drop flake8-import-order in samples noxfile (#275)
gcf-owl-bot[bot] Nov 27, 2022
6baccd1
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Dec 16, 2022
d45497f
chore(python): add support for python 3.11 (#279)
gcf-owl-bot[bot] Jan 6, 2023
b2a0051
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Jan 11, 2023
d1e5fd5
chore(deps): update dependency pytest to v7.2.1 (#283)
renovate-bot Jan 14, 2023
c4e044c
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Feb 9, 2023
3e292a4
feat: enable "rest" transport in Python for services supporting numer…
gcf-owl-bot[bot] Feb 16, 2023
bb8f52b
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Mar 1, 2023
1fa5e32
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Mar 1, 2023
ff09d7a
chore(deps): update dependency pytest to v7.2.2 (#301)
renovate-bot Mar 4, 2023
2a7932b
chore(deps): update dependency google-cloud-bigquery-connection to v1…
renovate-bot Apr 6, 2023
a827e5f
chore(deps): update dependency pytest to v7.3.0 (#309)
renovate-bot Apr 18, 2023
4c171b0
chore(deps): update dependency pytest to v7.3.1 (#310)
renovate-bot Apr 18, 2023
301dc84
chore(deps): update dependency pytest to v7.3.2 (#315)
renovate-bot Jun 12, 2023
58e89fe
Merge remote-tracking branch 'migration/main' into python-bigquery-co…
dizcology Jun 15, 2023
770059c
remove snippet/noxfile.py and omit python 3.6 in noxfile_config.py
dizcology Jun 15, 2023
f498581
update .github/CODEOWNERS and blunderbuss.yml
dizcology Jun 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
/texttospeech/**/* @GoogleCloudPlatform/dee-data-ai @GoogleCloudPlatform/python-samples-reviewers
/translate/**/* @GoogleCloudPlatform/dee-data-ai @GoogleCloudPlatform/python-samples-reviewers
/video/transcoder/* @GoogleCloudPlatform/dee-data-ai @GoogleCloudPlatform/python-samples-reviewers
/bigquery-connection/**/* @GoogleCloudPlatform/dee-data-ai @GoogleCloudPlatform/python-samples-reviewers

# Cloud SDK Databases & Data Analytics teams
# ---* Cloud Native DB
Expand Down
1 change: 1 addition & 0 deletions .github/blunderbuss.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ assign_issues_by:
- "api: texttospeech"
- "api: translate"
- "api: vision"
- "api: bigquery"
to:
- GoogleCloudPlatform/dee-data-ai

Expand Down
Empty file.
53 changes: 53 additions & 0 deletions bigquery-connection/snippets/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os

from google.cloud.bigquery_connection_v1.services import connection_service
import pytest


@pytest.fixture(scope="session")
def connection_client() -> connection_service.ConnectionServiceClient:
return connection_service.ConnectionServiceClient()


@pytest.fixture(scope="session")
def project_id() -> str:
return os.environ["GOOGLE_CLOUD_PROJECT"]


@pytest.fixture(scope="session")
def location() -> str:
return "US"


@pytest.fixture(scope="session")
def database() -> str:
return os.environ["MYSQL_DATABASE"]


@pytest.fixture(scope="session")
def cloud_sql_conn_name() -> str:
return os.environ["MYSQL_INSTANCE"]


@pytest.fixture(scope="session")
def mysql_username() -> str:
return os.environ["MYSQL_USER"]


@pytest.fixture(scope="session")
def mysql_password() -> str:
return os.environ["MYSQL_PASSWORD"]
66 changes: 66 additions & 0 deletions bigquery-connection/snippets/create_mysql_connection.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START bigqueryconnection_create_connection]
from google.cloud import bigquery_connection_v1 as bq_connection

"""This sample shows how to create a BigQuery connection with a Cloud SQL for MySQL database"""


def main() -> None:
# TODO(developer): Set all variables for your Cloud SQL for MySQL connection.
project_id = "your-project-id" # set project_id
location = "US" # set location
# See: https://cloud.google.com/bigquery/docs/locations for a list of
# available locations.
database = "my-database" # set database name
username = "my-username" # set database username
password = "my-password" # set database password
cloud_sql_conn_name = "" # set the name of your connection
transport = "grpc" # Set the transport to either "grpc" or "rest"

cloud_sql_credential = bq_connection.CloudSqlCredential(
{
"username": username,
"password": password,
}
)
cloud_sql_properties = bq_connection.CloudSqlProperties(
{
"type_": bq_connection.CloudSqlProperties.DatabaseType.MYSQL,
"database": database,
"instance_id": cloud_sql_conn_name,
"credential": cloud_sql_credential,
}
)
create_mysql_connection(project_id, location, cloud_sql_properties, transport)


def create_mysql_connection(
project_id: str,
location: str,
cloud_sql_properties: bq_connection.CloudSqlProperties,
transport: str,
) -> None:
connection = bq_connection.types.Connection({"cloud_sql": cloud_sql_properties})
client = bq_connection.ConnectionServiceClient(transport=transport)
parent = client.common_location_path(project_id, location)
request = bq_connection.CreateConnectionRequest(
{"parent": parent, "connection": connection}
)
response = client.create_connection(request)
print(f"Created connection successfully: {response.name}")


# [END bigqueryconnection_create_connection]
93 changes: 93 additions & 0 deletions bigquery-connection/snippets/create_mysql_connection_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import google.api_core.exceptions
from google.cloud import bigquery_connection_v1 as bq_connection
from google.cloud.bigquery_connection_v1.services import connection_service
import pytest
import test_utils.prefixer

from . import create_mysql_connection

connection_prefixer = test_utils.prefixer.Prefixer("py-bq-r", "snippets", separator="-")


@pytest.fixture(scope="session")
def location_path(
connection_client: connection_service.ConnectionServiceClient(),
project_id: str,
location: str,
) -> str:
return connection_client.common_location_path(project_id, location)


@pytest.fixture(scope="module", autouse=True)
def cleanup_connection(
connection_client: connection_service.ConnectionServiceClient, location_path: str
) -> None:
for connection in connection_client.list_connections(parent=location_path):
connection_id = connection.name.split("/")[-1]
if connection_prefixer.should_cleanup(connection_id):
connection_client.delete_connection(name=connection.name)


@pytest.fixture(scope="session")
def connection_id(
connection_client: connection_service.ConnectionServiceClient,
project_id: str,
location: str,
) -> str:
id_ = connection_prefixer.create_prefix()
yield id_

connection_name = connection_client.connection_path(project_id, location, id_)
try:
connection_client.delete_connection(name=connection_name)
except google.api_core.exceptions.NotFound:
pass


@pytest.mark.parametrize("transport", ["grpc", "rest"])
def test_create_mysql_connection(
capsys: pytest.CaptureFixture,
mysql_username: str,
mysql_password: str,
database: str,
cloud_sql_conn_name: str,
project_id: str,
location: str,
transport: str,
) -> None:
cloud_sql_credential = bq_connection.CloudSqlCredential(
{
"username": mysql_username,
"password": mysql_password,
}
)
cloud_sql_properties = bq_connection.CloudSqlProperties(
{
"type_": bq_connection.CloudSqlProperties.DatabaseType.MYSQL,
"database": database,
"instance_id": cloud_sql_conn_name,
"credential": cloud_sql_credential,
}
)
create_mysql_connection.create_mysql_connection(
project_id=project_id,
location=location,
cloud_sql_properties=cloud_sql_properties,
transport=transport,
)
out, _ = capsys.readouterr()
assert "Created connection successfully:" in out
38 changes: 38 additions & 0 deletions bigquery-connection/snippets/noxfile_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Default TEST_CONFIG_OVERRIDE for python repos.

# You can copy this file into your directory, then it will be inported from
# the noxfile.py.

# The source of truth:
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/noxfile_config.py

TEST_CONFIG_OVERRIDE = {
# You can opt out from the test for specific Python versions.
"ignored_versions": ["2.7", "3.6"],
# Old samples are opted out of enforcing Python type hints
# All new samples should feature them
"enforce_type_hints": True,
# An envvar key for determining the project id to use. Change it
# to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
# build specific Cloud project. You can also use your own string
# to use your own Cloud project.
"gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
# 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
# A dictionary you want to inject into your test. Don't put any
# secrets here. These values will override predefined values.
"envs": {},
}
33 changes: 33 additions & 0 deletions bigquery-connection/snippets/quickstart.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# [START bigqueryconnection_quickstart]

from google.cloud import bigquery_connection_v1 as bq_connection


def main(
project_id: str = "your-project-id", location: str = "US", transport: str = "grpc"
) -> None:
"""Prints details and summary information about connections for a given admin project and location"""
client = bq_connection.ConnectionServiceClient(transport=transport)
print(f"List of connections in project {project_id} in location {location}")
req = bq_connection.ListConnectionsRequest(
parent=client.common_location_path(project_id, location)
)
for connection in client.list_connections(request=req):
print(f"\tConnection {connection.friendly_name} ({connection.name})")


# [END bigqueryconnection_quickstart]
26 changes: 26 additions & 0 deletions bigquery-connection/snippets/quickstart_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import pytest

from . import quickstart


@pytest.mark.parametrize("transport", ["grpc", "rest"])
def test_quickstart(
capsys: pytest.CaptureFixture, project_id: str, location: str, transport: str
) -> None:
quickstart.main(project_id, location, transport)
out, _ = capsys.readouterr()
assert "List of connections in project" in out
2 changes: 2 additions & 0 deletions bigquery-connection/snippets/requirements-test.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
pytest==7.3.2
google-cloud-testutils==1.3.3
1 change: 1 addition & 0 deletions bigquery-connection/snippets/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
google-cloud-bigquery-connection==1.12.0