Calling Google Cloud APIs privately from on prem using Private Service Connect
20230913
Public/private bq rest calls public first Private Service Connect second
get auth token https://cloud.google.com/bigquery/docs/authorization https://cloud.google.com/docs/authentication/provide-credentials-adc#how-to https://github.com/GoogleCloudPlatform/shell-samples/blob/HEAD/bigquery/authorization/snippets.sh
gcloud auth application-default login
after we work with
gcloud auth application-default print-access-token
ACCESS_TOKEN="$(gcloud auth application-default print-access-token)"
create a default big query table - in this case one of my GPS data exports
201611185_gps_distinct_rollerblad.csv
biometric:benchmark michaelobrien$ export GOOGLE_CLOUD_PROJECT=bigquery-ol
biometric:benchmark michaelobrien$ curl -H "Authorization: Bearer $ACCESS_TOKEN" "https://www.googleapis.com/bigquery/v2/projects/$GOOGLE_CLOUD_PROJECT/datasets"
{
"kind": "bigquery#datasetList",
"etag": "irRvAdMvQcDWywbnuMPyBg==",
"datasets": [
{
"kind": "bigquery#dataset",
"id": "bigquery-ol:rollerblade",
"datasetReference": {
"datasetId": "rollerblade",
"projectId": "bigquery-ol"
},
"location": "northamerica-northeast1"
}
]
}
curl -H "Authorization: Bearer $ACCESS_TOKEN" https://bigquery.googleapis.com/bigquery/v2/projects/bigquery-ol/datasets/rollerblade
{
"kind": "bigquery#dataset",
"etag": "YGr4j2cVE8o77F0DFWySFg==",
"id": "bigquery-ol:rollerblade",
"selfLink": "https://bigquery.googleapis.com/bigquery/v2/projects/bigquery-ol/datasets/rollerblade",
"datasetReference": {
"datasetId": "rollerblade",
"projectId": "bigquery-ol"
},
...
}
biometric:benchmark michaelobrien$ curl -H "Authorization: Bearer $ACCESS_TOKEN" https://bigquery.googleapis.com/bigquery/v2/projects/bigquery-ol/datasets/rollerblade/tables/rollerblade/data?maxResults=2
{
"kind": "bigquery#tableDataList",
"etag": "wVKEep3xEdb51S3g1w7o3w==",
"totalRows": "4331",
"pageToken": "BHGUBPMJRIAQAAASAUIIBAEAAUNAICACCABCB77777777777757SUACKQAAQUSYKCYFAWYTJM5YXKZLSPEWW63ARQX7PZB65AAAAAEQLOJXWY3DFOJRGYYLEMUNCINLEHE2GIYTEHEWWMMJUMUWTIM3GMQWTQZJQMYWTEMJVGFRWEOJTME3DMMQSBNZG63DMMVZGE3DBMRSRUJBYG5RTQNZUGM3S2YZZGBSC2NBWGVTC2OJYGA4C2ZJXGY4WGMBTGRSGEMLE",
"rows": [
{
"f": [
{
"v": "45.424712"
},
{
"v": "-75.698209"
}
]
},
{
"f": [
{
"v": "45.424712"
},
{
"v": "-75.6949"
}
]
}
]
}
Cannot add the project "vpn-aws-obs" to ADC as the quota project because the account in ADC does not have the "serviceusage.services.use" permission on this project. You might receive a "quota_exceeded" or "API not enabled" error. Run $ gcloud auth application-default set-quota-project to add a quota project.
ubuntu@ip-10-51-1-185:~$ curl -H "Authorization: Bearer $ACCESS_TOKENcurl -H "Authorization: Bearer $ACCESS_TOKEN" https://bigquery.googleapis.com/bigquery/v2/projects/vpn-aws-obs/datasets/rollerblade/tables/rollerblade/data?maxResults=2
{
"kind": "bigquery#tableDataList",
"etag": "xTHaKWTlPPkd2/tRo1kgDQ==",
"totalRows": "4331",
"pageToken": "BHQG7VMKRIAQAAASAUIIBAEAAUNAICACCABCB77777777777757SUACKQAAQUSYKCYFAW5TQNYWWC53TFVXWE4YR5ADGWH7PAAAAAEQLOJXWY3DFOJRGYYLEMUNCINZRGMYTONRXGYWTEZRSGIWTIMRUGYWTQZLEMYWTINDGGM4WKNDEGQ4WGYISBNZG63DMMVZGE3DBMRSRUJBQMRRWGY3DHEZS2YRVGI3S2NBSG44C2OLBHE2C2YZVMQZWKNDEMRQTSZJW",
"rows": [
{
"f": [
{
"v": "45.424712"
},
{
"v": "-75.698209"
}
]
},
{
"f": [
{
"v": "45.424712"
},
{
"v": "-75.6949"
}
]
}
]
}