Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Allow setting BigQuery endpoint, for example to use bigquery emulator #28149

Closed
1 of 15 tasks
faisalhasnain opened this issue Aug 24, 2023 · 5 comments · Fixed by #32153
Closed
1 of 15 tasks

Comments

@faisalhasnain
Copy link

faisalhasnain commented Aug 24, 2023

What happened?

There is no way to specify root url for bigquery service like we have for pubsub (PubsubOptions.setPubsubRootUrl). In particular i was trying leverage bigquery-emulator for my local development environment and that was the only blocker.

Issue Priority

Priority: 3 (minor)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@pablomarti
Copy link

I am facing the same problem, I would like to be able to use the emulator. After checking the BigQueryIO class at public static <T> TypedRead<T> read(SerializableFunction<SchemaAndRecord, T> parseFn) it seems it could be possible by having a wrapper around setBigQueryServices?

@myhau
Copy link

myhau commented Jun 28, 2024

I would also like to use BigQuery emulator in our local test suite, but overriding base url in BigQueryIO does not seem to be supported.

I'm happy to add support for this, but wanted to know maintainers' opinion first.

What do you think?
cc: @kennknowles @ahmedabu98

Btw issue #28841 seems to be a duplicate of this one.

@kennknowles
Copy link
Member

I think this is a fine idea.

@kennknowles kennknowles changed the title [Bug]: Unable to use bigquery emulator with java bigquery io connector [Feature Request]: Allow setting BigQuery endpoint, for example to use bigquery emulator Jun 28, 2024
@kberezin-nshl
Copy link
Contributor

.take-issue

@tomaslink
Copy link

I am facing the same problem with Python. I see the PR added this feature only for Java. Should we create another separate Issue for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants