Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dataflow runner v2 does not support reading a side input twice #20726

Closed
damccorm opened this issue Jun 4, 2022 · 0 comments · Fixed by #24321
Closed

Dataflow runner v2 does not support reading a side input twice #20726

damccorm opened this issue Jun 4, 2022 · 0 comments · Fixed by #24321
Labels
bug dataflow P3 runners
Milestone
2.44.0 Release

Comments

@damccorm
Copy link
Contributor

damccorm commented Jun 4, 2022

To reproduce:

  1. Create a side input
  2. Read it twice

This will fail with --experiments=use_runner_v2 on the DataflowRunner.

See https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming_PR/13/

Imported from Jira BEAM-11858. Original Jira may contain additional context.
Reported by: kenn.
@lukecwik lukecwik mentioned this issue Nov 22, 2022
Merged
4 tasks
lukecwik added a commit to lukecwik/incubator-beam that referenced this issue Nov 22, 2022
@lukecwik
Enable streaming runner v2 tests that were forgotten to be enabled.
89401b2 
These tests are run internally on Google and have been passing for quite some time.

Fixes apache#19957
Fixes apache#20726
Fixes apache#20601
Fixes apache#20734
lukecwik added a commit that referenced this issue Nov 24, 2022
@lukecwik
Enable streaming runner v2 tests that were forgotten to be enabled. (#…
3b7e181 
…24321)

* Enable streaming runner v2 tests that were forgotten to be enabled.

These tests are run internally on Google and have been passing for quite some time.

Fixes #19957
Fixes #20726
Fixes #20601
Fixes #20734

* Explicitly enable streaming engine for runner based autosharding test
@github-actions github-actions bot added this to the 2.44.0 Release milestone Nov 24, 2022
ruslan-ikhsan pushed a commit to ruslan-ikhsan/beam that referenced this issue Nov 30, 2022
@lukecwik @ruslan-ikhsan
Enable streaming runner v2 tests that were forgotten to be enabled. (a…
445ec8d 
…pache#24321)

* Enable streaming runner v2 tests that were forgotten to be enabled.

These tests are run internally on Google and have been passing for quite some time.

Fixes apache#19957
Fixes apache#20726
Fixes apache#20601
Fixes apache#20734

* Explicitly enable streaming engine for runner based autosharding test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug dataflow P3 runners
Projects
None yet
Milestone
2.44.0 Release
Development

Successfully merging a pull request may close this issue.

Enable streaming runner v2 tests that were forgotten to be enabled. lukecwik/incubator-beam
1 participant
@damccorm