feat: Separate file parts #154
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
disq
commented
May 10, 2023
| ) | ||
|
|
||
| type Client struct { | ||
| spec *FileSpec | ||
|
|
||
| types.FileType |
There was a problem hiding this comment.
This is used for all interactions with the underlying writer. For read tests, separate Clients are used which are still below.
disq
commented
May 10, 2023
csv/write.go
Outdated
|
|
||
| var _ types.Handle = (*Handle)(nil) | ||
|
|
||
| func (cl *Client) WriteHeader(w io.Writer, s *arrow.Schema) (types.Handle, error) { |
There was a problem hiding this comment.
In this particular format/package, the csv header is written with the first call to .Write (and not in this fn) so the name might be misleading. Maybe rename the interface method to 'Begin' etc?
disq
force-pushed
the
feat/filetypes-separate-parts
branch
from
5b45f8e
to
8260f76
Compare
This was referenced May 16, 2023
disq
force-pushed
the
feat/filetypes-separate-parts
branch
2 times, most recently
from
efd328f
to
c274bcb
Compare
disq
force-pushed
the
feat/filetypes-separate-parts
branch
from
7a1d607
to
5b43e44
Compare
candiduslynx
approved these changes
Jun 6, 2023
kodiakhq bot
pushed a commit
that referenced
this pull request
Jun 14, 2023
🤖 I have created a release *beep* *boop* --- ## [3.2.0](v3.1.6...v3.2.0) (2023-06-14) ### Features * Separate file parts ([#154](#154)) ([3798fab](3798fab)) ### Bug Fixes * **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 4d76231 ([#198](#198)) ([aafa961](aafa961)) * **deps:** Update module github.com/cloudquery/plugin-sdk/v3 to v3.10.5 ([#201](#201)) ([0c8519c](0c8519c)) * **deps:** Update module github.com/cloudquery/plugin-sdk/v3 to v3.10.6 ([#202](#202)) ([a94a865](a94a865)) * Use buffered writing mode in parquet ([#203](#203)) ([5ad67f3](5ad67f3)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is 100% compatible with the 'old ways' because it still implements
WriteTableBatchFilewith unchanged signature.Readis not refactored due to the interface differences between parquet and the other types.