Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINOR: ParquetRewriter must close opened reader/stream #3002

Merged
merged 6 commits into from
Aug 31, 2024
Merged

MINOR: ParquetRewriter must close opened reader/stream #3002

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0691068
fix: open stream closed.
cetindogu Aug 28, 2024
0a1205a
fix: null check added before close()
cetindogu Aug 28, 2024
d83561d
fix: close() can throw IOException
cetindogu Aug 28, 2024
2ffd538
one line code line is needed for spotless library
cetindogu Aug 29, 2024
6dc7522
fix: white space is added after if
Aug 29, 2024
b06dd02
fix: mvn spotless:apply command is run.
Aug 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions parquet-hadoop/src/main/java/org/apache/parquet/hadoop/rewrite/ParquetRewriter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ private void openInputFiles(List<InputFile> inputFiles, ParquetConfiguration con
}

// Routines to get reader of next input file and set up relevant states
private void initNextReader() {
private void initNextReader() throws IOException {
if (reader != null) {
LOG.info("Finish rewriting input file: {}", reader.getFile());
}
Expand All @@ -252,7 +252,7 @@ private void initNextReader() {
originalCreatedBy = null;
return;
}

if (reader != null) reader.close();
reader = inputFiles.poll();
meta = reader.getFooter();
originalCreatedBy = meta.getFileMetaData().getCreatedBy();
Expand Down