Skip to content
This repository has been archived by the owner on Sep 27, 2019. It is now read-only.

Commit

Permalink
Fix CSV scanner for handling filter predicate on a VARCHAR column
Browse files Browse the repository at this point in the history
CSV scan does not care about delimiter in a CSV file, but query
processor should care the end of each column value, i.e., ‘\0’.

Because query parser parses each selection predicate by adding ‘\0’ at
the end of the predicate value, query processor cannot handle the query
correctly if we ignore the existence of delimiter in a raw tuple in CSV
file.

This simple solution fix the case by replacing delimiter to ‘\0’, and
thus StringCompare in type/type_util.h works correctly.
  • Loading branch information
RonyMin committed Nov 30, 2018
1 parent d5b9cf0 commit 3beb7a5
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion src/codegen/util/csv_scanner.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,9 @@ void CSVScanner::ProduceCSV(char *line) {
cols_[col_idx].len = static_cast<uint32_t>(col_end - col_begin);
cols_[col_idx].is_null = (cols_[col_idx].len == 0);

// Yoon-Min: replace delimiter to '\0' to fix a bug while comparing
// filter condition in a query and column value of VARCHAR type
*iter = 0;
// Eat delimiter, moving to next column
iter++;
}
Expand All @@ -372,4 +375,4 @@ void CSVScanner::ProduceCSV(char *line) {

} // namespace util
} // namespace codegen
} // namespace peloton
} // namespace peloton

0 comments on commit 3beb7a5

Please sign in to comment.