Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interpret 01000 sqlstate as string truncation #138

Merged

Conversation

jgoizueta
Copy link

@jgoizueta jgoizueta commented Nov 7, 2020

See https://app.clubhouse.io/cartoteam/story/116537

We've found a case where when reading a text field containing 178665 chars (a BigQuery Geography in WKT form), the text was being truncated to 8192 characters (MAXIMUM_BUFFER_SIZE, the chunk size in this case) because after the SQLGetData call, which was returning SQL_SUCCESS_WITH_INFO, the SQLState obtained with SQLGetDiagRec was "01000".
The value used by all drivers we've been using for string truncation is "01004", but
MS Docs say drivers can do whatever they wish:

Unlike return codes, the SQLSTATEs in this manual are guidelines, and drivers are not required to return them. Therefore, while drivers should return the proper SQLSTATE for any error or warning they are capable of detecting, applications should not count on this always occurring
...
Because SQLSTATEs are not returned reliably ...
...
When applications do base programming logic on SQLSTATEs, they should be prepared for the SQLSTATE not to be returned or for a different SQLSTATE to be returned. Exactly which SQLSTATEs are returned reliably can be based only on experience with numerous drivers.

🤦

Note that according to SQLGetData documentation 01000 is a General Warning ("Driver-specific informational message").

In the case found it seems clear that the meaning is string truncation and the full string is read if we interpret it so.
So this will interpret both the usual 01004 and 01000 as string truncation. It seems unlikely that we find a driver which uses 01000 with other meaning here 🤞

@jgoizueta
Copy link
Author

I looked into the Simba driver documentation and found no reference to 01000, but I need to check if there's some additional documentation

@jgoizueta
Copy link
Author

Maybe it's better to restore the string truncation detection we had before #131 where in the case of SQLGetData returning SQL_SUCCESS_WITH_INFO and the SQLState being different from 01S07 (fractional truncation) we interpret it as string truncation.

@jgoizueta
Copy link
Author

Hey, I'm going to merge this to test a new release package and fix this ASAP, but I still welcome CR about this, for awareness and discussing this problem.

@jgoizueta jgoizueta requested a review from Algunenano November 9, 2020 08:20
@jgoizueta jgoizueta merged commit b6a290a into master Nov 9, 2020
@jgoizueta jgoizueta deleted the feature/ch116537/emp-admin-geometry-invalid-when-bigquery branch November 9, 2020 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant