Forms with unicode file names fail upload

[yanokwa]

Test file: tést.xlsx.zip

Central reports Something went wrong: there was no request.

Here's what I know so far...

• Conversion with latest xlsform.getodk.org works, but Enketo preview fails with "Bad Request. Server URL is not a valid URL."
• Conversion with curl to latest pyxform-http works
  • curl --request POST --data-binary @/path/to/tést.xlsx http://127.0.0.1:5000/api/v1/convert
• Conversion with xls2xform works
• Uploading the XML to Central works and so does the Enketo preview
• Nothing obvious in Central logs

My gut says this is some interaction with Central and pyxform-http.

[matthew-white]

When I look at the browser console, I see the error message

Failed to execute 'setRequestHeader' on 'XMLHttpRequest': String contains non ISO-8859-1 code point.

Reading about it, I think the issue has to do with the X-XlsForm-FormId-Fallback header containing a non-ASCII character.

Is a form ID allowed to contain a non-ASCII character? If so, then I think we should encode the filename before specifying it in the header. Probably percent-encoding would be easiest? Frontend would need to encode the header, and pyxform-http would need to decode it, but I'm not sure that Backend would need to change.

We generally encode form IDs throughout Central, and Enketo links in Central use tokens separate from the form ID, so hopefully this would be a one-off change.

[lognaturel]

Is a form ID allowed to contain a non-ASCII character

Yes, it is.

Your analysis seems right to me and percent-encoding between frontend and pyxform-http seems like the right approach.

[matthew-white]

Sounds good! @yanokwa, are you able to add percent-decoding to pyxform-http? If so, I'll plan to add percent-encoding to Frontend.

[matthew-white]

As an update, as part of v1.2, we will have a new header, X-Action-Notes, that will need to be percent-encoded. Given that, I do think it makes for Frontend to percent-encode X-XlsForm-FormId-Fallback, as long as it's not hard for pyxform-http to decode it.

[lognaturel]

Related: getodk/collect#4554