Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UnicodeDecodeError when looking for failures from Xcode build logs #365

Merged
merged 3 commits into from
Oct 26, 2023

Conversation

priitlatt
Copy link
Contributor

@priitlatt priitlatt commented Oct 25, 2023

Backwards file reader codemagic.utilities.backwards_file_reader.iter_backwards can fail with UnicodeDecodeError if the file contains some invalid byte sequence. One case when this can happen is during reading Xcode build logs:

File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 206, in invoke_cli
  CliApp._running_app._invoke_action(args)
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 163, in _invoke_action
  return cli_action(**action_args)
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/cli/cli_app.py", line 458, in wrapper
  return func(self, *args, **kwargs)
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/tools/xcode_project.py", line 270, in build_ipa
  xcarchive = xcodebuild.archive(
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/models/xcodebuild.py", line 249, in archive
  errors = _XcodebuildLogErrorFinder(self.logs_path).find_failure_logs()
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/models/xcodebuild.py", line 440, in find_failure_logs
  failed_commands = self._get_failed_commands()
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/models/xcodebuild.py", line 390, in _get_failed_commands
  for line in self._backwards_log_iterator:
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/utilities/backwards_file_reader.py", line 49, in iter_backwards
  yield from _iter_backwards(fd, buffer_size, file_size=file_path.stat().st_size)
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/site-packages/codemagic/utilities/backwards_file_reader.py", line 19, in _iter_backwards
  buffer = file_descriptor.read(min(unprocessed_size, buffer_size))
File "/Users/builder/.pyenv/versions/3.8.13/lib/python3.8/codecs.py", line 322, in decode
  (result, consumed) = self._buffer_decode(data, self.errors, final)

UnicodeDecodeError('utf-8', ..., '1', 'invalid start byte')

Applied changes:

  • Support backwards line iteration from in-memory binary file descriptors.
  • Read the files in binary instead of text mode whenever possible to avoid such errors.
  • Ignore decoding errors when converting bytes back to strings.

Updated actions:

  • xcode-project build-ipa

@priitlatt priitlatt added the bug Something isn't working label Oct 25, 2023
@priitlatt priitlatt marked this pull request as ready for review October 25, 2023 11:33
…gfix/backwards-iterator-unicode-decode-errors
@priitlatt priitlatt merged commit 7b5d823 into master Oct 26, 2023
7 checks passed
@priitlatt priitlatt deleted the bugfix/backwards-iterator-unicode-decode-errors branch October 26, 2023 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant