[HOLD for payment 2023-10-27] [$500] Android - Scan IOU - A broken image is downloaded to the device when receipt cannot be downloaded

[izarutskaya]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Launch New Expensify app.
2. Create a Scan IOU.
3. Open the IOU details view.
4. Tap on the receipt > overflow menu > Download.
5. Close the error message.
6. Go to the device gallery.
   Note that a broken image is downloaded despite the error message.

Expected Result:

Nothing will be downloaded to the device since it shows 'Attachment cannot be downloaded' message.

Actual Result:

A broken image is downloaded to the device despite the 'Attachment cannot be downloaded' message.

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: v1.3.72-8

Reproducible in staging?: Y

Reproducible in production?: Y

If this was caught during regression testing, add the test name, ID and link from TestRail:

Email or phone of affected tester (no customers):

Logs: https://stackoverflow.com/c/expensify/questions/4856

Notes/Photos/Videos: Any additional supporting documentation

Bug6208754_Screen_Recording_20230921_105335_Gallery.mp4

Expensify/Expensify Issue URL:

Issue reported by: Applause-Internal Team

Slack conversation: @

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~0126f987f66d4dbcf0
• Upwork Job ID: 1705133271802134528
• Last Price Increase: 2023-09-22
• Automatic offers:
• bernhardoj | Contributor | 27229717

[melvin-bot]

Triggered auto assignment to @sakluger (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[sakluger]

Yep, that's a bug. @izarutskaya do you know what caused it to throw the error?

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~0126f987f66d4dbcf0

[melvin-bot]

Current assignee @sakluger is eligible for the External assigner, not assigning anyone new.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @mananjadhav (External)

[ayazalavi]

Proposal

Please re-state the problem that we are trying to solve in this issue.

As soon as user try downloading file then error is thrown along with junk file stored.

What is the root cause of that problem?

Root cause exists when we try to copy file in media store which does not exist.

App/src/libs/fileDownload/index.android.js

Lines 75 to 84 in 171259c

	return RNFetchBlob.MediaCollection.copyToMediaStore(
	{
	name: attachmentName,
	parentFolder: 'Expensify',
	mimeType: null,
	},
	'Download',
	attachmentPath,
	);
	})

This results in junk file saved.

What changes do you think we should make in order to solve the problem?

Use RNFetchBlob.fs.exists(attachmentPath) to check if file exists before trying to save the file.

Result:

0e23b2c6-cb5b-4260-b827-af117f8d6127.mp4

What alternative solutions did you explore? (Optional)

We can alternatively check for file system issues happening through the app and fix them all.

[ayazalavi]

Hi @mananjadhav,

I hope you're well. I would like to request your approval for my proposal. If you can provide the approval, I will be able to create the pull request (PR) within the next hour. Your prompt response would be greatly appreciated.

Thank you for your consideration.

Best regards,
Ayaz Alavi

[bernhardoj]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Trying to download a local receipt image will give an error but a broken file is saved on the device.

What is the root cause of that problem?

First of all, I don't know the reason why a broken file is saved even though the download fails. On the native side, the lib already check whether the file exists or not.

So, I want to focus on the other problem that is we can't download the local receipt image.

When we do a scan request, the receipt image will initially be a local file. The local file URI looks like this (android).

file:///data/user/0/com.expensify.chat.dev/cache/rn_image_picker_lib_temp_40c8937d-64c5-4f0b-9de8-0622caaed92b.png

Downloading a local file is basically copying the cached file above (that is selected from the gallery) to the Expensify/Download folder.

App/src/libs/fileDownload/index.android.js

Line 48 in 44568ba

let attachmentPath = isLocalFile ? url : undefined;

App/src/libs/fileDownload/index.android.js

Lines 75 to 83 in 44568ba

	return RNFetchBlob.MediaCollection.copyToMediaStore(
	{
	name: attachmentName,
	parentFolder: 'Expensify',
	mimeType: null,
	},
	'Download',
	attachmentPath,
	);

However, when we try to download the receipt image, it will always append an auth token to the file URI because isAuthTokenRequired is always true.

App/src/components/AttachmentModal.js

Lines 169 to 175 in 44568ba

	const downloadAttachment = useCallback(() => {
	let sourceURL = source;
	if (isAuthTokenRequired) {
	sourceURL = addEncryptedAuthTokenToURL(sourceURL);
	}
	fileDownload(sourceURL, file.name);

auth token is supposed to be used for remote resources only but we always set isAuthTokenRequired to true for receipt image.

App/src/components/Attachments/AttachmentCarousel/extractAttachmentsFromReport.js

Lines 55 to 59 in 71db96d

	if (TransactionUtils.hasReceipt(transaction)) {
	const {image} = ReceiptUtils.getThumbnailAndImageURIs(transaction.receipt.source, transaction.filename);
	attachments.unshift({
	source: tryResolveUrlFromApiRoot(image),
	isAuthTokenRequired: true,

so the local full URI would be like

file:///.....?encryptedAuthToken=asdWJWAPdad...

And because of that, react-native-blob-util is unable to find the file that we want to copy/move and so an error (file not found) is thrown.

What changes do you think we should make in order to solve the problem?

If it's a local file, set isAuthTokenRequired to false.

const isLocalFile = image.startsWith('blob:') || image.startsWith('file:');
isAuthTokenRequired: !isLocalFile,

the condition is taken from

App/src/libs/ReceiptUtils.js

Line 20 in 71db96d

if (isReceiptImage && (path.startsWith('blob:') || path.startsWith('file:'))) {

[mananjadhav]

I think considering the broken behavior @bernhardoj's proposal makes more sense. Tagging @shawnborton @marcaaron @sakluger to confirm.

🎀 👀 🎀 C+ reviewed.

[melvin-bot]

Triggered auto assignment to @marcaaron, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[shawnborton]

Hmm my take is, is it possible to fix the fact that the image can't be downloaded? If not, then I agree, we should not download a broken image after seeing that error message.

[ayazalavi]

@shawnborton @mananjadhav it is android only bug. same code is getting file downloaded just fine in other platforms so basically file name convention is not a bug here. As I mentioned in my proposal it is android file system bug and needs to solved only in index.android.js. Quickest solution is to check whether file exists i.e. android file system can read the given file path. If not then dont save the file otherwise file stored will be empty. It is bug in the code it is not checking whether file exists prior to saving it in media store.

[shawnborton]

Ah then if that is the case, let's try to fix the file not getting downloaded.

[ayazalavi]

@shawnborton yes I have proposed that already.

[ayazalavi]

@marcaaron @sakluger please provide your input about above comments.

[marcaaron]

Well, wow, I actually don't super understand the root cause here. @bernhardoj's explanation is pretty unclear.

I don't know the reason why a broken file is saved even though the download fails.

Can we not figure this out? Also why did the download fail? What do we need to understand this problem better?

The issue is, that if we try to download the receipt image and log the image URI, it contains an auth token, so the full URI would be like file:///.....?encryptedAuthToken=asdWJWAPdad...
So, it makes sense the lib can't find the file. (the receipt image will initially be a local file and later will be replaced with an image from the server, at least on the web)

I don't really understand what you are trying to say here. Where are we logging an image URI? And why does it "make sense" that the lib (which lib exactly?) can't find the file? 😕

I'm going to kick this one back to @mananjadhav to manage for now to try again. @mananjadhav Please work with the contributors here to put forward a complete proposal that clearly describes the problem and offers a solution.

[bernhardoj]

Updated my proposal to include more explanation.

Can we not figure this out?

I'm not trying to figure this out because the main issue is that we can't download the file.

Where are we logging an image URI?

It's just a log that I put to see the file URI.

[ayazalavi]

@marcaaron look at the video I attached with my proposal. Additionally copytomediastore copies existing file to media directory but if file does not exists then it will save empty file there. Android filesystem is not able to understand path provided, it is android only bug and exists due to OS incompatibility with the plugin . Simply using RNFetchBlob.fs.exists(attachmentPath) can avoid saving empty file which will sort out the issue.

[sakluger]

@mananjadhav can you please take a look at Marc and Shawn's concerns and advise how you think we should proceed here?

[sakluger]

@ayazalavi I don't review any proposals as I am not an engineer - I'm just helping to manage the GH issue and keep it moving forward.

@marcaaron do you agree with @ArekChr's recommendation to go with @bernhardoj's solution?

[marcaaron]

Sounds good. Yes I agree after reading the updated proposal.

[melvin-bot]

📣 @bernhardoj 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job
Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻
Keep in mind: Code of Conduct | Contributing 📖

[bernhardoj]

PR is ready

cc: @ArekChr

[jjcoffee]

Looks like this would also fix #29749, could you confirm @ArekChr or @bernhardoj?

[bernhardoj]

@jjcoffee yes, I tested and that issue is fixed too.

[mvtglobally]

Issue not reproducible during KI retests. (Third week)

[melvin-bot]

🎯 ⚡️ Woah @ArekChr / @bernhardoj, great job pushing this forwards! ⚡️

The pull request got merged within 3 working days of assignment, so this job is eligible for a 50% #urgency bonus 🎉

• when @bernhardoj got assigned: 2023-10-16 23:10:48 Z
• when the PR got merged: 2023-10-18 21:29:44 UTC

On to the next one 🚀

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.87-12 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Fix can't download scan receipt image #29738

If no regressions arise, payment will be issued on 2023-10-27. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

• @ArekChr does not require payment (Contractor)
• @bernhardoj requires payment offer (Contributor)

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@ArekChr] The PR that introduced the bug has been identified. Link to the PR:
• [@ArekChr] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@ArekChr] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@ArekChr] Determine if we should create a regression test for this bug.
• [@ArekChr] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@sakluger] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[ArekChr]

• Link to the PR: No bug identified.
• Link to comment: n/a
• Link to discussion: n/a
• Determine if we should create a regression test for this bug: We can create regression test.

Regresion test proposal:

1. Open app and enable offline mode.
2. Create a Scan IOU.
3. Open the IOU details view.
4. Tap on the receipt > overflow menu > Download.
5. Ensure that the image downloaded succesfully and verify if image is valid.

Do we agree 👍 or 👎

[sakluger]

Proposed regression test steps look good. 👍

I've paid out @bernhardoj, so we're good to close this one. Thanks everyone! 🚀