Images posted to server via fetch get their size inflated significantly

[thmsl]

React Native version:

System:

OS: macOS 10.15
CPU: (8) x64 Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz
Memory: 181.84 MB / 8.00 GB
Shell: 5.7.1 - /bin/zsh

Binaries:

Node: 10.16.3 - ~/.nvm/versions/node/v10.16.3/bin/node
Yarn: 1.17.3 - /usr/local/bin/yarn
npm: 6.9.0 - ~/.nvm/versions/node/v10.16.3/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman

SDKs:

iOS SDK:
  Platforms: iOS 13.1, DriverKit 19.0, macOS 10.15, tvOS 13.0, watchOS 6.0

IDEs:

Xcode: 11.1/11A1027 - /usr/bin/xcodebuild

npmPackages:

react: 16.8.1 => 16.8.1
react-native: 0.61.3 => 0.61.3

react-native-cli: 2.9.0

Description

After upgrading to RN 0.61.3, we observed that the hash of an image received on the server differs with the hash of the original picture on the phone.

The size is actually inflated significantly
jpg on the phone: 264KB
jpg received on the server: 628KB

This issue happens on iOS (13.1 tested only)
No issue detected on Android

Downgrading to 0.59 fixes the issue.

_body

  const formData = new FormData()
  formData.append(`files`, {
    uri: 'file://' + filepath,
    name: 'test.jpg',
    filename: 'test.jpg',
    type: 'application/octet-stream'
  })

Headers

      {
        Accept: 'application/json',
        // doesn't work with 'application/octet-stream' neither
        'Content-Type': 'multipart/form-data'
      }

Request

  // prepare form data
  // send post request
  fetch(`http://localhost:3000/upload`, 
  {
    method: 'POST',
    headers: {
      Accept: 'application/json',
      // doesn't work with 'application/octet-stream' neither
      'Content-Type': 'multipart/form-data'    
    },
    body: formData
  }).then(resp => {
    console.log('success: ' + JSON.stringify(resp))
  }).catch(err => {
    console.log('error: ' + err.message)
  })

The image size is unchanged until it is posted.
Image received by the server is modified.

Steps To Reproduce

Clone test repository

https://github.com/ivzeus/test-uploader

Start local node server to receive file uploads

The server sample is based on this repository

# in project root
cd server
npm i
node ./server.js

It starts a server at localhost:3000 and accepts file upload through /upload POST request. Uploaded files will be stored in server/uploads folder.

Run ReactNative sample

This sample was created by initializing a blank React Native project

To run the demo:

# in project root
npm i
cd ios
pod install
cd ..
npm run ios

• First click on DOWNLOAD IMG and DOWNLOAD BIN to save an image and a binary file to app storage (original image size: 260,062 bytes)

• Then try clicking on UPLOAD IMG or UPLOAD BIN and observe the server console

• Notice how the image file has been changed when uploaded, but not the binary file (new image size: 718,367 bytes)

[thmsl]

We are now refactoring using rn-fetch-blob instead of axios.

[tareqdayya]

same happens with me when trying to convert image to blob to send over network:

const img = await fetch(MY_IMAGE_URI);
const blob = await img.blob();
console.log(blob.size);

the blob size is 2x the size of the original picture. The problem appears to only affect ios. Android is fine.

I now use react-native-fetch-blob to work around this.

[builtbyproxy]

We are now refactoring using rn-fetch-blob instead of axios.

Hey @thmsl Could you please offer some insight into why this change was made?
Also, if we don't want this size increase how would you suggest we prevent it?

[cristianoccazinsp]

Facing the same issue on iOS. Uploaded image with fetch is inflated and loses all its metadata, what's going on? Why is the file even tampered with?

UPDATE: Just confirmed the issue. Doing a multipart POST request with an image results in the image being twice as big and lost metadata. Doing the exact same request with another library (e.g., rn-fetch-blob or RNBackgroundUpload) uploads the image as is. This seems to be only an iOS issue.

Please review this issue asap, this is quite critical and also suspicious that a network library modifies files before uploads.

[sanqi-med]

the reason is that：when we upload a jpg image，react native will use RCTImageLoader to load the image , in RCTImageLoader:

• (id)sendRequest:(NSURLRequest *)request withDelegate:(id)delegate
  {
  __block RCTImageLoaderCancellationBlock requestToken;
  requestToken = [self loadImageWithURLRequest:request callback:^(NSError *error, UIImage *image) {
  if (error) {
  [delegate URLRequest:requestToken didCompleteWithError:error];
  return;
  }

    NSString *mimeType = nil;
    NSData *imageData = nil;
    if (RCTImageHasAlpha(image.CGImage)) {
        mimeType = @"image/png";
        imageData = UIImagePNGRepresentation(image);
    } else {
        mimeType = @"image/jpeg";
        imageData = UIImageJPEGRepresentation(image, 1.0);
    }
  
    NSURLResponse *response = [[NSURLResponse alloc] initWithURL:request.URL
                                                        MIMEType:mimeType
                                           expectedContentLength:imageData.length
                                                textEncodingName:nil];
  
    [delegate URLRequest:requestToken didReceiveResponse:response];
    [delegate URLRequest:requestToken didReceiveData:imageData];
    [delegate URLRequest:requestToken didCompleteWithError:nil];
  

  }];

  return requestToken;
  }

when a jpg image upload,RCTImageLoader will use UIImageJPEGRepresentation(image, 1.0) to get the blob ,while UIImageJPEGRepresentation function get the size is different of the origin image size.that the reason why jpg image size is different, png size is same

[cristianoccazinsp]

Why does react native need all that? Can't it just treat the image as a regular file? Loading the whole image in memory and doing professing on it is definitely a bad idea; it should be treated as any file. El sáb., 21 de marzo de 2020 06:00, MrZhaoCn <notifications@github.com> escribió:

…

the reason is that：when we upload a jpg image，react native will use RCTImageLoader to load the image , in RCTImageLoader: - (id)sendRequest:(NSURLRequest *)request withDelegate:(id)delegate { __block RCTImageLoaderCancellationBlock requestToken; requestToken = [self loadImageWithURLRequest:request callback:^(NSError *error, UIImage *image) { if (error) { [delegate URLRequest:requestToken didCompleteWithError:error]; return; } NSString *mimeType = nil; NSData *imageData = nil; if (RCTImageHasAlpha(image.CGImage)) { mimeType = @"image/png"; imageData = UIImagePNGRepresentation(image); } else { mimeType = @"image/jpeg"; imageData = UIImageJPEGRepresentation(image, 1.0); } NSURLResponse *response = [[NSURLResponse alloc] initWithURL:request.URL MIMEType:mimeType expectedContentLength:imageData.length textEncodingName:nil]; [delegate URLRequest:requestToken didReceiveResponse:response]; [delegate URLRequest:requestToken didReceiveData:imageData]; [delegate URLRequest:requestToken didCompleteWithError:nil]; }]; return requestToken; } when a jpg image upload,RCTImageLoader will use UIImageJPEGRepresentation(image, 1.0) to get the blob ,while UIImageJPEGRepresentation function get the size is different of the origin image size.that the reason why jpg image size is different, png size is same — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#27099 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALU263F3JJ4PEOHEUCNHDOLRIR63VANCNFSM4JIOYWIA> .

[ganzik83]

https://stackoverflow.com/questions/56682109/react-native-through-upload-image-on-s3-bucket-using-aws-sdk

1. add node-module 'base64-arraybuffer'

yarn add base64-arraybuffer

import {decode} from 'base64-arraybuffer';
import RNFS from 'react-native-fs';

 const base64 = await RNFS.readFile(filePath, 'base64');
  // console.log('base64', base64); //get base64 data

  const arrayBuffer = decode(base64);
  //console.log('arrayBuffer', arrayBuffer.byteLength); // don't increase file size

[MayoudP]

@cristianoccazinsp Did you succeed to fixe it or found a workaround ?

[cristianoccazinsp]

@MayoudP I ended up using rn-fetch-blob to upload files.

[MayoudP]

@MayoudP I ended up using rn-fetch-blob to upload files.

I wasn't able to make this library works with a pre-signed URL for an upload on S3... By any chances, do you have any snippet code or example ?

[cristianoccazinsp]

Actually, now that I look again, I'm using https://github.com/Vydia/react-native-background-upload (my own fork actually https://github.com/cristianoccazinsp/react-native-background-upload) for iOS uploads, and regular fetch for Android.

I don't have an AWS example, but it shouldn't be any different, just a bunch of headers/query string parameters.

[awinograd]

My workaround using expo-file-system

  // Workaround for https://github.com/facebook/react-native/issues/27099
  if (Platform.OS === 'ios' && blob.type === 'image/jpeg') {
    const originalUri = uri;
    uri = `${FileSystem.documentDirectory}resumableUploadManager-${fileName}.toupload`;
    await FileSystem.copyAsync({ from: originalUri, to: uri });
    blob = new Blob([await (await fetch(uri)).blob()], { type: 'image/jpeg' });
  }

[stale]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may also label this issue as a "Discussion" or add it to the "Backlog" and I will leave it open. Thank you for your contributions.

[chrisbobbe]

Stale-bot, it looks like there hasn't been an official fix or workaround, so please don't close this. 🙂

[gensc004]

Hello everyone in the future still struggling with this. I was struggling with a similar issue trying to upload images to AWS S3 through a PreSigned PUT URL with metadata (specifically EXIF data like GPS coordinates). RNFS uploadFiles doesn't play nicely with signed S3 urls, so I was trying to use the built in XMLHttpRequest. I was able to upload files, but their metadata was just being silently removed due to things discussed earlier in this thread. Following through on @ganzik83 's train of thought, I was able to finally get metadata to stick through the upload process by converting the image to base64, fetching the base64, converting the result of the fetch into a blob, and then uploading that blob. The code ended up looking like this:

import RNFS from 'react-native-fs';

const uploadImageToSignedURL = async (signedURL, imagePath) => { // NOTE: In my case, Signed URL was configured with Content-Type: 'image/jpeg'
        const getBlob = async (fileUri) => {
            const base64 = await RNFS.readFile(fileUri, 'base64');
            const resp = await fetch(`data:image/jpeg;base64,${base64}`);
            const imageBody = await resp.blob();
            return imageBody;
        };

        const uploadImage = async (uploadUrl, imagePath) => {
            const imageBody = await getBlob(imagePath);

            return fetch(uploadUrl, {
                method: "PUT",
                body: imageBody,
            });
        };
        
        return uploadImage(signedURL, imagePath)
}

[ShogunPanda]

One final note on this.
As you can see, this has been fixed in 0.64.2.

If, by any chance, you are stuck in a lower version, I can tell you @awinograd solution works, but it can be overly simplified.
What Alec forgot to mention is that bug happens not due to the file content nor for the blob MIME but just due to the file extension.

Which means just renaming the file or copying to another destination with a suffix is enough.
Like:

const finalPath = image.path + '.toUpload';
await RNFS.copyFile(image.path, finalPath);

image = { ...image, uri: `file://${finalPath}`, path: finalPath };

Hope this helps!

[simonmitchell]

@ShogunPanda is this definitely resolved in 0.64.2? I can't find anything about it in the release notes! I'm not 100% sure I want to do the workaround you've posted as seems a bit hacky which could result in future bugs 😬

[ShogunPanda]

@simonmitchell It is hacky. :)

Unfortunately I couldn't update the project I'm currently on so I have no idea if 0.64.2 fixes for real or not. I trust RN devs on this.

[simonmitchell]

I tried in the end @ShogunPanda and seemed not to be fixed :( I think I'll go for hacky solution for now and try and keep an eye on when this issue is resolved! Thanks for the help ☺️

[tungduonghgg123]

It seems this issue still exist on latest react-native version 0.65.1

[arthuralee]

It's fixed in 0.66 (https://github.com/facebook/react-native/blob/main/CHANGELOG.md#v0660)

[mpatafio]

I'm experiencing this issue on react-native 0.66.4

[dmarkow]

Having the same issue on 0.66.4. Looks like the fix was reversed pretty quickly, so I'm wondering if it ever actually got included in a release...

[StampixSMO]

This is indeed still an issue in the latest release. It only happens when you have a file uri that has the jpeg extension however, as beautifully found out by @MrZhaoCn.

I find the easiest solution to be to rename the file with RNFS

const newUri = originalUri.replace(/\.jpe?g$/i, '.bin');
await RNFS.moveFile(originalUri, newUri);