Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assets uri's that begin with ph:// cause error #36136

Closed
norbusonam opened this issue Feb 13, 2023 · 32 comments
Closed

Assets uri's that begin with ph:// cause error #36136

norbusonam opened this issue Feb 13, 2023 · 32 comments
Labels
Component: Image p: Microsoft Partner: Microsoft Partner Platform: iOS iOS applications. Type: New Architecture Issues and PRs related to new architecture (Fabric/Turbo Modules)

Comments

@norbusonam
Copy link

norbusonam commented Feb 13, 2023

Description

When giving the Image component a uri beginning with ph:// like so:

<Image source={{ uri: "ph://*" }} />

the image fails to render:

eed5a1d1-5ffc-438c-96bf-ff3b79a8a6b5

In my scenario, I used expo-media-library to retrieve the uri's. I built my app with the old architecture, and this error did not persist.

Version

0.71.1

Output of npx react-native info

System:
OS: macOS 13.2
CPU: (8) arm64 Apple M1
Memory: 75.11 MB / 16.00 GB
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 18.13.0 - ~/.nvm/versions/node/v18.13.0/bin/node
Yarn: 1.22.19 - ~/.nvm/versions/node/v18.13.0/bin/yarn
npm: 8.19.3 - ~/.nvm/versions/node/v18.13.0/bin/npm
Watchman: 2023.01.30.00 - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.11.3 - /opt/homebrew/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1
Android SDK: Not Found
IDEs:
Android Studio: Not Found
Xcode: 14.2/14C18 - /usr/bin/xcodebuild
Languages:
Java: Not Found
npmPackages:
@react-native-community/cli: Not Found
react: 18.2.0 => 18.2.0
react-native: 0.71.1 => 0.71.1
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found

Steps to reproduce

  1. get a uri starting with ph:// by any means
  2. render an image with that uri

Snack, code example, screenshot, or link to a repository

Not sure how to switch a snack into the new architecture, but this code in the new architecture will fail:

https://snack.expo.dev/gv6bMd2MY?platform=ios

UPDATE: Providing a simpler example to convey that this is a react native issue and not due to any dependency. The following code fails on the new archatecure with the error above:

import React from 'react';
import { Image } from 'react-native';

const App = () => {
  return <Image source={{ uri: 'ph://B84E8479-475C-4727-A4A4-B77AA9980897' }} style={{ width: 100, height: 100 }} />;
};

export default App;

but it runs exactly as expected in the old one. It looks like the new architecture can not handle iOS's ph:// routes.

@norbusonam norbusonam added Needs: Triage 🔍 Type: New Architecture Issues and PRs related to new architecture (Fabric/Turbo Modules) labels Feb 13, 2023
@cortinico cortinico removed the Type: New Architecture Issues and PRs related to new architecture (Fabric/Turbo Modules) label Feb 13, 2023
@cortinico
Copy link
Contributor

In my scenario, I used expo-media-library to retrieve the uri's. I built my app with the old architecture, and this error did not persist.

As this is related to expo-media-library (which lives here https://github.com/expo/expo/tree/main/packages/expo-media-library) and their support for New Architecture, you should open this issue against Expo's repo.

@cortinico cortinico added Resolution: Issue in another tool or repo An issue that was opened against React Native but in reality is affecting another tool or library and removed Needs: Triage 🔍 labels Feb 13, 2023
@norbusonam
Copy link
Author

In my scenario, I used expo-media-library to retrieve the uri's. I built my app with the old architecture, and this error did not persist.

As this is related to expo-media-library (which lives here https://github.com/expo/expo/tree/main/packages/expo-media-library) and their support for New Architecture, you should open this issue against Expo's repo.

You don’t have to use expo-media-library. This issue persists with any ph://* uri

@norbusonam
Copy link
Author

norbusonam commented Feb 17, 2023

Could you please reopen this @cortinico? I boiled it down to something simpler to show that it is a react native issue, and not an expo-media-library issue:

running the following code fails on the new architecture:

import React from 'react';
import { Image } from 'react-native';

const App = () => {
  return <Image source={{ uri: 'ph://B84E8479-475C-4727-A4A4-B77AA9980897' }} style={{ width: 100, height: 100 }} />;
};

export default App;

but it runs exactly as expected in the old one. It looks like the new architecture can not handle iOS's ph:// routes.

I will update the original issue to make it clearer.

Thanks!

@cortinico cortinico reopened this Feb 17, 2023
@cortinico cortinico added Platform: iOS iOS applications. Type: New Architecture Issues and PRs related to new architecture (Fabric/Turbo Modules) and removed Resolution: Issue in another tool or repo An issue that was opened against React Native but in reality is affecting another tool or library labels Feb 17, 2023
@mingyeom1
Copy link

please save me!!!!!

@norbusonam
Copy link
Author

norbusonam commented Mar 1, 2023

for any one who needs a temporary workaround, I found that expo's image component works with ph:// routes.

cc @mingyeom1

@norbusonam
Copy link
Author

any update or ideas around what might cause this bug? I've been using expo-image in the meantime, but now I've run into the issue that expo-image does not render ph:// images when the user grants limited permissions (in both new and old architecture) 😞.

@lunaleaps
Copy link
Contributor

What does the ph:// prefix mean? What kind of permissions does the user need to grant?

@norbusonam
Copy link
Author

norbusonam commented Apr 15, 2023

What does the ph:// prefix mean? What kind of permissions does the user need to grant?

Just mean that the URI starts with ph://. These URI's route to PHAssets in iOS (basically just photo library assets).

That issue with permissions was just an expo-image thing.

The problem here is that React Native's core Image component errors when it tries to render any image with a ph:// on the new architecture. I confirmed this was a regression from the old architecture.

EDIT: To clarify, the user does need to grant either limited or full access to photo library to render a photo library image, but that is not the main issue here

@cortinico cortinico added p: Microsoft Partner: Microsoft Partner labels Apr 17, 2023
@norbusonam
Copy link
Author

linking to where the error is being triggered: code

@norbusonam
Copy link
Author

@react-native-camera-roll/camera-roll appears to be doing some handling of ph:// routes: code

maybe we can override getModuleInstanceFromClass in AppDelegate to add RNCAssetsLibraryRequestHandler to the list?

@norbusonam
Copy link
Author

Sharing some findings and a solution that worked for me:

Findings

According to this issue React Native no longer support's ph:// routes after @react-native-camera-roll/camera-roll was separated, and installing @react-native-camera-roll/camera-roll should integrate the proper handler. However, according to this issue the new architecture hard codes RCTURLRequestHandler, which means simply installing @react-native-camera-roll/camera-roll won't be enough to solve the problem.

Solution

Ultimetly, I was able to work around this by:

  1. install @react-native-camera-roll/camera-roll
  2. Override getModuleInstanceFromClass in your AppDelegate.mm to use @react-native-camera-roll/camera-roll's RNCAssetsLibraryRequestHandler handler:
...

#import "RNCAssetsLibraryRequestHandler.h"
#import <React/RCTImageLoader.h>
#import <React/RCTDataRequestHandler.h>
#import <React/RCTFileRequestHandler.h>
#import <React/RCTGIFImageDecoder.h>
#import <React/RCTHTTPRequestHandler.h>
#import <React/RCTLocalAssetImageLoader.h>
#import <React/RCTNetworking.h>

...

- (id<RCTTurboModule>)getModuleInstanceFromClass:(Class)moduleClass
{
  // Set up the default RCTImageLoader and RCTNetworking modules.
  if (moduleClass == RCTImageLoader.class) {
    return [[moduleClass alloc] initWithRedirectDelegate:nil
        loadersProvider:^NSArray<id<RCTImageURLLoader>> *(RCTModuleRegistry *) {
          return @[ [RCTLocalAssetImageLoader new] ];
        }
        decodersProvider:^NSArray<id<RCTImageDataDecoder>> *(RCTModuleRegistry *) {
          return @[ [RCTGIFImageDecoder new] ];
        }];
  } else if (moduleClass == RCTNetworking.class) {
    return [[moduleClass alloc] initWithHandlersProvider:^NSArray<id<RCTURLRequestHandler>> *(RCTModuleRegistry *) {
      return @[
        [RCTHTTPRequestHandler new],
        [RCTDataRequestHandler new],
        [RCTFileRequestHandler new],
        [RNCAssetsLibraryRequestHandler new],
      ];
    }];
  }
  // No custom initializer here.
  return [moduleClass new];
}

...

Extra Info

After these finding's I realized the reason ph:// routes were working on the old architecture was likely because expo provides a EXImageLoader. And since the new architecture hardcodes RCTURLRequestHandler, it did now work there.

@cipolleschi
Copy link
Contributor

Hi @norbusonam, thank you for opening the issue and to explore it further. We are aware of this limitation for the new Architecture, unfortunately. I see you found the PR trying to fix that, but we are not convinced by that fix yet.

Your override is a good workaround for the time being, while we find a better solution.

Copy link

github-actions bot commented Nov 9, 2023

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale There has been a lack of activity on this issue and it may be closed soon. label Nov 9, 2023
@cipolleschi
Copy link
Contributor

up, we don't want for this issue to be reaped. We will work on this next half.

@github-actions github-actions bot removed the Stale There has been a lack of activity on this issue and it may be closed soon. label Nov 10, 2023
@cipolleschi
Copy link
Contributor

I'm looking into this. I was not around when the old architecture was the only architecture...
What's the expected behavior? Was the old architecture able to open urls in the form of ph://<UUID>? Or was it just dying silently and now it is raising the redbox?

As far as I can see, we always relied on 3rd party libraries to load ph://. As @norbusonam mentioned, it's possible to use react-native-camera-roll. Thanks to #43046, libraries can now (from 0.74) self-register as handlers for ph://.

I'm trying to understand what's the expectation for a react native project with no libraries.

Perhaps @tsapeta, @WoLewicki or @mrousavy knows something about this? How was React Native behaving in the Old Architecture? Has it ever worked with ph:// urls out of the box?

@mrousavy
Copy link
Contributor

mrousavy commented Mar 5, 2024

I remember <Image> works with ph:// in old arch, I use it in ShadowLens. It should be supported by UIImage loading out of the box afaik.

@tsapeta
Copy link
Contributor

tsapeta commented Mar 6, 2024

I think it did work previously. We were using it in our examples for expo-media-library before we switched to expo-image.

After a quick look, I'm not actually sure why 😅 Rendering them would require using the Photos framework to load them, specifically its PHImageManager and PHAsset classes. It needs special handling since there are some more configuration options, e.g. whether to allow downloading the asset from iCloud and the requested quality (lower quality previews might be stored on device without the need to download them). Here is how we're doing that in expo-image: https://github.com/expo/expo/blob/main/packages/expo-image/ios/Loaders/PhotoLibraryAssetLoader.swift
The required permission is another thing.

Maybe the URL loader somehow supported it and treated as a network request?

After these finding's I realized the reason ph:// routes were working on the old architecture was likely because expo provides a EXImageLoader.

It's definitely not the case. This loader doesn't do anything beyond what RN's loaders support and we can actually remove it as it's no longer used.

@cipolleschi
Copy link
Contributor

@norbusonam @mrousavy @tsapeta
I spin up an app with 0.71.1, and the Old Architecture fails in the same way as the new one.

Repro:

npx react-native@0.71.1 init TestImage71_1 --version 0.71.1 #this installs pods for the old arch by default.
cd TestImage71_1/ios
open TestImage71_1.xcworkspace
cd ..
yarn start

I retrieved the ID by:

  1. Adding the NSPhotoLibraryUsageDescription in the capabilities in Xcode
  2. Adding the Photos framework in the Frameworks, Libraries and Embedded Contents in the General tab of the project in Xcode
  3. Adding this code to the app delegate to read the id:
#import <Photos/PHAsset.h>
//...
PHFetchResult<PHAsset *> * res = [PHAsset fetchAssetsWithOptions:nil];
PHAsset * first = res.firstObject;
NSLog(@"Id: %@", first.localIdentifier);

Then, replace the App.tsx content with the following:

import React from 'react';
import {
  SafeAreaView,
  ScrollView,
  StatusBar,
  useColorScheme,
  Image,
  View,
} from 'react-native';

import {Colors} from 'react-native/Libraries/NewAppScreen';

function App(): JSX.Element {
  const isDarkMode = useColorScheme() === 'dark';

  const backgroundStyle = {
    backgroundColor: isDarkMode ? Colors.darker : Colors.lighter,
  };

  return (
    <SafeAreaView style={backgroundStyle}>
      <StatusBar
        barStyle={isDarkMode ? 'light-content' : 'dark-content'}
        backgroundColor={backgroundStyle.backgroundColor}
      />
      <ScrollView
        contentInsetAdjustmentBehavior="automatic"
        style={backgroundStyle}>
        <View
          style={{
            backgroundColor: isDarkMode ? Colors.black : Colors.white,
          }}>
          <Image
            source={{uri: 'ph://106E99A1-4F6A-45A2-B320-B0AD4A8E8473'}}
            style={{width: 100, height: 100}}
          />
        </View>
      </ScrollView>
    </SafeAreaView>
  );
}

export default App;

For my simulator, 106E99A1-4F6A-45A2-B320-B0AD4A8E8473 is a valid id for an asset in the camera roll.

When I run the app in the Old Architecture, I get the same error message:
Screenshot 2024-03-06 at 10 29 26


Could it be that, in the Old Architecture, the libraries were able to self register their image handler in the list of handlers?
So, what I think was happening:

  1. yarn add react-native-cameraroll #and example of library that can handle ph:// url schemas
  2. Add NSPhotoLibraryUsageDescription
  3. Use `<Image source:{{ uri: "ph://*" }} />

And it looked like React Native is able to manage the ph:// directly, but it was actually a library handling that?

That fits with my understanding and as a difference between New/Old Arch. In fact, libraries couldn't, before 0.74, register themselves as URL Handlers/Image Loaders in the New Architecture.

@cipolleschi
Copy link
Contributor

Closing this as it is an intended React Native behaviour.

The Old Architecture is relying on some library to pick up the ph:// protocol. Libraries on the Old Architecture can register automatically to respond to custom URL protocols, that's why it looked like React Native was able to load images using the ph:// protocol.

In the New Architecture, before 0.74, libraries could not register themselves as URL handler, hence, this issue has been opened. In 0.74, thanks to this PR, libraries can now register themselves as URL handler, filling the gap between the Old and the New architecture.

@mrousavy
Copy link
Contributor

Hey @cipolleschi - this makes perfect sense, thanks for the explanation.

I'm not sure if I understand how to properly register custom image loaders (or URL handlers) now on the new architecture though? Is there a method I can use to register a custom handler in my AppDelegate?

RN CameraRoll still has that custom PH asset handler, but it doesn't work anymore. see react-native-cameraroll/react-native-cameraroll#631

@WoLewicki
Copy link
Contributor

@mrousavy I found this conversation: #36071 (comment). Seems like for some reason the links to commits are broken and lead to some weird, non-connected ones. I can quickly try to find the exact commit that introduces the changes @cipolleschi mentioned.

@WoLewicki
Copy link
Contributor

Looks like it is this one: #42923

@mrousavy
Copy link
Contributor

Got it!

For everyone else stumbling upon this issue, here's what I did:

  1. Create a file that implements the RCTImageURLLoader protocol, I'll call it RNCAssetsLibraryRequestHandler.h:
    #pragma once
    #import <Foundation/Foundation.h>
    #import <React/RCTImageURLLoader.h>
    
    @class PHPhotoLibrary;
    
    @interface RNCAssetsLibraryRequestHandler : NSObject <RCTImageURLLoader>
    
    @end
  2. Implement the two methods, and also treat it as an RCTBridgeModule (it has (NSString*)moduleName):
    #import "RNCAssetsLibraryRequestHandler.h"
    
    @implementation RNCAssetsLibraryRequestHandler
    
    - (BOOL)canLoadImageURL:(NSURL *)requestURL {
      return [requestURL.scheme isEqualToString:@"ph"];
    }
    
    - (RCTImageLoaderCancellationBlock)loadImageForURL:(NSURL *)imageURL
                                                  size:(CGSize)size
                                                 scale:(CGFloat)scale
                                            resizeMode:(RCTResizeMode)resizeMode
                                       progressHandler:(RCTImageLoaderProgressBlock)progressHandler
                                    partialLoadHandler:(RCTImageLoaderPartialLoadBlock)partialLoadHandler
                                     completionHandler:(RCTImageLoaderCompletionBlock)completionHandler {
      // TODO: Load ph:// asset here
    }
    
    + (NSString *)moduleName { 
      return @"RNCAssetsLibraryRequestHandler";
    }
    
    @end
  3. Add RNCAssetsLibraryRequestHandler to your lib's (or app's) codegen, there's a modulesConformingToProtocol option:
    "codegenConfig": {
      ...
      "ios": {
        "modulesConformingToProtocol": {
          "RCTImageURLLoader": "RNCAssetsLibraryRequestHandler"
        }
      },

Install pods & rebuild, and now the ph:// handler is registered 🎉

@WoLewicki I'll create a PR to CameraRoll :)

@mrousavy
Copy link
Contributor

@cortinico two things I wanna note here;

  1. Is what I'm doing fine? It seems like it's treating RNCAssetsLibraryRequestHandler as a RCTBridgeModule (at least that's the protocol that it's implementing through CodeGen) - is that still new arch stuff? For me it runs fine in bridgeless, so I was confused about this.
  2. Is there a documentation somewhere about all options you can pass to codegenConfig? I feel like this would've been easier to solve if there was a section in the docs about what you can actually do with codegenConfig, ideally something that automatically generates an API documentation based off of Flow/TS types of the codegen plugin.

@mrousavy
Copy link
Contributor

Ah - RCTImageURLLoader extends RCTBridgeModule. Is that correct? Does that need to be an RCTBridgeModule?
It won't have any methods or so anyways, so I'm thinking why not just have this be a simple ObjC class?

@mrousavy
Copy link
Contributor

I just created a PR in react-native-cameraroll to add custom Image URL loaders for ph:// and asset-library:// assets. react-native-cameraroll/react-native-cameraroll#632

So if you install react-native-cameraroll, ph:// assets can be loaded again in <Image> components on new arch/Fabric 🎉🥳

@khushal87
Copy link

khushal87 commented Jun 10, 2024

What is the fix for expo-media-library, though, since this issue seems to be relevant on version ~16.0.3 and with expo 51.0.11?

Just as a context, I am using the default Image component of React Native and I am on old architecture

@cha0sg0d
Copy link

What is the fix for expo-media-library, though, since this issue seems to be relevant on version ~16.0.3 and with expo 51.0.11?

Just as a context, I am using the default Image component of React Native and I am on old architecture

+1

@hamadmarhoon
Copy link

hamadmarhoon commented Jun 20, 2024

What is the fix for expo-media-library, though, since this issue seems to be relevant on version ~16.0.3 and with expo 51.0.11?

Just as a context, I am using the default Image component of React Native and I am on old architecture

Expo's image library is compatible with ph:// assets

You can also get the local URI of an asset by doing (MediaLibrary.getAssetInfoAsync(asset)).localUri

@tsapeta
Copy link
Contributor

tsapeta commented Jun 21, 2024

It turned out we had a regression causing the expo-media-library to not include the image loader for ph:// urls so the standard RN Image couldn't handle them. It's been fixed in version 16.0.4.

@nagisaando
Copy link

nagisaando commented Jun 22, 2024

I am using 16.0.4 for expo-media-library but still ph:// is not accepted by <Image>

But I found workaround as well. As #36136 (comment) says, (MediaLibrary.getAssetInfoAsync(asset)).localUri saved my life.

I left the workaround / error in my repo as well. Hopefully it helps someone

https://github.com/nagisaando/image-bug

@coolsoftwaretyler
Copy link

Hey folks, if anyone comes here on the new architecture, I just opened up expo/expo#33097, which I think might fix your issues once merged and shipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Image p: Microsoft Partner: Microsoft Partner Platform: iOS iOS applications. Type: New Architecture Issues and PRs related to new architecture (Fabric/Turbo Modules)
Projects
None yet
Development

No branches or pull requests