Missing Stream API causes problems in ReactNative

[oed]

I'm trying to get the ipfs-http-client running in react-native, but ran into an issue I wasn't able to easily solve. The problem stems from that the iso-stream-http uses the fetch stream API, which seems to be available in browsers, but is not in the RN implementation of fetch.

The problem is outlined in this stackoverflow post. It also seems that the RN team is not very interested in adding support for this.

I've tried various fetch replacements, but none of the seem to support the stream api. Now I'm wondering what the next step could be? Is there some other way around this perhaps? Happy to hear suggestions 🙂

[achingbrain]

Maybe @hugomrdias could polyfill the stream API in iso-stream-http for React Native?

Though I think longer term we are switching this module over to use ky instead - this has been done for ipfs.add in v35.0.0 and ipfs.cat in v36.0.0, do you see the same problem with those method calls in React Native?

[hugomrdias]

This should do the trick https://www.npmjs.com/package/web-streams-polyfill

[hugomrdias]

Question why are you getting the browser version of iso-stream-http? Are you bundling? You should be getting the node version

[oed]

Thanks for the quick replies!

Using ipfs.cat is not either unfortunately with a different error. The callback is called with the following error:

[TypeError: _iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"] is not a function. (In '_iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"]()', '_iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"]' is undefined)]

I've tried to polyfill Symbol, but it's not been effective.

I just tried the web-streams-polyfil, but still have the same issue. Main problem is that body isn't defined in the response. This causes this line to throw an error.

I'm not sure why I'm getting the browser version. Is there any way I can force it to use the node one? Is that even available in RN?

Edit:
I'm using ipfs-http-client@38.0.0

[hugomrdias]

Polyfill with this https://github.com/github/fetch/blob/master/fetch.js

[hugomrdias]

Plus this for iterator in your index.js
global.Symbol = require('core-js/es6/symbol');
require('core-js/fn/symbol/iterator');

[oed]

Thanks @hugomrdias I did try adding the symbol polyfill before with no improvements. Double checked now and got the same result in ipfs.cat.

The whatwg-fetch polyfill doesn't seem to be improving the situation for ipfs.dag.get either.

[hugomrdias]

Body needs to be defined now with that polyfill what's the error

[oed]

Maybe I'm doing something wrong then. I'm adding this to index.js:

global.fetch = require('whatwg-fetch').fetch

But i still get an error saying the same thing:

TypeError: response.body is undefined

Edit: I don't see body being defined anywhere in https://github.com/github/fetch/blob/master/fetch.js

[pcowgill]

@hugomrdias What do you make of that latest error @oed was running into? Thanks!

[pcowgill]

Though I think longer term we are switching this module over to use ky instead - this has been done for ipfs.add in v35.0.0 and ipfs.cat in v36.0.0, do you see the same problem with those method calls in React Native?

@achingbrain Can you speculate on the timeline for a move to ky or similar? 1 month? 3 months? 1 year? 3 years? Thanks!

[hugomrdias]

Have you tried the latest version?

[alanshaw]

Just following up on this - is the latest version working for you both @oed @pcowgill ?

[pcowgill]

@hugomrdias @alanshaw I'll give it a try today! Was there anything in particular that changed in the latest version that you had in mind that might address this?

[alanshaw]

🙄 my apologies I just dug in deeper and unless RN has fixed their fetch impl I think this will still be a problem.

[pcowgill]

@alanshaw Noted, that's what I thought. Thanks for looking into this!

[hugomrdias]

@pcowgill are you able test if using this https://www.npmjs.com/package/react-native-v8 instead of the default jsc help with this ?

[pcowgill]

The problem stems from that the iso-stream-http uses the fetch stream API

Though I think longer term we are switching this module over to use ky instead

Update: @achingbrain I see that iso-stream-http was removed from this lib in this commit, so the move to ky is complete.

But getting a fetch API working applies still, since ky uses the fetch api

[pcowgill]

@pcowgill are you able test if using this https://www.npmjs.com/package/react-native-v8 instead of the default jsc help with this ?

@hugomrdias Sure, I'll give that a try. Thanks.

[pcowgill]

@hugomrdias FYI this issue mentioned above:

[TypeError: _iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"] is not a function. (In '_iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"]()', '_iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"]' is undefined)]

is still a problem with the latest version of this package and the latest version of React Native. I'm pretty sure the solution isn't polyfilling Symbol or iterator, since polyfillying those doesn't change the error message. (I did use the core-js polyfill like was suggested above.)

The issue seems to be getting the for await ...of loop working for async iterators.

The problem steps from the use of the it packages like is done here in the js-ipfs-http-client codebase for most of the ipfs.{insertmethodhere} calls. The it packages are used indirectly via the converters here.

The it packages aren't built/transpiled using a tool like rollup, which means it's hard to use them in a project with a different Babel config. I created an issue for that here.

Here is where it-all uses for await...of

[pcowgill]

I am able to reproduce the undefined error above with a minimal snippet not using the ipfs-http-client dep and just using an async generator and iterator, which is a helpful clue:

App.tsx

      async function* generateSequence(start, end) {
        for (let i = start; i <= end; i++) {
          await new Promise(resolve => setTimeout(resolve, 1000));
          yield i;
        }
      }
      (async () => {
        let generator = generateSequence(1, 5);
        for await (let value of generator) {
          console.log(value); // 1, then 2, then 3, then 4, then 5
        }
      })().catch(e => {
        console.log("Async iterator error", e);
        console.log("Async iterator error stack", e.stack);
      });

[hugomrdias]

So the problem is async iterators? I don't understand the roll-up Babel issue why can't you transpile the iterators in your project?

[pcowgill]

@hugomrdias It seems that way, yes. Transpiling the iterators in our project is definitely an option worth considering, but if a goal for this library is wide adoption, including a build script in this repo and publishing prebuilt vanilla js code to npm that can be used in a project with any Babel config seems ideal for making it approachable for developers with minimal setup friction. That's what most libraries do these days, right?

[hugomrdias]

http-client works in react-native with these instruction

react-native

npx react-native init test-native-project
cd test-native-project
yarn add whatwg-url ipfs-http-client
yarn add rn-nodeify --dev
npx rn-nodeify --install --hack --yarn

Add "postinstall": "rn-nodeify --install --hack --yarn" to npm scripts

To connect to a ipfs daemon using localhost:5001

https://stackoverflow.com/questions/33704130/react-native-android-fetch-failing-on-connection-to-local-api/43277765#43277765

adb reverse tcp:5001 tcp:5001

or change ipfs config API to 0.0.0.0 connect using local IP

"Addresses": {
    "API": "/ip4/0.0.0.0/tcp/5001",
},

Test code

delete global.URL;
delete global.URLSearchParams;
import {polyfillGlobal} from 'react-native/Libraries/Utilities/PolyfillFunctions';

polyfillGlobal('URLSearchParams', () => require('whatwg-url').URLSearchParams);
polyfillGlobal('URL', () => require('whatwg-url').URL);

import './shim';

import {AppRegistry} from 'react-native';
import App from './App';
import {name as appName} from './app.json';

AppRegistry.registerComponent(appName, () => App);
console.log('yo');

const ipfsClient = require('ipfs-http-client');
const ipfs = ipfsClient('http://localhost:5001');

const test = async () => {
  try {
    console.log(await ipfs.id());
  } catch (error) {
    console.log(error);
  }
};

test();

[pcowgill]

@hugomrdias Thanks for looking into this! Have you tried with any other IPFS methods, like addFromUrl? ipfs.id() has been working all along.

[pcowgill]

The for await ... of syntax doesn't fail at compile time, but rather at runtime. So only methods like ipfs.addfromURL (and most other IPFS methods) that hit such lines in the ipfs codebase can be used. id doesn't appear to use an async iterator:
https://github.com/ipfs/js-ipfs-http-client/blob/master/src/id.js

If you modify the test function to:

const test = async () => {
  try {
    console.log(await ipfs.addFromURL("http://example.com/"));
  } catch (error) {
    console.log(error);
  }
};

you should be able to reproduce the async iterator error message.

[TypeError: _iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"] is not a function. (In '_iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"]()', '_iterator[typeof Symbol === "function" ? typeof Symbol === "function" ? Symbol.iterator : "@@iterator" : "@@iterator"]' is undefined)]

[pcowgill]

@hugomrdias and I chatted about trying to import the browser bundle for ipfs-http-client like this:

const ipfsClient = require("ipfs-http-client/dist/index.js");

and that does appear to get past the async iterator error!

The error message with the browser bundle is

[TypeError: undefined is not an object (evaluating 'body[typeof Symbol === "function" ? Symbol.asyncIterator : "@@asyncIterator"]')]

...which is different than the one before.

That new error occurs because the fetch available in a React Native env and the GitHub pollyfill for fetch both don't implement res.body as a getter for a ReadableStream.

I added a suggestion about error handling for the case when body is undefined in another issue here.

So if we can get fetch-readablestream working with a suitable polyfill for TextEncoder, we may have a viable solution here.

[pcowgill]

Thanks for transferring this issue. I think that's a great way to handle open issues now that most of the packages are moving into the js-ipfs monorepo.

For those following this issue for the state of using ipfs in React Native, I posted an update on the state of progress on that here:
#2813 (comment)

[hugomrdias]

Closing this one in favor of #2813

Thanks you all 🙏