Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: TextDecoder is not a constructor when handling multiaddresses in browser #909

Closed
da-kami opened this issue Apr 14, 2021 · 25 comments
Assignees
Labels

Comments

@da-kami
Copy link

da-kami commented Apr 14, 2021

  • Version: libp2p 0.30.12
  • Platform:
    • Firefox 87.0
    • Chrome `89.0.4389.114'
    • Platform: Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64 macOs Big Sur 11.2.3
  • Subsystem:

Type:

Bug

Severity:

Critical

Description:

When handling any kind of code that depends on https://github.com/multiformats/js-multibase the application crashes in the Browser with:

TypeError: TextDecoder is not a constructor
./node_modules/multiaddr/node_modules/multibase/src/util.js
node_modules/multiaddr/node_modules/multibase/src/util.js:6

  3 | // @ts-ignore
  4 | const { TextEncoder, TextDecoder } = require('web-encoding')
  5 | 
> 6 | const textDecoder = new TextDecoder()
  7 | /**
  8 |  * @param {ArrayBufferView|ArrayBuffer} bytes
  9 |  * @returns {string}

Test with Firefox and Chrome.

Using require for importing resulted in the same error.
Using multiaddr function like shown in the chat example https://github.com/libp2p/js-libp2p/blob/master/examples/chat/src/dialer.js#L33 resulted in the same error.

The problem seems to be related to how imports are handled in multibase.

Steps to reproduce the error:

I ran into this problem when building a PoC trying to connect to a rust-libp2p daemon with js-libp2p from the browser:
https://github.com/da-kami/js-libp2p-quote-poc

Note: I also did a pure Javascript version which failed with the same error.

Are we missing something for working with Mutiaddress in the browser or is this a bug?

@da-kami da-kami changed the title 'TypeError: TextDecoder is not a constructor' when handling multiaddresses in browser TypeError: TextDecoder is not a constructor when handling multiaddresses in browser Apr 14, 2021
@vasco-santos
Copy link
Member

vasco-santos commented Apr 14, 2021

Hello @da-kami
We are working on updating all the dependencies to not use web-encoding, since we really not need it #904 . Meanwhile, for this version we are blocked on Gozala/web-encoding#21

@vasco-santos vasco-santos added the status/in-progress In progress label Apr 14, 2021
@da-kami
Copy link
Author

da-kami commented Apr 15, 2021

Hello @da-kami
We are working on updating all the dependencies to not use web-encoding, since we really not need it #904 . Meanwhile, for this version we are blocked on Gozala/web-encoding#21

Thanks for the quick reply! I subscribed to #904 but after a quick look on the dependencies, I am not sure #904 will resolve not depending on web-encoding anymore:

As far as I could see #904 updates multiaddr to 9.0.1 which depends on multibase 0.4.2, but that still depends on web-encoding:

https://github.com/multiformats/js-multiaddr/blob/v9.0.1/package.json#L41
https://github.com/multiformats/js-multibase/blob/v4.0.2/package.json#L35

@da-kami
Copy link
Author

da-kami commented Apr 15, 2021

As a temporary quickfix we use yarn's resolutions feature to upgrade multibase and uint8arrays to the latest version which removes web-encoding:

https://github.com/da-kami/js-libp2p-quote-poc/blob/master/package.json#L23-L26

@vasco-santos
Copy link
Member

Thanks for the quick reply! I subscribed to #904 but after a quick look on the dependencies, I am not sure #904 will resolve not depending on web-encoding anymore:

As far as I could see #904 updates multiaddr to 9.0.1 which depends on multibase 0.4.2, but that still depends on web-encoding:

multibase@4.0.3 was already shipped removing web-encoding. You can already install libp2p@0.31.0-rc.0 and try it out

@vasco-santos vasco-santos self-assigned this Apr 15, 2021
@da-kami
Copy link
Author

da-kami commented Apr 16, 2021

Thanks for the quick reply! I subscribed to #904 but after a quick look on the dependencies, I am not sure #904 will resolve not depending on web-encoding anymore:

As far as I could see #904 updates multiaddr to 9.0.1 which depends on multibase 0.4.2, but that still depends on web-encoding:

multibase@4.0.3 was already shipped removing web-encoding. You can already install libp2p@0.31.0-rc.0 and try it out

Thanks! I gave 0.31.0-rc.0 it a try earlier and ran into some other issues. Will see to report them in detail once I find time to dig through.

@D4nte
Copy link
Contributor

D4nte commented Apr 16, 2021

Yes, facing an issue too when trying the release candidate:

../node_modules/libp2p/src/upgrader.js 315:14
Module parse failed: Cannot use keyword 'await' outside an async function (315:14)
File was processed with these loaders:
 * ./node_modules/babel-loader/lib/index.js
You may need an additional loader to handle the result of these loaders.
|             stream,
|             protocol
>           } = await mss.select(protocols);
|           if (this.metrics) this.metrics.trackStream({
|             stream,

@vasco-santos
Copy link
Member

The reported error is misleading as that await is wrapped by an async function: https://github.com/libp2p/js-libp2p/blob/master/src/upgrader.js#L257

I just tried our libp2p-in-the-browser example, which uses parcel+babel and does not have this issue. Per the original post, I see that you are using CRA for your project and this might be related to how dependencies are bundled.

We are checking the RC with js-ipfs that has an example using CRA as well, I will keep you posted if this leads to the same issue

@vasco-santos
Copy link
Member

@D4nte we found the bug for CRA. You can try the latest rc: 0.31.0-rc.3.

This was the fix. Somehow CRA with inline types comments understands an async function as a regular function 🤷🏼

FYI, we will be shipping the final release soon

@D4nte
Copy link
Contributor

D4nte commented Apr 21, 2021

@D4nte we found the bug for CRA. You can try the latest rc: 0.31.0-rc.3.

This was the fix. Somehow CRA with inline types comments understands an async function as a regular function 🤷🏼

FYI, we will be shipping the final release soon

@vasco-santos great thanks! I actually quickly tried rc.3 yesterday and it looked good but I had to log off so did not have time to double check. Will report once I do more testing.

@D4nte
Copy link
Contributor

D4nte commented Apr 21, 2021

Ok I can confirm the error is gone with rc.3 :)

@D4nte
Copy link
Contributor

D4nte commented Apr 27, 2021

I am now getting ReferenceError: TextDecoder is not defined when:

  • Using libp2p:0.31.0-rc.3 and
  • using react-scripts test (jest)

Is that a known issue?

@vasco-santos
Copy link
Member

Can you let me know your libp2p dependencies versions? Also do npm ls multibase

@D4nte
Copy link
Contributor

D4nte commented Apr 28, 2021

In the react app:

web-chat@0.1.0 /home/froyer/src/status-im/js-waku/web-chat
└─┬ peer-id@0.14.8
  ├─┬ cids@1.1.6
  │ └── multibase@4.0.4
  ├─┬ libp2p-crypto@0.19.4
  │ └── multibase@4.0.4 deduped
  ├─┬ multihashes@4.0.2
  │ └── multibase@4.0.4 deduped
  └─┬ uint8arrays@2.1.5
    └── multibase@4.0.4 deduped

In the js-waku library that is used by the react app and brings in libp2p:

js-waku@1.0.0 /home/froyer/src/status-im/js-waku
├─┬ libp2p-gossipsub@0.7.0
│ ├─┬ libp2p-interfaces@0.7.2
│ │ ├─┬ libp2p-crypto@0.18.0
│ │ │ └── multibase@3.1.2 deduped
│ │ ├─┬ multiaddr@8.1.2
│ │ │ └── multibase@3.1.2 deduped
│ │ ├── multibase@3.1.2
│ │ └─┬ multihashes@3.1.2
│ │   ├── multibase@3.1.2 deduped
│ │   └─┬ uint8arrays@2.1.3
│ │     └── multibase@4.0.2
│ ├─┬ peer-id@0.14.3
│ │ ├─┬ multihashes@3.1.2
│ │ │ └── multibase@3.1.2 deduped
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ uint8arrays@1.1.0
│   └── multibase@3.1.2 deduped
├─┬ libp2p-noise@2.0.5
│ ├─┬ libp2p-crypto@0.19.0
│ │ └── multibase@3.1.2 deduped
│ └─┬ uint8arrays@2.1.3
│   └── multibase@4.0.2
├─┬ libp2p-secio@0.13.1
│ ├─┬ libp2p-crypto@0.18.0
│ │ └── multibase@3.1.2 deduped
│ ├─┬ multiaddr@8.1.2
│ │ └── multibase@3.1.2 deduped
│ └─┬ multihashing-async@2.1.2
│   ├─┬ multihashes@4.0.2
│   │ ├── multibase@4.0.2
│   │ └─┬ uint8arrays@2.1.3
│   │   └── multibase@4.0.2 deduped
│   └─┬ uint8arrays@2.1.3
│     └── multibase@4.0.2
├─┬ libp2p-tcp@0.15.3
│ ├─┬ libp2p-utils@0.2.3
│ │ └─┬ multiaddr@8.1.2
│ │   └── multibase@3.1.2 deduped
│ ├─┬ mafmt@8.0.4
│ │ └─┬ multiaddr@8.1.2
│ │   └── multibase@3.1.2 deduped
│ └─┬ multiaddr@8.1.2
│   └── multibase@3.1.2 deduped
├─┬ libp2p-websockets@0.15.5
│ └─┬ multiaddr-to-uri@6.0.0
│   └─┬ multiaddr@8.1.2
│     └── multibase@3.1.2 deduped
├─┬ libp2p@0.31.0-rc.3
│ ├─┬ cids@1.1.6
│ │ ├── multibase@4.0.2
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2 deduped
│ ├─┬ libp2p-interfaces@0.10.1
│ │ ├── multibase@4.0.4
│ │ └─┬ uint8arrays@2.1.5
│ │   └── multibase@4.0.4 deduped
│ ├─┬ multicodec@3.0.1
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ multistream-select@2.0.0
│   └─┬ uint8arrays@2.1.5
│     └── multibase@4.0.4
└─┬ multiaddr@9.0.1
  ├── multibase@4.0.4

@vasco-santos
Copy link
Member

This error comes from outdated multibase. You can already update everything in there except gossipsub (still needs ChainSafe/js-libp2p-gossipsub#153 merged)

You can see on #912 what modules versions you should have. You can update everything except gossip, including new libp2p rc and you can experiment with gossipsub branch referenced for full functionality.

Sorry for the delay, we are working on guaranteeing everything is working properly with js-ipfs before shipping a final release

@D4nte
Copy link
Contributor

D4nte commented Apr 29, 2021

This error comes from outdated multibase. You can already update everything in there except gossipsub (still needs ChainSafe/js-libp2p-gossipsub#153 merged)

You can see on #912 what modules versions you should have. You can update everything except gossip, including new libp2p rc and you can experiment with gossipsub branch referenced for full functionality.

Looks like gossipsub is released too. However, it seems that peer-id is still bringing an old multibase version:

▶ npm ls multibase   
web-chat@0.1.0 /home/froyer/src/status-im/js-waku/web-chat
└─┬ peer-id@0.14.8
  ├─┬ cids@1.1.6
  │ └── multibase@4.0.4
  ├─┬ libp2p-crypto@0.19.4
  │ └── multibase@4.0.4 deduped
  ├─┬ multihashes@4.0.2
  │ └── multibase@4.0.4 deduped
  └─┬ uint8arrays@2.1.5
    └── multibase@4.0.4 deduped

Sorry for the delay, we are working on guaranteeing everything is working properly with js-ipfs before shipping a final release

No worries, thanks for the quick turn around!

@D4nte
Copy link
Contributor

D4nte commented Apr 29, 2021

Wait, which multibase version is the outdated one?

▶ npm ls multibase
js-waku@1.0.0 /home/froyer/src/status-im/js-waku
├─┬ libp2p-gossipsub@0.9.0
│ ├─┬ libp2p-interfaces@0.10.3
│ │ ├── multibase@4.0.4
│ │ ├─┬ multihashes@4.0.2
│ │ │ ├── multibase@4.0.2
│ │ │ └─┬ uint8arrays@2.1.3
│ │ │   └── multibase@4.0.2 deduped
│ │ └─┬ uint8arrays@2.1.5
│ │   └── multibase@4.0.4 deduped
│ ├─┬ peer-id@0.14.3
│ │ ├─┬ multihashes@3.1.2
│ │ │ └── multibase@3.1.2 deduped
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ uint8arrays@2.1.5
│   └── multibase@4.0.4
├─┬ libp2p-noise@3.0.0
│ ├─┬ libp2p-crypto@0.19.0
│ │ └── multibase@3.1.2
│ └─┬ uint8arrays@2.1.3
│   └── multibase@4.0.2
├─┬ libp2p-secio@0.13.1
│ ├─┬ libp2p-crypto@0.18.0
│ │ └── multibase@3.1.2 deduped
│ ├─┬ multiaddr@8.1.2
│ │ └── multibase@3.1.2 deduped
│ ├─┬ multihashing-async@2.1.2
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ uint8arrays@1.1.0
│   └── multibase@3.1.2 deduped
├─┬ libp2p@0.31.0
│ ├─┬ cids@1.1.6
│ │ ├── multibase@4.0.2
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2 deduped
│ ├─┬ interface-datastore@4.0.0
│ │ └─┬ uint8arrays@2.1.5
│ │   └── multibase@4.0.4
│ ├─┬ multicodec@3.0.1
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ multistream-select@2.0.0
│   └─┬ uint8arrays@2.1.5
│     └── multibase@4.0.4
└─┬ multiaddr@9.0.1
  ├── multibase@4.0.4
  └─┬ uint8arrays@2.1.5
    └── multibase@4.0.4 deduped

@D4nte
Copy link
Contributor

D4nte commented Apr 29, 2021

I having the issue with multibase 4.0.4, the latest version. This is not expected, right?

 FAIL  src/command.test.ts
  ● Test suite failed to run

    ReferenceError: TextDecoder is not defined

    > 1 | import { multiaddr } from 'multiaddr';
        | ^
      2 | import PeerId from 'peer-id';
      3 | import Waku from '../../build/main/lib/waku';
      4 |

      at Object.<anonymous> (../node_modules/multibase/src/util.js:3:21) <-- This is multibase 4.0.4
      at Object.<anonymous> (../node_modules/multibase/src/base.js:3:24)
      at Object.<anonymous> (../node_modules/multibase/src/constants.js:4:14)
      at Object.<anonymous> (../node_modules/multibase/src/index.js:7:19)
      at Object.<anonymous> (../node_modules/uint8arrays/to-string.js:3:32)
      at Object.<anonymous> (../node_modules/multiaddr/src/ip.js:4:28)
      at Object.<anonymous> (../node_modules/multiaddr/src/convert.js:3:12)
      at Object.<anonymous> (../node_modules/multiaddr/src/codec.js:3:17)
      at Object.<anonymous> (../node_modules/multiaddr/src/index.js:3:15)
      at Object.<anonymous> (src/command.ts:1:1)
      at Object.<anonymous> (src/command.test.ts:1:1)

@vasco-santos
Copy link
Member

vasco-santos commented Apr 29, 2021

Wait, which multibase version is the outdated one?

everything in multibase should be 4+

I having the issue with multibase 4.0.4, the latest version. This is not expected, right?

The error now looks different then before. What node version are you using? You should be using 14 or 15 by now?


A couple of other questions per latest dependency tree:

  • any reason to use secio? We deprecated it one year ago and it should not be used.
  • why do you have the PeerId version fixed? It should be in the latest as well

I think you need to reinstall the dependencies without your package lock, you have several versions that are not in the latest ones

If you cannot solve this issue with the latest issues, please let me know if I can try to replicate this locally?

@D4nte
Copy link
Contributor

D4nte commented Apr 30, 2021

Thanks. So there are still some multibase 3 coming from libp2p-noise for example:

▶ npm ls multibase
js-waku@1.0.0 /home/froyer/src/status-im/js-waku
├─┬ libp2p-gossipsub@0.9.0
│ ├─┬ libp2p-interfaces@0.10.3
│ │ ├── multibase@4.0.4
│ │ ├─┬ multihashes@4.0.2
│ │ │ ├── multibase@4.0.2
│ │ │ └─┬ uint8arrays@2.1.3
│ │ │   └── multibase@4.0.2 deduped
│ │ └─┬ uint8arrays@2.1.5
│ │   └── multibase@4.0.4 deduped
│ ├─┬ peer-id@0.14.3
│ │ ├─┬ multihashes@3.1.2
│ │ │ └── multibase@3.1.2 deduped
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ uint8arrays@2.1.5
│   └── multibase@4.0.4
├─┬ libp2p-noise@3.0.0
│ ├─┬ libp2p-crypto@0.19.0
│ │ ├── multibase@3.1.2
│ │ └─┬ uint8arrays@1.1.0
│ │   └── multibase@3.1.2 deduped
│ └─┬ uint8arrays@2.1.3
│   └── multibase@4.0.2
├─┬ libp2p@0.31.0
│ ├─┬ cids@1.1.6
│ │ ├── multibase@4.0.2
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2 deduped
│ ├─┬ interface-datastore@4.0.0
│ │ └─┬ uint8arrays@2.1.5
│ │   └── multibase@4.0.4
│ ├─┬ multicodec@3.0.1
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ ├─┬ multihashing-async@2.1.2
│ │ └─┬ uint8arrays@2.1.3
│ │   └── multibase@4.0.2
│ └─┬ multistream-select@2.0.0
│   └─┬ uint8arrays@2.1.5
│     └── multibase@4.0.4
└─┬ multiaddr@9.0.1
  ├── multibase@4.0.4
  └─┬ uint8arrays@2.1.5
    └── multibase@4.0.4 deduped

The error now looks different then before. What node version are you using? You should be using 14 or 15 by now?

The errors come from using react-scripts@4.0.3

  • any reason to use secio? We deprecated it one year ago and it should not be used.

Indeed, it was an unused dependency, now removed. Thanks!

  • why do you have the PeerId version fixed? It should be in the latest as well

Indeed, now removed.

@vasco-santos
Copy link
Member

You need to have some kind of lock file @D4nte

When you install libp2p-noise for example, it will install the latest of the 0.19.x series of libp2p-crypto, which will be 0.19.4 and will come with multibase 4. Same for gossipsub and its PeerId

@D4nte
Copy link
Contributor

D4nte commented Apr 30, 2021

You need to have some kind of lock file @D4nte

When you install libp2p-noise for example, it will install the latest of the 0.19.x series of libp2p-crypto, which will be 0.19.4 and will come with multibase 4. Same for gossipsub and its PeerId

Yes I have a package-lock.json file but thank you for the tip, I'll play around with that!

@vasco-santos
Copy link
Member

This should be fixed with the new libp2p release and all the latest libp2p modules. Feel free to re-open if this is still a problem for you

@D4nte
Copy link
Contributor

D4nte commented Jun 22, 2021

I am having the related issue again:

  ● Test suite failed to run

    ReferenceError: TextDecoder is not defined

    > 1 | import Libp2p, { Connection, Libp2pModules, Libp2pOptions } from 'libp2p';
        | ^
      2 | import Mplex from 'libp2p-mplex';
      3 | import { bytes } from 'libp2p-noise/dist/src/@types/basic';
      4 | import { Noise } from 'libp2p-noise/dist/src/noise';

      at Object.<anonymous> (../../node_modules/multibase/src/util.js:3:21)
      at Object.<anonymous> (../../node_modules/multibase/src/base.js:3:24)
      at Object.<anonymous> (../../node_modules/multibase/src/constants.js:4:14)
      at Object.<anonymous> (../../node_modules/multibase/src/index.js:7:19)
      at Object.<anonymous> (../../node_modules/multihashes/src/index.js:6:19)
      at Object.<anonymous> (../../node_modules/peer-id/src/index.js:7:12)
      at Object.<anonymous> (../../node_modules/libp2p/src/index.js:10:16)
      at Object.<anonymous> (../../src/lib/waku.ts:1:1)
      at Object.<anonymous> (../../src/index.ts:3:1)
      at Object.<anonymous> (src/App.tsx:5:1)
      at Object.<anonymous> (src/App.test.tsx:3:1)

When running npm run test:unit from an example folder examples/eth-dm.
The eth-dm project does not have multibase installed:

js-waku/examples/eth-dm  eth-dm ✔                                                           17h35m  
▶ npm ls multibase
eth-dm@0.1.0 /home/froyer/src/status-im/js-waku/examples/eth-dm
└── (empty)

And the parent project has latest multibase only:

src/status-im/js-waku  eth-dm ✔                                                             17h36m  
▶ npm ls multibase
js-waku@0.7.0 /home/froyer/src/status-im/js-waku
├─┬ libp2p-gossipsub@0.10.0
│ ├─┬ libp2p-interfaces@0.11.0
│ │ ├── multibase@4.0.4 deduped
│ │ └─┬ multihashes@4.0.2
│ │   └── multibase@4.0.4 deduped
│ └─┬ uint8arrays@2.1.5
│   └── multibase@4.0.4 deduped
├─┬ libp2p-noise@3.0.0
│ └─┬ libp2p-crypto@0.19.4
│   └── multibase@4.0.4 deduped
├─┬ libp2p@0.31.7
│ ├─┬ cids@1.1.7
│ │ └── multibase@4.0.4 deduped
│ └─┬ libp2p-interfaces@0.10.4
│   └── multibase@4.0.4 deduped
└─┬ multiaddr@9.0.1
  └── multibase@4.0.4

and the multibase line that returns the error is indeed 4.0.4:

js-waku/examples/eth-dm  eth-dm ✔                                                           17h37m  
▶ cat ../../node_modules/multibase/package.json | jq .version
"4.0.4"

The code can be found at waku-org/js-waku#212

What am I missing?

@vasco-santos
Copy link
Member

Is this still a problem? The PR was merged

From an initial observation, the error seems different, probably using the latest https://github.com/multiformats/js-multibase/blob/master/src/util.js#L3 . So what environment is this running?

@D4nte
Copy link
Contributor

D4nte commented Jun 23, 2021

It's running in react-scripts which I assume is the source of the problem.
Apparently this error can appear when using older node versions but I use node 16.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

No branches or pull requests

3 participants