Enable PubSub in the Browser

[daviddias]

Following: #493 (comment)

PubSub will be unavailable as a browser API until https://github.com/ipfs/http-api-spec/issues/116 happens.

[daviddias]

Update on this thread:

We currently have PubSub disabled on the browser as a way to prevent misunderstanding how the cancellation of the API works. Unfortunately, due to what is noted in -- #493 (comment) -- we simply can't cancel requests as go-ipfs expects in the browser. However, it seems to me that removing support for it entirely is not the optimal solution, as for a lot of use cases, only a few amount of PubSub channels will be open from a client, keeping the maintenance of those at bare minimum.

Proposal: enable PubSub and be explicit that cancels aren't actually being canceled, just silenced.

[mitar]

I think we should enable PubSub on client and document cancelation issue.

[Morantron]

I've found and ultra-hacky workaround to cancel fetch requests without XHR here ( https://github.com/Morantron/poor-mans-cancelable-fetch ).

The whole idea is to make requests in a separate WebWorker, and sending a signal to the worker to terminate itself, which also aborts the request

Eventually fetch requests will be able to be aborted ( whatwg/fetch#447 ), but the final API hasn't been decided yet.

I know it's super hackish, but if it works ™️ , it can serve as workaround until fetch requests are abortable.

[mitar]

Great hack! I like it!

[daviddias]

Oh wow! @Morantron wanna submit a PR with that feature? Can you make sure to add a check to see if the browser has WebWorker support?

[Morantron]

@diasdavid Sure! I'm gonna see if I can turn the hack into some kind of FetchController polyfill, and then make the needed changes in js-ipfs-api.

Looks like IE11 does not support fetch, so I guess PubSub won't work given that XHR does not play well with http streams.

[jwahyoung]

Hey, all, hate to ask a potentially obvious (and stupid) question.....but why not just use standard XHR which is cancellable? Believe me, I support the fetch standard and I love what it provides, but if we aren't using a service worker to intercept requests or anything like that, the main difference comes down to error handling.

XHR requests are cancellable and would solve the issue until the fetch standard includes true cancellable requests. The other alternative to this to cancel requests on the client side would be to run a service worker, and have the service worker respond to the request with a 400 Cancelled or something like that when a cancellation is requested. (Basically, you're manually sending a response to an initiated HTTP request to terminate it early.)

[daviddias]

@jedd-ahyoung xhr is not streamable, pubsub/subscribe is a stream. We need to use something like fetch or make this endpoint available Websockets.

[mitar]

Yes, why not websockets? Or dataframes from WebRTC?

[daviddias]

This is not a decision at the level of this client library, it is rather at the http-api spec level. Moving to a complete RPC API where WebSockets is used as a transport has been proposed: https://github.com/ipfs/http-api-spec/issues/116

[daviddias]

Update

The decision on this one is to either do:

a) Enable PubSub and fake subscriptions - #518 (comment)
b) Use WebWorkers to cancel subscriptions - #518 (comment)

Either is fine as a first iteration.

[nunofmn]

It's important to notice that Edge 16 and Firefox 57 now can cancel fetch requests by using AbortController and AbortSignal. But Chrome still doesn't support this API...

[victorb]

Seems Chrome has done some work on it (AbortController), last patch from February 5th this year https://bugs.chromium.org/p/chromium/issues/detail?id=750599

WebKit, no status yet: https://bugs.webkit.org/show_bug.cgi?id=174980

[victorb]

According to https://developer.mozilla.org/en-US/docs/Web/API/AbortController#Browser_compatibility most browsers now work with AbortController (except IE, shocker)

[shunkino]

Hi, I'm new to the ipfs community and impressed by the all works done so far.
I find the PubSub functionality very interesting and wish it be working in browsers too.

Here are my findings. Maybe I'm totally off the track, please let me know if anything is wrong.
In my understanding, the problem here is that we can't cancel requests in browser by the code bellow.
https://github.com/ipfs/js-ipfs-api/blob/master/src/pubsub.js#L94
But, as @nunofmn suggest if we can use AbortController, we can cancel the fetch.

Since js-ipfs-api uses stream-http as http, aborting function should be implemented using it's API. When looking at the code of the stream-http library, abort() to the client request is implemented at v2.7.2. Thus, required version of stream-http by package.json, v2.8.3, should already have this API. Looks like just calling abort() to the request object is enough(=same as node).

So, isn't this mean we can use same codebase as node to get this "cancellable fetch" working in the browser? I'd love to test my idea but I'm not sure how to do so yet, so just posting comment. Hope this helps.

[alanshaw]

This sounds reasonable to me @shunkino - I'd ❤️ to see a proposal PR 😄

[shunkino]

--- UPDATED ---
Find out that my code works on Firefox (Quantum 61.0.2 on MacOS)...
Research and status on the problem mentioned below were about Google Chrome (version 69.0.3497.81).

Still not sure why the test on Chrome failed...
---

I've made a change to the /src/pubsub.js according to my findings above. But something is still wrong, and I was debugging that these few days. Unfortunately, I'm stuck now and need help...

I modified the pubsub.js as below. It's basically adding condition to detect whether the browser have the AbortController functionality.
https://gist.github.com/shunkino/baf26def13aca1ae6949d3ff1e0f939b

Modified version seems to work properly inside browser, but failing the test in interface-ipfs-core.

    .pubsub.publish
      ✓ should error on string messags
      ✓ should publish message from buffer (42ms)
      ✓ should publish 10 times within time limit (251ms)
    .pubsub.subscribe
      single node
        1) should subscribe to one topic
        2) should subscribe to one topic (promised)
        3) should subscribe to one topic with options
        4) should subscribe to one topic with options (promised)
        5) should subscribe to topic multiple times with different handlers
        6) should allow discover option to be passed
      multiple connected nodes
        ✓ should receive messages from a different node (77ms)
        7) "after each" hook for "should receive messages from a different node"
    .pubsub.unsubscribe
      8) should subscribe and unsubscribe 10 times
    .pubsub.peers
      ✓ should not error when not subscribed to a topic (51ms)
      9) should not return extra peers
      10) should return peers for a topic - one peer
      11) should return peers for a topic - multiple peers
    .pubsub.ls
      ✓ should return an empty list when no topics are subscribed (14ms)
      12) should return a list with 1 subscribed topic
      13) should return a list with 3 subscribed topics

To identify why the test is failing, I created the small app based on the example.
(I deployed the test app to /ipfs/Qme1GWMpHuErWGcGXbF7NsDBNKB5yMsT8sSeRP3s8wopQ1/ 😜 You can debug it inside browser. Click testFunction button to test.)

Looks like callback here is not called.

Furthermore, Inside the unsubscribe(), callback given to the eos() was not called. I don't know anything about end-of-stream and once library it depends. Also it was difficult to dig deeper so I'm stack here now.

Suspicious factors:

• Event emitting from stream-http, readable-stream
• Function wrapper in eos(), once library

Same function worked perfectly in node.js. Any comment and advice are welcome, thank you.

[klueq]

Is there a way to enable pubsub locally?

[whyrusleeping]

any updates here?

[daviddias]

@shunkino did you happen to make a PR with your progress?