Filtering on brushend

[lastlegion]

This is basically for cases where filtering is happening asynchronously(via AJAX). Right now the filtering happens as the user drags the brush. Now if the filtering is computationally expensive, or say, happening remotely, this causes a great lag, often not allowing the brushing to happen smoothly.
With filtering on brush end, the filtering happens only when the brush is set.
I found a workaround, by removing the filtering code from brushing() function, and adding it to brushend()

[gordonwoodhull]

Yeah there should definitely be an option.

The weird thing is that right now I think dc.throttle unintentionally does what you're requesting, but with a lag, because it's actually debouncing not throttling - see this discussion:
https://groups.google.com/forum/m/#!topic/dc-js-user-group/3AlQht_8Eqs

I just want to point that out as a related problem.

[tehsenaus]

+1

[dubejf]

My workaround looks like this:

var b = chart.brush();
b.on('brushstart.custom', function() {
    chart.userIsBrushing = true;
});
b.on('brushend.custom', function() {
    chart.userIsBrushing = false;
    // update chart now... (chart 'filtered' event might or might not be called after brushend)
});

chart.on('filtered', function(chart) {
    if ( chart.userIsBrushing ) return;
    //...
});

[esjewett]

Hey Gordon - I came here from http://stackoverflow.com/questions/33298742/prevent-filtering-on-dc-js-bar-chart-until-mouse-up-event

One suggestion I'd make is that it's possible to auto-debounce and in my experience it works quite well, though it does change the semantics of brush/filter application slightly from dc.constants.EVENT_DELAY=0 (where visual display and filter state are hard-linked). The general idea is that you set up a promise and execute your filter asynchronously, first setting a flag to indicate a filter is running for that dimension. If another filter comes in while the first filter is still running, queue it to run next. If yet another filter comes in, queue it instead (the queue should be at most 1 filter long - this works because only one filter is applied to a dimension at a time, so all we really care about is the last one). Once the current filter is done, check if another is queued and if there is, run it.

Here's an example (it uses Angular's $q instead of Promise) - https://github.com/humanitiesplusdesign/palladio/blob/master/src/js/services/filter.js#L19 And yes, I should really probably be using requestAnimationFrame. Will switch over to that eventually :-)

I believe this is better than using dc.constants.EVENT_DELAY because it debounces but will run the final filter immediately when the current filter is complete. So, maybe it could be described as an adaptive throttle/debounce.

I think this could probably be implemented in the default filter handler. Interested?

[gordonwoodhull]

@esjewett, yes I think that would be helpful and almost always better than the current behavior.

I briefly tried to test what happens when your browser thread is eaten up by calculation, and couldn't figure it out. You definitely don't get mousemove events while computation is happening, but do you get a bunch of them once control returns? Or just the final one.

Unfortunately we have to think about both the case where crossfilter is stomping on interactivity and the case where filtering is happening in a different thread/process.

[esjewett]

@gordonwoodhull I think the behavior is undefined and depends on the job scheduling approach the browser and JS engine use. In Palladio we have seen an increase in mouse interactivity while filters are being processed using setTimeout, but it's still laggy and it seems to be different in different browsers for CPU-intensive filtering on the main thread. Safari seems to slip frames of mouse event interactivity in even while filters are processing, Chrome appears to only process mouse events in between filter processing (e.g. before the next setTimeout), and Firefox seems to behave like Chrome but a bit slower. If filters are actually being processed in another thread (as in lcadata.info), then there is no lag on the main thread but a similar solution is needed to avoid a backup of filter processing on the background process.

But the basic assumption for processing on the main thread, I think, is that you get a mouse events in bunches after filter processing either way. Under the proposed solution I am (I believe) executing both the 1st and last dimension.filter event of each bunch, and the rest get discarded. We have to execute the first because we don't know if subsequent dimension.filters are coming or not, and we have to execute the last because that's the one we actually want. If we also allow for a small delay, we might be able to avoid executing the first filter at all, but at the cost of that small delay on all filter actions. I guess?

[jsheldrake]

Hi @esjewett,

Hope you don't mind me jumping into this thread. We are experiencing another issue which is related to my previous question (http://stackoverflow.com/questions/33298742/prevent-filtering-on-dc-js-bar-chart-until-mouse-up-event).

As I mentioned, we are triggering some post-processing after the chart is filtered

chart.on('postRender', function() {
    chart.select('.brush').on("mouseup", function() {
        //additional data processing function called here
    });
});

However we are finding that sometimes this function is returning an incorrect result on filtering (in fact it's returning the same result as before filtering). It seems as if the callback is being called BEFORE crossfilter has finished calculating. The issue seems to happen intermittently and unpredictably.

Does this assumption seem feasible?

What we really want is an event that fires only when ALL crossfilter dimensions and groups have finished updating (if such a thing exists). We would then use this event to trigger our additional calculations.

Can you suggest a way to deal with this? Thanks in advance.

[gordonwoodhull]

@jsheldrake, right, mouseup is what triggers filtering so you won't see results at that point. You probably want chart.on('filtered', ...)

https://github.com/dc-js/dc.js/blob/develop/web/docs/api-latest.md#basemixinon--basemixin

[gordonwoodhull]

@esjewett, would a timeout of zero get us past any queued messages? Anyway, yours sounds like a good approach and a real improvement.

[esjewett]

@gordonwoodhull I don't think so. It seems like all outstanding events get processed before the next setTimeout function gets called. We call it with no default timeout (the delay argument setTimeout defaults to 0) and this is what seems to happen, to me anyway.

[jsheldrake]

Thanks @gordonwoodhull. We started using 'postRender' instead of 'filtered' in an attempt to fix the laggy brush, but as you pointed out, it caused another problem.

Now experimenting with using high values (~1000ms) for EVENT_DELAY which seems to be helping

[nreese]

I am using the 2.0.0-beta.19 and am still experiencing lag when brushing. It would be useful to have the ability to completely turn off filtering until the brushend event is fired.

How would you envision this implemented at the API level? Maybe a new property to specify when to apply the brush with a default value of BRUSHMOVE and setable to BRUSHEND?

For example,

dc.barChart()
  .brushOn(true)
  .brushApply(dc.consts.brushend)

[gordonwoodhull]

I think the original request is fulfilled by #898, although there are a lot of other useful ideas here.

[guilhermecgs]

Just to be clear: is this feature already merged into a release or a master branch?

[gordonwoodhull]

Nope, not yet. If you want to help this along, you can try out PR #898 and leave a review of the functionality and or code on that ticket. Thank you!