diff --git a/doc/user_guide/index.rst b/doc/user_guide/index.rst index 923dc0dfb3..b72dc3feec 100644 --- a/doc/user_guide/index.rst +++ b/doc/user_guide/index.rst @@ -105,6 +105,9 @@ These guides provide detail about specific additional features in HoloViews: `Working with renderers and plots `_ Using the ``Renderer`` and ``Plot`` classes for access to the plotting machinery. +`Using linked brushing to cross-filter complex datasets `_ + Explains how to use the `link_selections` helper to cross-filter multiple elements. + `Using Annotators to edit and label data `_ Explains how to use the `annotate` helper to edit and annotate elements with the help of drawing tools and editable tables. @@ -148,6 +151,7 @@ These guides provide detail about specific additional features in HoloViews: Plotting with matplotlib Plotting with plotly Working with Plot and Renderers + Linked Brushing Annotators Exporting and Archiving Continuous Coordinates diff --git a/examples/reference/elements/plotly/BoxWhiskers.ipynb b/examples/reference/elements/plotly/BoxWhisker.ipynb similarity index 100% rename from examples/reference/elements/plotly/BoxWhiskers.ipynb rename to examples/reference/elements/plotly/BoxWhisker.ipynb diff --git a/examples/user_guide/Linked_Brushing.ipynb b/examples/user_guide/Linked_Brushing.ipynb new file mode 100644 index 0000000000..10e38a27ce --- /dev/null +++ b/examples/user_guide/Linked_Brushing.ipynb @@ -0,0 +1,531 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import holoviews as hv\n", + "\n", + "from holoviews.util.transform import dim\n", + "from holoviews.selection import link_selections\n", + "from holoviews.operation import gridmatrix\n", + "from holoviews.operation.element import histogram\n", + "from holoviews import opts\n", + "\n", + "hv.extension('bokeh', 'plotly', width=100)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### JavaScript-based linked brushing\n", + "\n", + "Datasets very often have more dimensions than can be shown in a single plot, which is why HoloViews offers so many ways to show the data from each of these dimensions at once (via layouts, overlays, grids, holomaps, etc.). However, even once the data has been displayed, it can be difficult to relate data points between the various plots that are laid out together. For instance, \"is the outlier I can see in this x,y plot the same datapoint that stands out in this w,z plot\"? \"Are the datapoints with high x values in this plot also the ones with high w values in this other plot?\" Since points are not usually visibly connected between plots, answering such questions can be difficult and tedious, making it difficult to understand multidimensional datasets. [Linked brushing](https://infovis-wiki.net/wiki/Linking_and_Brushing) (also called \"brushing and linking\") offers an easy way to understand how data points and groups of them relate across different plots. Here \"brushing\" refers to selecting data points or ranges in one plot, with \"linking\" then highlighting those same points or ranges in other plots derived from the same data.\n", + "\n", + "As an example, consider the standard \"autompg\" dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from bokeh.sampledata.autompg import autompg\n", + "autompg" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This dataset contains specifications for 392 different types of car models from 1970 to 1982. Each car model represents a particular point in a nine-dimensional space, with a certain **mpg**, **cyl**, **displ**, **hp**, **weight**, **accel**, **yr**, **origin**, and **name**. We can use a [gridmatrix](http://holoviews.org/gallery/demos/bokeh/iris_density_grid.html) to see how each numeric dimension relates to the others:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "autompg_ds = hv.Dataset(autompg, ['yr', 'name', 'origin'])\n", + "\n", + "mopts = opts.Points(size=2, tools=['box_select','lasso_select'], active_tools=['box_select'])\n", + "\n", + "gridmatrix(autompg_ds, chart_type=hv.Points).opts(mopts)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "These plots show all sorts of interesting relationships already, such as that weight and horsepower are highly positively correlated (locate _weight_ along one axis and _hp_ along the other, and you can see that car models with high weight almost always have high horsepower and vice versa).\n", + "\n", + "What if we want to focus specifically on the subset of cars that have 4 cylinders (*cyl*)? You can do that by pre-filtering the dataframe in Python, but questions like that can be answered immediately using linked brushing, which is automatically supported by `gridmatrix` plots like this one. First, make sure the \"box select\" or \"lasso select\" tool is selected in the toolbar:\n", + "\n", + "\n", + "\n", + "Then pick one of the points plots labeled _cyl_ and use the selection tool to select all the values where _cyl_ is 4. All of those points in each plot should remain blue, while the points where _cyl_ is not 4 become more transparent: " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You should be able to see that 4-cylinder models have low displacement (*displ*), low horsepower (*hp*), and low weight, but tend to have higher fuel efficiency (*mpg*). Repeatedly selecting subsets of the data in this way can help you understand properties of a multidimensional dataset that may not be visible in the individual plots, without requiring coding and examining additional plots." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Python-based linked brushing\n", + "\n", + "The above example illustrates Bokeh's very useful automatic JavaScript-based [linked brushing](https://docs.bokeh.org/en/latest/docs/user_guide/interaction/linking.html#linked-brushing), which can be enabled for Bokeh plots sharing a common data source (as in the `gridmatrix` call) by simply adding a selection tool. However, this approach offers only a single type of selection, and is not available for Python-based data-processing pipelines such as those using [Datashader](15-Large_Data.ipynb).\n", + "\n", + "To get more power and flexibility (at the cost of requiring a Python server for deployment if you weren't already), HoloViews provides a Python-based implementation of linked brushing. HoloViews linked brushing lets you fully customize what elements are used and how linking behaves. Here, let's make a custom Layout displaying some Scatter plots for just a few of the available dimensions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "colors = hv.Cycle('Category10').values\n", + "dims = [\"cyl\", \"displ\", \"hp\", \"mpg\", \"weight\", \"yr\"]\n", + "\n", + "layout = hv.Layout([\n", + " hv.Points(autompg_ds, dims).opts(color=c)\n", + " for c, dims in zip(colors, [[d,'accel'] for d in dims])\n", + "])\n", + "\n", + "print(layout)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that we have a layout we can simply apply the `link_selections` operation to support linked brushing, automatically linking the selections across an arbitrary collection of plots that are derived from the same dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "link_selections(layout).opts(opts.Points(width=200, height=200)).cols(6)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The same `box_select` tool should now work as for the `gridmatrix` plot, but this time by calling back to Python. There are now many more options and capabilities available, as described below, but by default you can now also select additional regions in different elements, and the selected points will be those that match _all_ of the selections, so that you can precisely specify the data points of interest with constraints on _all_ dimensions at once. A bounding box will be shown for each selection, but only the overall selected points (across all selection dimensions) will be highlighted in each plot. You can use the reset tool to clear all the selections and start over." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Filter and selection modes\n", + "\n", + "Two parameters of `link_selections` control how the selections apply within a single element (the `selection_mode`) and across elements (the `cross_filter_mode`):\n", + "\n", + "* `selection_mode`: Determines how to combine successive selections on the same element, either `'overwrite'` (the default, allowing one selection per element), `'intersect'` (taking the intersection of all selections for that element), `'union'` (the combination of all selections for that element), or `'inverse'` (select all _but_ the selection region).\n", + "* `cross_filter_mode`: Determines how to combine selections across different elements, either `'overwrite'` (allows selecting on only a single element at a time) or `'intersect'` (the default, combining selections across all elements).\n", + "\n", + "To see how these work, we will create a number of views of the autompg dataset:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "w_accel_scatter = hv.Scatter(autompg_ds, 'weight', 'accel')\n", + "mpg_hist = histogram(autompg_ds, dimension='mpg', normed=False).opts(color=\"green\")\n", + "violin = hv.Violin(autompg_ds, [], 'hp')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will also capture an \"instance\" of the `link_selections` operation, which will allow us to access and set parameters on it even after we call it:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mpg_ls = link_selections.instance()\n", + "\n", + "mpg_ls(w_accel_scatter + mpg_hist + violin)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here you can select on both the Scatter plot and the Histogram. With these default settings, selecting on different elements computes the intersection of the two selections, allowing you to e.g. select only the points with high weight but mpg between 20 and 30. In the Scatter plot, the selected region will be shown as a rectangular bounding box, with the unselected points inside being transparent. On the histogram, data points selected on the histogram but not in other selections will be drawn in gray, data points not selected on either element will be transparent, and only those points that are selected in _both_ plots will be shown in the default blue color. The Violin plot does not itself allow selections, but it will update to show the distribution of the selected points, with the original distribution being lighter (more transparent) behind it for comparison. Here, selecting high weights and intermediate mpg gives points with a lower range of horsepower in the Violin plot.\n", + "\n", + "The way this all works is for each selection to be collected into a shared \"selection expression\" that is then applied by every linked plot after any change to a selection:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "mpg_ls.selection_expr" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "e.g. a box selection on the *weight*,*accel* scatter element might look like this:\n", + "\n", + "```\n", + "(((dim('weight') >= (3125.237)) & (dim('weight') <= (3724.860))) & (dim('accel') >= (13.383))) & (dim('accel') <= (19.678))\n", + "```\n", + "\n", + "Additional selections in other plots add to this list of filters if enabled, while additional selections within the same plot are combined with an operator that depends on the `selection_mode`.\n", + "\n", + "To better understand how to configure linked brushing, let's create a [Panel](https://panel.holoviz.org) that makes widgets for the parameters of the `linked_selection` operation and lets us explore their effect interactively. Play around with different `cross_filter_mode` and `selection_mode` settings and observe their effects (hitting reset when needed to get back to an unselected state):" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import panel as pn\n", + "\n", + "mpg_lsp = link_selections.instance()\n", + "\n", + "params = pn.Param(mpg_lsp, parameters=[\n", + " 'cross_filter_mode', 'selection_mode', 'show_regions',\n", + " 'selected_color', 'unselected_alpha', 'unselected_color'])\n", + "\n", + "pn.Row(params, mpg_lsp(w_accel_scatter + mpg_hist + violin))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Index-based selections\n", + "\n", + "So far we have worked entirely using range-based selections, which result in selection expressions based only on the axis ranges selected, not the actual data points. Range-based selection requires that all selectable dimensions are present on the datasets behind every plot, so that the selection expression can be evaluated to filter every plot down to the correct set of data points. Range-based selections also only support the `box_select` tool, as they are filtering the data based on a rectangular region of the visible space in that plot. (Of course, you can still combine multiple such boxes to build up to selections of other shapes, with `selection_mode='union'`.)\n", + "\n", + "You can also choose to use index-based selections, which generate expressions based not on axis ranges but on values of one or more index columns (selecting individual, specific data points, as for the Bokeh JavaScript-based linked brushing). For index-based selections, plots can be linked as long as the datasets underlying each plot all have those index columns, so that expressions generated from a selection on one plot can be applied to all of the plots. Ordinarily the index columns should be unique in combination (e.g. Firstname,Lastname), each specifying one particular data point out of your data so that it can be correlated across all plots.\n", + "\n", + "To use index-based selections, specify the `index_cols` that are present across your elements. In the example below we will load the shapes and names of counties from the US state of Texas and their corresponding unemployment rates. We then generate a choropleth plot and a histogram plot both displaying the unemployment rate." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from bokeh.sampledata.us_counties import data as counties\n", + "from bokeh.sampledata.unemployment import data as unemployment\n", + "\n", + "counties = [dict(county, Unemployment=unemployment[cid])\n", + " for cid, county in counties.items()\n", + " if county[\"state\"] == \"tx\"]\n", + "\n", + "choropleth = hv.Polygons(counties, ['lons', 'lats'], [('detailed name', 'County'), 'Unemployment'])\n", + "hist = choropleth.hist('Unemployment', adjoin=False, normed=False)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To link the two we will specify the `'detailed name'` column as the `index_cols`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "linked_choropleth = link_selections(choropleth + hist, index_cols=['detailed name'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now that the two plots are linked we can display them and select individual polygons by tapping or apply a box selection on the histogram:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "linked_choropleth.opts(\n", + " hv.opts.Polygons(tools=['hover', 'tap', 'box_select'], xaxis=None, yaxis=None,\n", + " show_grid=False, show_frame=False, width=500, height=500,\n", + " color='Unemployment', colorbar=True, line_color='white'),\n", + " hv.opts.Histogram(width=500, height=500)\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This type of linked brushing will work even for datasets not including the latitude and longitude axes of the choropleth plot, because each selected county resolves not to a geographic region but to a county name, which can then be used to index into any other dataset that includes the county name for each data point." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Styling selections\n", + "\n", + "By default, unselected objects will be indicated using a lower alpha value specified using the `unselected_alpha` keyword argument, which keeps unselected points the same color but makes them fade into the background. That way it should be safe to call `link_selections` on a plot without altering its visual appearance by default; you'll only see a visible difference once you select something. An alternative is to specify `selected_color` and an `unselected_color`, which can provide a more vivid contrast between the two states. To make sure both colors are visible ensure you also need to override the `unselected_alpha`:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "link_selections(w_accel_scatter + mpg_hist, selected_color='#ff0000', unselected_alpha=1, unselected_color='#90FF90')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Plotly support\n", + "\n", + "Linked brushing also works with the Plotly backend, which, unlike Bokeh, has support for rendering 3D plots:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "hv.Store.set_current_backend('plotly')\n", + "\n", + "ds = hv.Dataset(autompg)\n", + "\n", + "sel = link_selections.instance(\n", + " selected_color='#bf0000', unselected_color='#ff9f9f', unselected_alpha=1\n", + ")\n", + "\n", + "scatter1 = hv.Scatter(ds, 'weight', 'accel')\n", + "scatter2 = hv.Scatter(ds, 'mpg', 'displ')\n", + "scatter3d = hv.Scatter3D(ds, ['mpg', 'hp', 'weight'])\n", + "table = hv.Table(ds, ['name', 'origin', 'yr'], 'mpg')\n", + "\n", + "sel(scatter1 + scatter2 + scatter3d + table, selection_expr=((dim('origin')==1) & (dim('mpg') >16))).cols(2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "hv.Store.set_current_backend('bokeh')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note that Plotly does not yet support _selecting_ in 3D, but you should be able to provide 2D views for selecting alongside the 3D plots." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Operations\n", + "\n", + "One of the major advantages linked selections in HoloViews provide over using plotting libraries directly is the fact that HoloViews keeps track of the full pipeline of operations that have been applied to a dataset, allowing selections to be applied to the original dataset and then replaying the entire processing pipeline.\n", + "\n", + "In the example below, we'll use an example from [Datashader](https://datashader.org/getting_started/Pipeline.html) that has a sum of five normal distributions of different widths, each with its own range of values and its own category:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import datashader as ds\n", + "import holoviews.operation.datashader as hd\n", + "\n", + "num = 100000\n", + "np.random.seed(1)\n", + "\n", + "dists = {\n", + " cat: pd.DataFrame({\n", + " 'x': np.random.normal(x, s, num), \n", + " 'y': np.random.normal(y, s, num), \n", + " 'val': np.random.normal(val, 1.5, num), \n", + " 'cat': cat\n", + " }) for x, y, s, val, cat in \n", + " [( 2, 2, 0.03, 10, \"d1\"), \n", + " ( 2, -2, 0.10, 20, \"d2\"), \n", + " ( -2, -2, 0.50, 30, \"d3\"), \n", + " ( -2, 2, 1.00, 40, \"d4\"), \n", + " ( 0, 0, 3.00, 50, \"d5\")]\n", + "}\n", + "\n", + "points = hv.Points(pd.concat(dists), ['x', 'y'], ['val', 'cat'])\n", + "datashaded = hd.datashade(points, aggregator=ds.count_cat('cat'))\n", + "spreaded = hd.dynspread(datashaded, threshold=0.50, how='over')\n", + "\n", + "# Declare dim expression to color by cluster\n", + "dim_expr = ((0.1+hv.dim('val')/10).round()).categorize(hv.Cycle('Set1').values)\n", + "histogram = points.hist(num_bins=60, adjoin=False, normed=False).opts(color=dim_expr)\n", + "\n", + "link_selections(spreaded + histogram)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Here you can select a group of points on the datashaded plot (left) and see that the histogram updates to show that subset of points, and similarly you can select one or more groups in the histogram and see that the corresponding group of points is highlighted on the left. (Here we've color-coded the histogram to make it easy to see that relationship, i.e. that the small dot of red points has a `val` around 10, and the large cloud of orange points has a `val` around 50; usually such relationships won't be so easy to see!) Each time you make such a selection on either plot, the entire 2D spatial aggregation pipeline from Datashader is re-run on the first plot and the entire 1D aggregation by value is run on the second plot, allowing you to see how the subset of `x`,`y` and/or `val` values relates to the dataset as a whole. At no point is the actual data (100,000 points here) sent to the browser, but the browser still allows selecting against the original dataset so that all aggregate views of the data accurately reflect the selection." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Supported elements\n", + "\n", + "Not all elements can be used with the `link_selections` function, and those that do can support either being selected on (for range and/or index selections), displaying selections, or neither. Below we show most elements supporting range based selections:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "colors = hv.Cycle('Category10').values\n", + "\n", + "area = autompg_ds.aggregate('yr', function=np.mean).to(hv.Area, 'yr', 'weight')\n", + "bivariate = hv.Bivariate(autompg_ds, ['mpg', 'accel'], []).opts(show_legend=False)\n", + "box_whisker = hv.BoxWhisker(autompg_ds, 'cyl', 'accel').sort()\n", + "curve = autompg_ds.aggregate('yr', function=np.mean).to(hv.Curve, 'yr', 'mpg')\n", + "spread = autompg_ds.aggregate('yr', function=np.mean, spreadfn=np.std).to(hv.Spread, 'yr', ['mpg', 'mpg_std'])\n", + "distribution = hv.Distribution(autompg_ds, 'weight')\n", + "img = rasterize(hv.Points(autompg_ds, ['hp', 'displ']), dynamic=False, width=20, height=20)\n", + "heatmap = hv.HeatMap(autompg_ds, ['yr', 'origin'], 'accel').aggregate(function=np.mean)\n", + "hextiles = hv.HexTiles(autompg_ds, ['weight', 'displ'], []).opts(gridsize=20)\n", + "hist = autompg_ds.hist('displ', adjoin=False, normed=False)\n", + "scatter = hv.Scatter(autompg_ds, 'mpg', 'hp')\n", + "violin = hv.Violin(autompg_ds, 'origin', 'mpg').sort()\n", + "\n", + "link_selections(\n", + " area + bivariate + box_whisker + curve +\n", + " distribution + heatmap + hextiles + hist +\n", + " img + scatter + spread + violin\n", + ").opts(\n", + " opts.Area(color=colors[0]),\n", + " opts.Bivariate(cmap='Blues'),\n", + " opts.BoxWhisker(box_color=colors[1]),\n", + " opts.Curve(color=colors[2]),\n", + " opts.Distribution(color=colors[3]),\n", + " opts.HexTiles(cmap='Purples'),\n", + " opts.HeatMap(cmap='Greens'),\n", + " opts.Histogram(color=colors[4]),\n", + " opts.Image(cmap='Reds'),\n", + " opts.Scatter(color=colors[5]),\n", + " opts.Spread(color=colors[6]),\n", + " opts.Violin(violin_fill_color=colors[7]),\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Annotations (Arrow, Bounds, Box, Ellipse, HLine, HSpan, HSpan, Slope, Spline, Text, VLine, VSpan) are not selectable and are thus not applicable for link_selections. Most non-annotation elements are supported:\n", + "\n", + "\n", + "#### Elements that can be selected on using both range and index-based selections\n", + "\n", + "- `Area`\n", + "- `Bivariate`\n", + "- `Curve`\n", + "- `BoxWhisker`\n", + "- `Distribution`\n", + "- `HeatMap`\n", + "- `HexTiles`\n", + "- `HSV`\n", + "- `Histogram`\n", + "- `Image`\n", + "- `Spread`\n", + "- `Scatter`\n", + "- `Points`\n", + "- `RGB`\n", + "- `VectorField`\n", + "- `Violin`\n", + "\n", + "#### Elements that can be selected on using only index-based selections\n", + "\n", + "- `Contours`\n", + "- `Polygons`\n", + "- `Table`\n", + "\n", + "#### Elements that cannot be selected, but can display selections created on other elements:\n", + "\n", + "- `Labels`\n", + "- `Scatter3d`\n", + "- `ErrorBars`\n", + "\n", + "#### Not supported\n", + "\n", + "- `Bars` (Not currently implemented due to complications with stacked and multi-level bars)\n", + "- `Path` (Not currently supported due to complications masking partial paths)" + ] + } + ], + "metadata": { + "language_info": { + "name": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/holoviews/core/accessors.py b/holoviews/core/accessors.py index a0c0cf92f2..50275d9426 100644 --- a/holoviews/core/accessors.py +++ b/holoviews/core/accessors.py @@ -97,7 +97,8 @@ class Apply(object): def __init__(self, obj, mode=None): self._obj = obj - def __call__(self, function, streams=[], link_inputs=True, dynamic=None, **kwargs): + def __call__(self, function, streams=[], link_inputs=True, dynamic=None, + per_element=False, **kwargs): """Applies a function to all (Nd)Overlay or Element objects. Any keyword arguments are passed through to the function. If @@ -122,6 +123,10 @@ def __call__(self, function, streams=[], link_inputs=True, dynamic=None, **kwarg supplied, an instance parameter is supplied as a keyword argument, or the supplied function is a parameterized method. + per_element (bool, optional): Whether to apply per element + By default apply works on the leaf nodes, which + includes both elements and overlays. If set it will + apply directly to elements. kwargs (dict, optional): Additional keyword arguments Keyword arguments which will be supplied to the function. @@ -131,6 +136,7 @@ def __call__(self, function, streams=[], link_inputs=True, dynamic=None, **kwarg contained (Nd)Overlay or Element objects. """ from .dimension import ViewableElement + from .element import Element from .spaces import HoloMap, DynamicMap from ..util import Dynamic @@ -164,7 +170,8 @@ def function(object, **kwargs): kwargs = {k: v.param.value if isinstance(v, Widget) else v for k, v in kwargs.items()} - applies = isinstance(self._obj, ViewableElement) + spec = Element if per_element else ViewableElement + applies = isinstance(self._obj, spec) params = {p: val for p, val in kwargs.items() if isinstance(val, param.Parameter) and isinstance(val.owner, param.Parameterized)} @@ -299,6 +306,46 @@ def _filter_cache(self, dmap, kdims): filtered.append((key, value)) return filtered + def _transform_dimension(self, kdims, vdims, dimension): + if dimension in kdims: + idx = kdims.index(dimension) + dimension = self._obj.kdims[idx] + elif dimension in vdims: + idx = vdims.index(dimension) + dimension = self._obj.vdims[idx] + return dimension + + def _create_expression_transform(self, kdims, vdims, exclude=[]): + from .dimension import dimension_name + from ..util.transform import dim + + def _transform_expression(expression): + if dimension_name(expression.dimension) in exclude: + dimension = expression.dimension + else: + dimension = self._transform_dimension( + kdims, vdims, expression.dimension + ) + expression = expression.clone(dimension) + ops = [] + for op in expression.ops: + new_op = dict(op) + new_args = [] + for arg in op['args']: + if isinstance(arg, dim): + arg = _transform_expression(arg) + new_args.append(arg) + new_op['args'] = tuple(new_args) + new_kwargs = {} + for kw, kwarg in op['kwargs'].items(): + if isinstance(kwarg, dim): + kwarg = _transform_expression(kwarg) + new_kwargs[kw] = kwarg + new_op['kwargs'] = new_kwargs + ops.append(new_op) + expression.ops = ops + return expression + return _transform_expression def __call__(self, specs=None, **dimensions): """ @@ -324,13 +371,15 @@ def __call__(self, specs=None, **dimensions): kdims = self.replace_dimensions(obj.kdims, dimensions) vdims = self.replace_dimensions(obj.vdims, dimensions) zipped_dims = zip(obj.kdims+obj.vdims, kdims+vdims) - renames = {pk.name: nk for pk, nk in zipped_dims if pk != nk} + renames = {pk.name: nk for pk, nk in zipped_dims if pk.name != nk.name} if self.mode == 'dataset': data = obj.data if renames: data = obj.interface.redim(obj, renames) - clone = obj.clone(data, kdims=kdims, vdims=vdims) + transform = self._create_expression_transform(kdims, vdims, list(renames.values())) + transforms = obj._transforms + [transform] + clone = obj.clone(data, kdims=kdims, vdims=vdims, transforms=transforms) if self._obj.dimensions(label='name') == clone.dimensions(label='name'): # Ensure that plot_id is inherited as long as dimension # name does not change diff --git a/holoviews/core/data/__init__.py b/holoviews/core/data/__init__.py index 200c876f61..75c23bb98f 100644 --- a/holoviews/core/data/__init__.py +++ b/holoviews/core/data/__init__.py @@ -283,14 +283,21 @@ def __init__(self, data, kdims=None, vdims=None, **kwargs): input_data = data dataset_provided = 'dataset' in kwargs input_dataset = kwargs.pop('dataset', None) - input_pipeline = kwargs.pop( - 'pipeline', None - ) + input_pipeline = kwargs.pop('pipeline', None) + input_transforms = kwargs.pop('transforms', None) if isinstance(data, Element): pvals = util.get_param_values(data) kwargs.update([(l, pvals[l]) for l in ['group', 'label'] if l in pvals and l not in kwargs]) + if isinstance(data, Dataset): + if not dataset_provided and data._dataset is not None: + input_dataset = data._dataset + if input_pipeline is None: + input_pipeline = data.pipeline + if input_transforms is None: + input_transforms = data._transforms + kwargs.update(process_dimensions(kdims, vdims)) kdims, vdims = kwargs.get('kdims'), kwargs.get('vdims') @@ -316,6 +323,7 @@ def __init__(self, data, kdims=None, vdims=None, **kwargs): operations=input_pipeline.operations + [init_op], output_type=type(self), ) + self._transforms = input_transforms or [] # Handle initializing the dataset property. self._dataset = None @@ -331,7 +339,6 @@ def dataset(self): """ The Dataset that this object was created from """ - from . import Dataset if self._dataset is None: datatype = list(util.unique_iterator(self.datatype+Dataset.datatype)) dataset = Dataset(self, _validate_vdims=False, datatype=datatype) @@ -526,7 +533,7 @@ def select(self, selection_expr=None, selection_specs=None, **selection): selection = {dim_name: sel for dim_name, sel in selection.items() if dim_name in self.dimensions()+['selection_mask']} if (selection_specs and not any(self.matches(sp) for sp in selection_specs) - or (not selection and not selection_expr)): + or (not selection and not selection_expr)): return self # Handle selection dim expression diff --git a/holoviews/core/data/array.py b/holoviews/core/data/array.py index 883356fc76..da03c3e932 100644 --- a/holoviews/core/data/array.py +++ b/holoviews/core/data/array.py @@ -18,6 +18,8 @@ class ArrayInterface(Interface): datatype = 'array' + named = False + @classmethod def dimension_type(cls, dataset, dim): return dataset.data.dtype.type @@ -123,9 +125,7 @@ def sort(cls, dataset, by=[], reverse=False): @classmethod - def values( - cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False - ): + def values(cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False): data = dataset.data dim_idx = dataset.get_dimension_index(dim) if data.ndim == 1: @@ -136,6 +136,13 @@ def values( return values + @classmethod + def mask(cls, dataset, mask, mask_value=np.nan): + masked = np.copy(dataset.data) + masked[mask] = mask_value + return masked + + @classmethod def reindex(cls, dataset, kdims=None, vdims=None): # DataFrame based tables don't need to be reindexed diff --git a/holoviews/core/data/dask.py b/holoviews/core/data/dask.py index 53fec2e39e..8dca6c321e 100644 --- a/holoviews/core/data/dask.py +++ b/holoviews/core/data/dask.py @@ -90,15 +90,7 @@ def sort(cls, dataset, by=[], reverse=False): return dataset.data @classmethod - def values( - cls, - dataset, - dim, - expanded=True, - flat=True, - compute=True, - keep_index=False, - ): + def values(cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False): dim = dataset.get_dimension(dim) data = dataset.data[dim.name] if not expanded: diff --git a/holoviews/core/data/dictionary.py b/holoviews/core/data/dictionary.py index 4cdbd8acd0..6a4d38501f 100644 --- a/holoviews/core/data/dictionary.py +++ b/holoviews/core/data/dictionary.py @@ -224,6 +224,16 @@ def concat(cls, datasets, dimensions, vdims): return OrderedDict([(d.name, np.concatenate(columns[d.name])) for d in dims]) + @classmethod + def mask(cls, dataset, mask, mask_value=np.nan): + masked = OrderedDict() + for k, v in dataset.data.items(): + new_array = np.copy(dataset.data[k]) + new_array[mask] = mask_value + masked[k] = new_array + return masked + + @classmethod def sort(cls, dataset, by=[], reverse=False): by = [dataset.get_dimension(d).name for d in by] @@ -246,10 +256,8 @@ def range(cls, dataset, dimension): @classmethod - def values( - cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False - ): - dim = dataset.get_dimension(dim).name + def values(cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False): + dim = dataset.get_dimension(dim, strict=True).name values = dataset.data.get(dim) if isscalar(values): if not expanded: diff --git a/holoviews/core/data/grid.py b/holoviews/core/data/grid.py index a135db42a1..0ede8fb7c2 100644 --- a/holoviews/core/data/grid.py +++ b/holoviews/core/data/grid.py @@ -58,6 +58,7 @@ def init(cls, eltype, data, kdims, vdims): ndims = len(kdims) dimensions = [dimension_name(d) for d in kdims+vdims] vdim_tuple = tuple(dimension_name(vd) for vd in vdims) + if isinstance(data, tuple): if (len(data) != len(dimensions) and len(data) == (ndims+1) and len(data[-1].shape) == (ndims+1)): @@ -66,18 +67,34 @@ def init(cls, eltype, data, kdims, vdims): data[vdim_tuple] = value_array else: data = {d: v for d, v in zip(dimensions, data)} - elif isinstance(data, list) and data == []: - data = OrderedDict([(d, []) for d in dimensions]) + elif (isinstance(data, list) and data == []): + if len(kdims) == 1: + data = OrderedDict([(d, []) for d in dimensions]) + else: + data = OrderedDict([(d.name, np.array([])) for d in kdims]) + if len(vdims) == 1: + data[vdims[0].name] = np.zeros((0, 0)) + else: + data[vdim_tuple] = np.zeros((0, 0, len(vdims))) elif not any(isinstance(data, tuple(t for t in interface.types if t is not None)) for interface in cls.interfaces.values()): data = {k: v for k, v in zip(dimensions, zip(*data))} elif isinstance(data, np.ndarray): - if data.ndim == 1: - if eltype._auto_indexable_1d and len(kdims)+len(vdims)>1: - data = np.column_stack([np.arange(len(data)), data]) - else: - data = np.atleast_2d(data).T - data = {k: data[:,i] for i,k in enumerate(dimensions)} + if data.shape == (0, 0) and len(vdims) == 1: + array = data + data = OrderedDict([(d.name, np.array([])) for d in kdims]) + data[vdims[0].name] = array + elif data.shape == (0, 0, len(vdims)): + array = data + data = OrderedDict([(d.name, np.array([])) for d in kdims]) + data[vdim_tuple] = array + else: + if data.ndim == 1: + if eltype._auto_indexable_1d and len(kdims)+len(vdims)>1: + data = np.column_stack([np.arange(len(data)), data]) + else: + data = np.atleast_2d(data).T + data = {k: data[:, i] for i, k in enumerate(dimensions)} elif isinstance(data, list) and data == []: data = {d: np.array([]) for d in dimensions[:ndims]} data.update({d: np.empty((0,) * ndims) for d in dimensions[ndims:]}) @@ -245,9 +262,9 @@ def _infer_interval_breaks(cls, coord, axis=0): if sys.version_info.major == 2 and len(coord) and isinstance(coord[0], (dt.datetime, dt.date)): # np.diff does not work on datetimes in python 2 coord = coord.astype('datetime64') - if len(coord) == 0: + if coord.shape[axis] == 0: return np.array([], dtype=coord.dtype) - if len(coord) > 1: + if coord.shape[axis] > 1: deltas = 0.5 * np.diff(coord, axis=axis) else: deltas = np.array([0.5]) @@ -393,9 +410,7 @@ def ndloc(cls, dataset, indices): @classmethod - def values( - cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False - ): + def values(cls, dataset, dim, expanded=True, flat=True, compute=True, keep_index=False): dim = dataset.get_dimension(dim, strict=True) if dim in dataset.vdims or dataset.data[dim.name].ndim > 1: vdim_tuple = cls.packed(dataset) @@ -598,6 +613,29 @@ def select(cls, dataset, selection_mask=None, **selection): return data + @classmethod + def mask(cls, dataset, mask, mask_val=np.nan): + mask = cls.canonicalize(dataset, mask) + packed = cls.packed(dataset) + masked = OrderedDict(dataset.data) + if packed: + masked = dataset.data[packed].copy() + try: + masked[mask] = mask_val + except ValueError: + masked = masked.astype('float') + masked[mask] = mask_val + else: + for vd in dataset.vdims: + masked[vd.name] = marr = masked[vd.name].copy() + try: + marr[mask] = mask_val + except ValueError: + masked[vd.name] = marr = marr.astype('float') + marr[mask] = mask_val + return masked + + @classmethod def sample(cls, dataset, samples=[]): """ diff --git a/holoviews/core/data/image.py b/holoviews/core/data/image.py index f82230f141..6640fdcbad 100644 --- a/holoviews/core/data/image.py +++ b/holoviews/core/data/image.py @@ -20,6 +20,8 @@ class ImageInterface(GridInterface): datatype = 'image' + named = False + @classmethod def init(cls, eltype, data, kdims, vdims): if kdims is None: @@ -199,6 +201,13 @@ def values( return None + @classmethod + def mask(cls, dataset, mask, mask_val=np.nan): + masked = dataset.data.copy().astype('float') + masked[np.flipud(mask)] = mask_val + return masked + + @classmethod def select(cls, dataset, selection_mask=None, **selection): """ diff --git a/holoviews/core/data/interface.py b/holoviews/core/data/interface.py index 7d0e6a135f..c8a0697980 100644 --- a/holoviews/core/data/interface.py +++ b/holoviews/core/data/interface.py @@ -158,6 +158,9 @@ class Interface(param.Parameterized): # Denotes whether the interface expects ragged data multi = False + # Whether the interface stores the names of the underlying dimensions + named = True + @classmethod def loaded(cls): """ @@ -229,7 +232,7 @@ def initialize(cls, eltype, data, kdims, vdims, datatype=None): datatype = eltype.datatype interface = data.interface - if interface.datatype in datatype and interface.datatype in eltype.datatype: + if interface.datatype in datatype and interface.datatype in eltype.datatype and interface.named: data = data.data elif interface.multi and any(cls.interfaces[dt].multi for dt in datatype if dt in cls.interfaces): data = [d for d in data.interface.split(data, None, None, 'columns')] diff --git a/holoviews/core/data/multipath.py b/holoviews/core/data/multipath.py index 19998673bd..0be135beba 100644 --- a/holoviews/core/data/multipath.py +++ b/holoviews/core/data/multipath.py @@ -214,6 +214,8 @@ def select(cls, dataset, selection_mask=None, **selection): from ...element import Polygons if not dataset.data: return dataset.data + elif selection_mask is not None: + return [d for b, d in zip(selection_mask, dataset.data) if b] ds = cls._inner_dataset_template(dataset) skipped = (Polygons._hole_key,) if hasattr(ds.interface, 'geo_column'): diff --git a/holoviews/core/data/pandas.py b/holoviews/core/data/pandas.py index 51fb1fa279..692452d820 100644 --- a/holoviews/core/data/pandas.py +++ b/holoviews/core/data/pandas.py @@ -254,6 +254,13 @@ def reindex(cls, dataset, kdims=None, vdims=None): return dataset.data + @classmethod + def mask(cls, dataset, mask, mask_value=np.nan): + masked = dataset.data.copy() + masked[mask] = mask_value + return masked + + @classmethod def redim(cls, dataset, dimensions): column_renames = {k: v.name for k, v in dimensions.items()} diff --git a/holoviews/core/data/xarray.py b/holoviews/core/data/xarray.py index ddc1704407..c7b77f823f 100644 --- a/holoviews/core/data/xarray.py +++ b/holoviews/core/data/xarray.py @@ -474,6 +474,27 @@ def reindex(cls, dataset, kdims=None, vdims=None): def sort(cls, dataset, by=[], reverse=False): return dataset + @classmethod + def mask(cls, dataset, mask, mask_val=np.nan): + packed = cls.packed(dataset) + masked = dataset.data.copy() + if packed: + data_coords = list(dataset.data.dims)[:-1] + mask = cls.canonicalize(dataset, mask, data_coords) + try: + masked.values[mask] = mask_val + except ValueError: + masked = masked.astype('float') + masked.values[mask] = mask_val + else: + orig_mask = mask + for vd in dataset.vdims: + data_coords = list(dataset.data[vd.name].dims) + mask = cls.canonicalize(dataset, orig_mask, data_coords) + masked[vd.name] = marr = masked[vd.name].astype('float') + marr.values[mask] = mask_val + return masked + @classmethod def select(cls, dataset, selection_mask=None, **selection): validated = {} diff --git a/holoviews/core/dimension.py b/holoviews/core/dimension.py index 1403e7668d..b19448e2e6 100644 --- a/holoviews/core/dimension.py +++ b/holoviews/core/dimension.py @@ -965,7 +965,7 @@ def get_dimension(self, dimension, default=None, strict=False): if isinstance(dimension, Dimension): dims = [d for d in all_dims if dimension == d] if strict and not dims: - raise KeyError("Dimension %r not found." % dimension) + raise KeyError("%r not found." % dimension) elif dims: return dims[0] else: @@ -973,6 +973,7 @@ def get_dimension(self, dimension, default=None, strict=False): else: dimension = dimension_name(dimension) name_map = {dim.spec: dim for dim in all_dims} + name_map.update({dim.name: dim for dim in all_dims}) name_map.update({dim.label: dim for dim in all_dims}) name_map.update({util.dimension_sanitizer(dim.name): dim for dim in all_dims}) if strict and dimension not in name_map: diff --git a/holoviews/core/element.py b/holoviews/core/element.py index abf8c7be3d..5f3b8c41c8 100644 --- a/holoviews/core/element.py +++ b/holoviews/core/element.py @@ -29,7 +29,7 @@ class Element(ViewableElement, Composable, Overlayable): _selection_streams = () def _get_selection_expr_for_stream_value(self, **kwargs): - return None, None + return None, None, None def hist(self, dimension=None, num_bins=20, bin_range=None, adjoin=True, **kwargs): @@ -378,6 +378,8 @@ class Element3D(Element2D): __abstract = True + _selection_streams = () + class Collator(NdMapping): """ diff --git a/holoviews/core/options.py b/holoviews/core/options.py index b1f05f70f2..af5f944c43 100644 --- a/holoviews/core/options.py +++ b/holoviews/core/options.py @@ -100,6 +100,13 @@ def lookup_options(obj, group, backend): return node + +class CallbackError(RuntimeError): + """ + An error raised during a callback. + """ + + class SkipRendering(Exception): """ A SkipRendering exception in the plotting code will make the display diff --git a/holoviews/core/spaces.py b/holoviews/core/spaces.py index fb12c52be2..93dd8521b1 100644 --- a/holoviews/core/spaces.py +++ b/holoviews/core/spaces.py @@ -625,6 +625,10 @@ class Callable(param.Parameterized): The list of inputs the callable function is wrapping. Used to allow deep access to streams in chained Callables.""") + operation_kwargs = param.Dict(default={}, constant=True, doc=""" + Potential dynamic keyword arguments associated with the + operation.""") + link_inputs = param.Boolean(default=True, doc=""" If the Callable wraps around other DynamicMaps in its inputs, determines whether linked streams attached to the inputs are @@ -1160,11 +1164,6 @@ def clone(self, data=None, shared_data=True, new_type=None, link=True, Returns: Cloned object """ - if 'link_inputs' in overrides: - self.param.warning( - 'link_inputs argument to the clone method is deprecated, ' - 'use the more general link argument instead.') - link = link and overrides.pop('link_inputs', True) callback = overrides.pop('callback', self.callback) if data is None and shared_data: data = self.data @@ -1177,9 +1176,13 @@ def clone(self, data=None, shared_data=True, new_type=None, link=True, # Ensure the clone references this object to ensure # stream sources are inherited if clone.callback is self.callback: + from ..operation import function with util.disable_constant(clone): - clone.callback = clone.callback.clone(inputs=[self], - link_inputs=link) + op = function.instance(fn=lambda x, **kwargs: x) + clone.callback = clone.callback.clone( + inputs=[self], link_inputs=link, operation=op, + operation_kwargs={} + ) return clone diff --git a/holoviews/core/util.py b/holoviews/core/util.py index 2af181afe1..07302ca316 100644 --- a/holoviews/core/util.py +++ b/holoviews/core/util.py @@ -1106,6 +1106,20 @@ def unique_iterator(seq): yield item +def lzip(*args): + """ + zip function that returns a list. + """ + return list(zip(*args)) + + +def unique_zip(*args): + """ + Returns a unique list of zipped values. + """ + return list(unique_iterator(zip(*args))) + + def unique_array(arr): """ Returns an array of unique values in the input order. diff --git a/holoviews/element/annotation.py b/holoviews/element/annotation.py index 07b2af4b35..c56330d8ed 100644 --- a/holoviews/element/annotation.py +++ b/holoviews/element/annotation.py @@ -164,7 +164,6 @@ def __init__(self, slope, y_intercept, kdims=None, vdims=None, **params): (slope, y_intercept), slope=slope, y_intercept=y_intercept, kdims=kdims, vdims=vdims, **params) - @classmethod def from_scatter(cls, element, **kwargs): """Returns a Slope element given an element of x/y-coordinates @@ -192,16 +191,16 @@ class VSpan(Annotation): group = param.String(default='VSpan', constant=True) - x1 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, doc=""" + x1 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, allow_None=True, doc=""" The start x-position of the VSpan which must be numeric or a timestamp.""") - x2 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, doc=""" + x2 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, allow_None=True, doc=""" The end x-position of the VSpan which must be numeric or a timestamp.""") __pos_params = ['x1', 'x2'] - def __init__(self, x1, x2, **params): - super(VSpan, self).__init__([x1, x2], **params) + def __init__(self, x1=None, x2=None, **params): + super(VSpan, self).__init__([x1, x2], x1=x1, x2=x2, **params) def dimension_values(self, dimension, expanded=True, flat=True): """Return the values along the requested dimension. @@ -228,16 +227,16 @@ class HSpan(Annotation): group = param.String(default='HSpan', constant=True) - y1 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, doc=""" + y1 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, allow_None=True, doc=""" The start y-position of the VSpan which must be numeric or a timestamp.""") - y2 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, doc=""" + y2 = param.ClassSelector(default=0, class_=(Number,) + datetime_types, allow_None=True, doc=""" The end y-position of the VSpan which must be numeric or a timestamp.""") __pos_params = ['y1', 'y2'] - def __init__(self, y1, y2, **params): - super(HSpan, self).__init__([y1, y2], **params) + def __init__(self, y1=None, y2=None, **params): + super(HSpan, self).__init__([y1, y2], y1=y1, y2=y2, **params) def dimension_values(self, dimension, expanded=True, flat=True): """Return the values along the requested dimension. diff --git a/holoviews/element/chart.py b/holoviews/element/chart.py index 88f57d45af..b32aeebf35 100644 --- a/holoviews/element/chart.py +++ b/holoviews/element/chart.py @@ -1,12 +1,12 @@ import numpy as np import param -from ..streams import BoundsXY from ..core import util from ..core import Dimension, Dataset, Element2D from ..core.data import GridInterface -from .geom import Points, VectorField # noqa: backward compatible import -from .stats import BoxWhisker # noqa: backward compatible import +from ..streams import SelectionXY +from .geom import Rectangles, Points, VectorField # noqa: backward compatible import +from .selection import Selection1DExpr, Selection2DExpr class Chart(Dataset, Element2D): @@ -48,59 +48,18 @@ def __getitem__(self, index): return super(Chart, self).__getitem__(index) -class Chart2dSelectionExpr(object): - """ - Mixin class for Cartesian 2D Chart elements to add basic support for - SelectionExpr streams. - """ - _selection_streams = (BoundsXY,) - - def _get_selection_expr_for_stream_value(self, **kwargs): - from ..util.transform import dim - - invert_axes = self.opts.get('plot').kwargs.get('invert_axes', False) - - if kwargs.get('bounds', None): - x0, y0, x1, y1 = kwargs['bounds'] - - # Handle invert_xaxis/invert_yaxis - if y0 > y1: - y0, y1 = y1, y0 - if x0 > x1: - x0, x1 = x1, x0 - - if invert_axes: - ydim = self.kdims[0] - xdim = self.vdims[0] - else: - xdim = self.kdims[0] - ydim = self.vdims[0] - - bbox = { - xdim.name: (x0, x1), - ydim.name: (y0, y1), - } - - selection_expr = ( - (dim(xdim) >= x0) & (dim(xdim) <= x1) & - (dim(ydim) >= y0) & (dim(ydim) <= y1) - ) - - return selection_expr, bbox - return None, None - - -class Scatter(Chart2dSelectionExpr, Chart): +class Scatter(Selection2DExpr, Chart): """ Scatter is a Chart element representing a set of points in a 1D coordinate system where the key dimension maps to the points location along the x-axis while the first value dimension represents the location of the point along the y-axis. """ + group = param.String(default='Scatter', constant=True) -class Curve(Chart2dSelectionExpr, Chart): +class Curve(Selection1DExpr, Chart): """ Curve is a Chart element representing a line in a 1D coordinate system where the key dimension maps on the line x-coordinate and @@ -111,21 +70,22 @@ class Curve(Chart2dSelectionExpr, Chart): group = param.String(default='Curve', constant=True) -class ErrorBars(Chart2dSelectionExpr, Chart): +class ErrorBars(Selection1DExpr, Chart): """ ErrorBars is a Chart element representing error bars in a 1D coordinate system where the key dimension corresponds to the - location along the x-axis and the first value dimension - corresponds to the location along the y-axis and one or two - extra value dimensions corresponding to the symmetric or + location along the x-axis and the first value dimension + corresponds to the location along the y-axis and one or two + extra value dimensions corresponding to the symmetric or asymetric errors either along x-axis or y-axis. If two value - dimensions are given, then the last value dimension will be - taken as symmetric errors. If three value dimensions are given + dimensions are given, then the last value dimension will be + taken as symmetric errors. If three value dimensions are given then the last two value dimensions will be taken as negative and positive errors. By default the errors are defined along y-axis. A parameter `horizontal`, when set `True`, will define the errors along the x-axis. """ + group = param.String(default='ErrorBars', constant=True, doc=""" A string describing the quantity measured by the ErrorBars object.""") @@ -220,7 +180,7 @@ class Histogram(Chart): _binned = True - _selection_streams = (BoundsXY,) + _selection_streams = (SelectionXY,) def __init__(self, data, edges=None, **params): if data is None: @@ -241,52 +201,96 @@ def _get_selection_expr_for_stream_value(self, **kwargs): invert_axes = self.opts.get('plot').kwargs.get('invert_axes', False) - if kwargs.get('bounds', None): - if invert_axes: - y0, x0, y1, x1 = kwargs['bounds'] - else: - x0, y0, x1, y1 = kwargs['bounds'] - - # Handle invert_xaxis/invert_yaxis - if y0 > y1: - y0, y1 = y1, y0 - if x0 > x1: - x0, x1 = x1, x0 - - xdim = self.kdims[0] - ydim = self.vdims[0] - - edges = self.edges - centers = self.dimension_values(xdim) - heights = self.dimension_values(ydim) - - selected_mask = ( - (centers >= x0) & (centers <= x1) & - (heights >= y0) & (heights <= y1) - ) + ds = self.dataset + if kwargs.get('bounds', None) is None: + el = ds.clone([]) if ds.interface.gridded else ds.iloc[:0] + return None, None, self.pipeline(el) - selected_bins = (np.arange(len(centers))[selected_mask] + 1).tolist() - if not selected_bins: - return None, None + if invert_axes: + y0, x0, y1, x1 = kwargs['bounds'] + else: + x0, y0, x1, y1 = kwargs['bounds'] - selection_expr = ( - dim(xdim).digitize(edges).isin(selected_bins) + # Handle invert_xaxis/invert_yaxis + if y0 > y1: + y0, y1 = y1, y0 + if x0 > x1: + x0, x1 = x1, x0 + + xdim = self.kdims[0] + ydim = self.vdims[0] + + edges = self.edges + centers = self.dimension_values(xdim) + heights = self.dimension_values(ydim) + + selected_mask = ( + (centers >= x0) & (centers <= x1) & + (heights >= y0) & (heights <= y1) + ) + + selected_bins = (np.arange(len(centers))[selected_mask] + 1).tolist() + if not selected_bins: + el = ds.clone([]) if ds.interface.gridded else ds.iloc[:0] + return None, None, self.pipeline(el) + + bbox = { + xdim.name: ( + edges[max(0, min(selected_bins) - 1)], + edges[min(len(edges - 1), max(selected_bins))], + ), + } + index_cols = kwargs.get('index_cols') + if index_cols: + shape = dim(self.dataset.get_dimension(index_cols[0]), np.shape) + index_cols = [dim(self.dataset.get_dimension(c), np.ravel) for c in index_cols] + sel = self.dataset.clone(datatype=['dataframe', 'dictionary']).select(**bbox) + vals = dim(index_cols[0], util.unique_zip, *index_cols[1:]).apply( + sel, expanded=True, flat=True ) - + contains = dim(index_cols[0], util.lzip, *index_cols[1:]).isin(vals, object=True) + selection_expr = dim(contains, np.reshape, shape) + region = None + else: + selection_expr = dim(xdim).digitize(edges).isin(selected_bins) if selected_bins[-1] == len(centers): # Handle values exactly on the upper boundary selection_expr = selection_expr | (dim(xdim) == edges[-1]) - - bbox = { - xdim.name: ( - edges[max(0, min(selected_bins) - 1)], - edges[min(len(edges - 1), max(selected_bins))], - ), - } - - return selection_expr, bbox - - return None, None + if ds.interface.gridded: + mask = selection_expr.apply(ds, expanded=True, flat=False) + ds = ds.clone(ds.interface.mask(ds, mask)) + else: + ds = ds.select(selection_expr) + region = self.pipeline(ds) + return selection_expr, bbox, region + + @staticmethod + def _merge_regions(region1, region2, operation): + if region1 is None: + if operation == 'inverse': + return region2.clone(data=( + (region2.edges, + np.zeros_like(region2.dimension_values(1))) + )) + else: + return region2 + + if operation == 'overwrite': + return region2 + elif operation == 'inverse': + y = region1.dimension_values(1).copy() + y[region2.dimension_values(1) > 0] = 0 + return region1.clone(data=(region1.edges, y)) + elif operation == 'intersect': + op = np.min + elif operation == 'union': + op = np.max + + return region1.clone(data=( + region1.edges, + op(np.stack([region1.dimension_values(1), + region2.dimension_values(1)], axis=1), axis=1) + )) def __setstate__(self, state): """ @@ -315,7 +319,7 @@ def edges(self): return self.interface.coords(self, self.kdims[0], edges=True) -class Spikes(Chart2dSelectionExpr, Chart): +class Spikes(Selection1DExpr, Chart): """ Spikes is a Chart element which represents a number of discrete spikes, events or observations in a 1D coordinate system. The key @@ -335,6 +339,7 @@ class Spikes(Chart2dSelectionExpr, Chart): _auto_indexable_1d = False + class Area(Curve): """ Area is a Chart element representing the area under a curve or diff --git a/holoviews/element/chart3d.py b/holoviews/element/chart3d.py index d43876db0e..233852de5a 100644 --- a/holoviews/element/chart3d.py +++ b/holoviews/element/chart3d.py @@ -37,6 +37,9 @@ def __init__(self, data, kdims=None, vdims=None, extents=None, **params): extents = extents if extents else (None, None, None, None, None, None) Image.__init__(self, data, kdims=kdims, vdims=vdims, extents=extents, **params) + def _get_selection_expr_for_stream_value(self, **kwargs): + expr, bbox, _ = super(Surface, self)._get_selection_expr_for_stream_value(**kwargs) + return expr, bbox, None class TriSurface(Element3D, Points): diff --git a/holoviews/element/geom.py b/holoviews/element/geom.py index 3211b6b879..5410fdd1d2 100644 --- a/holoviews/element/geom.py +++ b/holoviews/element/geom.py @@ -3,7 +3,7 @@ import param from ..core import Dimension, Dataset, Element2D -from ..streams import BoundsXY +from .selection import Selection2DExpr, SelectionGeomExpr class Geometry(Dataset, Element2D): @@ -27,47 +27,7 @@ class Geometry(Dataset, Element2D): __abstract = True -class GeometrySelectionExpr(object): - """ - Mixin class for Geometry elements to add basic support for - SelectionExpr streams. - """ - _selection_streams = (BoundsXY,) - - def _get_selection_expr_for_stream_value(self, **kwargs): - from ..util.transform import dim - - invert_axes = self.opts.get('plot').kwargs.get('invert_axes', False) - - if kwargs.get('bounds', None): - x0, y0, x1, y1 = kwargs['bounds'] - - # Handle invert_xaxis/invert_yaxis - if y0 > y1: - y0, y1 = y1, y0 - if x0 > x1: - x0, x1 = x1, x0 - - if invert_axes: - ydim, xdim = self.kdims[:2] - else: - xdim, ydim = self.kdims[:2] - - bbox = { - xdim.name: (x0, x1), - ydim.name: (y0, y1), - } - - selection_expr = ( - (dim(xdim) >= x0) & (dim(xdim) <= x1) & - (dim(ydim) >= y0) & (dim(ydim) <= y1) - ) - - return selection_expr, bbox - return None, None - - -class Points(GeometrySelectionExpr, Geometry): +class Points(Selection2DExpr, Geometry): """ Points represents a set of coordinates in 2D space, which may optionally be associated with any number of value dimensions. @@ -78,7 +38,7 @@ class Points(GeometrySelectionExpr, Geometry): _auto_indexable_1d = True -class VectorField(GeometrySelectionExpr, Geometry): +class VectorField(Selection2DExpr, Geometry): """ A VectorField represents a set of vectors in 2D space with an associated angle, as well as an optional magnitude and any number @@ -93,7 +53,7 @@ class VectorField(GeometrySelectionExpr, Geometry): Dimension('Magnitude')], bounds=(1, None)) -class Segments(Geometry): +class Segments(SelectionGeomExpr, Geometry): """ Segments represent a collection of lines in 2D space. """ @@ -106,7 +66,7 @@ class Segments(Geometry): coordinates in 2D space.""") -class Rectangles(Geometry): +class Rectangles(SelectionGeomExpr, Geometry): """ Rectangles represent a collection of axis-aligned rectangles in 2D space. """ diff --git a/holoviews/element/graphs.py b/holoviews/element/graphs.py index bed4d7ea95..8fcf8eca7f 100644 --- a/holoviews/element/graphs.py +++ b/holoviews/element/graphs.py @@ -149,6 +149,7 @@ def __init__(self, data, kdims=None, vdims=None, **params): node_info = None if edgepaths is not None and not isinstance(edgepaths, self.edge_type): edgepaths = self.edge_type(edgepaths) + self._nodes = nodes self._edgepaths = edgepaths super(Graph, self).__init__(edges, kdims=kdims, vdims=vdims, **params) @@ -251,7 +252,7 @@ def clone(self, data=None, shared_data=True, new_type=None, link=True, *args, **overrides) - def select(self, selection_specs=None, selection_mode='edges', **selection): + def select(self, selection_expr=None, selection_specs=None, selection_mode='edges', **selection): """ Allows selecting data by the slices, sets and scalar values along a particular dimension. The indices should be supplied as @@ -265,17 +266,28 @@ def select(self, selection_specs=None, selection_mode='edges', **selection): connected to the selected nodes. To select only edges between the selected nodes set the selection_mode to 'nodes'. """ + from ..util.transform import dim + if selection_expr is not None and not isinstance(selection_expr, dim): + raise ValueError("""\ +The first positional argument to the Dataset.select method is expected to be a +holoviews.util.transform.dim expression. Use the selection_specs keyword +argument to specify a selection specification""") + selection = {dim: sel for dim, sel in selection.items() if dim in self.dimensions('ranges')+['selection_mask']} if (selection_specs and not any(self.matches(sp) for sp in selection_specs) - or not selection): + or (not selection and not selection_expr)): return self index_dim = self.nodes.kdims[2].name dimensions = self.kdims+self.vdims node_selection = {index_dim: v for k, v in selection.items() if k in self.kdims} - nodes = self.nodes.select(**dict(selection, **node_selection)) + if selection_expr: + mask = selection_expr.apply(self.nodes, compute=False, keep_index=True) + nodes = self.nodes[mask] + else: + nodes = self.nodes.select(**dict(selection, **node_selection)) selection = {k: v for k, v in selection.items() if k in dimensions} # Compute mask for edges if nodes were selected on @@ -364,8 +376,12 @@ def nodes(self): Computes the node positions the first time they are requested if no explicit node information was supplied. """ + if self._nodes is None: + from ..operation.element import chain self._nodes = layout_nodes(self, only_nodes=True) + self._nodes._dataset = None + self._nodes._pipeline = chain.instance() return self._nodes diff --git a/holoviews/element/path.py b/holoviews/element/path.py index 0c5ba62057..ec19381090 100644 --- a/holoviews/element/path.py +++ b/holoviews/element/path.py @@ -13,6 +13,7 @@ from ..core.dimension import Dimension, asdim from ..core.util import OrderedDict, disable_constant from .geom import Geometry +from .selection import SelectionIndexExpr class Path(Geometry): @@ -83,6 +84,8 @@ def __init__(self, data, kdims=None, vdims=None, **params): def __getitem__(self, key): + if isinstance(key, np.ndarray): + return self.select(selection_mask=np.squeeze(key)) if key in self.dimensions(): return self.dimension_values(key) if not isinstance(key, tuple) or len(key) == 1: key = (key, slice(None)) @@ -202,7 +205,7 @@ def __setstate__(self, state): -class Contours(Path): +class Contours(SelectionIndexExpr, Path): """ The Contours element is a subtype of a Path which is characterized by the fact that each path geometry may only be associated with @@ -245,6 +248,14 @@ class Contours(Path): _level_vdim = Dimension('Level') # For backward compatibility + def _get_selection_expr_for_stream_value(self, **kwargs): + expr, _, _ = super(Contours, self)._get_selection_expr_for_stream_value(**kwargs) + if expr: + region = self.pipeline(self.dataset.select(expr)) + else: + region = self.iloc[:0] + return expr, _, region + def __init__(self, data, kdims=None, vdims=None, **params): data = [] if data is None else data if params.get('level') is not None: diff --git a/holoviews/element/raster.py b/holoviews/element/raster.py index d8c5469c32..853ad86e4d 100644 --- a/holoviews/element/raster.py +++ b/holoviews/element/raster.py @@ -11,6 +11,7 @@ from ..core.boundingregion import BoundingRegion, BoundingBox from ..core.sheetcoords import SheetCoordinateSystem, Slice from .chart import Curve +from .geom import Selection2DExpr from .graphs import TriMesh from .tabular import Table from .util import compute_slice_bounds, categorical_aggregate2d @@ -213,7 +214,7 @@ def _coord2matrix(self, coord): -class Image(Dataset, Raster, SheetCoordinateSystem): +class Image(Selection2DExpr, Dataset, Raster, SheetCoordinateSystem): """ Image represents a regularly sampled 2D grid of an underlying continuous space of intensity values, which will be colormapped on @@ -314,18 +315,20 @@ def __init__(self, data, kdims=None, vdims=None, bounds=None, extents=None, if self.interface is ImageInterface and not isinstance(data, (np.ndarray, Image)): data_bounds = self.bounds.lbrt() - l, b, r, t = bounds.lbrt() - xdensity = xdensity if xdensity else util.compute_density(l, r, dim1, self._time_unit) - ydensity = ydensity if ydensity else util.compute_density(b, t, dim2, self._time_unit) - if not util.isfinite(xdensity) or not util.isfinite(ydensity): - raise ValueError('Density along Image axes could not be determined. ' - 'If the data contains only one coordinate along the ' - 'x- or y-axis ensure you declare the bounds and/or ' - 'density.') + non_finite = all(not util.isfinite(v) for v in bounds.lbrt()) + if non_finite: + bounds = BoundingBox(points=((0, 0), (0, 0))) + xdensity = xdensity or 1 + ydensity = ydensity or 1 + else: + l, b, r, t = bounds.lbrt() + xdensity = xdensity if xdensity else util.compute_density(l, r, dim1, self._time_unit) + ydensity = ydensity if ydensity else util.compute_density(b, t, dim2, self._time_unit) SheetCoordinateSystem.__init__(self, bounds, xdensity, ydensity) + if non_finite: + self.bounds = BoundingBox(points=((np.nan, np.nan), (np.nan, np.nan))) self._validate(data_bounds, supplied_bounds) - def _validate(self, data_bounds, supplied_bounds): if len(self.shape) == 3: if self.shape[2] != len(self.vdims): @@ -803,7 +806,7 @@ def rgb(self): **params) -class QuadMesh(Dataset, Element2D): +class QuadMesh(Selection2DExpr, Dataset, Element2D): """ A QuadMesh represents 2D rectangular grid expressed as x- and y-coordinates defined as 1D or 2D arrays. Unlike the Image type @@ -908,7 +911,7 @@ def trimesh(self): -class HeatMap(Dataset, Element2D): +class HeatMap(Selection2DExpr, Dataset, Element2D): """ HeatMap represents a 2D grid of categorical coordinates which can be computed from a sparse tabular representation. A HeatMap does diff --git a/holoviews/element/selection.py b/holoviews/element/selection.py new file mode 100644 index 0000000000..1bc5f181b1 --- /dev/null +++ b/holoviews/element/selection.py @@ -0,0 +1,251 @@ +""" +Defines mix-in classes to handle support for linked brushing on +elements. +""" + +import numpy as np + +from ..core import util, NdOverlay +from ..streams import SelectionXY, Selection1D +from ..util.transform import dim +from .annotation import HSpan, VSpan + + +class SelectionIndexExpr(object): + + _selection_dims = None + + _selection_streams = (Selection1D,) + + def _get_selection_expr_for_stream_value(self, **kwargs): + index = kwargs.get('index') + index_cols = kwargs.get('index_cols') + if index is None or index_cols is None: + expr = None + else: + get_shape = dim(self.dataset.get_dimension(index_cols[0]), np.shape) + index_cols = [dim(self.dataset.get_dimension(c), np.ravel) for c in index_cols] + vals = dim(index_cols[0], util.unique_zip, *index_cols[1:]).apply( + self.iloc[index], expanded=True, flat=True + ) + contains = dim(index_cols[0], util.lzip, *index_cols[1:]).isin(vals, object=True) + expr = dim(contains, np.reshape, get_shape) + return expr, None, None + + @staticmethod + def _merge_regions(region1, region2, operation): + return None + + +class Selection2DExpr(object): + """ + Mixin class for Cartesian 2D elements to add basic support for + SelectionExpr streams. + """ + + _selection_dims = 2 + + _selection_streams = (SelectionXY,) + + def _get_selection(self, **kwargs): + xcats, ycats = None, None + x0, y0, x1, y1 = kwargs['bounds'] + if 'x_selection' in kwargs: + xsel = kwargs['x_selection'] + if isinstance(xsel, list): + xcats = xsel + x0, x1 = int(round(x0)), int(round(x1)) + ysel = kwargs['y_selection'] + if isinstance(ysel, list): + ycats = ysel + y0, y1 = int(round(y0)), int(round(y1)) + + # Handle invert_xaxis/invert_yaxis + if x0 > x1: + x0, x1 = x1, x0 + if y0 > y1: + y0, y1 = y1, y0 + + return (x0, x1), xcats, (y0, y1), ycats + + def _get_index_expr(self, index_cols, bbox): + get_shape = dim(self.dataset.get_dimension(index_cols[0]), np.shape) + index_cols = [dim(self.dataset.get_dimension(c), np.ravel) for c in index_cols] + sel = self.dataset.clone(datatype=['dataframe', 'dictionary']).select(**bbox) + vals = dim(index_cols[0], util.unique_zip, *index_cols[1:]).apply( + sel, expanded=True, flat=True + ) + contains = dim(index_cols[0], util.lzip, *index_cols[1:]).isin(vals, object=True) + return dim(contains, np.reshape, get_shape) + + def _get_selection_expr_for_stream_value(self, **kwargs): + from .geom import Rectangles + from .graphs import Graph + + invert_axes = self.opts.get('plot').kwargs.get('invert_axes', False) + + if kwargs.get('bounds') is None and kwargs.get('x_selection') is None: + return None, None, Rectangles([]) + + (x0, x1), xcats, (y0, y1), ycats = self._get_selection(**kwargs) + xsel = xcats or (x0, x1) + ysel = ycats or (y0, y1) + + if isinstance(self, Graph): + xdim, ydim = self.nodes.dimensions()[:2] + else: + xdim, ydim = self.dimensions()[:2] + + if invert_axes: + xdim, ydim = ydim, xdim + + bbox = {xdim.name: xsel, ydim.name: ysel} + index_cols = kwargs.get('index_cols') + if index_cols: + selection_expr = self._get_index_expr(index_cols, bbox) + region_element = None + else: + if xcats: + xexpr = dim(xdim).isin(xcats) + else: + xexpr = (dim(xdim) >= x0) & (dim(xdim) <= x1) + if ycats: + yexpr = dim(ydim).isin(ycats) + else: + yexpr = (dim(ydim) >= y0) & (dim(ydim) <= y1) + selection_expr = (xexpr & yexpr) + region_element = Rectangles([(x0, y0, x1, y1)]) + return selection_expr, bbox, region_element + + @staticmethod + def _merge_regions(region1, region2, operation): + if region1 is None or operation == "overwrite": + return region2 + return region1.clone(region1.interface.concatenate([region1, region2])) + + +class SelectionGeomExpr(Selection2DExpr): + + def _get_selection_expr_for_stream_value(self, **kwargs): + from .geom import Rectangles + + if kwargs.get('bounds') is None and kwargs.get('x_selection') is None: + return None, None, Rectangles([]) + + invert_axes = self.opts.get('plot').kwargs.get('invert_axes', False) + + (x0, x1), xcats, (y0, y1), ycats = self._get_selection(**kwargs) + xsel = xcats or (x0, x1) + ysel = ycats or (y0, y1) + + x0dim, y0dim, x1dim, y1dim = self.kdims + if invert_axes: + x0dim, x1dim, y0dim, y1dim = y0dim, y1dim, x0dim, x1dim + + bbox = {x0dim.name: xsel, y0dim.name: ysel, x1dim.name: xsel, y1dim.name: ysel} + index_cols = kwargs.get('index_cols') + if index_cols: + selection_expr = self._get_index_expr(index_cols, bbox) + region_element = None + else: + x0expr = (dim(x0dim) >= x0) & (dim(x0dim) <= x1) + y0expr = (dim(y0dim) >= y0) & (dim(y0dim) <= y1) + x1expr = (dim(x1dim) >= x0) & (dim(x1dim) <= x1) + y1expr = (dim(y1dim) >= y0) & (dim(y1dim) <= y1) + selection_expr = (x0expr & y0expr & x1expr & y1expr) + region_element = Rectangles([(x0, y0, x1, y1)]) + return selection_expr, bbox, region_element + + +class Selection1DExpr(Selection2DExpr): + """ + Mixin class for Cartesian 1D Chart elements to add basic support for + SelectionExpr streams. + """ + + _selection_dims = 1 + + _inverted_expr = False + + def _get_selection_expr_for_stream_value(self, **kwargs): + invert_axes = self.opts.get('plot').kwargs.get('invert_axes', False) + if ((invert_axes and not self._inverted_expr) or (not invert_axes and self._inverted_expr)): + region_el = HSpan + else: + region_el = VSpan + + if kwargs.get('bounds', None) is None: + region = None if 'index_cols' in kwargs else NdOverlay({0: region_el()}) + return None, None, region + + x0, y0, x1, y1 = kwargs['bounds'] + + # Handle invert_xaxis/invert_yaxis + if y0 > y1: + y0, y1 = y1, y0 + if x0 > x1: + x0, x1 = x1, x0 + + if len(self.dimensions()) == 1: + xdim = self.dimensions()[0] + ydim = None + else: + xdim, ydim = self.dimensions()[:2] + + if invert_axes: + x0, x1, y0, y1 = y0, y1, x0, x1 + cat_kwarg = 'y_selection' + else: + cat_kwarg = 'x_selection' + + if self._inverted_expr: + if ydim is not None: xdim = ydim + x0, x1 = y0, y1 + cat_kwarg = ('y' if invert_axes else 'x') + '_selection' + cats = kwargs.get(cat_kwarg) + + bbox = {xdim.name: (x0, x1)} + if cats is not None and len(self.kdims) == 1: + bbox[self.kdims[0].name] = cats + index_cols = kwargs.get('index_cols') + if index_cols: + selection_expr = self._get_index_expr(index_cols, bbox) + region_element = None + else: + if isinstance(cats, list) and xdim in self.kdims[:1]: + selection_expr = dim(xdim).isin(cats) + else: + selection_expr = ((dim(xdim) >= x0) & (dim(xdim) <= x1)) + if isinstance(cats, list) and len(self.kdims) == 1: + selection_expr &= dim(self.kdims[0]).isin(cats) + region_element = NdOverlay({0: region_el(x0, x1)}) + return selection_expr, bbox, region_element + + @staticmethod + def _merge_regions(region1, region2, operation): + if region1 is None or operation == "overwrite": + return region2 + data = [d.data for d in region1] + [d.data for d in region2] + prev = len(data) + new = None + while prev != new: + prev = len(data) + contiguous = [] + for l, u in data: + if not util.isfinite(l) or not util.isfinite(u): + continue + overlap = False + for i, (pl, pu) in enumerate(contiguous): + if l >= pl and l <= pu: + pu = max(u, pu) + overlap = True + elif u <= pu and u >= pl: + pl = min(l, pl) + overlap = True + if overlap: + contiguous[i] = (pl, pu) + if not overlap: + contiguous.append((l, u)) + new = len(contiguous) + data = contiguous + return NdOverlay([(i, region1.last.clone(l, u)) for i, (l, u) in enumerate(data)]) diff --git a/holoviews/element/stats.py b/holoviews/element/stats.py index 4f61c88fe1..5f3b82f2dd 100644 --- a/holoviews/element/stats.py +++ b/holoviews/element/stats.py @@ -4,7 +4,8 @@ from ..core.dimension import Dimension, process_dimensions from ..core.data import Dataset from ..core.element import Element, Element2D -from ..core.util import get_param_values, OrderedDict +from ..core.util import get_param_values, unique_iterator, OrderedDict +from .selection import Selection1DExpr, Selection2DExpr class StatisticsElement(Dataset, Element2D): @@ -36,6 +37,19 @@ def __init__(self, data, kdims=None, vdims=None, **params): else: self.vdims = process_dimensions(None, vdims)['vdims'] + @property + def dataset(self): + """ + The Dataset that this object was created from + """ + from . import Dataset + if self._dataset is None: + datatype = list(unique_iterator(self.datatype+Dataset.datatype)) + dataset = Dataset(self, vdims=[], datatype=datatype) + return dataset + else: + return self._dataset + def range(self, dim, data_range=True, dimension_range=True): """Return the lower and upper bounds of values along dimension. @@ -150,23 +164,23 @@ def columns(self, dimensions=None): -class Bivariate(StatisticsElement): +class Bivariate(Selection2DExpr, StatisticsElement): """ Bivariate elements are containers for two dimensional data, which is to be visualized as a kernel density estimate. The data should be supplied in a tabular format of x- and y-columns. """ + group = param.String(default="Bivariate", constant=True) + kdims = param.List(default=[Dimension('x'), Dimension('y')], bounds=(2, 2)) vdims = param.List(default=[Dimension('Density')], bounds=(0,1)) - group = param.String(default="Bivariate", constant=True) - -class Distribution(StatisticsElement): +class Distribution(Selection1DExpr, StatisticsElement): """ Distribution elements provides a representation for a one-dimensional distribution which can be visualized as a kernel @@ -174,14 +188,15 @@ class Distribution(StatisticsElement): and will use the first column. """ - kdims = param.List(default=[Dimension('Value')], bounds=(1, 1)) - group = param.String(default='Distribution', constant=True) + kdims = param.List(default=[Dimension('Value')], bounds=(1, 1)) + vdims = param.List(default=[Dimension('Density')], bounds=(0, 1)) -class BoxWhisker(Dataset, Element2D): + +class BoxWhisker(Selection1DExpr, Dataset, Element2D): """ BoxWhisker represent data as a distributions highlighting the median, mean and various percentiles. It may have a single value @@ -191,10 +206,12 @@ class BoxWhisker(Dataset, Element2D): group = param.String(default='BoxWhisker', constant=True) - kdims = param.List(default=[], bounds=(0,None)) + kdims = param.List(default=[], bounds=(0, None)) vdims = param.List(default=[Dimension('y')], bounds=(1,1)) + _inverted_expr = True + class Violin(BoxWhisker): """ @@ -207,7 +224,7 @@ class Violin(BoxWhisker): group = param.String(default='Violin', constant=True) -class HexTiles(Dataset, Element2D): +class HexTiles(Selection2DExpr, Dataset, Element2D): """ HexTiles is a statistical element with a visual representation that renders a density map of the data values as a hexagonal grid. @@ -221,3 +238,4 @@ class HexTiles(Dataset, Element2D): kdims = param.List(default=[Dimension('x'), Dimension('y')], bounds=(2, 2)) + diff --git a/holoviews/element/tabular.py b/holoviews/element/tabular.py index 06cac7e2a0..bff59938b3 100644 --- a/holoviews/element/tabular.py +++ b/holoviews/element/tabular.py @@ -4,6 +4,7 @@ from ..core import OrderedDict, Element, Dataset, Tabular from ..core.dimension import Dimension, dimension_name +from .selection import SelectionIndexExpr class ItemTable(Element): @@ -27,17 +28,14 @@ class ItemTable(Element): group = param.String(default="ItemTable", constant=True) - @property def rows(self): return len(self.vdims) - @property def cols(self): return 2 - def __init__(self, data, **params): if data is None: data = [] @@ -128,7 +126,7 @@ def cell_type(self, row, col): -class Table(Dataset, Tabular): +class Table(SelectionIndexExpr, Dataset, Tabular): """ Table is a Dataset type, which gets displayed in a tabular format and is convertible to most other Element types. diff --git a/holoviews/operation/datashader.py b/holoviews/operation/datashader.py index 83091ed888..df66ad7900 100644 --- a/holoviews/operation/datashader.py +++ b/holoviews/operation/datashader.py @@ -1346,13 +1346,12 @@ class rasterize(AggregationOperation): """ aggregator = param.ClassSelector(class_=(ds.reductions.Reduction, basestring), - default=None) + default='default') interpolation = param.ObjectSelector( - default='bilinear', objects=['linear', 'nearest', 'bilinear', None, False], doc=""" + default='default', objects=['default', 'linear', 'nearest', 'bilinear', None, False], doc=""" The interpolation method to apply during rasterization. - Defaults to linear interpolation and None and False are aliases - of each other.""") + Default depends on element type""") _transforms = [(Image, regrid), (Polygons, geometry_rasterize), @@ -1374,7 +1373,7 @@ class rasterize(AggregationOperation): (Points, aggregate), (Curve, aggregate), (Path, aggregate), - (type(None), shade) # To handles parameters of datashade + (type(None), shade) # To handle parameters of datashade ] def _process(self, element, key=None): @@ -1382,7 +1381,15 @@ def _process(self, element, key=None): all_allowed_kws = set() all_supplied_kws = set() for predicate, transform in self._transforms: - op_params = dict({k: v for k, v in self.p.items() + merged_param_values = dict(self.param.get_param_values(), **self.p) + + # If aggregator or interpolation are 'default', pop parameter so + # datashader can choose the default aggregator itself + for k in ['aggregator', 'interpolation']: + if merged_param_values.get(k, None) == 'default': + merged_param_values.pop(k) + + op_params = dict({k: v for k, v in merged_param_values.items() if not (v is None and k == 'aggregator')}, dynamic=False) extended_kws = dict(op_params, **self.p.extra_keywords()) diff --git a/holoviews/plotting/bokeh/annotation.py b/holoviews/plotting/bokeh/annotation.py index 971f36cce7..51487d733c 100644 --- a/holoviews/plotting/bokeh/annotation.py +++ b/holoviews/plotting/bokeh/annotation.py @@ -22,6 +22,7 @@ from ...element import HLine, VLine, VSpan from ..plot import GenericElementPlot from .element import AnnotationPlot, ElementPlot, CompositeElementPlot, ColorbarPlot +from .selection import BokehOverlaySelectionDisplay from .styles import text_properties, line_properties, fill_properties from .plot import BokehPlot from .util import date_to_integer @@ -70,6 +71,7 @@ def get_extents(self, element, ranges=None, range_type='combined'): + class LabelsPlot(ColorbarPlot, AnnotationPlot): show_legend = param.Boolean(default=False, doc=""" @@ -87,6 +89,8 @@ class LabelsPlot(ColorbarPlot, AnnotationPlot): allow_None=True, doc=""" Deprecated in favor of color style mapping, e.g. `color=dim('color')`""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = text_properties + ['cmap', 'angle', 'visible'] _nonvectorized_styles = ['cmap'] @@ -167,6 +171,8 @@ def get_extents(self, element, ranges=None, range_type='combined'): ranges[dim]['soft'] = loc, loc return super(LineAnnotationPlot, self).get_extents(element, ranges, range_type) + def _get_factors(self, element, ranges): + return [], [] class BoxAnnotationPlot(ElementPlot, AnnotationPlot): @@ -201,6 +207,9 @@ def _init_glyph(self, plot, mapping, properties): plot.renderers.append(box) return None, box + def _get_factors(self, element, ranges): + return [], [] + class SlopePlot(ElementPlot, AnnotationPlot): diff --git a/holoviews/plotting/bokeh/callbacks.py b/holoviews/plotting/bokeh/callbacks.py index 1758027d10..961515e497 100644 --- a/holoviews/plotting/bokeh/callbacks.py +++ b/holoviews/plotting/bokeh/callbacks.py @@ -1,25 +1,30 @@ from __future__ import absolute_import, division, unicode_literals from collections import defaultdict +from functools import partial import param import numpy as np + from bokeh.models import ( CustomJS, FactorRange, DatetimeAxis, ToolbarBox, Range1d, DataRange1d, PolyDrawTool, BoxEditTool, PolyEditTool, FreehandDrawTool, PointDrawTool ) +from panel.io.state import state from panel.callbacks import PeriodicCallback from pyviz_comms import JS_CALLBACK from ...core import OrderedDict +from ...core.options import CallbackError from ...core.util import dimension_sanitizer, isscalar, dt64_to_dt +from ...element import Table from ...streams import (Stream, PointerXY, RangeXY, Selection1D, RangeX, RangeY, PointerX, PointerY, BoundsX, BoundsY, Tap, SingleTap, DoubleTap, MouseEnter, MouseLeave, PlotSize, Draw, BoundsXY, PlotReset, BoxEdit, PointDraw, PolyDraw, PolyEdit, CDSStream, - FreehandDraw, CurveEdit) + FreehandDraw, CurveEdit, SelectionXY) from ..links import Link, RectanglesTableLink, DataLink, RangeToolLink, SelectionLink, VertexTableLink from ..plot import GenericElementPlot, GenericOverlayPlot from .util import convert_timestamp @@ -70,14 +75,18 @@ def __init__(self, plot, streams, source, **params): self.plot = plot self.streams = streams if plot.renderer.mode != 'server': + if plot.pane: + on_error = partial(plot.pane._on_error, plot.root) + else: + on_error = None self.comm = plot.renderer.comm_manager.get_client_comm(on_msg=self.on_msg) + self.comm._on_error = on_error else: self.comm = None self.source = source self.handle_ids = defaultdict(dict) self.reset() - def cleanup(self): self.reset() self.handle_ids = None @@ -137,6 +146,12 @@ def on_msg(self, msg): try: Stream.trigger(streams) + except CallbackError as e: + if self.plot.root and self.plot.root.ref['id'] in state._handles: + handle, _ = state._handles[self.plot.root.ref['id']] + handle.update({'text/html': str(e)}, raw=True) + else: + raise e except Exception as e: raise e finally: @@ -858,6 +873,51 @@ def _process_msg(self, msg): return {} +class SelectionXYCallback(BoundsCallback): + """ + Converts a bounds selection to numeric or categorical x-range + and y-range selections. + """ + + def _process_msg(self, msg): + msg = super(SelectionXYCallback, self)._process_msg(msg) + if 'bounds' not in msg: + return msg + el = self.plot.current_frame + x0, y0, x1, y1 = msg['bounds'] + x_range = self.plot.handles['x_range'] + if isinstance(x_range, FactorRange): + x0, x1 = int(round(x0)), int(round(x1)) + xfactors = x_range.factors[x0: x1] + if x_range.tags and x_range.tags[0]: + xdim = el.get_dimension(x_range.tags[0][0][0]) + if xdim and hasattr(el, 'interface'): + dtype = el.interface.dtype(el, xdim) + try: + xfactors = list(np.array(xfactors).astype(dtype)) + except: + pass + msg['x_selection'] = xfactors + else: + msg['x_selection'] = (x0, x1) + y_range = self.plot.handles['y_range'] + if isinstance(y_range, FactorRange): + y0, y1 = int(round(y0)), int(round(y1)) + yfactors = y_range.factors[y0: y1] + if y_range.tags and y_range.tags[0]: + ydim = el.get_dimension(y_range.tags[0][0][0]) + if ydim and hasattr(el, 'interface'): + dtype = el.interface.dtype(el, ydim) + try: + yfactors = list(np.array(yfactors).astype(dtype)) + except: + pass + msg['y_selection'] = yfactors + else: + msg['y_selection'] = (y0, y1) + return msg + + class BoundsXCallback(Callback): """ Returns the bounds of a xbox_select tool. @@ -912,8 +972,15 @@ class Selection1DCallback(Callback): on_changes = ['indices'] def _process_msg(self, msg): + el = self.plot.current_frame if 'index' in msg: msg = {'index': [int(v) for v in msg['index']]} + if isinstance(el, Table): + # Ensure that explicitly applied selection does not + # trigger new events + sel = el.opts.get('style').kwargs.get('selection') + if sel is not None and list(sel) == msg['index']: + return {} return self._transform(msg) else: return {} @@ -1242,6 +1309,7 @@ def initialize(self, plot_id=None): callbacks[BoundsY] = BoundsYCallback callbacks[Selection1D] = Selection1DCallback callbacks[PlotSize] = PlotSizeCallback +callbacks[SelectionXY] = SelectionXYCallback callbacks[Draw] = DrawCallback callbacks[PlotReset] = ResetCallback callbacks[CDSStream] = CDSCallback diff --git a/holoviews/plotting/bokeh/chart.py b/holoviews/plotting/bokeh/chart.py index b02c5d9131..5c0056f1f3 100644 --- a/holoviews/plotting/bokeh/chart.py +++ b/holoviews/plotting/bokeh/chart.py @@ -9,8 +9,6 @@ from bokeh.models.tools import BoxSelectTool from bokeh.transform import jitter -from ...plotting.bokeh.selection import BokehOverlaySelectionDisplay -from ...selection import NoOpSelectionDisplay from ...core.data import Dataset from ...core.dimension import dimension_name from ...core.util import ( @@ -21,6 +19,7 @@ from ..mixins import AreaMixin, BarsMixin, SpikesMixin from ..util import compute_sizes, get_min_distance from .element import ElementPlot, ColorbarPlot, LegendPlot +from .selection import BokehOverlaySelectionDisplay from .styles import (expand_batched_style, line_properties, fill_properties, mpl_to_bokeh, rgb2hex) from .util import bokeh_version, categorize_array @@ -55,14 +54,14 @@ class PointPlot(LegendPlot, ColorbarPlot): Function applied to size values before applying scaling, to remove values lower than zero.""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = (['cmap', 'palette', 'marker', 'size', 'angle', 'visible'] + line_properties + fill_properties) _plot_methods = dict(single='scatter', batched='scatter') _batched_style_opts = line_properties + fill_properties + ['size', 'marker', 'angle'] - selection_display = BokehOverlaySelectionDisplay() - def _get_size_data(self, element, ranges, style): data, mapping = {}, {} sdim = element.get_dimension(self.size_index) @@ -218,6 +217,8 @@ class VectorFieldPlot(ColorbarPlot): transforms using the magnitude option, e.g. `dim('Magnitude').norm()`.""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = line_properties + ['scale', 'cmap', 'visible'] _nonvectorized_styles = ['scale', 'cmap', 'visible'] @@ -333,11 +334,13 @@ class CurvePlot(ElementPlot): default is 'linear', other options include 'steps-mid', 'steps-pre' and 'steps-post'.""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = line_properties + ['visible'] - _nonvectorized_styles = line_properties + ['visible'] - _plot_methods = dict(single='line', batched='multi_line') _batched_style_opts = line_properties + _nonvectorized_styles = line_properties + ['visible'] + _plot_methods = dict(single='line', batched='multi_line') def get_data(self, element, ranges, style): xidx, yidx = (1, 0) if self.invert_axes else (0, 1) @@ -405,7 +408,7 @@ class HistogramPlot(ColorbarPlot): _nonvectorized_styles = ['line_dash', 'visible'] - selection_display = BokehOverlaySelectionDisplay() + selection_display = BokehOverlaySelectionDisplay(color_prop=['color', 'fill_color']) def get_data(self, element, ranges, style): if self.invert_axes: @@ -504,7 +507,10 @@ def _init_glyph(self, plot, mapping, properties): class ErrorPlot(ColorbarPlot): - style_opts = line_properties + ['lower_head', 'upper_head', 'visible'] + style_opts = ([ + p for p in line_properties if p.split('_')[0] not in + ('hover', 'selection', 'nonselection', 'muted') + ] + ['lower_head', 'upper_head', 'visible']) _nonvectorized_styles = ['line_dash', 'visible'] @@ -512,9 +518,7 @@ class ErrorPlot(ColorbarPlot): _plot_methods = dict(single=Whisker) - # selection_display should be changed to BokehOverlaySelectionDisplay - # when #3950 is fixed - selection_display = NoOpSelectionDisplay() + selection_display = BokehOverlaySelectionDisplay() def get_data(self, element, ranges, style): mapping = dict(self._mapping) @@ -567,12 +571,12 @@ class SpreadPlot(ElementPlot): padding = param.ClassSelector(default=(0, 0.1), class_=(int, float, tuple)) + selection_display = BokehOverlaySelectionDisplay() + style_opts = line_properties + fill_properties + ['visible'] _no_op_style = style_opts - _plot_methods = dict(single='patch') - _stream_data = False # Plot does not support streaming data def _split_area(self, xs, lower, upper): @@ -623,6 +627,8 @@ class AreaPlot(AreaMixin, SpreadPlot): padding = param.ClassSelector(default=(0, 0.1), class_=(int, float, tuple)) + selection_display = BokehOverlaySelectionDisplay() + _stream_data = False # Plot does not support streaming data def get_data(self, element, ranges, style): @@ -661,12 +667,12 @@ class SpikesPlot(SpikesMixin, ColorbarPlot): allow_None=True, doc=""" Deprecated in favor of color style mapping, e.g. `color=dim('color')`""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = (['color', 'cmap', 'palette', 'visible'] + line_properties) _plot_methods = dict(single='segment') - selection_display = BokehOverlaySelectionDisplay() - def _get_axis_dims(self, element): if 'spike_length' in self.lookup_options(element, 'plot').options: return [element.dimensions()[0], None, None] @@ -748,6 +754,8 @@ class BarPlot(BarsMixin, ColorbarPlot, LegendPlot): allow_None=True, doc=""" Deprecated in favor of color style mapping, e.g. `color=dim('color')`""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = (line_properties + fill_properties + ['width', 'bar_width', 'cmap', 'visible']) @@ -759,8 +767,6 @@ class BarPlot(BarsMixin, ColorbarPlot, LegendPlot): # Declare that y-range should auto-range if not bounded _y_range_type = Range1d - selection_display = BokehOverlaySelectionDisplay() - def _axis_properties(self, axis, key, plot, dimension=None, ax_mapping={'x': 0, 'y': 1}): props = super(BarPlot, self)._axis_properties(axis, key, plot, dimension, ax_mapping) diff --git a/holoviews/plotting/bokeh/element.py b/holoviews/plotting/bokeh/element.py index ea46a7edac..b389ce969b 100644 --- a/holoviews/plotting/bokeh/element.py +++ b/holoviews/plotting/bokeh/element.py @@ -1323,6 +1323,14 @@ def _update_glyphs(self, element, ranges, style): else: data, mapping, style = self.get_data(element, ranges, style) + # Include old data if source static + if self.static_source: + for k, v in source.data.items(): + if k not in data: + data[k] = v + elif not len(data[k]) and len(source.data): + data[k] = source.data[k] + with abbreviated_exception(): style = self._apply_transforms(element, data, ranges, style) @@ -1368,7 +1376,10 @@ def update_frame(self, key, ranges=None, plot=None, element=None): style = self.lookup_options(style_element, 'style') self.style = style.max_cycles(max_cycles) if max_cycles else style - ranges = self.compute_ranges(self.hmap, key, ranges) + if not self.overlaid: + ranges = self.compute_ranges(self.hmap, key, ranges) + else: + self.ranges.update(ranges) self.param.set_param(**self.lookup_options(style_element, 'plot').options) ranges = util.match_spec(style_element, ranges) self.current_ranges = ranges diff --git a/holoviews/plotting/bokeh/geometry.py b/holoviews/plotting/bokeh/geometry.py index 605e9b2715..6f8f4204bb 100644 --- a/holoviews/plotting/bokeh/geometry.py +++ b/holoviews/plotting/bokeh/geometry.py @@ -4,6 +4,7 @@ from ..mixins import GeomMixin from .element import ColorbarPlot, LegendPlot +from .selection import BokehOverlaySelectionDisplay from .styles import line_properties, fill_properties @@ -12,6 +13,9 @@ class SegmentPlot(GeomMixin, ColorbarPlot): Segments are lines in 2D space where each two each dimensions specify a (x, y) node of the line. """ + + selection_display = BokehOverlaySelectionDisplay() + style_opts = line_properties + ['cmap'] _nonvectorized_styles = ['cmap'] @@ -25,10 +29,14 @@ def get_data(self, element, ranges, style): mapping = dict(x0='x0', x1='x1', y0='y0', y1='y1') return (data, mapping, style) + def _get_factors(self, element, ranges): + return [], [] class RectanglesPlot(GeomMixin, LegendPlot, ColorbarPlot): + selection_display = BokehOverlaySelectionDisplay() + style_opts = ['cmap', 'visible'] + line_properties + fill_properties _plot_methods = dict(single='rect') _batched_style_opts = line_properties + fill_properties @@ -43,4 +51,5 @@ def get_data(self, element, ranges, style): mapping = {'x': 'x', 'y': 'y', 'width': 'width', 'height': 'height'} return data, mapping, style - + def _get_factors(self, element, ranges): + return [], [] diff --git a/holoviews/plotting/bokeh/heatmap.py b/holoviews/plotting/bokeh/heatmap.py index 10c48fca77..0a243dfc65 100644 --- a/holoviews/plotting/bokeh/heatmap.py +++ b/holoviews/plotting/bokeh/heatmap.py @@ -9,6 +9,7 @@ from ...core.util import is_nan, dimension_sanitizer from ...core.spaces import HoloMap from .element import ColorbarPlot, CompositeElementPlot +from .selection import BokehOverlaySelectionDisplay from .styles import line_properties, fill_properties, text_properties @@ -56,6 +57,8 @@ class HeatMapPlot(ColorbarPlot): ['ymarks_' + p for p in line_properties] + ['cmap', 'color', 'dilate', 'visible'] + line_properties + fill_properties) + selection_display = BokehOverlaySelectionDisplay() + @classmethod def is_radial(cls, heatmap): heatmap = heatmap.last if isinstance(heatmap, HoloMap) else heatmap @@ -72,6 +75,8 @@ def get_data(self, element, ranges, style): cmapper = self._get_colormapper(element.vdims[0], element, ranges, style) if 'line_alpha' not in style and 'line_width' not in style: style['line_alpha'] = 0 + style['selection_line_alpha'] = 0 + style['nonselection_line_alpha'] = 0 elif 'line_color' not in style: style['line_color'] = 'white' diff --git a/holoviews/plotting/bokeh/hex_tiles.py b/holoviews/plotting/bokeh/hex_tiles.py index 2d52eb0f06..091dc89ee8 100644 --- a/holoviews/plotting/bokeh/hex_tiles.py +++ b/holoviews/plotting/bokeh/hex_tiles.py @@ -14,6 +14,7 @@ from ...element import HexTiles from ...util.transform import dim from .element import ColorbarPlot +from .selection import BokehOverlaySelectionDisplay from .styles import line_properties, fill_properties @@ -135,19 +136,22 @@ class HexTilesPlot(ColorbarPlot): allow_None=True, doc=""" Index of the dimension from which the sizes will the drawn.""") - style_opts = ['cmap', 'color', 'scale', 'visible'] + line_properties + fill_properties + selection_display = BokehOverlaySelectionDisplay() - _plot_methods = dict(single='hex_tile') + style_opts = ['cmap', 'color', 'scale', 'visible'] + line_properties + fill_properties _nonvectorized_styles = ['cmap', 'line_dash'] + _plot_methods = dict(single='hex_tile') def get_extents(self, element, ranges, range_type='combined'): xdim, ydim = element.kdims[:2] ranges[xdim.name]['data'] = xdim.range ranges[ydim.name]['data'] = ydim.range xdim, ydim = element.dataset.kdims[:2] - ranges[xdim.name]['hard'] = xdim.range - ranges[ydim.name]['hard'] = ydim.range + if xdim.name in ranges: + ranges[xdim.name]['hard'] = xdim.range + if ydim.name in ranges: + ranges[ydim.name]['hard'] = ydim.range return super(HexTilesPlot, self).get_extents(element, ranges, range_type) def _hover_opts(self, element): diff --git a/holoviews/plotting/bokeh/path.py b/holoviews/plotting/bokeh/path.py index 5c0e929856..aca9b10529 100644 --- a/holoviews/plotting/bokeh/path.py +++ b/holoviews/plotting/bokeh/path.py @@ -10,6 +10,7 @@ from ...util.transform import dim from .callbacks import PolyDrawCallback, PolyEditCallback from .element import ColorbarPlot, LegendPlot +from .selection import BokehOverlaySelectionDisplay from .styles import (expand_batched_style, line_properties, fill_properties, mpl_to_bokeh, validate) from .util import bokeh_version, multi_polygons_data @@ -290,3 +291,5 @@ class PolygonPlot(ContourPlot): _plot_methods = dict(single='patches', batched='patches') _batched_style_opts = line_properties + fill_properties _color_style = 'fill_color' + + selection_display = BokehOverlaySelectionDisplay() diff --git a/holoviews/plotting/bokeh/plot.py b/holoviews/plotting/bokeh/plot.py index 92ecd921a9..234c9dab37 100644 --- a/holoviews/plotting/bokeh/plot.py +++ b/holoviews/plotting/bokeh/plot.py @@ -560,7 +560,7 @@ def _create_subplots(self, layout, ranges): else: subplot = plotting_class(view, dimensions=self.dimensions, show_title=False, subplot=True, - renderer=self.renderer, + renderer=self.renderer, root=self.root, ranges=frame_ranges, uniform=self.uniform, keys=self.keys, **dict(opts, **kwargs)) collapsed_layout[coord] = (subplot.layout @@ -808,7 +808,7 @@ def _create_subplots(self, layout, positions, layout_dimensions, ranges, num=0): subplot = plot_type(element, keys=self.keys, dimensions=self.dimensions, layout_dimensions=layout_dimensions, - ranges=ranges, subplot=True, + ranges=ranges, subplot=True, root=self.root, uniform=self.uniform, layout_num=num, renderer=self.renderer, **dict({'shared_axes': self.shared_axes}, diff --git a/holoviews/plotting/bokeh/raster.py b/holoviews/plotting/bokeh/raster.py index 625e55dd46..5f9627013f 100644 --- a/holoviews/plotting/bokeh/raster.py +++ b/holoviews/plotting/bokeh/raster.py @@ -8,6 +8,7 @@ from ...core.util import cartesian_product, dimension_sanitizer, isfinite from ...element import Raster from .element import ElementPlot, ColorbarPlot +from .selection import BokehOverlaySelectionDisplay from .styles import line_properties, fill_properties, mpl_to_bokeh from .util import colormesh @@ -27,6 +28,8 @@ class RasterPlot(ColorbarPlot): _plot_methods = dict(single='image') + selection_display = BokehOverlaySelectionDisplay() + def _hover_opts(self, element): xdim, ydim = element.kdims tooltips = [(xdim.pprint_label, '$x'), (ydim.pprint_label, '$y')] @@ -130,6 +133,8 @@ class RGBPlot(ElementPlot): _plot_methods = dict(single='image_rgba') + selection_display = BokehOverlaySelectionDisplay() + def _hover_opts(self, element): xdim, ydim = element.kdims return [(xdim.pprint_label, '$x'), (ydim.pprint_label, '$y'), @@ -203,6 +208,8 @@ class QuadMeshPlot(ColorbarPlot): show_legend = param.Boolean(default=False, doc=""" Whether to show legend for the plot.""") + selection_display = BokehOverlaySelectionDisplay() + style_opts = ['cmap', 'color', 'visible'] + line_properties + fill_properties _nonvectorized_styles = style_opts diff --git a/holoviews/plotting/bokeh/selection.py b/holoviews/plotting/bokeh/selection.py index 35bdd58180..1763be53e5 100644 --- a/holoviews/plotting/bokeh/selection.py +++ b/holoviews/plotting/bokeh/selection.py @@ -1,30 +1,95 @@ -from ...selection import OverlaySelectionDisplay +import numpy as np + from ...core.options import Store +from ...core.overlay import NdOverlay +from ...selection import OverlaySelectionDisplay, SelectionDisplay + + +class TabularSelectionDisplay(SelectionDisplay): + + def _build_selection(self, el, exprs, **kwargs): + opts = {} + if exprs[1]: + opts['selected'] = np.where(exprs[1].apply(el.dataset, expanded=True, flat=True))[0] + return el.opts(clone=True, backend='bokeh', **opts) + + def build_selection(self, selection_streams, hvobj, operations, region_stream=None): + sel_streams = [selection_streams.exprs_stream] + hvobj = hvobj.apply(self._build_selection, streams=sel_streams, per_element=True) + for op in operations: + hvobj = op(hvobj) + return hvobj class BokehOverlaySelectionDisplay(OverlaySelectionDisplay): """ Overlay selection display subclass for use with bokeh backend """ - def _build_element_layer( - self, element, layer_color, selection_expr=True - ): - element, visible = self._select(element, selection_expr) + def _build_element_layer(self, element, layer_color, layer_alpha, **opts): backend_options = Store.options(backend='bokeh') - style_options = backend_options[(type(element).name,)]['style'] + el_name = type(element).name + style_options = backend_options[(el_name,)]['style'] + allowed = style_options.allowed_keywords + + merged_opts = {opt_name: layer_alpha for opt_name in allowed + if 'alpha' in opt_name} + if el_name in ('HeatMap', 'QuadMesh'): + merged_opts = {k: v for k, v in merged_opts.items() if 'line_' not in k} + elif layer_color is None: + # Keep current color (including color from cycle) + for color_prop in self.color_props: + current_color = element.opts.get(group="style")[0].get(color_prop, None) + if current_color: + merged_opts.update({color_prop: current_color}) + else: + # set color + merged_opts.update(self._get_color_kwarg(layer_color)) - def alpha_opts(alpha): - options = dict() + for opt in ('cmap', 'colorbar'): + if opt in opts and opt in allowed: + merged_opts[opt] = opts[opt] - for opt_name in style_options.allowed_keywords: - if 'alpha' in opt_name: - options[opt_name] = alpha + filtered = {k: v for k, v in merged_opts.items() if k in allowed} + return element.opts(backend='bokeh', clone=True, tools=['box_select'], + **filtered) - return options + def _style_region_element(self, region_element, unselected_color): + from ..util import linear_gradient + backend_options = Store.options(backend="bokeh") + if isinstance(region_element, NdOverlay): + element_name = type(region_element.last).name + else: + element_name = type(region_element).name + style_options = backend_options[(element_name,)]['style'] + allowed = style_options.allowed_keywords + options = {} + for opt_name in allowed: + if 'alpha' in opt_name: + options[opt_name] = 1.0 - layer_alpha = 1.0 if visible else 0.0 - merged_opts = dict(self._get_color_kwarg(layer_color), **alpha_opts(layer_alpha)) - layer_element = element.options(tools=['box_select'], **merged_opts) + if element_name != "Histogram": + # Darken unselected color + if unselected_color: + region_color = linear_gradient(unselected_color, "#000000", 9)[3] + options["color"] = region_color + if element_name == 'Rectangles': + options["line_width"] = 1 + options["fill_alpha"] = 0 + options["selection_fill_alpha"] = 0 + options["nonselection_fill_alpha"] = 0 + elif "Span" in element_name: + unselected_color = unselected_color or "#e6e9ec" + region_color = linear_gradient(unselected_color, "#000000", 9)[1] + options["color"] = region_color + options["fill_alpha"] = 0.2 + options["selection_fill_alpha"] = 0.2 + options["nonselection_fill_alpha"] = 0.2 + else: + # Darken unselected color slightly + unselected_color = unselected_color or "#e6e9ec" + region_color = linear_gradient(unselected_color, "#000000", 9)[1] + options["fill_color"] = region_color + options["color"] = region_color - return layer_element + return region_element.opts(element_name, backend='bokeh', clone=True, **options) diff --git a/holoviews/plotting/bokeh/stats.py b/holoviews/plotting/bokeh/stats.py index 7c361c5c26..2f92930775 100644 --- a/holoviews/plotting/bokeh/stats.py +++ b/holoviews/plotting/bokeh/stats.py @@ -89,7 +89,7 @@ class BoxWhiskerPlot(CompositeElementPlot, ColorbarPlot, LegendPlot): _stream_data = False # Plot does not support streaming data - selection_display = BokehOverlaySelectionDisplay() + selection_display = BokehOverlaySelectionDisplay(color_prop='box_color') def get_extents(self, element, ranges, range_type='combined'): return super(BoxWhiskerPlot, self).get_extents( diff --git a/holoviews/plotting/bokeh/tabular.py b/holoviews/plotting/bokeh/tabular.py index 8d14fd69c9..d036d566a5 100644 --- a/holoviews/plotting/bokeh/tabular.py +++ b/holoviews/plotting/bokeh/tabular.py @@ -14,6 +14,7 @@ from ...core.util import dimension_sanitizer, isdatetime from ..plot import GenericElementPlot from .plot import BokehPlot +from .selection import TabularSelectionDisplay class TablePlot(BokehPlot, GenericElementPlot): @@ -30,7 +31,9 @@ class TablePlot(BokehPlot, GenericElementPlot): width = param.Number(default=400) - style_opts = ['row_headers', 'selectable', 'editable', + selection_display = TabularSelectionDisplay() + + style_opts = ['row_headers', 'selectable', 'selected', 'editable', 'sortable', 'fit_columns', 'scroll_to_selection', 'index_position', 'visible'] @@ -66,6 +69,8 @@ def initialize_plot(self, ranges=None, plot=None, plots=None, source=None): source = self._init_datasource(data) self.handles['source'] = self.handles['cds'] = source self.handles['selected'] = source.selected + if 'selected' in style: + source.selected.indices = list(style['selected']) columns = self._get_columns(element, data) style['reorderable'] = False @@ -135,4 +140,6 @@ def update_frame(self, key, ranges=None, plot=None): data, _, style = self.get_data(element, ranges, style) columns = self._get_columns(element, data) self.handles['table'].columns = columns + if 'selected' in style: + source.selected.indices = list(style['selected']) self._update_datasource(source, data) diff --git a/holoviews/plotting/plot.py b/holoviews/plotting/plot.py index 65166eac22..a8e803403f 100644 --- a/holoviews/plotting/plot.py +++ b/holoviews/plotting/plot.py @@ -6,16 +6,20 @@ from __future__ import absolute_import import threading +import uuid import warnings -from itertools import groupby, product from collections import Counter, defaultdict +from functools import partial +from itertools import groupby, product import numpy as np import param +from panel.config import config from panel.io.notebook import push from panel.io.state import state +from pyviz_comms import JupyterComm from ..selection import NoOpSelectionDisplay from ..core import OrderedDict @@ -118,19 +122,33 @@ def document(self, doc): if plot is not None: plot.document = doc - @property def pane(self): return self._pane @pane.setter def pane(self, pane): + if (config.console_output != 'disable' and self.root and + self.root.ref['id'] not in state._handles and + isinstance(self.comm, JupyterComm)): + from IPython.display import display + handle = display(display_id=uuid.uuid4().hex) + state._handles[self.root.ref['id']] = (handle, []) + self._pane = pane if self.subplots: for plot in self.subplots.values(): if plot is not None: plot.pane = pane - + if not plot.root: + continue + for cb in getattr(plot, 'callbacks', []): + if hasattr(pane, '_on_error') and cb.comm: + cb.comm._on_error = partial(pane._on_error, plot.root.ref['id']) + elif self.root: + for cb in getattr(self, 'callbacks', []): + if hasattr(pane, '_on_error') and cb.comm: + cb.comm._on_error = partial(pane._on_error, self.root.ref['id']) @property def comm(self): @@ -286,6 +304,9 @@ def __init__(self, selector, plot_classes, allow_mismatch=False): interface = self._define_interface(self.plot_classes.values(), allow_mismatch) self.style_opts, self.plot_options = interface + def selection_display(self, obj): + plt_class = self.get_plot_class(obj) + return getattr(plt_class, 'selection_display', None) def _define_interface(self, plots, allow_mismatch): parameters = [{k:v.precedence for k,v in plot.param.params().items() @@ -843,7 +864,7 @@ def _get_projection(cls, obj): [CompositeOverlay, Element], keyfn=isoverlay) from_overlay = not all(p is None for p in opts.get(True, {}).get('projection', [])) - projections = opts.get(from_overlay).get('projection', []) + projections = opts.get(from_overlay, {}).get('projection', []) custom_projs = [p for p in projections if p is not None] if len(set(custom_projs)) > 1: raise Exception("An axis may only be assigned one projection type") @@ -1580,7 +1601,7 @@ def _create_subplot(self, key, obj, streams, ranges): renderer=self.renderer, adjoined=self.adjoined, stream_sources=self.stream_sources, projection=self.projection, fontscale=self.fontscale, - zorder=zorder, **passed_handles) + zorder=zorder, root=self.root, **passed_handles) return plottype(obj, **plotopts) diff --git a/holoviews/plotting/plotly/callbacks.py b/holoviews/plotting/plotly/callbacks.py index 0d136214c6..cffcad09e2 100644 --- a/holoviews/plotting/plotly/callbacks.py +++ b/holoviews/plotting/plotly/callbacks.py @@ -3,7 +3,8 @@ from param.parameterized import add_metaclass from ...streams import ( - Stream, Selection1D, RangeXY, RangeX, RangeY, BoundsXY, BoundsX, BoundsY + Stream, Selection1D, RangeXY, RangeX, RangeY, BoundsXY, BoundsX, BoundsY, + SelectionXY ) from .util import _trace_to_subplot @@ -206,6 +207,7 @@ class RangeYCallback(RangeCallback): callbacks = Stream._callbacks['plotly'] callbacks[Selection1D] = Selection1DCallback +callbacks[SelectionXY] = BoundsXYCallback callbacks[BoundsXY] = BoundsXYCallback callbacks[BoundsX] = BoundsXCallback callbacks[BoundsY] = BoundsYCallback diff --git a/holoviews/plotting/plotly/chart.py b/holoviews/plotting/plotly/chart.py index 013a21d8d3..6e08d7e8e0 100644 --- a/holoviews/plotting/plotly/chart.py +++ b/holoviews/plotting/plotly/chart.py @@ -271,7 +271,10 @@ def get_data(self, element, ranges, style): ydim = element.vdims[0] values = element.interface.coords(element, ydim) edges = element.interface.coords(element, xdim) - binwidth = edges[1] - edges[0] + if len(edges) < 2: + binwidth = 0 + else: + binwidth = edges[1] - edges[0] if self.invert_axes: ys = edges diff --git a/holoviews/plotting/plotly/chart3d.py b/holoviews/plotting/plotly/chart3d.py index 1ea55866a7..986ee4156c 100644 --- a/holoviews/plotting/plotly/chart3d.py +++ b/holoviews/plotting/plotly/chart3d.py @@ -44,6 +44,8 @@ class SurfacePlot(Chart3DPlot, ColorbarPlot): style_opts = ['visible', 'alpha', 'lighting', 'lightposition', 'cmap'] + selection_display = PlotlyOverlaySelectionDisplay(supports_region=False) + def graph_options(self, element, ranges, style): opts = super(SurfacePlot, self).graph_options(element, ranges, style) copts = self.get_color_opts(element.vdims[0], element, ranges, style) @@ -89,6 +91,8 @@ class TriSurfacePlot(Chart3DPlot, ColorbarPlot): style_opts = ['cmap', 'edges_color', 'facecolor'] + selection_display = PlotlyOverlaySelectionDisplay(supports_region=False) + def get_data(self, element, ranges, style): try: from scipy.spatial import Delaunay diff --git a/holoviews/plotting/plotly/selection.py b/holoviews/plotting/plotly/selection.py index ad3904086d..22d6b0d46c 100644 --- a/holoviews/plotting/plotly/selection.py +++ b/holoviews/plotting/plotly/selection.py @@ -7,20 +7,66 @@ class PlotlyOverlaySelectionDisplay(OverlaySelectionDisplay): """ Overlay selection display subclass for use with plotly backend """ - def _build_element_layer( - self, element, layer_color, selection_expr=True - ): - element, visible = self._select(element, selection_expr) + def _build_element_layer(self, element, layer_color, layer_alpha, **opts): backend_options = Store.options(backend='plotly') style_options = backend_options[(type(element).name,)]['style'] + allowed = style_options.allowed_keywords - if 'selectedpoints' in style_options.allowed_keywords: + if 'selectedpoints' in allowed: shared_opts = dict(selectedpoints=False) else: shared_opts = dict() - merged_opts = dict(self._get_color_kwarg(layer_color), **shared_opts) - layer_element = element.options(visible=visible, **merged_opts) + merged_opts = dict(shared_opts) - return layer_element + if 'opacity' in allowed: + merged_opts['opacity'] = layer_alpha + elif 'alpha' in allowed: + merged_opts['alpha'] = layer_alpha + + if layer_color is not None: + # set color + merged_opts.update(self._get_color_kwarg(layer_color)) + else: + # Keep current color (including color from cycle) + for color_prop in self.color_props: + current_color = element.opts.get(group="style")[0].get(color_prop, None) + if current_color: + merged_opts.update({color_prop: current_color}) + + for opt in ('cmap', 'colorbar'): + if opt in opts and opt in allowed: + merged_opts[opt] = opts[opt] + + filtered = {k: v for k, v in merged_opts.items() if k in allowed} + return element.opts(clone=True, backend='plotly', **filtered) + + def _style_region_element(self, region_element, unselected_color): + from ..util import linear_gradient + backend_options = Store.options(backend="plotly") + element_name = type(region_element).name + style_options = backend_options[(type(region_element).name,)]['style'] + allowed_keywords = style_options.allowed_keywords + options = {} + + if element_name != "Histogram": + # Darken unselected color + if unselected_color: + region_color = linear_gradient(unselected_color, "#000000", 9)[3] + if "line_width" in allowed_keywords: + options["line_width"] = 1 + else: + # Darken unselected color slightly + unselected_color = unselected_color or "#e6e9ec" + region_color = linear_gradient(unselected_color, "#000000", 9)[1] + + if "color" in allowed_keywords and unselected_color: + options["color"] = region_color + elif "line_color" in allowed_keywords and unselected_color: + options["line_color"] = region_color + + if "selectedpoints" in allowed_keywords: + options["selectedpoints"] = False + + return region_element.opts(clone=True, backend='plotly', **options) diff --git a/holoviews/plotting/plotly/tabular.py b/holoviews/plotting/plotly/tabular.py index 099b2d7452..b9d12ad65a 100644 --- a/holoviews/plotting/plotly/tabular.py +++ b/holoviews/plotting/plotly/tabular.py @@ -18,7 +18,7 @@ class TablePlot(ElementPlot): _style_key = 'cells' - selection_display = ColorListSelectionDisplay(color_prop='fill') + selection_display = ColorListSelectionDisplay(color_prop='fill', backend='plotly') def get_data(self, element, ranges, style): header = dict(values=[d.pprint_label for d in element.dimensions()]) diff --git a/holoviews/plotting/util.py b/holoviews/plotting/util.py index c0c83a1e80..cbd8e5d873 100644 --- a/holoviews/plotting/util.py +++ b/holoviews/plotting/util.py @@ -12,7 +12,7 @@ from ..core import (HoloMap, DynamicMap, CompositeOverlay, Layout, Overlay, GridSpace, NdLayout, NdOverlay) -from ..core.options import Cycle +from ..core.options import CallbackError, Cycle from ..core.ndmapping import item_check from ..core.spaces import get_nested_streams from ..core.util import (match_spec, wrap_tuple, basestring, get_overlay_spec, @@ -273,7 +273,7 @@ def get_plot_frame(map_obj, key_map, cached=False): return map_obj[key] except KeyError: return None - except StopIteration as e: + except (StopIteration, CallbackError) as e: raise e except Exception: print(traceback.format_exc()) @@ -1075,8 +1075,8 @@ def dim_range_key(eldim): """ if isinstance(eldim, dim): dim_name = repr(eldim) - if dim_name.startswith("'") and dim_name.endswith("'"): - dim_name = dim_name[1:-1] + if dim_name.startswith("dim('") and dim_name.endswith("')"): + dim_name = dim_name[5:-2] else: dim_name = eldim.name return dim_name diff --git a/holoviews/selection.py b/holoviews/selection.py index 39292dae2e..b34968d43a 100644 --- a/holoviews/selection.py +++ b/holoviews/selection.py @@ -1,28 +1,35 @@ from collections import namedtuple -import copy + import numpy as np import param from param.parameterized import bothmethod -from .core import OperationCallable, Overlay +from .core.dimension import OrderedDict from .core.element import Element, Layout -from .core.options import Store -from .streams import SelectionExpr, Stream +from .core.options import CallbackError, Store +from .core.overlay import NdOverlay, Overlay +from .core.spaces import GridSpace +from .streams import SelectionExpr, PlotReset, Stream from .operation.element import function -from .util import Dynamic, DynamicMap -from .plotting.util import initialize_dynamic, linear_gradient +from .util import DynamicMap +from .util.transform import dim + + +class _Cmap(Stream): + cmap = param.Parameter(default=None, allow_None=True) + -_Cmap = Stream.define('Cmap', cmap=[]) -_Alpha = Stream.define('Alpha', alpha=1.0) _Exprs = Stream.define('Exprs', exprs=[]) -_Colors = Stream.define('Colors', colors=[]) +_Styles = Stream.define('Styles', colors=[], alpha=1.) +_RegionElement = Stream.define("RegionElement", region_element=None) + _SelectionStreams = namedtuple( - 'SelectionStreams', 'colors_stream exprs_stream cmap_streams alpha_streams' + 'SelectionStreams', + 'style_stream exprs_stream cmap_streams ' ) - class _base_link_selections(param.ParameterizedFunction): """ Baseclass for linked selection functions. @@ -35,16 +42,21 @@ class _base_link_selections(param.ParameterizedFunction): subclasses to control whether new selections override prior selections or whether they are combined with prior selections """ + @bothmethod def instance(self_or_cls, **params): inst = super(_base_link_selections, self_or_cls).instance(**params) # Init private properties inst._selection_expr_streams = [] + inst._reset_streams = [] # Init selection streams inst._selection_streams = self_or_cls._build_selection_streams(inst) + # Init dict of region streams + inst._region_streams = {} + return inst def _register(self, hvobj): @@ -52,12 +64,24 @@ def _register(self, hvobj): Register an Element of DynamicMap that may be capable of generating selection expressions in response to user interaction events """ - expr_stream = SelectionExpr(source=hvobj) + # Create stream that produces element that displays region of selection + if getattr(hvobj, "_selection_streams", ()): + self._region_streams[hvobj] = _RegionElement() + + # Create SelectionExpr stream + expr_stream = SelectionExpr(source=hvobj, index_cols=self.index_cols) expr_stream.add_subscriber( lambda **kwargs: self._expr_stream_updated(hvobj, **kwargs) ) self._selection_expr_streams.append(expr_stream) + # Create PlotReset stream + reset_stream = PlotReset(source=hvobj) + reset_stream.add_subscriber( + lambda **kwargs: setattr(self, 'selection_expr', None) + ) + self._reset_streams.append(reset_stream) + def __call__(self, hvobj, **kwargs): # Apply kwargs as params self.param.set_param(**kwargs) @@ -67,82 +91,59 @@ def __call__(self, hvobj, **kwargs): return hvobj_selection - def _selection_transform( - self, - hvobj, - operations=(), - ): + def _selection_transform(self, hvobj, operations=()): """ Transform an input HoloViews object into a dynamic object with linked selections enabled. """ + from .plotting.util import initialize_dynamic if isinstance(hvobj, DynamicMap): - initialize_dynamic(hvobj) - - if len(hvobj.callback.inputs) == 1 and hvobj.callback.operation: - child_hvobj = hvobj.callback.inputs[0] - if isinstance(hvobj.callback, OperationCallable): - next_op = hvobj.callback.operation + callback = hvobj.callback + ninputs = len(callback.inputs) + if ninputs == 1: + child_hvobj = callback.inputs[0] + if callback.operation: + next_op = {'op': callback.operation, 'kwargs': callback.operation_kwargs} else: - fn = hvobj.callback.operation - next_op = function.instance(fn=fn) + fn = function.instance(fn=callback.callable) + next_op = {'op': fn, 'kwargs': callback.operation_kwargs} new_operations = (next_op,) + operations + return self._selection_transform(child_hvobj, new_operations) + elif ninputs == 2: + return Overlay([self._selection_transform(el) + for el in hvobj.callback.inputs]).collate() - # Recurse on child with added operation - return self._selection_transform( - hvobj=child_hvobj, - operations=new_operations, - ) - elif hvobj.type == Overlay and not hvobj.streams: - # Process overlay inputs individually and then overlay again - overlay_elements = hvobj.callback.inputs - new_hvobj = self._selection_transform(overlay_elements[0]) - for overlay_element in overlay_elements[1:]: - new_hvobj = new_hvobj * self._selection_transform(overlay_element) - - return new_hvobj - elif issubclass(hvobj.type, Element): + initialize_dynamic(hvobj) + if issubclass(hvobj.type, Element): self._register(hvobj) - chart = Store.registry[Store.current_backend][hvobj.type] - return chart.selection_display.build_selection( - self._selection_streams, hvobj, operations + return chart.selection_display(hvobj).build_selection( + self._selection_streams, hvobj, operations, + self._region_streams.get(hvobj, None), ) else: # This is a DynamicMap that we don't know how to recurse into. return hvobj - elif isinstance(hvobj, Element): - element = hvobj.clone(link=False) - # Register hvobj to receive selection expression callbacks - self._register(element) - - chart = Store.registry[Store.current_backend][type(element)] - try: - return chart.selection_display.build_selection( - self._selection_streams, element, operations - ) - except AttributeError: - # In case chart doesn't have selection_display defined - return element - - elif isinstance(hvobj, (Layout, Overlay)): - new_hvobj = hvobj.clone(shared_data=False) - for k, v in hvobj.items(): - new_hvobj[k] = self._selection_transform( - v, operations + chart = Store.registry[Store.current_backend][type(hvobj)] + if getattr(chart, 'selection_display', None): + element = hvobj.clone(link=False) + self._register(element) + return chart.selection_display(element).build_selection( + self._selection_streams, element, operations, + self._region_streams.get(element, None), ) - - # collate if available. Needed for Overlay - try: + return hvobj + elif isinstance(hvobj, (Layout, Overlay, NdOverlay, GridSpace)): + data = OrderedDict([(k, self._selection_transform(v, operations)) + for k, v in hvobj.items()]) + new_hvobj = hvobj.clone(data) + if hasattr(new_hvobj, 'collate'): new_hvobj = new_hvobj.collate() - except AttributeError: - pass - return new_hvobj else: - # Unsupported object + # Unsupported object return hvobj @classmethod @@ -153,7 +154,7 @@ def _build_selection_streams(cls, inst): """ raise NotImplementedError() - def _expr_stream_updated(self, hvobj, selection_expr, bbox): + def _expr_stream_updated(self, hvobj, selection_expr, bbox, region_element): """ Called when one of the registered HoloViews objects produces a new selection expression. Subclasses should override this method, and @@ -169,15 +170,62 @@ def _expr_stream_updated(self, hvobj, selection_expr, bbox): class link_selections(_base_link_selections): - selection_expr = param.Parameter(default=None) - unselected_color = param.Color(default="#99a6b2") # LightSlateGray - 65% - selected_color = param.Color(default="#DC143C") # Crimson + """ + Operation which automatically links selections between elements + in the supplied HoloViews object. Can be used a single time or + be used as an instance to apply the linked selections across + multiple objects. + """ + + cross_filter_mode = param.Selector( + ['overwrite', 'intersect'], default='intersect', doc=""" + Determines how to combine selections across different + elements.""") + + index_cols = param.List(default=None, doc=""" + If provided, selection switches to index mode where all queries + are expressed solely in terms of discrete values along the + index_cols. All Elements given to link_selections must define the index_cols, either as explicit dimensions or by sharing an underlying Dataset that defines them.""") + + selection_expr = param.Parameter(default=None, doc=""" + dim expression of the current selection or None to indicate + that everything is selected.""") + + selected_color = param.Color(default=None, allow_None=True, doc=""" + Color of selected data, or None to use the original color of + each element.""") + + selection_mode = param.Selector( + ['overwrite', 'intersect', 'union', 'inverse'], default='overwrite', doc=""" + Determines how to combine successive selections on the same + element.""") + + show_regions = param.Boolean(default=True, doc=""" + Whether to highlight the selected regions.""") + + unselected_alpha = param.Magnitude(default=0.1, doc=""" + Alpha of unselected data.""") + + unselected_color = param.Color(default=None, doc=""" + Color of unselected data.""") + + @bothmethod + def instance(self_or_cls, **params): + inst = super(link_selections, self_or_cls).instance(**params) + + # Initialize private properties + inst._obj_selections = {} + inst._obj_regions = {} + inst._reset_regions = True + + return inst @classmethod def _build_selection_streams(cls, inst): # Colors stream - colors_stream = _Colors( - colors=[inst.unselected_color, inst.selected_color] + style_stream = _Styles( + colors=[inst.unselected_color, inst.selected_color], + alpha=inst.unselected_alpha ) # Cmap streams @@ -187,43 +235,31 @@ def _build_selection_streams(cls, inst): ] def update_colors(*_): - colors_stream.event( - colors=[inst.unselected_color, inst.selected_color] - ) + colors = [inst.unselected_color, inst.selected_color] + style_stream.event(colors=colors, alpha=inst.unselected_alpha) cmap_streams[0].event(cmap=inst.unselected_cmap) - cmap_streams[1].event(cmap=inst.selected_cmap) + if cmap_streams[1] is not None: + cmap_streams[1].event(cmap=inst.selected_cmap) - inst.param.watch( - update_colors, - parameter_names=['unselected_color', 'selected_color'] - ) + inst.param.watch(update_colors,['unselected_color', 'selected_color', 'unselected_alpha']) # Exprs stream exprs_stream = _Exprs(exprs=[True, None]) def update_exprs(*_): exprs_stream.event(exprs=[True, inst.selection_expr]) + # Reset regions + if inst._reset_regions: + for k, v in inst._region_streams.items(): + inst._region_streams[k].event(region_element=None) + inst._obj_selections.clear() + inst._obj_regions.clear() - inst.param.watch( - update_exprs, - parameter_names=['selection_expr'] - ) - - # Alpha streams - alpha_streams = [ - _Alpha(alpha=255), - _Alpha(alpha=inst._selected_alpha), - ] - - def update_alphas(*_): - alpha_streams[1].event(alpha=inst._selected_alpha) - - inst.param.watch(update_alphas, parameter_names=['selection_expr']) + inst.param.watch(update_exprs, ['selection_expr']) return _SelectionStreams( - colors_stream=colors_stream, + style_stream=style_stream, exprs_stream=exprs_stream, - alpha_streams=alpha_streams, cmap_streams=cmap_streams, ) @@ -232,6 +268,8 @@ def unselected_cmap(self): """ The datashader colormap for unselected data """ + if self.unselected_color is None: + return None return _color_to_cmap(self.unselected_color) @property @@ -239,18 +277,70 @@ def selected_cmap(self): """ The datashader colormap for selected data """ - return _color_to_cmap(self.selected_color) + return None if self.selected_color is None else _color_to_cmap(self.selected_color) - @property - def _selected_alpha(self): - if self.selection_expr: - return 255 - else: - return 0 - - def _expr_stream_updated(self, hvobj, selection_expr, bbox): + def _expr_stream_updated(self, hvobj, selection_expr, bbox, region_element): if selection_expr: + if self.cross_filter_mode == "overwrite": + # clear other regions and selections + for k, v in self._region_streams.items(): + if k is not hvobj: + self._region_streams[k].event(region_element=None) + self._obj_regions.pop(k, None) + self._obj_selections.pop(k, None) + + # Update selection expression + if hvobj not in self._obj_selections or self.selection_mode == "overwrite": + if self.selection_mode == "inverse": + self._obj_selections[hvobj] = ~selection_expr + else: + self._obj_selections[hvobj] = selection_expr + else: + if self.selection_mode == "intersect": + self._obj_selections[hvobj] &= selection_expr + elif self.selection_mode == "union": + self._obj_selections[hvobj] |= selection_expr + else: # inverse + self._obj_selections[hvobj] &= ~selection_expr + + # Update region + if self.show_regions: + if isinstance(hvobj, DynamicMap): + el_type = hvobj.type + else: + el_type = hvobj + + region_element = el_type._merge_regions( + self._obj_regions.get(hvobj, None), region_element, self.selection_mode + ) + self._obj_regions[hvobj] = region_element + else: + region_element = None + + # build combined selection + selection_exprs = list(self._obj_selections.values()) + if self.index_cols: + if len(selection_exprs) > 1: + vals = set.intersection(*(set(expr.ops[2]['args'][0]) for expr in selection_exprs)) + old = selection_exprs[0] + selection_expr = dim('new') + selection_expr.dimension = old.dimension + selection_expr.ops = list(old.ops) + selection_expr.ops[2] = dict(selection_expr.ops[2], args=(list(vals),)) + else: + selection_expr = selection_exprs[0] + for expr in selection_exprs[1:]: + selection_expr = selection_expr & expr + + # Set _reset_regions to False so that plot regions aren't automatically + # cleared when self.selection_expr is set. + self._reset_regions = False self.selection_expr = selection_expr + self._reset_regions = True + + # update this region stream + if self._region_streams.get(hvobj, None) is not None: + self._region_streams[hvobj].event(region_element=region_element) class SelectionDisplay(object): @@ -260,7 +350,11 @@ class SelectionDisplay(object): element) into a HoloViews object that represents the current selection state. """ - def build_selection(self, selection_streams, hvobj, operations): + + def __call__(self, element): + return self + + def build_selection(self, selection_streams, hvobj, operations, region_stream=None): raise NotImplementedError() @@ -269,7 +363,8 @@ class NoOpSelectionDisplay(SelectionDisplay): Selection display class that returns input element unchanged. For use with elements that don't support displaying selections. """ - def build_selection(self, selection_streams, hvobj, operations): + + def build_selection(self, selection_streams, hvobj, operations, region_stream=None): return hvobj @@ -278,103 +373,127 @@ class OverlaySelectionDisplay(SelectionDisplay): Selection display base class that represents selections by overlaying colored subsets on top of the original element in an Overlay container. """ - def __init__(self, color_prop='color', is_cmap=False): - self.color_prop = color_prop + + def __init__(self, color_prop='color', is_cmap=False, supports_region=True): + if not isinstance(color_prop, (list, tuple)): + self.color_props = [color_prop] + else: + self.color_props = color_prop self.is_cmap = is_cmap + self.supports_region = supports_region def _get_color_kwarg(self, color): - return {self.color_prop: [color] if self.is_cmap else color} + return {color_prop: [color] if self.is_cmap else color + for color_prop in self.color_props} - def build_selection(self, selection_streams, hvobj, operations): - layers = [] - num_layers = len(selection_streams.colors_stream.colors) + def build_selection(self, selection_streams, hvobj, operations, region_stream=None): + from .element import Histogram + + num_layers = len(selection_streams.style_stream.colors) if not num_layers: - return Overlay(items=[]) + return Overlay() + layers = [] for layer_number in range(num_layers): - build_layer = self._build_layer_callback(layer_number) - sel_streams = [selection_streams.colors_stream, - selection_streams.exprs_stream] - - if isinstance(hvobj, DynamicMap): - def apply_map( - obj, - build_layer=build_layer, - colors=None, - exprs=None, - **kwargs - ): - return obj.map( - lambda el: build_layer(el, colors, exprs), - specs=Element, - clone=True, - ) - - layer = Dynamic( - hvobj, - operation=apply_map, - streams=hvobj.streams + sel_streams, - link_inputs=True, - ) - else: - layer = Dynamic( - hvobj, - operation=build_layer, - streams=sel_streams, - ) - + streams = [selection_streams.exprs_stream] + obj = hvobj.clone(link=False) if layer_number == 1 else hvobj + layer = obj.apply( + self._build_layer_callback, streams=streams, + layer_number=layer_number, per_element=True + ) layers.append(layer) # Wrap in operations for op in operations: + op, kws = op['op'], op['kwargs'] for layer_number in range(num_layers): - streams = copy.copy(op.streams) - - if 'cmap' in op.param: - streams += [selection_streams.cmap_streams[layer_number]] - - if 'alpha' in op.param: - streams += [selection_streams.alpha_streams[layer_number]] - + streams = list(op.streams) + cmap_stream = selection_streams.cmap_streams[layer_number] + kwargs = dict(kws) + + # Handle cmap as an operation parameter + if 'cmap' in op.param or 'cmap' in kwargs: + if layer_number == 0 or (op.cmap is None and kwargs.get('cmap') is None): + streams += [cmap_stream] + else: + @param.depends(cmap=cmap_stream.param.cmap) + def update_cmap(cmap, default=op.cmap, kw=kwargs.get('cmap')): + return cmap or kw or default + kwargs['cmap'] = update_cmap new_op = op.instance(streams=streams) - layers[layer_number] = new_op(layers[layer_number]) - - # build overlay - result = layers[0] - for layer in layers[1:]: - result *= layer - return result - - def _build_layer_callback(self, layer_number): - def _build_layer(element, colors, exprs, **_): - layer_element = self._build_element_layer( - element, colors[layer_number], exprs[layer_number] + layers[layer_number] = new_op(layers[layer_number], **kwargs) + + for layer_number in range(num_layers): + layer = layers[layer_number] + cmap_stream = selection_streams.cmap_streams[layer_number] + streams = [selection_streams.style_stream, cmap_stream] + layer = layer.apply( + self._apply_style_callback, layer_number=layer_number, + streams=streams, per_element=True ) + layers[layer_number] = layer + + # Build region layer + if region_stream is not None and self.supports_region: + def update_region(element, region_element, colors, **kwargs): + unselected_color = colors[0] + if region_element is None: + region_element = element._get_selection_expr_for_stream_value()[2] + return self._style_region_element(region_element, unselected_color) + + streams = [region_stream, selection_streams.style_stream] + region = hvobj.clone(link=False).apply(update_region, streams) + if getattr(hvobj, '_selection_dims', None) == 1 or isinstance(hvobj, Histogram): + layers.insert(1, region) + else: + layers.append(region) + return Overlay(layers).collate() - return layer_element + def _build_layer_callback(self, element, exprs, layer_number, **kwargs): + return self._select(element, exprs[layer_number]) - return _build_layer + def _apply_style_callback(self, element, layer_number, colors, cmap, alpha, **kwargs): + opts = {} + if layer_number == 0: + opts['colorbar'] = False + else: + alpha = 1 + if cmap is not None: + opts['cmap'] = cmap + color = colors[layer_number] if colors else None + return self._build_element_layer(element, color, alpha, **opts) + + def _build_element_layer(self, element, layer_color, layer_alpha, selection_expr=True): + raise NotImplementedError() - def _build_element_layer( - self, element, layer_color, selection_expr=True - ): + def _style_region_element(self, region_element, unselected_cmap): raise NotImplementedError() @staticmethod def _select(element, selection_expr): + from .element import Curve, Spread from .util.transform import dim if isinstance(selection_expr, dim): + dataset = element.dataset try: - element = element.pipeline( - element.dataset.select(selection_expr=selection_expr) - ) + if dataset.interface.gridded: + mask = selection_expr.apply(dataset, expanded=True, flat=False) + selection = dataset.clone(dataset.interface.mask(dataset, ~mask)) + elif isinstance(element, (Curve, Spread)) and hasattr(dataset.interface, 'mask'): + mask = selection_expr.apply(dataset) + selection = dataset.clone(dataset.interface.mask(dataset, ~mask)) + else: + selection = dataset.select(selection_expr=selection_expr) + element = element.pipeline(selection) + except KeyError as e: + key_error = str(e).replace('"', '').replace('.', '') + raise CallbackError("linked_selection aborted because it could not " + "display selection for all elements: %s on '%r'." + % (key_error, element)) except Exception as e: - print(e) - raise - visible = True - else: - visible = bool(selection_expr) - return element, visible + raise CallbackError("linked_selection aborted because it could not " + "display selection for all elements: %s." % e) + return element class ColorListSelectionDisplay(SelectionDisplay): @@ -382,65 +501,40 @@ class ColorListSelectionDisplay(SelectionDisplay): Selection display class for elements that support coloring by a vectorized color list. """ - def __init__(self, color_prop='color'): - self.color_prop = color_prop - def build_selection(self, selection_streams, hvobj, operations): - def _build_selection(el, colors, exprs, **_): + def __init__(self, color_prop='color', alpha_prop='alpha', backend=None): + self.color_props = [color_prop] + self.alpha_props = [alpha_prop] + self.backend = backend + def build_selection(self, selection_streams, hvobj, operations, region_stream=None): + def _build_selection(el, colors, alpha, exprs, **kwargs): + from .plotting.util import linear_gradient + ds = el.dataset selection_exprs = exprs[1:] unselected_color = colors[0] - selected_colors = colors[1:] - n = len(el.dimension_values(0)) + # Use darker version of unselected_color if not selected color provided + unselected_color = unselected_color or "#e6e9ec" + backup_clr = linear_gradient(unselected_color, "#000000", 7)[2] + selected_colors = [c or backup_clr for c in colors[1:]] + n = len(ds) + clrs = np.array([unselected_color] + list(selected_colors)) - if not any(selection_exprs): - colors = [unselected_color] * n - else: - clrs = np.array( - [unselected_color] + list(selected_colors)) - - color_inds = np.zeros(n, dtype='int8') - - for i, expr, color in zip( - range(1, len(clrs)), - selection_exprs, - selected_colors - ): - color_inds[expr.apply(el)] = i + color_inds = np.zeros(n, dtype='int8') - colors = clrs[color_inds] - - return el.options(**{self.color_prop: colors}) + for i, expr, color in zip(range(1, len(clrs)), selection_exprs, selected_colors): + if not expr: + color_inds[:] = i + else: + color_inds[expr.apply(ds)] = i - sel_streams = [selection_streams.colors_stream, - selection_streams.exprs_stream] + colors = clrs[color_inds] + color_opts = {color_prop: colors for color_prop in self.color_props} + return el.pipeline(ds).opts(backend=self.backend, clone=True, **color_opts) - if isinstance(hvobj, DynamicMap): - def apply_map( - obj, - colors=None, - exprs=None, - **kwargs - ): - return obj.map( - lambda el: _build_selection(el, colors, exprs), - specs=Element, - clone=True, - ) - - hvobj = Dynamic( - hvobj, - operation=apply_map, - streams=hvobj.streams + sel_streams, - link_inputs=True, - ) - else: - hvobj = Dynamic( - hvobj, - operation=_build_selection, - streams=sel_streams - ) + sel_streams = [selection_streams.style_stream, selection_streams.exprs_stream] + hvobj = hvobj.apply(_build_selection, streams=sel_streams, per_element=True) for op in operations: hvobj = op(hvobj) @@ -452,9 +546,10 @@ def _color_to_cmap(color): """ Create a light to dark cmap list from a base color """ + from .plotting.util import linear_gradient # Lighten start color by interpolating toward white start_color = linear_gradient("#ffffff", color, 7)[2] # Darken end color by interpolating toward black end_color = linear_gradient(color, "#000000", 7)[2] - return [start_color, end_color] + return linear_gradient(start_color, end_color, 64) diff --git a/holoviews/streams.py b/holoviews/streams.py index afbfef21cf..75b3817fa1 100644 --- a/holoviews/streams.py +++ b/holoviews/streams.py @@ -139,7 +139,16 @@ def trigger(cls, streams): items = [stream.contents.items() for stream in set(streams)] union = [kv for kvs in items for kv in kvs] klist = [k for k, _ in union] - key_clashes = set([k for k in klist if klist.count(k) > 1]) + key_clashes = [] + for k, v in union: + key_count = klist.count(k) + try: + value_count = union.count((k, v)) + except Exception: + # If we can't compare values we assume they are not equal + value_count = 1 + if key_count > 1 and key_count > value_count and k not in key_clashes: + key_clashes.append(k) if key_clashes: print('Parameter name clashes for keys %r' % key_clashes) @@ -763,30 +772,33 @@ def __init__(self, parameterized, parameters=None, watch=True, **params): class SelectionExpr(Stream): selection_expr = param.Parameter(default=None, constant=True) + bbox = param.Dict(default=None, constant=True) + region_element = param.Parameter(default=None, constant=True) + def __init__(self, source, **params): from .element import Element from .core.spaces import DynamicMap from .plotting.util import initialize_dynamic + self._index_cols = params.pop('index_cols', None) + if isinstance(source, DynamicMap): initialize_dynamic(source) - if isinstance(source, Element) or ( - isinstance(source, DynamicMap) and - issubclass(source.type, Element) - ): + if ((isinstance(source, DynamicMap) and issubclass(source.type, Element)) or + isinstance(source, Element)): self._source_streams = [] super(SelectionExpr, self).__init__(source=source, **params) self._register_chart(source) else: - raise ValueError(""" -The source of SelectionExpr must be an instance of an Element subclass, -or a DynamicMap that returns such an instance - Received value of type {typ}: {val}""".format( - typ=type(source), val=source - )) + raise ValueError( + "The source of SelectionExpr must be an instance of an " + "Element subclass or a DynamicMap that returns such an " + "instance. Received value of type {typ}: {val}".format( + typ=type(source), val=source) + ) def _register_chart(self, hvobj): from .core.spaces import DynamicMap @@ -803,15 +815,22 @@ def _set_expr(**params): element = hvobj.values()[-1] else: element = hvobj - selection_expr, bbox = \ - element._get_selection_expr_for_stream_value(**params); - - self.event(selection_expr=selection_expr, bbox=bbox) + params = dict(params, index_cols=self._index_cols) + selection_expr, bbox, region_element = \ + element._get_selection_expr_for_stream_value(**params) + for expr_transform in element._transforms[::-1]: + if selection_expr is not None: + selection_expr = expr_transform(selection_expr) + + self.event( + selection_expr=selection_expr, + bbox=bbox, + region_element=region_element, + ) for stream_type in selection_streams: stream = stream_type(source=hvobj) self._source_streams.append(stream) - stream.add_subscriber(_set_expr) def _unregister_chart(self): @@ -993,6 +1012,24 @@ class BoundsXY(LinkedStream): Bounds defined as (left, bottom, right, top) tuple.""") +class SelectionXY(BoundsXY): + """ + A stream representing the selection along the x-axis and y-axis. + Unlike a BoundsXY stream, this stream returns range or categorical + selections. + """ + + x_selection = param.ClassSelector(class_=(tuple, list), allow_None=True, + constant=True, doc=""" + The current selection along the x-axis, either a numerical range + defined as a tuple or a list of categories.""") + + y_selection = param.ClassSelector(class_=(tuple, list), allow_None=True, + constant=True, doc=""" + The current selection along the y-axis, either a numerical range + defined as a tuple or a list of categories.""") + + class BoundsX(LinkedStream): """ A stream representing the bounds of a box selection as an diff --git a/holoviews/tests/core/data/testgridinterface.py b/holoviews/tests/core/data/testgridinterface.py index a5321251e7..6cab4a593d 100644 --- a/holoviews/tests/core/data/testgridinterface.py +++ b/holoviews/tests/core/data/testgridinterface.py @@ -42,6 +42,14 @@ def test_dataset_dataframe_init_hm_alias(self): Dataset(pd.DataFrame({'x':self.xs, 'x2':self.xs_2}), kdims=['x'], vdims=['x2']) + def test_dataset_empty_constructor(self): + ds = Dataset([], ['x', 'y'], ['z']) + assert ds.interface.shape(ds, gridded=True) == (0, 0) + + def test_dataset_multi_vdim_empty_constructor(self): + ds = Dataset([], ['x', 'y'], ['z1', 'z2', 'z3']) + assert all(ds.dimension_values(vd, flat=False).shape == (0, 0) for vd in ds.vdims) + def test_irregular_grid_data_values(self): nx, ny = 20, 5 xs, ys = np.meshgrid(np.arange(nx)+0.5, np.arange(ny)+0.5) @@ -266,6 +274,46 @@ def test_reindex_2d_grid_to_1d(self): with DatatypeContext([self.datatype, 'dictionary' , 'dataframe'], Dataset): self.assertEqual(ds, Dataset(self.dataset_grid.columns(), 'x', 'z')) + def test_mask_2d_array(self): + array = np.random.rand(4, 3) + ds = Dataset(([0, 1, 2], [1, 2, 3, 4], array), ['x', 'y'], 'z') + mask = np.array([[1, 1, 0], [1, 0, 1], [0, 1, 1], [1, 0, 1]], dtype='bool') + masked = ds.clone(ds.interface.mask(ds, mask)) + masked_array = masked.dimension_values(2, flat=False) + expected = array.copy() + expected[mask] = np.nan + self.assertEqual(masked_array, expected) + + def test_mask_2d_array_x_reversed(self): + array = np.random.rand(4, 3) + ds = Dataset(([0, 1, 2][::-1], [1, 2, 3, 4], array[:, ::-1]), ['x', 'y'], 'z') + mask = np.array([[1, 1, 0], [1, 0, 1], [0, 1, 1], [1, 0, 1]], dtype='bool') + masked = ds.clone(ds.interface.mask(ds, mask)) + masked_array = masked.dimension_values(2, flat=False) + expected = array.copy() + expected[mask] = np.nan + self.assertEqual(masked_array, expected) + + def test_mask_2d_array_y_reversed(self): + array = np.random.rand(4, 3) + ds = Dataset(([0, 1, 2], [1, 2, 3, 4][::-1], array[::-1]), ['x', 'y'], 'z') + mask = np.array([[1, 1, 0], [1, 0, 1], [0, 1, 1], [1, 0, 1]], dtype='bool') + masked = ds.clone(ds.interface.mask(ds, mask)) + masked_array = masked.dimension_values(2, flat=False) + expected = array.copy() + expected[mask] = np.nan + self.assertEqual(masked_array, expected) + + def test_mask_2d_array_xy_reversed(self): + array = np.random.rand(4, 3) + ds = Dataset(([0, 1, 2][::-1], [1, 2, 3, 4][::-1], array[::-1, ::-1]), ['x', 'y'], 'z') + mask = np.array([[1, 1, 0], [1, 0, 1], [0, 1, 1], [1, 0, 1]], dtype='bool') + masked = ds.clone(ds.interface.mask(ds, mask)) + masked_array = masked.dimension_values(2, flat=False) + expected = array.copy() + expected[mask] = np.nan + self.assertEqual(masked_array, expected) + class GridInterfaceTests(BaseGridInterfaceTests): diff --git a/holoviews/tests/core/data/testxarrayinterface.py b/holoviews/tests/core/data/testxarrayinterface.py index fe899ad193..292f1b4adc 100644 --- a/holoviews/tests/core/data/testxarrayinterface.py +++ b/holoviews/tests/core/data/testxarrayinterface.py @@ -236,11 +236,22 @@ def test_select_dropped_dimensions_restoration(self): self.assertEqual(t.data.dims , dict(chain=1,value=8)) self.assertEqual(t.data.stuff.shape , (1,8)) + def test_mask_2d_array_transposed(self): + array = np.random.rand(4, 3) + da = xr.DataArray(array.T, coords={'x': [0, 1, 2], 'y': [0, 1, 2, 3]}, dims=['x', 'y']) + ds = Dataset(da, ['x', 'y'], 'z') + mask = np.array([[1, 1, 0], [1, 0, 1], [0, 1, 1], [1, 0, 1]], dtype='bool') + masked = ds.clone(ds.interface.mask(ds, mask)) + masked_array = masked.dimension_values(2, flat=False) + expected = array.copy() + expected[mask] = np.nan + self.assertEqual(masked_array, expected) + + # Disabled tests for NotImplemented methods def test_dataset_array_init_hm(self): "Tests support for arrays (homogeneous)" raise SkipTest("Not supported") - # Disabled tests for NotImplemented methods def test_dataset_add_dimensions_values_hm(self): raise SkipTest("Not supported") diff --git a/holoviews/tests/element/test_selection.py b/holoviews/tests/element/test_selection.py new file mode 100644 index 0000000000..71864e0e54 --- /dev/null +++ b/holoviews/tests/element/test_selection.py @@ -0,0 +1,384 @@ +""" +Test cases for the Comparisons class over the Chart elements +""" + +from unittest import SkipTest + +import numpy as np + +from holoviews.core import NdOverlay +from holoviews.core.options import Store +from holoviews.element import ( + Area, BoxWhisker, Curve, Distribution, HSpan, Image, Points, + Rectangles, RGB, Scatter, Segments, Violin, VSpan +) +from holoviews.element.comparison import ComparisonTestCase + + +class TestSelection1DExpr(ComparisonTestCase): + + def setUp(self): + try: + import holoviews.plotting.bokeh # noqa + except: + raise SkipTest("Bokeh selection tests require bokeh.") + super(TestSelection1DExpr, self).setUp() + self._backend = Store.current_backend + Store.set_current_backend('bokeh') + + def tearDown(self): + Store.current_backend = self._backend + + def test_area_selection_numeric(self): + area = Area([3, 2, 1, 3, 4]) + expr, bbox, region = area._get_selection_expr_for_stream_value(bounds=(1, 0, 3, 2)) + self.assertEqual(bbox, {'x': (1, 3)}) + self.assertEqual(expr.apply(area), np.array([False, True, True, True, False])) + self.assertEqual(region, NdOverlay({0: VSpan(1, 3)})) + + def test_area_selection_numeric_inverted(self): + area = Area([3, 2, 1, 3, 4]).opts(invert_axes=True) + expr, bbox, region = area._get_selection_expr_for_stream_value(bounds=(0, 1, 2, 3)) + self.assertEqual(bbox, {'x': (1, 3)}) + self.assertEqual(expr.apply(area), np.array([False, True, True, True, False])) + self.assertEqual(region, NdOverlay({0: HSpan(1, 3)})) + + def test_area_selection_categorical(self): + area = Area((['B', 'A', 'C', 'D', 'E'], [3, 2, 1, 3, 4])) + expr, bbox, region = area._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 3), x_selection=['B', 'A', 'C'] + ) + self.assertEqual(bbox, {'x': ['B', 'A', 'C']}) + self.assertEqual(expr.apply(area), np.array([True, True, True, False, False])) + self.assertEqual(region, NdOverlay({0: VSpan(0, 2)})) + + def test_area_selection_numeric_index_cols(self): + area = Area([3, 2, 1, 3, 2]) + expr, bbox, region = area._get_selection_expr_for_stream_value( + bounds=(1, 0, 3, 2), index_cols=['y'] + ) + self.assertEqual(bbox, {'x': (1, 3)}) + self.assertEqual(expr.apply(area), np.array([False, True, True, False, True])) + self.assertEqual(region, None) + + def test_curve_selection_numeric(self): + curve = Curve([3, 2, 1, 3, 4]) + expr, bbox, region = curve._get_selection_expr_for_stream_value(bounds=(1, 0, 3, 2)) + self.assertEqual(bbox, {'x': (1, 3)}) + self.assertEqual(expr.apply(curve), np.array([False, True, True, True, False])) + self.assertEqual(region, NdOverlay({0: VSpan(1, 3)})) + + def test_curve_selection_categorical(self): + curve = Curve((['B', 'A', 'C', 'D', 'E'], [3, 2, 1, 3, 4])) + expr, bbox, region = curve._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 3), x_selection=['B', 'A', 'C'] + ) + self.assertEqual(bbox, {'x': ['B', 'A', 'C']}) + self.assertEqual(expr.apply(curve), np.array([True, True, True, False, False])) + self.assertEqual(region, NdOverlay({0: VSpan(0, 2)})) + + def test_curve_selection_numeric_index_cols(self): + curve = Curve([3, 2, 1, 3, 2]) + expr, bbox, region = curve._get_selection_expr_for_stream_value( + bounds=(1, 0, 3, 2), index_cols=['y'] + ) + self.assertEqual(bbox, {'x': (1, 3)}) + self.assertEqual(expr.apply(curve), np.array([False, True, True, False, True])) + self.assertEqual(region, None) + + def test_box_whisker_single(self): + box_whisker = BoxWhisker(list(range(10))) + expr, bbox, region = box_whisker._get_selection_expr_for_stream_value( + bounds=(0, 3, 1, 7) + ) + self.assertEqual(bbox, {'y': (3, 7)}) + self.assertEqual(expr.apply(box_whisker), np.array([ + False, False, False, True, True, True, True, True, False, False + ])) + self.assertEqual(region, NdOverlay({0: HSpan(3, 7)})) + + def test_box_whisker_single_inverted(self): + box = BoxWhisker(list(range(10))).opts(invert_axes=True) + expr, bbox, region = box._get_selection_expr_for_stream_value( + bounds=(3, 0, 7, 1) + ) + self.assertEqual(bbox, {'y': (3, 7)}) + self.assertEqual(expr.apply(box), np.array([ + False, False, False, True, True, True, True, True, False, False + ])) + self.assertEqual(region, NdOverlay({0: VSpan(3, 7)})) + + def test_box_whisker_cats(self): + box_whisker = BoxWhisker((['A', 'A', 'A', 'B', 'B', 'C', 'C', 'C', 'C', 'C'], list(range(10))), 'x', 'y') + expr, bbox, region = box_whisker._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 7), x_selection=['A', 'B'] + ) + self.assertEqual(bbox, {'y': (1, 7), 'x': ['A', 'B']}) + self.assertEqual(expr.apply(box_whisker), np.array([ + False, True, True, True, True, False, False, False, False, False + ])) + self.assertEqual(region, NdOverlay({0: HSpan(1, 7)})) + + def test_box_whisker_cats_index_cols(self): + box_whisker = BoxWhisker((['A', 'A', 'A', 'B', 'B', 'C', 'C', 'C', 'C', 'C'], list(range(10))), 'x', 'y') + expr, bbox, region = box_whisker._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 7), x_selection=['A', 'B'], index_cols=['x'] + ) + self.assertEqual(bbox, {'y': (1, 7), 'x': ['A', 'B']}) + self.assertEqual(expr.apply(box_whisker), np.array([ + True, True, True, True, True, False, False, False, False, False + ])) + self.assertEqual(region, None) + + def test_violin_single(self): + violin = Violin(list(range(10))) + expr, bbox, region = violin._get_selection_expr_for_stream_value( + bounds=(0, 3, 1, 7) + ) + self.assertEqual(bbox, {'y': (3, 7)}) + self.assertEqual(expr.apply(violin), np.array([ + False, False, False, True, True, True, True, True, False, False + ])) + self.assertEqual(region, NdOverlay({0: HSpan(3, 7)})) + + def test_violin_single_inverted(self): + violin = Violin(list(range(10))).opts(invert_axes=True) + expr, bbox, region = violin._get_selection_expr_for_stream_value( + bounds=(3, 0, 7, 1) + ) + self.assertEqual(bbox, {'y': (3, 7)}) + self.assertEqual(expr.apply(violin), np.array([ + False, False, False, True, True, True, True, True, False, False + ])) + self.assertEqual(region, NdOverlay({0: VSpan(3, 7)})) + + def test_violin_cats(self): + violin = Violin((['A', 'A', 'A', 'B', 'B', 'C', 'C', 'C', 'C', 'C'], list(range(10))), 'x', 'y') + expr, bbox, region = violin._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 7), x_selection=['A', 'B'] + ) + self.assertEqual(bbox, {'y': (1, 7), 'x': ['A', 'B']}) + self.assertEqual(expr.apply(violin), np.array([ + False, True, True, True, True, False, False, False, False, False + ])) + self.assertEqual(region, NdOverlay({0: HSpan(1, 7)})) + + def test_violin_cats_index_cols(self): + violin = Violin((['A', 'A', 'A', 'B', 'B', 'C', 'C', 'C', 'C', 'C'], list(range(10))), 'x', 'y') + expr, bbox, region = violin._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 7), x_selection=['A', 'B'], index_cols=['x'] + ) + self.assertEqual(bbox, {'y': (1, 7), 'x': ['A', 'B']}) + self.assertEqual(expr.apply(violin), np.array([ + True, True, True, True, True, False, False, False, False, False + ])) + self.assertEqual(region, None) + + def test_distribution_single(self): + dist = Distribution(list(range(10))) + expr, bbox, region = dist._get_selection_expr_for_stream_value( + bounds=(3, 0, 7, 1) + ) + self.assertEqual(bbox, {'Value': (3, 7)}) + self.assertEqual(expr.apply(dist), np.array([ + False, False, False, True, True, True, True, True, False, False + ])) + self.assertEqual(region, NdOverlay({0: VSpan(3, 7)})) + + def test_distribution_single_inverted(self): + dist = Distribution(list(range(10))).opts(invert_axes=True) + expr, bbox, region = dist._get_selection_expr_for_stream_value( + bounds=(0, 3, 1, 7) + ) + self.assertEqual(bbox, {'Value': (3, 7)}) + self.assertEqual(expr.apply(dist), np.array([ + False, False, False, True, True, True, True, True, False, False + ])) + self.assertEqual(region, NdOverlay({0: HSpan(3, 7)})) + + +class TestSelection2DExpr(ComparisonTestCase): + + def setUp(self): + try: + import holoviews.plotting.bokeh # noqa + except: + raise SkipTest("Bokeh selection tests require bokeh.") + super(TestSelection2DExpr, self).setUp() + self._backend = Store.current_backend + Store.set_current_backend('bokeh') + + def tearDown(self): + Store.current_backend = self._backend + + def test_points_selection_numeric(self): + points = Points([3, 2, 1, 3, 4]) + expr, bbox, region = points._get_selection_expr_for_stream_value(bounds=(1, 0, 3, 2)) + self.assertEqual(bbox, {'x': (1, 3), 'y': (0, 2)}) + self.assertEqual(expr.apply(points), np.array([False, True, True, False, False])) + self.assertEqual(region, Rectangles([(1, 0, 3, 2)])) + + def test_points_selection_numeric_inverted(self): + points = Points([3, 2, 1, 3, 4]).opts(invert_axes=True) + expr, bbox, region = points._get_selection_expr_for_stream_value(bounds=(0, 1, 2, 3)) + self.assertEqual(bbox, {'x': (1, 3), 'y': (0, 2)}) + self.assertEqual(expr.apply(points), np.array([False, True, True, False, False])) + self.assertEqual(region, Rectangles([(0, 1, 2, 3)])) + + def test_points_selection_categorical(self): + points = Points((['B', 'A', 'C', 'D', 'E'], [3, 2, 1, 3, 4])) + expr, bbox, region = points._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 3), x_selection=['B', 'A', 'C'], y_selection=None + ) + self.assertEqual(bbox, {'x': ['B', 'A', 'C'], 'y': (1, 3)}) + self.assertEqual(expr.apply(points), np.array([True, True, True, False, False])) + self.assertEqual(region, Rectangles([(0, 1, 2, 3)])) + + def test_points_selection_numeric_index_cols(self): + points = Points([3, 2, 1, 3, 2]) + expr, bbox, region = points._get_selection_expr_for_stream_value( + bounds=(1, 0, 3, 2), index_cols=['y'] + ) + self.assertEqual(bbox, {'x': (1, 3), 'y': (0, 2)}) + self.assertEqual(expr.apply(points), np.array([False, False, True, False, False])) + self.assertEqual(region, None) + + def test_scatter_selection_numeric(self): + scatter = Scatter([3, 2, 1, 3, 4]) + expr, bbox, region = scatter._get_selection_expr_for_stream_value(bounds=(1, 0, 3, 2)) + self.assertEqual(bbox, {'x': (1, 3), 'y': (0, 2)}) + self.assertEqual(expr.apply(scatter), np.array([False, True, True, False, False])) + self.assertEqual(region, Rectangles([(1, 0, 3, 2)])) + + def test_scatter_selection_numeric_inverted(self): + scatter = Scatter([3, 2, 1, 3, 4]).opts(invert_axes=True) + expr, bbox, region = scatter._get_selection_expr_for_stream_value(bounds=(0, 1, 2, 3)) + self.assertEqual(bbox, {'x': (1, 3), 'y': (0, 2)}) + self.assertEqual(expr.apply(scatter), np.array([False, True, True, False, False])) + self.assertEqual(region, Rectangles([(0, 1, 2, 3)])) + + def test_scatter_selection_categorical(self): + scatter = Scatter((['B', 'A', 'C', 'D', 'E'], [3, 2, 1, 3, 4])) + expr, bbox, region = scatter._get_selection_expr_for_stream_value( + bounds=(0, 1, 2, 3), x_selection=['B', 'A', 'C'], y_selection=None + ) + self.assertEqual(bbox, {'x': ['B', 'A', 'C'], 'y': (1, 3)}) + self.assertEqual(expr.apply(scatter), np.array([True, True, True, False, False])) + self.assertEqual(region, Rectangles([(0, 1, 2, 3)])) + + def test_scatter_selection_numeric_index_cols(self): + scatter = Scatter([3, 2, 1, 3, 2]) + expr, bbox, region = scatter._get_selection_expr_for_stream_value( + bounds=(1, 0, 3, 2), index_cols=['y'] + ) + self.assertEqual(bbox, {'x': (1, 3), 'y': (0, 2)}) + self.assertEqual(expr.apply(scatter), np.array([False, False, True, False, False])) + self.assertEqual(region, None) + + def test_image_selection_numeric(self): + img = Image(([0, 1, 2], [0, 1, 2, 3], np.random.rand(4, 3))) + expr, bbox, region = img._get_selection_expr_for_stream_value(bounds=(0.5, 1.5, 2.1, 3.1)) + self.assertEqual(bbox, {'x': (0.5, 2.1), 'y': (1.5, 3.1)}) + self.assertEqual(expr.apply(img, expanded=True, flat=False), np.array([ + [False, False, False], + [False, False, False], + [False, True, True], + [False, True, True] + ])) + self.assertEqual(region, Rectangles([(0.5, 1.5, 2.1, 3.1)])) + + def test_image_selection_numeric_inverted(self): + img = Image(([0, 1, 2], [0, 1, 2, 3], np.random.rand(4, 3))).opts(invert_axes=True) + expr, bbox, region = img._get_selection_expr_for_stream_value(bounds=(1.5, 0.5, 3.1, 2.1)) + self.assertEqual(bbox, {'x': (0.5, 2.1), 'y': (1.5, 3.1)}) + self.assertEqual(expr.apply(img, expanded=True, flat=False), np.array([ + [False, False, False], + [False, False, False], + [False, True, True], + [False, True, True] + ])) + self.assertEqual(region, Rectangles([(1.5, 0.5, 3.1, 2.1)])) + + def test_rgb_selection_numeric(self): + img = RGB(([0, 1, 2], [0, 1, 2, 3], np.random.rand(4, 3, 3))) + expr, bbox, region = img._get_selection_expr_for_stream_value(bounds=(0.5, 1.5, 2.1, 3.1)) + self.assertEqual(bbox, {'x': (0.5, 2.1), 'y': (1.5, 3.1)}) + self.assertEqual(expr.apply(img, expanded=True, flat=False), np.array([ + [False, False, False], + [False, False, False], + [False, True, True], + [False, True, True] + ])) + self.assertEqual(region, Rectangles([(0.5, 1.5, 2.1, 3.1)])) + + def test_rgb_selection_numeric_inverted(self): + img = RGB(([0, 1, 2], [0, 1, 2, 3], np.random.rand(4, 3, 3))).opts(invert_axes=True) + expr, bbox, region = img._get_selection_expr_for_stream_value(bounds=(1.5, 0.5, 3.1, 2.1)) + self.assertEqual(bbox, {'x': (0.5, 2.1), 'y': (1.5, 3.1)}) + self.assertEqual(expr.apply(img, expanded=True, flat=False), np.array([ + [False, False, False], + [False, False, False], + [False, True, True], + [False, True, True] + ])) + self.assertEqual(region, Rectangles([(1.5, 0.5, 3.1, 2.1)])) + + + +class TestSelectionGeomExpr(ComparisonTestCase): + + def setUp(self): + try: + import holoviews.plotting.bokeh # noqa + except: + raise SkipTest("Bokeh selection tests require bokeh.") + super(TestSelectionGeomExpr, self).setUp() + self._backend = Store.current_backend + Store.set_current_backend('bokeh') + + def tearDown(self): + Store.current_backend = self._backend + + def test_rect_selection_numeric(self): + rect = Rectangles([(0, 1, 2, 3), (1, 3, 1.5, 4), (2.5, 4.2, 3.5, 4.8)]) + expr, bbox, region = rect._get_selection_expr_for_stream_value(bounds=(0.5, 0.9, 3.4, 4.9)) + self.assertEqual(bbox, {'x0': (0.5, 3.4), 'y0': (0.9, 4.9), 'x1': (0.5, 3.4), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(rect), np.array([False, True, False])) + self.assertEqual(region, Rectangles([(0.5, 0.9, 3.4, 4.9)])) + expr, bbox, region = rect._get_selection_expr_for_stream_value(bounds=(0, 0.9, 3.5, 4.9)) + self.assertEqual(bbox, {'x0': (0, 3.5), 'y0': (0.9, 4.9), 'x1': (0, 3.5), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(rect), np.array([True, True, True])) + self.assertEqual(region, Rectangles([(0, 0.9, 3.5, 4.9)])) + + def test_rect_selection_numeric_inverted(self): + rect = Rectangles([(0, 1, 2, 3), (1, 3, 1.5, 4), (2.5, 4.2, 3.5, 4.8)]).opts(invert_axes=True) + expr, bbox, region = rect._get_selection_expr_for_stream_value(bounds=(0.9, 0.5, 4.9, 3.4)) + self.assertEqual(bbox, {'x0': (0.5, 3.4), 'y0': (0.9, 4.9), 'x1': (0.5, 3.4), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(rect), np.array([False, True, False])) + self.assertEqual(region, Rectangles([(0.9, 0.5, 4.9, 3.4)])) + expr, bbox, region = rect._get_selection_expr_for_stream_value(bounds=(0.9, 0, 4.9, 3.5)) + self.assertEqual(bbox, {'x0': (0, 3.5), 'y0': (0.9, 4.9), 'x1': (0, 3.5), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(rect), np.array([True, True, True])) + self.assertEqual(region, Rectangles([(0.9, 0, 4.9, 3.5)])) + + def test_segments_selection_numeric(self): + segs = Segments([(0, 1, 2, 3), (1, 3, 1.5, 4), (2.5, 4.2, 3.5, 4.8)]) + expr, bbox, region = segs._get_selection_expr_for_stream_value(bounds=(0.5, 0.9, 3.4, 4.9)) + self.assertEqual(bbox, {'x0': (0.5, 3.4), 'y0': (0.9, 4.9), 'x1': (0.5, 3.4), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(segs), np.array([False, True, False])) + self.assertEqual(region, Rectangles([(0.5, 0.9, 3.4, 4.9)])) + expr, bbox, region = segs._get_selection_expr_for_stream_value(bounds=(0, 0.9, 3.5, 4.9)) + self.assertEqual(bbox, {'x0': (0, 3.5), 'y0': (0.9, 4.9), 'x1': (0, 3.5), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(segs), np.array([True, True, True])) + self.assertEqual(region, Rectangles([(0, 0.9, 3.5, 4.9)])) + + def test_segs_selection_numeric_inverted(self): + segs = Segments([(0, 1, 2, 3), (1, 3, 1.5, 4), (2.5, 4.2, 3.5, 4.8)]).opts(invert_axes=True) + expr, bbox, region = segs._get_selection_expr_for_stream_value(bounds=(0.9, 0.5, 4.9, 3.4)) + self.assertEqual(bbox, {'x0': (0.5, 3.4), 'y0': (0.9, 4.9), 'x1': (0.5, 3.4), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(segs), np.array([False, True, False])) + self.assertEqual(region, Rectangles([(0.9, 0.5, 4.9, 3.4)])) + expr, bbox, region = segs._get_selection_expr_for_stream_value(bounds=(0.9, 0, 4.9, 3.5)) + self.assertEqual(bbox, {'x0': (0, 3.5), 'y0': (0.9, 4.9), 'x1': (0, 3.5), 'y1': (0.9, 4.9)}) + self.assertEqual(expr.apply(segs), np.array([True, True, True])) + self.assertEqual(region, Rectangles([(0.9, 0, 4.9, 3.5)])) diff --git a/holoviews/tests/operation/testdatashader.py b/holoviews/tests/operation/testdatashader.py index c170ec0373..65aa6d7bd1 100644 --- a/holoviews/tests/operation/testdatashader.py +++ b/holoviews/tests/operation/testdatashader.py @@ -353,10 +353,16 @@ def test_segments_aggregate_count(self): expected = Image((xs, ys, arr), vdims='count') self.assertEqual(agg, expected) - def test_segments_aggregate_sum(self): + def test_segments_aggregate_sum(self, instance=False): segments = Segments([(0, 1, 4, 1, 2), (1, 0, 1, 4, 4)], vdims=['value']) - agg = rasterize(segments, width=4, height=4, dynamic=False, - aggregator='sum') + if instance: + agg = rasterize.instance( + width=10, height=10, dynamic=False, aggregator='sum' + )(segments, width=4, height=4) + else: + agg = rasterize( + segments, width=4, height=4, dynamic=False, aggregator='sum' + ) xs = [0.5, 1.5, 2.5, 3.5] ys = [0.5, 1.5, 2.5, 3.5] na = np.nan @@ -369,6 +375,9 @@ def test_segments_aggregate_sum(self): expected = Image((xs, ys, arr), vdims='value') self.assertEqual(agg, expected) + def test_segments_aggregate_sum_instance(self): + self.test_segments_aggregate_sum(instance=True) + def test_segments_aggregate_dt_count(self): segments = Segments([ (0, dt.datetime(2016, 1, 2), 4, dt.datetime(2016, 1, 2)), diff --git a/holoviews/tests/plotting/bokeh/testviolinplot.py b/holoviews/tests/plotting/bokeh/testviolinplot.py index 12e2b48d4b..3b06aaede3 100644 --- a/holoviews/tests/plotting/bokeh/testviolinplot.py +++ b/holoviews/tests/plotting/bokeh/testviolinplot.py @@ -177,8 +177,8 @@ def test_violin_split_op_single(self): source = plot.handles['patches_1_source'] glyph = plot.handles['patches_1_glyph'] cmapper = plot.handles['violin_color_mapper'] - self.assertEqual(source.data["'a'"], ['0', '1']) - self.assertEqual(glyph.fill_color, {'field': "'a'", 'transform': cmapper}) + self.assertEqual(source.data["dim('a')"], ['0', '1']) + self.assertEqual(glyph.fill_color, {'field': "dim('a')", 'transform': cmapper}) def test_violin_box_linear_color_op(self): a = np.repeat(np.arange(5), 5) diff --git a/holoviews/tests/testselection.py b/holoviews/tests/testselection.py index de08486f2d..57925b843d 100644 --- a/holoviews/tests/testselection.py +++ b/holoviews/tests/testselection.py @@ -3,8 +3,12 @@ import holoviews as hv import pandas as pd +from holoviews.core.util import unicode, basestring from holoviews.core.options import Store +from holoviews.element import ErrorBars, Points, Rectangles, Table +from holoviews.plotting.util import linear_gradient from holoviews.selection import link_selections +from holoviews.streams import SelectionXY from holoviews.element.comparison import ComparisonTestCase try: @@ -15,6 +19,10 @@ ds_skip = skipIf(datashade is None, "Datashader not available") +unselected_color = "#ff0000" +box_region_color = linear_gradient(unselected_color, "#000000", 9)[3] +hist_region_color = linear_gradient(unselected_color, "#000000", 9)[1] + class TestLinkSelections(ComparisonTestCase): def setUp(self): @@ -26,98 +34,107 @@ def setUp(self): {'x': [1, 2, 3], 'y': [0, 3, 2], 'e': [1, 1.5, 2], - }, + }, columns=['x', 'y', 'e'] ) def element_color(self, element): raise NotImplementedError - def element_visible(self, element): - raise NotImplementedError - - def check_base_scatter_like(self, base_scatter, lnk_sel, data=None): + def check_base_points_like(self, base_points, lnk_sel, data=None): if data is None: data = self.data self.assertEqual( - self.element_color(base_scatter), + self.element_color(base_points), lnk_sel.unselected_color ) - self.assertTrue(self.element_visible(base_scatter)) - self.assertEqual(base_scatter.data, data) + self.assertEqual(base_points.data, data) - def check_overlay_scatter_like(self, overlay_scatter, lnk_sel, data): - self.assertEqual( - self.element_color(overlay_scatter), - lnk_sel.selected_color - ) + @staticmethod + def expected_selection_color(element, lnk_sel): + if lnk_sel.selected_color is not None: + expected_color = lnk_sel.selected_color + else: + expected_color = element.opts.get(group='style')[0].get('color') + return expected_color + + def check_overlay_points_like(self, overlay_points, lnk_sel, data): self.assertEqual( - self.element_visible(overlay_scatter), - len(data) != len(self.data) + self.element_color(overlay_points), + self.expected_selection_color(overlay_points, lnk_sel), ) - self.assertEqual(overlay_scatter.data, data) + self.assertEqual(overlay_points.data, data) - def test_scatter_selection(self, dynamic=False): - scatter = hv.Scatter(self.data, kdims='x', vdims='y') + def test_points_selection(self, dynamic=False, show_regions=True): + points = Points(self.data) if dynamic: - # Convert scatter to DynamicMap that returns the element - scatter = hv.util.Dynamic(scatter) + # Convert points to DynamicMap that returns the element + points = hv.util.Dynamic(points) - lnk_sel = link_selections.instance() - linked = lnk_sel(scatter) + lnk_sel = link_selections.instance(show_regions=show_regions, + unselected_color='#ff0000') + linked = lnk_sel(points) current_obj = linked[()] # Check initial state of linked dynamic map self.assertIsInstance(current_obj, hv.Overlay) + unselected, selected, region = current_obj.values() # Check initial base layer - self.check_base_scatter_like(current_obj.Scatter.I, lnk_sel) + self.check_base_points_like(unselected, lnk_sel) # Check selection layer - self.check_overlay_scatter_like(current_obj.Scatter.II, lnk_sel, self.data) + self.check_overlay_points_like(selected, lnk_sel, self.data) # Perform selection of second and third point boundsxy = lnk_sel._selection_expr_streams[0]._source_streams[0] - self.assertIsInstance(boundsxy, hv.streams.BoundsXY) + self.assertIsInstance(boundsxy, hv.streams.SelectionXY) boundsxy.event(bounds=(0, 1, 5, 5)) - current_obj = linked[()] + unselected, selected, region = linked[()].values() # Check that base layer is unchanged - self.check_base_scatter_like(current_obj.Scatter.I, lnk_sel) + self.check_base_points_like(unselected, lnk_sel) # Check selection layer - self.check_overlay_scatter_like(current_obj.Scatter.II, lnk_sel, self.data.iloc[1:]) + self.check_overlay_points_like(selected, lnk_sel, self.data.iloc[1:]) + + if show_regions: + self.assertEqual(region, Rectangles([(0, 1, 5, 5)])) + else: + self.assertEqual(region, Rectangles([])) - def test_scatter_selection_dynamic(self): - self.test_scatter_selection(dynamic=True) + def test_points_selection_hide_region(self): + self.test_points_selection(show_regions=False) - def test_layout_selection_scatter_table(self): - scatter = hv.Scatter(self.data, kdims='x', vdims='y') - table = hv.Table(self.data) - lnk_sel = link_selections.instance() - linked = lnk_sel(scatter + table) + def test_points_selection_dynamic(self): + self.test_points_selection(dynamic=True) + + def test_layout_selection_points_table(self): + points = Points(self.data) + table = Table(self.data) + lnk_sel = link_selections.instance( + selected_color="#aa0000", unselected_color='#ff0000' + ) + linked = lnk_sel(points + table) current_obj = linked[()] - # Check initial base scatter - self.check_base_scatter_like( - current_obj[0][()].Scatter.I, + # Check initial base points + self.check_base_points_like( + current_obj[0][()].Points.I, lnk_sel ) - # Check initial selection scatter - self.check_overlay_scatter_like( - current_obj[0][()].Scatter.II, - lnk_sel, - self.data - ) + # Check initial selection points + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data) # Check initial table self.assertEqual( self.element_color(current_obj[1][()]), - [lnk_sel.unselected_color] * len(self.data) + [lnk_sel.selected_color] * len(self.data) ) # Select first and third point @@ -125,18 +142,15 @@ def test_layout_selection_scatter_table(self): boundsxy.event(bounds=(0, 0, 4, 2)) current_obj = linked[()] - # Check base scatter - self.check_base_scatter_like( - current_obj[0][()].Scatter.I, + # Check base points + self.check_base_points_like( + current_obj[0][()].Points.I, lnk_sel ) - # Check selection scatter - self.check_overlay_scatter_like( - current_obj[0][()].Scatter.II, - lnk_sel, - self.data.iloc[[0, 2]] - ) + # Check selection points + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data.iloc[[0, 2]]) # Check selected table self.assertEqual( @@ -148,11 +162,11 @@ def test_layout_selection_scatter_table(self): ] ) - def test_overlay_scatter_errorbars(self, dynamic=False): - scatter = hv.Scatter(self.data, kdims='x', vdims='y') - error = hv.ErrorBars(self.data, kdims='x', vdims=['y', 'e']) - lnk_sel = link_selections.instance() - overlay = scatter * error + def test_overlay_points_errorbars(self, dynamic=False): + points = Points(self.data) + error = ErrorBars(self.data, kdims='x', vdims=['y', 'e']) + lnk_sel = link_selections.instance(unselected_color='#ff0000') + overlay = points * error if dynamic: overlay = hv.util.Dynamic(overlay) @@ -160,16 +174,12 @@ def test_overlay_scatter_errorbars(self, dynamic=False): current_obj = linked[()] # Check initial base layers - self.check_base_scatter_like(current_obj.Scatter.I, lnk_sel) - self.check_base_scatter_like(current_obj.ErrorBars.I, lnk_sel) + self.check_base_points_like(current_obj.Points.I, lnk_sel) + self.check_base_points_like(current_obj.ErrorBars.I, lnk_sel) # Check initial selection layers - self.check_overlay_scatter_like( - current_obj.Scatter.II, lnk_sel, self.data - ) - self.check_overlay_scatter_like( - current_obj.ErrorBars.II, lnk_sel, self.data - ) + self.check_overlay_points_like(current_obj.Points.II, lnk_sel, self.data) + self.check_overlay_points_like(current_obj.ErrorBars.II, lnk_sel, self.data) # Select first and third point boundsxy = lnk_sel._selection_expr_streams[0]._source_streams[0] @@ -177,42 +187,39 @@ def test_overlay_scatter_errorbars(self, dynamic=False): current_obj = linked[()] # Check base layers haven't changed - self.check_base_scatter_like(current_obj.Scatter.I, lnk_sel) - self.check_base_scatter_like(current_obj.ErrorBars.I, lnk_sel) + self.check_base_points_like(current_obj.Points.I, lnk_sel) + self.check_base_points_like(current_obj.ErrorBars.I, lnk_sel) # Check selected layers - self.check_overlay_scatter_like( - current_obj.Scatter.II, lnk_sel, self.data.iloc[[0, 2]] - ) - self.check_overlay_scatter_like( - current_obj.ErrorBars.II, lnk_sel, self.data.iloc[[0, 2]] - ) + self.check_overlay_points_like(current_obj.Points.II, lnk_sel, + self.data.iloc[[0, 2]]) + self.check_overlay_points_like(current_obj.ErrorBars.II, lnk_sel, + self.data.iloc[[0, 2]]) - def test_overlay_scatter_errorbars_dynamic(self): - self.test_overlay_scatter_errorbars(dynamic=True) + def test_overlay_points_errorbars_dynamic(self): + self.test_overlay_points_errorbars(dynamic=True) @ds_skip def test_datashade_selection(self): - scatter = hv.Scatter(self.data, kdims='x', vdims='y') - layout = scatter + dynspread(datashade(scatter)) + points = Points(self.data) + layout = points + dynspread(datashade(points)) - lnk_sel = link_selections.instance() + lnk_sel = link_selections.instance(unselected_color='#ff0000') linked = lnk_sel(layout) current_obj = linked[()] - # Check base scatter layer - self.check_base_scatter_like(current_obj[0][()].Scatter.I, lnk_sel) + # Check base points layer + self.check_base_points_like(current_obj[0][()].Points.I, lnk_sel) # Check selection layer - self.check_overlay_scatter_like( - current_obj[0][()].Scatter.II, lnk_sel, self.data - ) + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data) # Check RGB base layer self.assertEqual( current_obj[1][()].RGB.I, dynspread( - datashade(scatter, cmap=lnk_sel.unselected_cmap, alpha=255) + datashade(points, cmap=lnk_sel.unselected_cmap, alpha=255) )[()] ) @@ -220,29 +227,28 @@ def test_datashade_selection(self): self.assertEqual( current_obj[1][()].RGB.II, dynspread( - datashade(scatter, cmap=lnk_sel.selected_cmap, alpha=0) + datashade(points, cmap=lnk_sel.selected_cmap, alpha=255) )[()] ) # Perform selection of second and third point boundsxy = lnk_sel._selection_expr_streams[0]._source_streams[0] - self.assertIsInstance(boundsxy, hv.streams.BoundsXY) + self.assertIsInstance(boundsxy, SelectionXY) boundsxy.event(bounds=(0, 1, 5, 5)) current_obj = linked[()] - # Check that base scatter layer is unchanged - self.check_base_scatter_like(current_obj[0][()].Scatter.I, lnk_sel) + # Check that base points layer is unchanged + self.check_base_points_like(current_obj[0][()].Points.I, lnk_sel) - # Check scatter selection layer - self.check_overlay_scatter_like( - current_obj[0][()].Scatter.II, lnk_sel, self.data.iloc[1:] - ) + # Check points selection layer + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data.iloc[1:]) # Check that base RGB layer is unchanged self.assertEqual( current_obj[1][()].RGB.I, dynspread( - datashade(scatter, cmap=lnk_sel.unselected_cmap, alpha=255) + datashade(points, cmap=lnk_sel.unselected_cmap, alpha=255) )[()] ) @@ -251,47 +257,352 @@ def test_datashade_selection(self): current_obj[1][()].RGB.II, dynspread( datashade( - scatter.iloc[1:], cmap=lnk_sel.selected_cmap, alpha=255 + points.iloc[1:], cmap=lnk_sel.selected_cmap, alpha=255 ) )[()] ) - def test_scatter_selection_streaming(self): + def test_points_selection_streaming(self): buffer = hv.streams.Buffer(self.data.iloc[:2], index=False) - scatter = hv.DynamicMap(hv.Scatter, streams=[buffer]) - lnk_sel = link_selections.instance() - linked = lnk_sel(scatter) + points = hv.DynamicMap(Points, streams=[buffer]) + lnk_sel = link_selections.instance(unselected_color='#ff0000') + linked = lnk_sel(points) # Perform selection of first and (future) third point boundsxy = lnk_sel._selection_expr_streams[0]._source_streams[0] - self.assertIsInstance(boundsxy, hv.streams.BoundsXY) + self.assertIsInstance(boundsxy, hv.streams.SelectionXY) boundsxy.event(bounds=(0, 0, 4, 2)) current_obj = linked[()] # Check initial base layer - self.check_base_scatter_like( - current_obj.Scatter.I, lnk_sel, self.data.iloc[:2] + self.check_base_points_like( + current_obj.Points.I, lnk_sel, self.data.iloc[:2] ) # Check selection layer - self.check_overlay_scatter_like( - current_obj.Scatter.II, lnk_sel, self.data.iloc[[0]] - ) + self.check_overlay_points_like(current_obj.Points.II, lnk_sel, + self.data.iloc[[0]]) # Now stream third point to the DynamicMap buffer.send(self.data.iloc[[2]]) current_obj = linked[()] # Check initial base layer - self.check_base_scatter_like( - current_obj.Scatter.I, lnk_sel, self.data + self.check_base_points_like( + current_obj.Points.I, lnk_sel, self.data ) # Check selection layer - self.check_overlay_scatter_like( - current_obj.Scatter.II, lnk_sel, self.data.iloc[[0, 2]] + self.check_overlay_points_like(current_obj.Points.II, lnk_sel, + self.data.iloc[[0, 2]]) + + def do_crossfilter_points_histogram( + self, selection_mode, cross_filter_mode, + selected1, selected2, selected3, selected4, + points_region1, points_region2, points_region3, points_region4, + hist_region2, hist_region3, hist_region4, show_regions=True, dynamic=False + ): + points = Points(self.data) + hist = points.hist('x', adjoin=False, normed=False, num_bins=5) + + if dynamic: + # Convert points to DynamicMap that returns the element + hist_orig = hist + points = hv.util.Dynamic(points) + else: + hist_orig = hist + + lnk_sel = link_selections.instance( + selection_mode=selection_mode, + cross_filter_mode=cross_filter_mode, + show_regions=show_regions, + selected_color='#00ff00', + unselected_color='#ff0000' + ) + linked = lnk_sel(points + hist) + current_obj = linked[()] + + # Check initial base points + self.check_base_points_like( + current_obj[0][()].Points.I, + lnk_sel ) + # Check initial selection overlay points + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data) + + # Initial region bounds all None + self.assertEqual(len(current_obj[0][()].Curve.I), 0) + + # Check initial base histogram + base_hist = current_obj[1][()].Histogram.I + self.assertEqual( + self.element_color(base_hist), lnk_sel.unselected_color + ) + self.assertEqual(base_hist.data, hist_orig.data) + + # No selection region + region_hist = current_obj[1][()].Histogram.II + self.assertEqual(region_hist.data, hist_orig.pipeline(hist_orig.dataset.iloc[:0]).data) + + # Check initial selection overlay Histogram + selection_hist = current_obj[1][()].Histogram.III + self.assertEqual( + self.element_color(selection_hist), + self.expected_selection_color(selection_hist, lnk_sel) + ) + self.assertEqual(selection_hist.data, hist_orig.data) + + # (1) Perform selection on points of points [1, 2] + points_boundsxy = lnk_sel._selection_expr_streams[0]._source_streams[0] + self.assertIsInstance(points_boundsxy, SelectionXY) + points_boundsxy.event(bounds=(1, 1, 4, 4)) + + # Get current object + current_obj = linked[()] + + # Check base points unchanged + self.check_base_points_like( + current_obj[0][()].Points.I, + lnk_sel + ) + + # Check points selection overlay + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data.iloc[selected1]) + + # Check points region bounds + region_bounds = current_obj[0][()].Rectangles.I + self.assertEqual(region_bounds, Rectangles(points_region1)) + + if show_regions: + self.assertEqual( + self.element_color(region_bounds), + box_region_color + ) + + # Check histogram bars selected + selection_hist = current_obj[1][()].Histogram.III + self.assertEqual( + selection_hist.data, hist_orig.pipeline(hist_orig.dataset.iloc[selected1]).data + ) + + # (2) Perform selection on histogram bars [0, 1] + hist_boundsxy = lnk_sel._selection_expr_streams[1]._source_streams[0] + self.assertIsInstance(hist_boundsxy, SelectionXY) + hist_boundsxy.event(bounds=(0, 0, 2.5, 2)) + + points_unsel, points_sel, points_region = current_obj[0][()].values() + + # Check points selection overlay + self.check_overlay_points_like(points_sel, lnk_sel, self.data.iloc[selected2]) + + self.assertEqual(points_region, Rectangles(points_region2)) + + # Check base histogram unchanged + base_hist, region_hist, sel_hist = current_obj[1][()].values() + self.assertEqual(self.element_color(base_hist), lnk_sel.unselected_color) + self.assertEqual(base_hist.data, hist_orig.data) + + # Check selection region covers first and second bar + if show_regions: + self.assertEqual(self.element_color(region_hist), hist_region_color) + self.assertEqual( + region_hist.data, hist_orig.pipeline(hist_orig.dataset.iloc[hist_region2]).data + ) + + # Check histogram selection overlay + self.assertEqual( + self.element_color(sel_hist), + self.expected_selection_color(sel_hist, lnk_sel) + ) + self.assertEqual( + sel_hist.data, hist_orig.pipeline(hist_orig.dataset.iloc[selected2]).data + ) + + # (3) Perform selection on points points [0, 2] + points_boundsxy = lnk_sel._selection_expr_streams[0]._source_streams[0] + self.assertIsInstance(points_boundsxy, SelectionXY) + points_boundsxy.event(bounds=(0, 0, 4, 2.5)) + + # Check selection overlay points contains only second point + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data.iloc[selected3]) + + # Check points region bounds + region_bounds = current_obj[0][()].Rectangles.I + self.assertEqual(region_bounds, Rectangles(points_region3)) + + # Check second and third histogram bars selected + selection_hist = current_obj[1][()].Histogram.III + self.assertEqual( + selection_hist.data, hist_orig.pipeline(hist_orig.dataset.iloc[selected3]).data + ) + + # Check selection region covers first and second bar + region_hist = current_obj[1][()].Histogram.II + self.assertEqual( + region_hist.data, + hist_orig.pipeline(hist_orig.dataset.iloc[hist_region3]).data + ) + + # (4) Perform selection of bars [1, 2] + hist_boundsxy = lnk_sel._selection_expr_streams[1]._source_streams[0] + self.assertIsInstance(hist_boundsxy, SelectionXY) + hist_boundsxy.event(bounds=(1.5, 0, 3.5, 2)) + + # Check points selection overlay + self.check_overlay_points_like(current_obj[0][()].Points.II, lnk_sel, + self.data.iloc[selected4]) + + # Check points region bounds + region_bounds = current_obj[0][()].Rectangles.I + self.assertEqual(region_bounds, Rectangles(points_region4)) + + # Check bar selection region + region_hist = current_obj[1][()].Histogram.II + if show_regions: + self.assertEqual( + self.element_color(region_hist), hist_region_color + ) + self.assertEqual( + region_hist.data, + hist_orig.pipeline(hist_orig.dataset.iloc[hist_region4]).data + ) + + # Check bar selection overlay + selection_hist = current_obj[1][()].Histogram.III + self.assertEqual( + self.element_color(selection_hist), + self.expected_selection_color(selection_hist, lnk_sel) + ) + self.assertEqual( + selection_hist.data, hist_orig.pipeline(hist_orig.dataset.iloc[selected4]).data + ) + + # cross_filter_mode="overwrite" + def test_points_histogram_overwrite_overwrite(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="overwrite", cross_filter_mode="overwrite", + selected1=[1, 2], selected2=[0, 1], selected3=[0, 2], selected4=[1, 2], + points_region1=[(1, 1, 4, 4)], + points_region2=[], + points_region3=[(0, 0, 4, 2.5)], + points_region4=[], + hist_region2=[0, 1], hist_region3=[], hist_region4=[1, 2], + dynamic=dynamic + ) + + def test_points_histogram_overwrite_overwrite_dynamic(self): + self.test_points_histogram_overwrite_overwrite(dynamic=True) + + def test_points_histogram_intersect_overwrite(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="intersect", cross_filter_mode="overwrite", + selected1=[1, 2], selected2=[0, 1], selected3=[0, 2], selected4=[1, 2], + points_region1=[(1, 1, 4, 4)], + points_region2=[], + points_region3=[(0, 0, 4, 2.5)], + points_region4=[], + hist_region2=[0, 1], hist_region3=[], hist_region4=[1, 2], + dynamic=dynamic + ) + + def test_points_histogram_intersect_overwrite_dynamic(self): + self.test_points_histogram_intersect_overwrite(dynamic=True) + + def test_points_histogram_union_overwrite(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="union", cross_filter_mode="overwrite", + selected1=[1, 2], selected2=[0, 1], selected3=[0, 2], selected4=[1, 2], + points_region1=[(1, 1, 4, 4)], + points_region2=[], + points_region3=[(0, 0, 4, 2.5)], + points_region4=[], + hist_region2=[0, 1], hist_region3=[], hist_region4=[1, 2], + dynamic=dynamic + ) + + def test_points_histogram_union_overwrite_dynamic(self): + self.test_points_histogram_union_overwrite(dynamic=True) + + # cross_filter_mode="intersect" + def test_points_histogram_overwrite_intersect(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="overwrite", cross_filter_mode="intersect", + selected1=[1, 2], selected2=[1], selected3=[0], selected4=[2], + points_region1=[(1, 1, 4, 4)], + points_region2=[(1, 1, 4, 4)], + points_region3=[(0, 0, 4, 2.5)], + points_region4=[(0, 0, 4, 2.5)], + hist_region2=[0, 1], hist_region3=[0, 1], hist_region4=[1, 2], + dynamic=dynamic + ) + + def test_points_histogram_overwrite_intersect_dynamic(self): + self.test_points_histogram_overwrite_intersect(dynamic=True) + + def test_points_histogram_overwrite_intersect_hide_region(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="overwrite", cross_filter_mode="intersect", + selected1=[1, 2], selected2=[1], selected3=[0], selected4=[2], + points_region1=[], + points_region2=[], + points_region3=[], + points_region4=[], + hist_region2=[], hist_region3=[], hist_region4=[], + show_regions=False, dynamic=dynamic + ) + + def test_points_histogram_overwrite_intersect_hide_region_dynamic(self): + self.test_points_histogram_overwrite_intersect_hide_region(dynamic=True) + + def test_points_histogram_intersect_intersect(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="intersect", cross_filter_mode="intersect", + selected1=[1, 2], selected2=[1], selected3=[], selected4=[], + points_region1=[(1, 1, 4, 4)], + points_region2=[(1, 1, 4, 4)], + points_region3=[(1, 1, 4, 4), (0, 0, 4, 2.5)], + points_region4=[(1, 1, 4, 4), (0, 0, 4, 2.5)], + hist_region2=[0, 1], hist_region3=[0, 1], hist_region4=[1], + dynamic=dynamic + ) + + def test_points_histogram_intersect_intersect_dynamic(self): + self.test_points_histogram_intersect_intersect(dynamic=True) + + def test_points_histogram_union_intersect(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="union", cross_filter_mode="intersect", + selected1=[1, 2], selected2=[1], selected3=[0, 1], selected4=[0, 1, 2], + points_region1=[(1, 1, 4, 4)], + points_region2=[(1, 1, 4, 4)], + points_region3=[(1, 1, 4, 4), (0, 0, 4, 2.5)], + points_region4=[(1, 1, 4, 4), (0, 0, 4, 2.5)], + hist_region2=[0, 1], hist_region3=[0, 1], hist_region4=[0, 1, 2], + dynamic=dynamic + ) + + def test_points_histogram_union_intersect_dynamic(self): + self.test_points_histogram_union_intersect(dynamic=True) + + def test_points_histogram_inverse_intersect(self, dynamic=False): + self.do_crossfilter_points_histogram( + selection_mode="inverse", cross_filter_mode="intersect", + selected1=[0], selected2=[], selected3=[], selected4=[], + points_region1=[(1, 1, 4, 4)], + points_region2=[(1, 1, 4, 4)], + points_region3=[(1, 1, 4, 4), (0, 0, 4, 2.5)], + points_region4=[(1, 1, 4, 4), (0, 0, 4, 2.5)], + hist_region2=[], hist_region3=[], hist_region4=[], + dynamic=dynamic + ) + + def test_points_histogram_inverse_intersect_dynamic(self): + self.test_points_histogram_inverse_intersect(dynamic=True) + # Backend implementations class TestLinkSelectionsPlotly(TestLinkSelections): @@ -307,20 +618,19 @@ def setUp(self): def tearDown(self): Store.current_backend = self._backend - def element_color(self, element): - if isinstance(element, hv.Table): + def element_color(self, element, color_prop=None): + if isinstance(element, Table): color = element.opts.get('style').kwargs['fill'] + elif isinstance(element, Rectangles): + color = element.opts.get('style').kwargs['line_color'] else: color = element.opts.get('style').kwargs['color'] - if isinstance(color, str): + if isinstance(color, (basestring, unicode)): return color else: return list(color) - def element_visible(self, element): - return element.opts.get('style').kwargs['visible'] - class TestLinkSelectionsBokeh(TestLinkSelections): def setUp(self): @@ -338,22 +648,19 @@ def tearDown(self): def element_color(self, element): color = element.opts.get('style').kwargs['color'] - if isinstance(color, str): + if isinstance(color, (basestring, unicode)): return color else: return list(color) - def element_visible(self, element): - return element.opts.get('style').kwargs['alpha'] > 0 - @skip("Coloring Bokeh table not yet supported") - def test_layout_selection_scatter_table(self): + def test_layout_selection_points_table(self): pass @skip("Bokeh ErrorBars selection not yet supported") - def test_overlay_scatter_errorbars(self): + def test_overlay_points_errorbars(self): pass @skip("Bokeh ErrorBars selection not yet supported") - def test_overlay_scatter_errorbars_dynamic(self): + def test_overlay_points_errorbars_dynamic(self): pass diff --git a/holoviews/tests/teststreams.py b/holoviews/tests/teststreams.py index ce47ce3046..bf9b65c7bc 100644 --- a/holoviews/tests/teststreams.py +++ b/holoviews/tests/teststreams.py @@ -883,7 +883,7 @@ def setUp(self): extension("bokeh") def test_selection_expr_stream_scatter_points(self): - for element_type in [Scatter, Points, Curve]: + for element_type in [Scatter, Points]: # Create SelectionExpr on element element = element_type(([1, 2, 3], [1, 5, 10])) expr_stream = SelectionExpr(element) @@ -900,8 +900,7 @@ def test_selection_expr_stream_scatter_points(self): # Check SelectionExpr values self.assertEqual( repr(expr_stream.selection_expr), - repr((dim('x') >= 1) & (dim('x') <= 3) & - (dim('y') >= 1) & (dim('y') <= 4)) + repr(((dim('x')>=1)&(dim('x')<=3))&((dim('y')>=1)&(dim('y')<=4))) ) self.assertEqual( expr_stream.bbox, @@ -909,7 +908,7 @@ def test_selection_expr_stream_scatter_points(self): ) def test_selection_expr_stream_invert_axes(self): - for element_type in [Scatter, Points, Curve]: + for element_type in [Scatter, Points]: # Create SelectionExpr on element element = element_type(([1, 2, 3], [1, 5, 10])).opts(invert_axes=True) expr_stream = SelectionExpr(element) @@ -926,8 +925,7 @@ def test_selection_expr_stream_invert_axes(self): # Check SelectionExpr values self.assertEqual( repr(expr_stream.selection_expr), - repr((dim('y') >= 1) & (dim('y') <= 3) & - (dim('x') >= 1) & (dim('x') <= 4)) + repr(((dim('y')>=1)&(dim('y')<=3))&((dim('x')>=1)&(dim('x')<=4))) ) self.assertEqual( expr_stream.bbox, @@ -935,7 +933,7 @@ def test_selection_expr_stream_invert_axes(self): ) def test_selection_expr_stream_invert_xaxis_yaxis(self): - for element_type in [Scatter, Points, Curve]: + for element_type in [Scatter, Points]: # Create SelectionExpr on element element = element_type(([1, 2, 3], [1, 5, 10])).opts( @@ -956,8 +954,7 @@ def test_selection_expr_stream_invert_xaxis_yaxis(self): # Check SelectionExpr values self.assertEqual( repr(expr_stream.selection_expr), - repr((dim('x') >= 1) & (dim('x') <= 3) & - (dim('y') >= 1) & (dim('y') <= 4)) + repr(((dim('x')>=1)&(dim('x')<=3))&((dim('y')>=1)&(dim('y')<=4))) ) self.assertEqual( expr_stream.bbox, @@ -1063,7 +1060,7 @@ def test_selection_expr_stream_hist_invert_xaxis_yaxis(self): self.assertEqual(expr_stream.bbox, {'x': (2.5, 5.5)}) def test_selection_expr_stream_dynamic_map(self): - for element_type in [Scatter, Points, Curve]: + for element_type in [Scatter, Points]: # Create SelectionExpr on element dmap = Dynamic(element_type(([1, 2, 3], [1, 5, 10]))) expr_stream = SelectionExpr(dmap) @@ -1080,8 +1077,7 @@ def test_selection_expr_stream_dynamic_map(self): # Check SelectionExpr values self.assertEqual( repr(expr_stream.selection_expr), - repr((dim('x') >= 1) & (dim('x') <= 3) & - (dim('y') >= 1) & (dim('y') <= 4)) + repr(((dim('x')>=1)&(dim('x')<=3))&((dim('y')>=1)&(dim('y')<=4))) ) self.assertEqual( expr_stream.bbox, diff --git a/holoviews/tests/util/testtransform.py b/holoviews/tests/util/testtransform.py index a14e32d947..76bbe3d68a 100644 --- a/holoviews/tests/util/testtransform.py +++ b/holoviews/tests/util/testtransform.py @@ -27,9 +27,11 @@ def setUp(self): self.repeating = pd.Series( ['A', 'B', 'C', 'A', 'B', 'C', 'A', 'B', 'C', 'A'] ) + self.booleans = self.repeating == 'A' self.dataset = Dataset( - (self.linear_ints, self.linear_floats, self.negative, self.repeating), - ['int', 'float', 'negative', 'categories'] + (self.linear_ints, self.linear_floats, + self.negative, self.repeating, self.booleans), + ['int', 'float', 'negative', 'categories', 'booleans'] ) if dd is None: @@ -116,6 +118,10 @@ def test_neg_transform(self): expr = -dim('negative') self.check_apply(expr, self.linear_floats) + def test_inv_transform(self): + expr = ~dim('booleans') + self.check_apply(expr, ~self.booleans) + # Binary operators def test_add_transform(self): @@ -224,6 +230,18 @@ def test_norm_transform(self): expr = dim('int').norm() self.check_apply(expr, (self.linear_ints-1)/9.) + def test_iloc_transform_int(self): + expr = dim('int').iloc[1] + self.check_apply(expr, self.linear_ints[1]) + + def test_iloc_transform_slice(self): + expr = dim('int').iloc[1:3] + self.check_apply(expr, self.linear_ints[1:3], skip_dask=True) + + def test_iloc_transform_list(self): + expr = dim('int').iloc[[1, 3, 5]] + self.check_apply(expr, self.linear_ints[[1, 3, 5]], skip_dask=True) + def test_bin_transform(self): expr = dim('int').bin([0, 5, 10]) expected = pd.Series( @@ -293,7 +311,7 @@ def test_multi_dim_expression(self): # Repr method def test_dim_repr(self): - self.assertEqual(repr(dim('float')), "'float'") + self.assertEqual(repr(dim('float')), "dim('float')") def test_unary_op_repr(self): self.assertEqual(repr(-dim('float')), "-dim('float')") diff --git a/holoviews/util/__init__.py b/holoviews/util/__init__.py index ae19b8160d..bea2cdfdc4 100644 --- a/holoviews/util/__init__.py +++ b/holoviews/util/__init__.py @@ -17,8 +17,8 @@ from ..core.operation import Operation from ..core.util import basestring, merge_options_to_dict, OrderedDict from ..core.operation import OperationCallable -from ..core.spaces import Callable from ..core import util +from ..operation.element import function from ..streams import Stream, Params from .settings import OutputSettings, list_formats, list_backends @@ -972,14 +972,13 @@ def dynamic_operation(*key, **kwargs): key, obj = resolve(key, kwargs) return apply(obj, *key, **kwargs) - if isinstance(self.p.operation, Operation): - return OperationCallable(dynamic_operation, inputs=[map_obj], - link_inputs=self.p.link_inputs, - operation=self.p.operation) - else: - return Callable(dynamic_operation, inputs=[map_obj], - link_inputs=self.p.link_inputs, - operation=apply) + operation = self.p.operation + if not isinstance(operation, Operation): + operation = function.instance(fn=apply) + return OperationCallable(dynamic_operation, inputs=[map_obj], + link_inputs=self.p.link_inputs, + operation=operation, + operation_kwargs=self.p.kwargs) def _make_dynamic(self, hmap, dynamic_fn, streams): diff --git a/holoviews/util/transform.py b/holoviews/util/transform.py index 79b224588f..8c38eb7ec5 100644 --- a/holoviews/util/transform.py +++ b/holoviews/util/transform.py @@ -1,22 +1,17 @@ from __future__ import division import operator + from types import BuiltinFunctionType, BuiltinMethodType, FunctionType, MethodType import numpy as np from ..core.dimension import Dimension from ..core.util import basestring, unique_iterator -from ..element import Graph - -function_types = ( - BuiltinFunctionType, BuiltinMethodType, FunctionType, - MethodType, np.ufunc) - def _maybe_map(numpy_fn): def fn(values, *args, **kwargs): - series_like = hasattr(values, 'index') + series_like = hasattr(values, 'index') and not isinstance(values, list) map_fn = (getattr(values, 'map_partitions', None) or getattr(values, 'map_blocks', None)) if map_fn: @@ -71,6 +66,24 @@ def lognorm(values, min=None, max=None): return (np.log(values) - min) / (max-min) +class iloc(object): + """Implements integer array indexing for dim expressions. + """ + + __name__ = 'iloc' + + def __init__(self, dim_expr): + self.expr = dim_expr + self.index = slice(None) + + def __getitem__(self, index): + self.index = index + return dim(self.expr, self) + + def __call__(self, values): + return values[self.index] + + @_maybe_map def bin(values, bins, labels=None): """Bins data into declared bins @@ -140,6 +153,15 @@ def categorize(values, categories, default=None): astype = _maybe_map(np.asarray) round_ = _maybe_map(np.round) +def _python_isin(array, values): + return [v in values for v in array] + +python_isin = _maybe_map(_python_isin) + +function_types = ( + BuiltinFunctionType, BuiltinMethodType, FunctionType, + MethodType, np.ufunc, iloc) + class dim(object): """ @@ -150,7 +172,7 @@ class dim(object): """ _binary_funcs = { - operator.add: '+', operator.and_: '&', operator.eq: '=', + operator.add: '+', operator.and_: '&', operator.eq: '==', operator.floordiv: '//', operator.ge: '>=', operator.gt: '>', operator.le: '<=', operator.lshift: '<<', operator.lt: '<', operator.mod: '%', operator.mul: '*', operator.ne: '!=', @@ -166,8 +188,10 @@ class dim(object): categorize: 'categorize', digitize: 'digitize', isin: 'isin', + python_isin: 'isin', astype: 'astype', round_: 'round', + iloc: 'iloc' } _numpy_funcs = { @@ -206,6 +230,20 @@ def __init__(self, obj, *args, **kwargs): 'reverse': kwargs.pop('reverse', False)}] self.ops = ops + + def clone(self, dimension=None, ops=None): + """ + Creates a clone of the dim expression optionally overriding + the dim and ops. + """ + if dimension is None: + dimension = self.dimension + new_dim = dim(dimension) + if ops is None: + ops = list(self.ops) + new_dim.ops = ops + return new_dim + @classmethod def register(cls, key, function): """ @@ -227,6 +265,9 @@ def pipe(cls, func, *args, **kwargs): args[k] = dim(arg) return dim(args[0], func, *args[1:], **kwargs) + def __hash__(self): + return hash(repr(self)) + # Builtin functions def __abs__(self): return dim(self, abs) def __round__(self, ndigits=None): @@ -236,6 +277,7 @@ def __round__(self, ndigits=None): # Unary operators def __neg__(self): return dim(self, operator.neg) def __not__(self): return dim(self, operator.not_) + def __invert__(self): return dim(self, operator.inv) def __pos__(self): return dim(self, operator.pos) # Binary operators @@ -299,8 +341,15 @@ def log10(self, *args, **kwargs): return dim(self, np.log10, *args, **kwargs) ## Custom functions def astype(self, dtype): return dim(self, astype, dtype=dtype) def round(self, decimals=0): return dim(self, round_, decimals=decimals) - def digitize(self, *args, **kwargs): return dim(self, digitize, *args, **kwargs) - def isin(self, *args, **kwargs): return dim(self, isin, *args, **kwargs) + def digitize(self, *args, **kwargs): return dim(self, digitize, *args, **kwargs) + def isin(self, *args, **kwargs): + if kwargs.pop('object', None): + return dim(self, python_isin, *args, **kwargs) + return dim(self, isin, *args, **kwargs) + + @property + def iloc(self): + return iloc(self) def bin(self, bins, labels=None): """Bins continuous values. @@ -365,6 +414,8 @@ def applies(self, dataset): Dataset, i.e. whether all referenced dimensions can be resolved. """ + from ..element import Graph + if isinstance(self.dimension, dim): applies = self.dimension.applies(dataset) else: @@ -380,16 +431,8 @@ def applies(self, dataset): applies &= arg.applies(dataset) return applies - def apply( - self, - dataset, - flat=False, - expanded=None, - ranges={}, - all_values=False, - keep_index=False, - compute=True, - ): + def apply(self, dataset, flat=False, expanded=None, ranges={}, all_values=False, + keep_index=False, compute=True): """Evaluates the transform on the supplied dataset. Args: @@ -409,6 +452,8 @@ def apply( Returns: values: NumPy array computed by evaluating the expression """ + from ..element import Graph + dimension = self.dimension if expanded is None: expanded = not ((dataset.interface.gridded and dimension in dataset.kdims) or @@ -455,11 +500,11 @@ def apply( def __repr__(self): op_repr = "'%s'" % self.dimension - for o in self.ops: - if 'dim(' in op_repr: - prev = '{repr}' if op_repr.endswith(')') else '({repr})' + for i, o in enumerate(self.ops): + if i == 0: + prev = 'dim({repr}' else: - prev = 'dim({repr})' + prev = '({repr}' fn = o['fn'] ufunc = isinstance(fn, np.ufunc) args = ', '.join([repr(r) for r in o['args']]) if o['args'] else '' @@ -470,7 +515,9 @@ def __repr__(self): if o['reverse']: format_string = '{args}{fn}'+prev else: - format_string = prev+'{fn}{args}' + format_string = prev+'){fn}{args}' + if any(isinstance(a, dim) for a in o['args']): + format_string = format_string.replace('{args}', '({args})') elif fn in self._unary_funcs: fn_name = self._unary_funcs[fn] format_string = '{fn}' + prev @@ -481,10 +528,12 @@ def __repr__(self): format_string = '{fn}'+prev elif fn in self._numpy_funcs: fn_name = self._numpy_funcs[fn] - format_string = prev+'.{fn}(' + format_string = prev+').{fn}(' + elif isinstance(fn, iloc): + format_string = prev+').iloc[{0}]'.format(repr(fn.index)) elif fn in self._custom_funcs: fn_name = self._custom_funcs[fn] - format_string = prev+'.{fn}(' + format_string = prev+').{fn}(' elif ufunc: fn_name = str(fn)[8:-2] if not (prev.startswith('dim') or prev.endswith(')')): @@ -494,14 +543,21 @@ def __repr__(self): if fn_name in dir(np): format_string = '.'.join([self._namespaces['numpy'], format_string]) else: - format_string = prev+', {fn}' + format_string = 'dim(' + prev+', {fn}' if args: - format_string += ', {args}' + if not format_string.endswith('('): + format_string += ', ' + format_string += '{args}' if kwargs: format_string += ', {kwargs}' elif kwargs: format_string += '{kwargs}' - format_string += ')' op_repr = format_string.format(fn=fn_name, repr=op_repr, args=args, kwargs=kwargs) + if op_repr.count('(') - op_repr.count(')') > 0: + op_repr += ')' + if not self.ops: + op_repr = 'dim({repr})'.format(repr=op_repr) + if op_repr.count('(') - op_repr.count(')') > 0: + op_repr += ')' return op_repr