From cc7d64178cd0bc3173fd615db692a5a45ecbc3b7 Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 11:49:32 -0700 Subject: [PATCH 01/10] Document how to use regrid for image data --- examples/user_guide/15-Large_Data.ipynb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index 755c895f3b..ee2ab1461b 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -159,7 +159,9 @@ "source": [ "In all three of the above plots, `rasterize()` is being called to aggregate the data (a large set of x,y locations) into a rectangular grid, with each grid cell counting up the number of points that fall into it. In the plot on the left, only `rasterize()` is done, and the resulting numeric array of counts is passed to Bokeh for colormapping. Bokeh can then use dynamic (client-side, browser-based) operations in JavaScript, allowing users to have dynamic control over even static HTML plots. For instance, in this case, users can use the Box Select tool and select a range of the histogram shown, dynamically remapping the colors used in the plot to cover the selected range.\n", "\n", - "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. You can also try the lower-level ``hv.aggregate()`` (for points and lines) and ``hv.regrid()` (for image/raster data) operations, which may provide more control." + "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` gives more control over how the data is aggregated.\n", + "\n", + "When working with already gridded data, such as images, it is not always necessary to perform the rasterization/aggregation step. If you just want to increase the performance by automatically rerender the image as the zoom level changes, `regrid()` can be used. Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior is often not desirable when working with images; to instead use a fixed range for colormap the `clim` parameter can be passed on to the `bokeh` backend via the `opts()` method." ] }, { From 2ef0c58bbd3b0bb6d7828eeda14fdc4093b810b7 Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 11:50:24 -0700 Subject: [PATCH 02/10] Add example for auto and fixed colorrange --- examples/user_guide/15-Large_Data.ipynb | 27 ++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index ee2ab1461b..24cf3442a3 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -25,7 +25,7 @@ "import holoviews as hv\n", "from holoviews import opts\n", "import datashader as ds\n", - "from holoviews.operation.datashader import datashade, shade, dynspread, rasterize\n", + "from holoviews.operation.datashader import datashade, shade, dynspread, rasterize, regrid\n", "from holoviews.operation import decimate\n", "\n", "hv.extension('bokeh','matplotlib')\n", @@ -164,6 +164,31 @@ "When working with already gridded data, such as images, it is not always necessary to perform the rasterization/aggregation step. If you just want to increase the performance by automatically rerender the image as the zoom level changes, `regrid()` can be used. Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior is often not desirable when working with images; to instead use a fixed range for colormap the `clim` parameter can be passed on to the `bokeh` backend via the `opts()` method." ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "n = 10_000\n", + "\n", + "# Strong signal on top\n", + "x = rs.pareto(n, n)\n", + "y = x + rs.standard_normal(n)\n", + "im1, *_ = np.histogram2d(x, y, bins=60)\n", + "\n", + "# Weak signal in the middle\n", + "x2 = rs.standard_normal(n)\n", + "y2 = 5 * x + 10 * rs.standard_normal(n)\n", + "im2, *_ = np.histogram2d(x2, y2, bins=60)\n", + "\n", + "im = im1 + im2\n", + "hv_im = hv.Image(im).opts(active_tools=['wheel_zoom'])\n", + "auto_scale_grid = regrid(hv_im).opts(title='Automatic color range rescaling')\n", + "fixed_scale_grid = regrid(hv_im).opts(title='Fixed color range', clim=(im.min(), im.max()))\n", + "auto_scale_grid + fixed_scale_grid" + ] + }, { "cell_type": "markdown", "metadata": {}, From 7c3b6a99a68a07e57fcb0f1cad0b0cce12cde1d0 Mon Sep 17 00:00:00 2001 From: Joel Ostblom Date: Fri, 16 Aug 2019 16:01:52 -0700 Subject: [PATCH 03/10] Remove mentioned of not needing rasterization on images Co-Authored-By: James A. Bednar --- examples/user_guide/15-Large_Data.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index 24cf3442a3..b32b2300c5 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -161,7 +161,7 @@ "\n", "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` gives more control over how the data is aggregated.\n", "\n", - "When working with already gridded data, such as images, it is not always necessary to perform the rasterization/aggregation step. If you just want to increase the performance by automatically rerender the image as the zoom level changes, `regrid()` can be used. Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior is often not desirable when working with images; to instead use a fixed range for colormap the `clim` parameter can be passed on to the `bokeh` backend via the `opts()` method." + "Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior may not be desirable when working with images; to instead use a fixed range for colormap the `clim` parameter can be passed on to the `bokeh` backend via the `opts()` method." ] }, { From f4ce55f54852e084020b9859278363525a697504 Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 16:05:57 -0700 Subject: [PATCH 04/10] Substitute rasterize for regrid --- examples/user_guide/15-Large_Data.ipynb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index b32b2300c5..d2969c6a44 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -25,7 +25,7 @@ "import holoviews as hv\n", "from holoviews import opts\n", "import datashader as ds\n", - "from holoviews.operation.datashader import datashade, shade, dynspread, rasterize, regrid\n", + "from holoviews.operation.datashader import datashade, shade, dynspread, rasterize\n", "from holoviews.operation import decimate\n", "\n", "hv.extension('bokeh','matplotlib')\n", @@ -184,8 +184,8 @@ "\n", "im = im1 + im2\n", "hv_im = hv.Image(im).opts(active_tools=['wheel_zoom'])\n", - "auto_scale_grid = regrid(hv_im).opts(title='Automatic color range rescaling')\n", - "fixed_scale_grid = regrid(hv_im).opts(title='Fixed color range', clim=(im.min(), im.max()))\n", + "auto_scale_grid = rasterize(hv_im).opts(title='Automatic color range rescaling')\n", + "fixed_scale_grid = rasterize(hv_im).opts(title='Fixed color range', clim=(im.min(), im.max()))\n", "auto_scale_grid + fixed_scale_grid" ] }, From 250a24ee8cdb0bb09c46acca3213e5f32147f2b1 Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 16:06:30 -0700 Subject: [PATCH 05/10] Add missing definition of rs --- examples/user_guide/15-Large_Data.ipynb | 1 + 1 file changed, 1 insertion(+) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index d2969c6a44..f9aeeb0beb 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -173,6 +173,7 @@ "n = 10_000\n", "\n", "# Strong signal on top\n", + "rs = np.random.RandomState(101010)\n", "x = rs.pareto(n, n)\n", "y = x + rs.standard_normal(n)\n", "im1, *_ = np.histogram2d(x, y, bins=60)\n", From f5c3a691312b19f37113e885cb48262b21f70a4a Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 16:15:24 -0700 Subject: [PATCH 06/10] Clarify wording --- examples/user_guide/15-Large_Data.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index f9aeeb0beb..b6924add2f 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -161,7 +161,7 @@ "\n", "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` gives more control over how the data is aggregated.\n", "\n", - "Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior may not be desirable when working with images; to instead use a fixed range for colormap the `clim` parameter can be passed on to the `bokeh` backend via the `opts()` method." + "Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior may not be desirable when working with images; to instead use a fixed colormap range, the `clim` parameter can be passed to the `bokeh` backend via the `opts()` method." ] }, { From 63796cce29cc7e2d9faca99c8336053e3cb2c23c Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 16:15:49 -0700 Subject: [PATCH 07/10] Make variable name more explicit --- examples/user_guide/15-Large_Data.ipynb | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index b6924add2f..78cb54a422 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -176,17 +176,17 @@ "rs = np.random.RandomState(101010)\n", "x = rs.pareto(n, n)\n", "y = x + rs.standard_normal(n)\n", - "im1, *_ = np.histogram2d(x, y, bins=60)\n", + "img1, *_ = np.histogram2d(x, y, bins=60)\n", "\n", "# Weak signal in the middle\n", "x2 = rs.standard_normal(n)\n", "y2 = 5 * x + 10 * rs.standard_normal(n)\n", - "im2, *_ = np.histogram2d(x2, y2, bins=60)\n", + "img2, *_ = np.histogram2d(x2, y2, bins=60)\n", "\n", - "im = im1 + im2\n", - "hv_im = hv.Image(im).opts(active_tools=['wheel_zoom'])\n", - "auto_scale_grid = rasterize(hv_im).opts(title='Automatic color range rescaling')\n", - "fixed_scale_grid = rasterize(hv_im).opts(title='Fixed color range', clim=(im.min(), im.max()))\n", + "img = img1 + img2\n", + "hv_img = hv.Image(img).opts(active_tools=['wheel_zoom'])\n", + "auto_scale_grid = rasterize(hv_img).opts(title='Automatic color range rescaling')\n", + "fixed_scale_grid = rasterize(hv_img).opts(title='Fixed color range', clim=(img.min(), img.max()))\n", "auto_scale_grid + fixed_scale_grid" ] }, From e8b3d89eb141e6d8a1a8e101b7a39d7251e83163 Mon Sep 17 00:00:00 2001 From: Joel Ostblom Date: Fri, 16 Aug 2019 16:20:11 -0700 Subject: [PATCH 08/10] Include mention of regrid again Co-Authored-By: Philipp Rudiger --- examples/user_guide/15-Large_Data.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index 78cb54a422..9331e16122 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -159,7 +159,7 @@ "source": [ "In all three of the above plots, `rasterize()` is being called to aggregate the data (a large set of x,y locations) into a rectangular grid, with each grid cell counting up the number of points that fall into it. In the plot on the left, only `rasterize()` is done, and the resulting numeric array of counts is passed to Bokeh for colormapping. Bokeh can then use dynamic (client-side, browser-based) operations in JavaScript, allowing users to have dynamic control over even static HTML plots. For instance, in this case, users can use the Box Select tool and select a range of the histogram shown, dynamically remapping the colors used in the plot to cover the selected range.\n", "\n", - "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` gives more control over how the data is aggregated.\n", + "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` and ``regrid`` operations give more control over how the data is aggregated.\n", "\n", "Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior may not be desirable when working with images; to instead use a fixed colormap range, the `clim` parameter can be passed to the `bokeh` backend via the `opts()` method." ] From 76ca5c22bacd99817b2edcac99bcba86422d8aff Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 16:35:17 -0700 Subject: [PATCH 09/10] Be explicit about when opts is used --- examples/user_guide/15-Large_Data.ipynb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index 9331e16122..34a75bfe82 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -159,9 +159,9 @@ "source": [ "In all three of the above plots, `rasterize()` is being called to aggregate the data (a large set of x,y locations) into a rectangular grid, with each grid cell counting up the number of points that fall into it. In the plot on the left, only `rasterize()` is done, and the resulting numeric array of counts is passed to Bokeh for colormapping. Bokeh can then use dynamic (client-side, browser-based) operations in JavaScript, allowing users to have dynamic control over even static HTML plots. For instance, in this case, users can use the Box Select tool and select a range of the histogram shown, dynamically remapping the colors used in the plot to cover the selected range.\n", "\n", - "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` and ``regrid`` operations give more control over how the data is aggregated.\n", + "The other two plots should be identical. In both cases, the numerical array output of `rasterize()` is mapped into RGB colors by Datashader itself, in Python (\"server-side\"), which allows special Datashader computations like the histogram-equalization in the above plots and the \"spreading\" discussed below. The `shade()` and `datashade()` operations accept a `cmap` argument that lets you control the colormap used, which can be selected to match the HoloViews/Bokeh `cmap` option but is strictly independent of it. See ``hv.help(rasterize)``, ``hv.help(shade)``, and ``hv.help(datashade)`` for options that can be selected, and the [Datashader web site](http://datashader.org) for all the details. The lower-level `aggregate()` and `regrid()` give more control over how the data is aggregated.\n", "\n", - "Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior may not be desirable when working with images; to instead use a fixed colormap range, the `clim` parameter can be passed to the `bokeh` backend via the `opts()` method." + "Since datashader only sends the data currently in view to the plotting backend, the default behavior is to rescale colormap to the range of the visible data as the zoom level changes. This behavior may not be desirable when working with images; to instead use a fixed colormap range, the `clim` parameter can be passed to the `bokeh` backend via the `opts()` method. Note that this approach works with `rasterize()` where the colormapping is done by the `bokeh` backend. With `datashade()`, the colormapping is done with the `shade()` function which takes a `clims` parameter directly instead of passing additional parameters to the backend via `opts()`." ] }, { From b795c9955d6ae8ca1a35dd4f943ca3f03e58e731 Mon Sep 17 00:00:00 2001 From: joelostblom Date: Fri, 16 Aug 2019 16:35:59 -0700 Subject: [PATCH 10/10] Include gif and supress notebook output --- examples/user_guide/15-Large_Data.ipynb | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/examples/user_guide/15-Large_Data.ipynb b/examples/user_guide/15-Large_Data.ipynb index 34a75bfe82..2569ec7612 100644 --- a/examples/user_guide/15-Large_Data.ipynb +++ b/examples/user_guide/15-Large_Data.ipynb @@ -187,7 +187,14 @@ "hv_img = hv.Image(img).opts(active_tools=['wheel_zoom'])\n", "auto_scale_grid = rasterize(hv_img).opts(title='Automatic color range rescaling')\n", "fixed_scale_grid = rasterize(hv_img).opts(title='Fixed color range', clim=(img.min(), img.max()))\n", - "auto_scale_grid + fixed_scale_grid" + "auto_scale_grid + fixed_scale_grid; # Output supressed and gif shown below instead" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![](http://assets.holoviews.org/gifs/guides/user_guide/Large_Data/rasterize_color_range.gif)" ] }, {