Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added an example in the user guide on using operations on 2d elements. #2316

Merged
merged 3 commits into from
Feb 9, 2018
Merged

Added an example in the user guide on using operations on 2d elements. #2316

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3fc154c
Added an example in the user guide on using operations on 2d elements.
drs251 Feb 7, 2018
2227fc9
Fixed type
drs251 Feb 7, 2018
0ff0f45
Cleared up the code and improved the explanation
drs251 Feb 9, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions examples/user_guide/10-Transforming_Elements.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,6 +298,105 @@
"source": [
"Using a few additional lines we have now evaluated the operation over a number of different parameters values, allowing us to process the data with different smoothing parameters. In addition, by interacting with this visualization we can gain a better understanding of the operation parameters as well as gain insights into the structure of the underlying data.\n",
"\n",
"## Operations on 2D elements\n",
"\n",
"Let's look at another example of operations in action, this time applying a simple filter to an `Image`. The basic idea is the same as above, although accessing the values to be transformed is a bit more complicated. First, we prepare an example image:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%output backend=\"matplotlib\"\n",
"\n",
"from scipy.misc import ascent\n",
"\n",
"stairs_image = hv.Image(ascent()[200:500, :], bounds=[0, 0, ascent().shape[1], 300], label=\"stairs\")\n",
"stairs_image"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We'll define a simple ``Operation``, which takes an ``Image`` and applies a high-pass or low-pass filter. We then use this to build a ``HoloMap`` of images filtered with different sigma values:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%output backend=\"matplotlib\"\n",
"\n",
"from scipy import ndimage\n",
"\n",
"class image_filter(hv.Operation):\n",
" \n",
" sigma = param.Number(default=5)\n",
" \n",
" type_ = param.String(default=\"low-pass\")\n",
"\n",
" def _process(self, element, key=None):\n",
" # we don't need the x and y coordinates in this example, \n",
" # but we could access them like this\n",
" x_coords = element.dimension_values(0, expanded=False)\n",
" y_coords = element.dimension_values(1, expanded=False)\n",
" \n",
" # setting flat=False will preserve the matrix shape\n",
" data = element.dimension_values(2, flat=False)[::-1]\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to:

data = element.dimension_values(2, flat=False)

And then apply my suggestion below.

" \n",
" if self.p.type_ == \"high-pass\":\n",
" new_data = data - ndimage.gaussian_filter(data, self.p.sigma)\n",
" else:\n",
" new_data = ndimage.gaussian_filter(data, self.p.sigma)\n",
" \n",
" # make an exact copy of all setting and axes etc., just with different data:\n",
" element = element.clone(data=new_data)\n",
" element = element.relabel(element.label + \" ({} filtered)\".format(self.p.type_))\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change these two lines to:

label = element.label + " ({} filtered)".format(self.p.type_)
element = element.clone((xs, ys, new_data), label=label)

" return element\n",
"\n",
"stairs_map = hv.HoloMap({sigma: image_filter(stairs_image, sigma=sigma)\n",
" for sigma in range(0, 12, 1)}, kdims=\"sigma\")\n",
"stairs_map"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Just as in the previous example, it is quite straight-forward to build a HoloMap containing results for different parameter values. Inside the ``_process()`` method, the given parameters can be accessed as ``self.p.<parameter-name>`` (note that ``self.<parameter_name>`` always contains the default value!). Since we did not specify the ``_type`` parameter, it defaulted to \"low-pass\".\n",
"\n",
"There are some peculiarities when applying operations to two-dimensional elements:\n",
"\n",
"- Understanding the ``dimension_values()`` method: In principle, one could use ``element.data`` to access the element's data, however, using ``dimension_values()`` means one does not have to worry about different data formats for different elements. The first parameter specifies the dimension to be returned. Values 0 and 1 refer to the horizontal and vertical axis, respectively. For ``Image`` and ``Raster`` elements, 2 refers to the magnitude, while for ``RGB``, 2, 3 and 4 refer to the red, green and blue channels. Setting ``expanded=False`` yields only the axis, while the default setting ``expanded=True`` returns a value for every pixel. Specifying ``flat=False`` means that the data's matrix shape will be preserved, which is what we need for this kind of filter.\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some things that should maybe be clarified:

  1. While it is true that the data format depends on the element type, all elements also support a variety of different formats, which are defined by the datatype. In the background a number of Interface classes allow elements to access and manipulate the data in different formats, be that numpy arrays, dask arrays, xarrays or whatever else. We need to add a user guide about interfaces eventually so maybe just say something like: "In principle, one could use element.data to access the element's data, however, since HoloViews can wrap a wide range of data formats dimension_values() provides an API that lets you access the data without having to worry about the type of the data.

  2. While it is true that dimension_values allows using integers to reference which values you want, it is primarily about the dimensions. On a 2D Element 0 and 1 correspond to the two key dimensions, while subsequent values refer to the value dimensions, basically it's defined as element.dimensions()[index]. So maybe say something like:

On a 2D element like an Image or Raster the first two dimensions reference the key dimensions, so passing an index of 0 or 1 will return the x- and y-axis values respectively. Any subsequent dimensions will be value dimensions, e.g. on an Image index value 2 will refer to the intensity values and on an RGB index values 2, 3, and 4 will return the Red, Green and Blue intensities instead.

"- The image data returned by ``dimension_values()`` must be flipped along the first axis due to conflicting conventions.\n",
"- ``Image`` and related classes come with convenient methods to convert between matrix indices and data coordinates and vice versa: ``matrix2sheet()`` and ``sheet2matrix()``. This is useful when searching for features such as peaks.\n",
"\n",
"A very powerful aspect of operations is the fact that they understand Holoviews data structures. This means it is very straight-forward to apply an operation to every element in a container. As an example, well apply an additional high-pass filter to our HoloMap:"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll

(or "let's")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed it (but now I see I made a typo in the commit message...)

]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%output backend=\"matplotlib\"\n",
"\n",
"image_filter(stairs_map, type_=\"high-pass\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note, that the sigma value for the high-pass filter has defaulted to 5, and the sigma value in the HoloMap still corresponds to the original low-pass filter.\n",
"\n",
"\n",
"## Benefits of using ``Operation``\n",
"\n",
"Now that we have seen some operations in action we can get some appreciation of what makes them useful. When working with data interactively we often end up applying a lot of ad-hoc data transforms, which provides maximum flexibility but is neither reproducible nor maintainable. Operations allow us to encapsulate analysis code using a well defined interface that is well suited for building complex analysis pipelines:\n",
Expand Down