style.bar: add support for axis=None (tablewise application instead of rowwise or columnwise) #21548

soxofaan · 2018-06-19T23:08:25Z

eliminate code duplication related to style.bar with different align modes
add support for axis=None
fix minor bug with align 'zero' and width < 100
make generated CSS gradients more compact
Closes NotImplementedError: 'AnnAssign' nodes are not implemented but numpy-like syntax works #21525
tests added / passed
passes git diff origin/master -u -- "*.py" | flake8 --diff
whatsnew entry

soxofaan · 2018-06-19T23:12:59Z

example with axis=0 (top) and axis=None (bottom):

soxofaan · 2018-06-20T07:56:40Z

related for style.background_gradient PR: #21259

codecov · 2018-06-20T08:11:12Z

Codecov Report

Merging #21548 into master will increase coverage by <.01%.
The diff coverage is 97.5%.

@@            Coverage Diff             @@
##           master   #21548      +/-   ##
==========================================
+ Coverage   92.05%   92.05%   +<.01%     
==========================================
  Files         169      169              
  Lines       50733    50736       +3     
==========================================
+ Hits        46702    46706       +4     
+ Misses       4031     4030       -1

Flag	Coverage Δ
#multiple	`90.46% <97.5%> (ø)`	⬆️
#single	`42.24% <5%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/formats/style.py	`96.42% <97.5%> (+0.26%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68273a7...e32c333. Read the comment docs.

jreback · 2018-06-20T09:58:01Z

can you search the issues to see if we have something an for this? IIRC we do

soxofaan · 2018-06-20T14:43:51Z

@jreback did a quick search but couldn't find anything yet

note to self: #21526 should be straightforward to include in another iteration of this issue

…andas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact

pep8speaks · 2018-06-24T22:15:36Z

Hello @soxofaan! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on August 21, 2018 at 14:03 Hours UTC

soxofaan · 2018-06-24T22:19:46Z

I added a commit to handle issue #21526

TomAugspurger · 2018-06-27T15:11:32Z

@soxofaan can you update the original post to include Closes #21525

will try to review this sometime soon.

soxofaan · 2018-06-28T08:40:26Z

Maybe also to be considered in this pull request:
I find the align mode names mid and zero very confusing:

mid claims "midpoint is centered in the cell", but if all values are strictly positive or negative, this is not the case as zero is forced into the value range
zero is about "zero is centered in the cell", which I would have called center (because there already is left), but this sounds a lot like mid of course.

Also: the left mode can lead to poor visualisation and is a bad choice as default IMHO. The bars do not start from zero at the left but from the minimum value. When this minimum is negative (or all values are negative), you get in the territory of poor/confusing visualisation.

In my refactor, I kept the original modes to avoid "bike shedding" issues, but I think I should at least raise the issue here.
What do you think?

…andas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact

soxofaan · 2018-07-11T15:45:52Z

(rebased)

TomAugspurger · 2018-07-12T12:47:33Z

pandas/io/formats/style.py

-        Color the background ``color`` proportional to the values in each
-        column.
+        Draw bars in the cell backgrounds, with a width proportional to
+        the values


First sentence should be less than 80 characters and end with a .

TomAugspurger · 2018-07-12T12:47:42Z

pandas/io/formats/style.py

        Excludes non-numeric data by default.

        Parameters
        ----------
        subset: IndexSlice, default None
            a valid slice for ``data`` to limit the style application to
-        axis: int
+        axis: int or None


"or None" -> "optional"

TomAugspurger · 2018-07-12T12:48:19Z

pandas/io/formats/style.py

@@ -1127,40 +1064,35 @@ def bar(self, subset=None, axis=0, color='#d65f5f', width=100,
            - 'mid' : the center of the cell is at (max-min)/2, or
              if values are all negative (positive) the zero is aligned
              at the right (left) of the cell
+        vmin: float or None: minimum bar value, defining the left hand limit


"or None" -> "optional"

Description should start on the next line.

https://pandas.pydata.org/pandas-docs/stable/contributing_docstring.html

TomAugspurger · 2018-07-12T12:49:20Z

pandas/tests/io/formats/test_style.py

@@ -350,10 +350,10 @@ def test_bar_align_left(self):
            (0, 0): ['width: 10em', ' height: 80%'],
            (1, 0): ['width: 10em', ' height: 80%',
                     'background: linear-gradient('
-                     '90deg,#d65f5f 50.0%, transparent 0%)'],
+                     '90deg,#d65f5f 50.0%, transparent 50.0%)'],


Why are these tests changing?

The original implementation generated gradient stops that were invalid: the positions (percentages) should not decrease. In the example above the first stop (#d65f5f) is at 50% and de second (transparent) was originally at 0%, but it's more correct to have the second one explicitly at 50% as well. (Browsers are expected to guess the correct intention of these incorrect cases, but you never know of course.)

In other test cases, the original css was also bit redudant, e.g.:

transparent 0%, transparent 10.0%, #5fba7d 10.0%

can be simplified to

transparent 10.0%, #5fba7d 10.0%

or even worse, sometimes things like these were part of the gradient: transparent 0%, transparent 0.0%, #5fba7d 0.0%, which is completely redundant.

The new implementation improves on all these points, so original tests were adapted.

And there was also a test case (test_bar_align_zero_pos_and_neg) with bar size bug in the original implementation when width < 100, which is also fixed in new implementation

…andas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact

soxofaan · 2018-07-12T22:09:04Z

@TomAugspurger : I addressed the docstring style remarks and rebased

TomAugspurger · 2018-07-24T16:06:52Z

Thanks.

There's a linting failure: https://travis-ci.org/pandas-dev/pandas/jobs/403324861#L2928.

If you could fix that, then +1 from me.

…andas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact

soxofaan · 2018-07-26T14:14:28Z

@TomAugspurger done and rebased

TomAugspurger · 2018-08-12T19:58:00Z

doc/source/whatsnew/v0.24.0.txt

@@ -571,6 +571,7 @@ Other
 - :meth: `~pandas.io.formats.style.Styler.background_gradient` now takes a ``text_color_threshold`` parameter to automatically lighten the text color based on the luminance of the background color. This improves readability with dark background colors without the need to limit the background colormap range. (:issue:`21258`)
 - Require at least 0.28.2 version of ``cython`` to support read-only memoryviews (:issue:`21688`)
 - :meth: `~pandas.io.formats.style.Styler.background_gradient` now also supports tablewise application (in addition to rowwise and columnwise) with ``axis=None`` (:issue:`15204`)
+- :meth: `~pandas.io.formats.style.Styler.bar` now also supports tablewise application (in addition to rowwise and columnwise) with ``axis=None`` and setting clipping range with ``vmin`` and ``vmax`` (:issue:`21548` and :issue:`21526`)


No space between :meth: and the method.

Ok, I'll fix that in the preceding two bullet points as well then

TomAugspurger · 2018-08-12T20:00:57Z

pandas/io/formats/style.py

+        """Draw bar chart in dataframe cells"""
+
+        # Get input value range.
+        smin = s.values.min() if vmin is None else vmin


What are the types of s? Series or DataFrame?

We don't want .values.min, since if it has any missing values the output will be missing. Presumably, we'll want to skip missing values here...

Pandas doesn't yet support reducing over all axes with min / max, so you'll likely need to handle series / frame separately here

Good point. Apparently the original implementation doesn't handle this properly either

TomAugspurger · 2018-08-12T20:01:48Z

pandas/io/formats/style.py

@@ -1129,38 +1064,37 @@ def bar(self, subset=None, axis=0, color='#d65f5f', width=100,
              at the right (left) of the cell

              .. versionadded:: 0.20.0
+        vmin: float, optional


Can you add ..versionadded for both of these.

…andas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact

soxofaan · 2018-08-14T08:56:38Z

@TomAugspurger done and rebased

TomAugspurger · 2018-08-18T19:55:40Z

pandas/io/formats/style.py

+        """Draw bar chart in dataframe cells"""
+
+        # Get input value range.
+        smin = s.min().min() if vmin is None else vmin


This will fail for object dtypes, which may happen in this context. Even though it's slightly wordier, I'd prefer

smin = s.min() if vmin is None else vmin if isinstance(smin, ABCSeries): smin = smin.min()

to reduce it down to a scalar.

LGTM otherwise.

…andas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact

soxofaan · 2018-08-21T12:28:10Z

@TomAugspurger done and rebased

TomAugspurger · 2018-08-30T12:47:03Z

Thanks @soxofaan!

* ENH: Styler.bar: add support for tablewise application with axis=None pandas-dev#21548 - eliminate code duplication related to style.bar with different align modes - add support for axis=None - fix minor bug with align 'zero' and width < 100 - make generated CSS gradients more compact * ENH: Styler.bar: add support for vmin/vmax pandas-dev#21526 * ENH: Styler.bar: properly support NaNs * ENH: Styler.bar: docstring improvements

soxofaan changed the title ~~style.bar: add support for axis=None~~ style.bar: add support for axis=None (tablewise application instead of rowwise or columnwise) Jun 20, 2018

jreback added Output-Formatting __repr__ of pandas objects, to_string IO HTML read_html, to_html, Styler.apply, Styler.applymap labels Jun 20, 2018

soxofaan force-pushed the style_bar_axis_none branch from 9a0362f to 6a5a1e5 Compare June 24, 2018 22:15

soxofaan force-pushed the style_bar_axis_none branch from 6a5a1e5 to 55476c1 Compare June 24, 2018 22:18

soxofaan mentioned this pull request Jun 24, 2018

ENH: allow ranges to be set on Styler .bar() #21526

Closed

soxofaan force-pushed the style_bar_axis_none branch from 55476c1 to 336f4bc Compare July 11, 2018 15:45

TomAugspurger reviewed Jul 12, 2018

View reviewed changes

soxofaan force-pushed the style_bar_axis_none branch from 9822573 to 2694732 Compare July 12, 2018 22:06

soxofaan force-pushed the style_bar_axis_none branch from 2694732 to 706da47 Compare July 26, 2018 14:13

TomAugspurger reviewed Aug 12, 2018

View reviewed changes

TomAugspurger added this to the 0.24.0 milestone Aug 12, 2018

soxofaan force-pushed the style_bar_axis_none branch from 706da47 to 44a85b7 Compare August 14, 2018 08:54

TomAugspurger reviewed Aug 18, 2018

View reviewed changes

soxofaan added 3 commits August 21, 2018 14:19

ENH: Styler.bar: add support for vmin/vmax pandas-dev#21526

6b4e2c2

ENH: Styler.bar: properly support NaNs

1c34755

soxofaan force-pushed the style_bar_axis_none branch from 44a85b7 to 1c34755 Compare August 21, 2018 12:20

ENH: Styler.bar: docstring improvements

e32c333

TomAugspurger merged commit 6b83df9 into pandas-dev:master Aug 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

style.bar: add support for axis=None (tablewise application instead of rowwise or columnwise) #21548

style.bar: add support for axis=None (tablewise application instead of rowwise or columnwise) #21548

soxofaan commented Jun 19, 2018 •

edited

Loading

soxofaan commented Jun 19, 2018

soxofaan commented Jun 20, 2018

codecov bot commented Jun 20, 2018 •

edited

Loading

jreback commented Jun 20, 2018 •

edited

Loading

soxofaan commented Jun 20, 2018

pep8speaks commented Jun 24, 2018 •

edited

Loading

soxofaan commented Jun 24, 2018

TomAugspurger commented Jun 27, 2018

soxofaan commented Jun 28, 2018

soxofaan commented Jul 11, 2018

TomAugspurger Jul 12, 2018

TomAugspurger Jul 12, 2018

TomAugspurger Jul 12, 2018

TomAugspurger Jul 12, 2018

soxofaan Jul 12, 2018

soxofaan Jul 12, 2018

soxofaan commented Jul 12, 2018

TomAugspurger commented Jul 24, 2018

soxofaan commented Jul 26, 2018

TomAugspurger Aug 12, 2018

soxofaan Aug 14, 2018

TomAugspurger Aug 12, 2018

soxofaan Aug 14, 2018

TomAugspurger Aug 12, 2018

soxofaan commented Aug 14, 2018

TomAugspurger Aug 18, 2018

soxofaan commented Aug 21, 2018

TomAugspurger commented Aug 30, 2018

style.bar: add support for axis=None (tablewise application instead of rowwise or columnwise) #21548

style.bar: add support for axis=None (tablewise application instead of rowwise or columnwise) #21548

Conversation

soxofaan commented Jun 19, 2018 • edited Loading

soxofaan commented Jun 19, 2018

soxofaan commented Jun 20, 2018

codecov bot commented Jun 20, 2018 • edited Loading

Codecov Report

jreback commented Jun 20, 2018 • edited Loading

soxofaan commented Jun 20, 2018

pep8speaks commented Jun 24, 2018 • edited Loading

Comment last updated on August 21, 2018 at 14:03 Hours UTC

soxofaan commented Jun 24, 2018

TomAugspurger commented Jun 27, 2018

soxofaan commented Jun 28, 2018

soxofaan commented Jul 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soxofaan commented Jul 12, 2018

TomAugspurger commented Jul 24, 2018

soxofaan commented Jul 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soxofaan commented Aug 14, 2018

Choose a reason for hiding this comment

soxofaan commented Aug 21, 2018

TomAugspurger commented Aug 30, 2018

soxofaan commented Jun 19, 2018 •

edited

Loading

codecov bot commented Jun 20, 2018 •

edited

Loading

jreback commented Jun 20, 2018 •

edited

Loading

pep8speaks commented Jun 24, 2018 •

edited

Loading