Minify output to front-end

[PeterDaveHello]

No description provided.

[nobodxbodon]

It seems to minify everything. That would be perfect, but maybe a lower hanging fruit is to minify just the css and js files mentioned in #4262?

[PeterDaveHello]

hmmm ... the testing environment seems to be different with Dockerfile, I'm now trying to fix the build.

[chadwhitacre]

We need tests for this, and also an envvar switch to turn it off in development.

[PeterDaveHello]

@whit537 I'm still trying to fix that error on CI, I didn't get it on my docker development environment lol ...

[chadwhitacre]

There are one or two intermittent bugs that crop up on Travis but this doesn't like that them. Hmm ... 🤔

[chadwhitacre]

I do get failures locally:

______________________ Tests.test_distributing_doesnt_redirect_when_money_is_available _______________________
Traceback (most recent call last):
  File "/Users/whit537/personal/gratipay/gratipay.com/tests/py/test_www_team_distributing.py", line 15, in test_distributing_doesnt_redirect_when_money_is_available
    assert self.client.GET('/TheEnterprise/distributing/').code == 200
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/aspen/testing/client.py", line 104, in GET
    def GET(self, *a, **kw):    return self.hit('GET', *a, **kw)
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/aspen/testing/client.py", line 128, in hit
    , return_after=return_after
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/aspen/website.py", line 62, in respond
    , _return_after=return_after
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/algorithm.py", line 288, in run
    new_state = function(**deps.as_kwargs)
  File "/Users/whit537/personal/gratipay/gratipay.com/gratipay/minify.py", line 9, in minify_output
    response.body = htmlmin.minify(unicode(response.body), 'utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 6207: ordinal not in range(128)
============================ 19 failed, 737 passed, 36 xfailed in 136.43 seconds =============================
[gratipay] $

[chadwhitacre]

Interesting! The test case passes in isolation:

[gratipay] $ envtest tests/py/test_www_team_distributing.py test_distributing_doesnt_redirect_when_money_is_available
============================================ test session starts =============================================
platform darwin -- Python 2.7.12 -- py-1.4.26 -- pytest-2.6.4
plugins: cache, cov
ERROR: file not found: test_distributing_doesnt_redirect_when_money_is_available

==============================================  in 0.01 seconds ==============================================
[gratipay] $

[chadwhitacre]

Sike. Wrong command line. :-)

[chadwhitacre]

Fails in isolation. 👍

================================================== FAILURES ==================================================
______________________ Tests.test_distributing_doesnt_redirect_when_money_is_available _______________________
Traceback (most recent call last):
  File "/Users/whit537/personal/gratipay/gratipay.com/tests/py/test_www_team_distributing.py", line 15, in test_distributing_doesnt_redirect_when_money_is_available
    assert self.client.GET('/TheEnterprise/distributing/').code == 200
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/aspen/testing/client.py", line 104, in GET
    def GET(self, *a, **kw):    return self.hit('GET', *a, **kw)
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/aspen/testing/client.py", line 128, in hit
    , return_after=return_after
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/aspen/website.py", line 62, in respond
    , _return_after=return_after
  File "/Users/whit537/personal/gratipay/gratipay.com/env/lib/python2.7/site-packages/algorithm.py", line 288, in run
    new_state = function(**deps.as_kwargs)
  File "/Users/whit537/personal/gratipay/gratipay.com/gratipay/minify.py", line 9, in minify_output
    response.body = htmlmin.minify(unicode(response.body), 'utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 6207: ordinal not in range(128)
============ 5 tests deselected by '-ktest_distributing_doesnt_redirect_when_money_is_available' =============
=================================== 1 failed, 5 deselected in 3.41 seconds ===================================
[gratipay] $

[chadwhitacre]

@PeterDaveHello Yeah, the issue is that unicode(b) is trying to convert the bytestring b to a unicode object with a default codec of ASCII. You can be explicit about the codec with b.decode('utf8').

As to why this bug doesn't appear in your docker instance: what's the output of python -c 'import sys; print(sys.getdefaultencoding())'?

$ python -c 'import sys; print(sys.getdefaultencoding())'
utf-8
$ source env/bin/activate
[gratipay] $ python -c 'import sys; print(sys.getdefaultencoding())'
ascii
[gratipay] $

[PeterDaveHello]

@whit537 I got ascii from python -c 'import sys; print(sys.getdefaultencoding())', thanks for the help and info, I'll try to fix it soon

[PeterDaveHello]

Trying to isolate different minify process but strangely failed on Travis CI ... hmmm ...

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

[chadwhitacre]

I got ascii from python -c 'import sys; print(sys.getdefaultencoding())'

Interesting! I would expect a Python with ascii as the default encoding to exhibit the test failure. I'm not sure what else to look for in your docker environment to see what fails. I would start interactive debugging at the point of failure to see if I could find anything unusual in the data types or something

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Generally this means we need to restart the build. Do you have permissions to do that on Travis?

[PeterDaveHello]

@whit537 yes I have the permission, but looks the last two builds have the same problem, so I assume restart will not help.

[chadwhitacre]

Sorry I missed this in my notifications (I've been using Octobox, not sure if that explains it). The latest failures are due to our Python linter, which checks for unused imports.

[chadwhitacre]

In other words, ca352de removes a number of calls, without also removing the imports associated with those calls.

[PeterDaveHello]

ca352de is actually a try to isolate the different minify process, I'm not sure how much time I can spend on fixing all this out, if someone wants to take this, please feel free to let me know, sorry that I keep this not solved for almost a month ...

[chadwhitacre]

Closing for now.