Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting Unicode #8777

Closed
bartoostlander opened this issue Mar 13, 2019 · 12 comments
Closed

Getting Unicode #8777

bartoostlander opened this issue Mar 13, 2019 · 12 comments
Assignees
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Output Workaround

Comments

@bartoostlander
Copy link

bartoostlander commented Mar 13, 2019

Hi Team, I get the error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 172: ordinal not in range(128)
Logged from file util.py, line 28

when I execute "az storage blob exists".

To reproduce

export AZURE_STORAGE_ACCOUNT="xxxxxxx"
export AZURE_STORAGE_ACCESS_KEY="yyyyyyy"
az storage blob exists --container-name mycontainer --name node_1.lock

Expected behavior
In the past, this returned an object or string with _true_ or _false_ in it. Since early this year it throws a UnicodeEncodeError error in util.py.

Environment summary
azure-cli 2.0.59, bash shell, Red Hat Enterprise Linux Server release 7.6 (Maipo) running Python 2.7.5 [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on a server in Azure SouthCentral, and on an identical one in Azure West.

Additional context
The script running the 'offending' command is an essential step in our Disaster Recovery procedure and runs nightly.

@bartoostlander
Copy link
Author

As it turns out, running the following command before running any Azure CLI command solves the problem:

export LANG=en_US.UTF-8

@bartoostlander
Copy link
Author

Running the following command before running any Azure CLI commands avoids the problem:

export LANG=en_US.UTF-8

@bartoostlander
Copy link
Author

I spoke too soon. Issue still exists... :-(

@PatMyron
Copy link

related: #8312, #6408, #4418, #1277

Pats-MacBook-Pro  ~ $ az resource list > /dev/null 
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 868, in emit
    msg = self.format(record)
  File "/usr/local/lib/python2.7/site-packages/knack/log.py", line 78, in format
    msg = logging.StreamHandler.format(self, record)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 741, in format
    return fmt.format(record)
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py", line 469, in format
    s = self._fmt % record.__dict__
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 172: ordinal not in range(128)
Logged from file util.py, line 26

Pats-MacBook-Pro  ~ $ which python pip az
/usr/local/bin/python
/usr/local/bin/pip
/usr/local/bin/az

Pats-MacBook-Pro  ~ $ python --version
Python 2.7.16

Pats-MacBook-Pro  ~ $ pip --version
pip 19.0.3 from /usr/local/lib/python2.7/site-packages/pip (python 2.7)

Pats-MacBook-Pro  ~ $ uname -a
Darwin Pats-MacBook-Pro.local 18.2.0 Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST 2018; root:xnu-4903.231.4~2/RELEASE_X86_64 x86_64

Pats-MacBook-Pro  ~ $ az --version
azure-cli (2.0.41)

acr (2.0.28)
acs (2.1.3)
advisor (0.5.1)
ams (0.1.1)
appservice (0.1.36)
backup (1.1.2)
batch (3.2.4)
batchai (0.3.1)
billing (0.1.8)
cdn (0.0.14)
cloud (2.0.15)
cognitiveservices (0.1.13)
command-modules-nspkg (2.0.1)
configure (2.0.17)
consumption (0.3.1)
container (0.2.1)
core (2.0.41)
cosmosdb (0.1.21)
dla (0.1.0)
dls (0.0.22)
dms (0.0.1)
eventgrid (0.1.12)
eventhubs (0.1.3)
extension (0.1.0)
feedback (2.1.3)
find (0.2.11)
interactive (0.3.25)
iot (0.1.21)
keyvault (2.0.23)
lab (0.0.22)
monitor (0.1.8)
network (2.1.5)
nspkg (3.0.2)
profile (2.0.27)
rdbms (0.2.5)
redis (0.2.14)
reservations (0.2.1)
resource (2.0.32)
role (2.0.26)
servicebus (0.1.2)
servicefabric (0.0.12)
sql (2.0.27)
storage (2.0.35)
vm (2.0.35)

Python location '/usr/local/opt/python@2/bin/python2.7'
Extensions directory '/Users/patmy/.azure/cliextensions'

Python (Darwin) 2.7.16 (default, Mar  4 2019, 09:01:38) 
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)]

Legal docs and information: aka.ms/AzureCliLegal

@PatMyron
Copy link

PatMyron commented Mar 13, 2019

Worked around this on my Mac by forcing the use of azure-cli from Homebrew instead of Pip:

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-macos

brew update && brew install azure-cli && brew link --overwrite azure-cli

@marstr
Copy link
Member

marstr commented Mar 13, 2019

@bartoostlander, seems like the workaround is effectively "use Python3 for the Azure CLI". This is easy for @PatMyron, who is installing with brew because Python3 is the default in that environment.

However, since you're coming from Red Hat, I'll assume you install from the RPM that we publish. That makes it trickier, as my understanding is that Python3 won't be available through first party repositories until EL8.

Do you have access to a python3 package through third party repositories?

@marstr marstr self-assigned this Mar 13, 2019
@williexu williexu added bug This issue requires a change to an existing behavior in the product in order to be resolved. Output labels Mar 13, 2019
@bartoostlander
Copy link
Author

@marstr , I installed Python 3.6 from the RedHat SCL. I can then run "scl enable rh-python36 bash" and run my script (that does "az storage blob exists"). Echoing "which python" in that script confirms that Python 3.6 is active. However, the "az storage" command on the next line still gives the same error, referring to Python 2.7!

Apparently, I need to do something to convince the Azure CLI to use the currently active Python version. What could that be?

@marstr
Copy link
Member

marstr commented Mar 14, 2019

Yep, that's expected! We try to isolate ourselves from system-python by running in a virtualenv. Which in general is a great thing, but in this particular case it makes hard to break-open your install to change things.

(While not what we usually recommend) In this case the easiest thing will be to install directly from PyPI using pip by doing the following:

# remove the old azure-cli install for cleanliness
yum erase -y azure-cli

# get python3 if you don't already have it (@bartoostlander can skip this)

## your SCL may vary if you're not using CentOS
yum install -y centos-release-scl
yum install -y rh-python36
scl enable rh-python36 bash

# Create a virtualenv where you would like to install the azure-cli
## Make sure that python invokes python3 before executing this step. If it doesn't, run the scl enable from above; this is because scl enable isn't durable between bash sessions.
cd ${yourDesiredLocation}
python -m virtualenv azure-cli

# Install the Azure-CLI from PyPI into this virtualenv
./azure-cli/bin/python -m pip install azure-cli

# Create a launcher for the azure-cli, and make it executable
cat > /usr/bin/az << EOM
#!/usr/bin/env bash
${yourDesiredLocation}/azure-cli/bin/python -m azure.cli "\$@"
EOM

chmod +x /usr/bin/az

To remove this installation, all you'll have to do is delete ${yourDesiredLocation}/azure-cli and /usr/bin/az.

@bartoostlander
Copy link
Author

Aha! You just saved me from going doing a rabbit hole. I'll follow your instructions. Your responsiveness is highly appreciated!

@bartoostlander
Copy link
Author

Preliminary observation: it works... I'll close this issue after further validation. Thank you!

@bartoostlander
Copy link
Author

Success. Thank you, @marstr !

@sdkks
Copy link

sdkks commented Apr 8, 2019

Worked around this on my Mac by forcing the use of azure-cli from Homebrew instead of Pip:

docs.microsoft.com/en-us/cli/azure/install-azure-cli-macos

brew update && brew install azure-cli && brew link --overwrite azure-cli

brew install azure-cli exits non zero because symlink can't be created. Maybe:
brew update && brew install azure-cli || brew link --overwrite azure-cli ?

@haroldrandom haroldrandom added bug This issue requires a change to an existing behavior in the product in order to be resolved. Output Workaround labels Oct 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Output Workaround
Projects
None yet
Development

No branches or pull requests

7 participants
@PatMyron @williexu @marstr @sdkks @haroldrandom @bartoostlander and others