py3: let the repr of numbers be a string

[fchapoton]

and not forced to be "bytes"

replaced by #24223

CC: @embray @jdemeyer

Component: python3

Keywords: unicode

Author: Frédéric Chapoton

Issue created by migration from https://trac.sagemath.org/ticket/23812

[fchapoton]

Commit: f34394d

[fchapoton]

Branch: u/chapoton/23812

[sagetrac-git]

Changed commit from f34394d to f093217

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f093217

py3: str repr for integers

[jdemeyer]

comment:3

I wonder if we should have a general function to convert from bytes to str and back in Sage. For example, like in cypari2: https://github.com/defeo/cypari2/blob/master/cypari2/string_utils.pyx

[fchapoton]

comment:4

It seems that we also need to change the repr of rational numbers..

[fchapoton]

comment:5

ping ?

[jdemeyer]

comment:6

As I said, I think we should have a general solution instead of this.

[fchapoton]

comment:7

There is

src/sage/misc/six.py:def u(x):

but this is not doing exactly that.

[embray]

comment:8

I don't think this is the right answer. There a dozens of places where a similar conversion is needed. There needs to be a generic function for this kind of thing.

[fchapoton]

comment:9

This is maybe not the definitive word. If you feel you can do better quickly, please do.

Nethertheless, there are not so many casting to bytes:

git grep -c "<bytes>" *.pyx
src/sage/libs/ecl.pyx:2
src/sage/libs/homfly.pyx:1
src/sage/libs/pynac/pynac.pyx:1
src/sage/numerical/backends/cvxopt_backend.pyx:1
src/sage/numerical/backends/cvxopt_sdp_backend.pyx:1
src/sage/numerical/backends/ppl_backend.pyx:1
src/sage/rings/integer.pyx:1

and integer.pyx is certainly the most important place where we need to correct the casting to bytes.

[jdemeyer]

comment:10

Replying to @fchapoton:

Nethertheless, there are not so many casting to bytes:

I think most of the time, the bytes is implicit because it's just what corresponds to char * in C. For example, in code like

def f():
    cdef char* cstr = ....
    return cstr

Cython will automatically convert cstr to Python bytes. Since Sage deals with a lot of C libraries, I would guess that this happens in a lot of places.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

60782ff	Merge branch 'u/chapoton/23812' in 8.1.b5
6f8a6cf	trac 23812 also for rationals
f0bafbb	Merge branch 'u/chapoton/23812' of trac.sagemath.org:sage into 23812
8cb8955	Merge branch '23812' into integer branch

[sagetrac-git]

Changed commit from f093217 to 8cb8955

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

ad28790

more utf8 compatibility

[sagetrac-git]

Changed commit from 8cb8955 to ad28790

[fchapoton]

comment:13

what do you think of the new branch ?

[fchapoton]

comment:14

see also #23648

[embray]

comment:15

Yeah, this really needs some opposite of string_to_bytes() from #23648. This isn't a great approach. And although in the vast majority of cases we can assume that the return values in these cases will be in ASCII, it's a bad pattern to repeat, because if anything happens to not return ASCII using this pattern you'll get something like this on Python 2:

>>> str(b'\xe2\x98\x83'.decode('utf8'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2603' in position 0: ordinal not in range(128)

[fchapoton]

Changed keywords from none to unicode

[fchapoton]

comment:17

I would appreciate if Erik or another expert of unicode issues works on the matter here and all related tickets dealing with unicode adaptations for python3. I understand when one tells me that my solutions are not good, but please then propose another branch.

[embray]

comment:18

Replying to @fchapoton:

I would appreciate if Erik or another expert of unicode issues works on the matter here and all related tickets dealing with unicode adaptations for python3. I understand when one tells me that my solutions are not good, but please then propose another branch.

I hear you-- I need to get my Python 3 build working again but I do want to help with this stuff. Please feel free to assign such tickets to me.

[fchapoton]

comment:19

I have created a new ticket that just wants to introduce the conversion tools: #24186.

[embray]

comment:20

Oops, this patch is now for some reason deleting all of rationals.pyx?

I'm going to see if I can rework this with my new string conversion functions.

[fchapoton]

comment:21

Replying to @embray:

Oops, this patch is now for some reason deleting all of rationals.pyx?

probably a known bug in trac git plugin

I'm going to see if I can rework this with my new string conversion functions.

ok. Beware also of #24136

[fchapoton]

comment:23

let us get rid of this one as duplicate

[fchapoton]

comment:24

ping ? any objection to close as invalid ?

[fchapoton]

Changed branch from u/chapoton/23812 to none

[fchapoton]

Changed commit from ad28790 to none