"cannot import name 'quantiles' from 'statistics'" in Joinmarket 0.9.7

[TheBitcoinProf]

After installing the newest Joinmarket, version 0.9.7, I get the following when trying to run python wallet-tool.py wallet.jmdat

Traceback (most recent call last):
File "wallet-tool.py", line 3, in
from jmclient import wallet_tool_main
File "/home/pi/Important/joinmarket/joinmarket-clientserver-0.9.7/jmclient/jmclient/init.py", line 74, in
from .bond_calc import get_bond_values
File "/home/pi/Important/joinmarket/joinmarket-clientserver-0.9.7/jmclient/jmclient/bond_calc.py", line 6, in
from statistics import quantiles
ImportError: cannot import name 'quantiles' from 'statistics' (/usr/lib/python3.7/statistics.py)

I have Python 3.7.3 on a Raspbian machine
I installed Joinmarket by downloading the tar.gz and running install.sh - everything worked fine as always.

Any idea why this is happening? Seems like a python version thing, but python3 should work.

[PulpCattel]

Yep, https://docs.python.org/3/library/statistics.html#statistics.quantiles
It requires Python 3.8, sad life.

I'm not familiar with Raspbian at all, if updating Python is not an option, as a quick solution you can comment (#) out the import from statistics import quantiles (or remove the entire bond_calc.py file if you don't care about the new bond-calculator.py script at all).

[TheBitcoinProf]

Awesome, thanks for the tip, I'll update it. Might be worth adding the 3.8+ requirement to the installation readme

[PulpCattel]

I think JM wants to support 3.7 still (not sure for how long), EOL is coming I think.
The proper solution, if 3.7 is still supported, is of course to replace the function with something else.

[kristapsk]

I think JM wants to support 3.7 still (not sure for how long)

We still have python_requires='>=3.6'. :) Somebody needs to look at this, how hard this is to solve. Either needs to be fixed or we could raise our Python version requirement.

[AdamISZ]

@PulpCattel I guess there's a relatively natural solution here: First, I see from the relevant import in jmclient.__init__.py that there is no usage of get_bond_values outside the bond calculator script itself, and the test, but clearly it's because of the test that you preferred it to be exported as part of the package. You could maybe not do that, but even if we stick with that option, we can just wrap a try/except around the import and let it fail.

Indeed it would be silly to have this stop 3.7 working, so it's an oversight on our part. I guess it comes out of the fact that our test suite doesn't run the actual user level scripts.

[PulpCattel]

but clearly it's because of the test that you preferred it to be exported as part of the package

That also, but mostly the idea was/is to reuse get_bond_values in other places (RPC, orderbook).
If this is still reasonable, I guess it's not terrible to keep the package level import and add a try block.

wrap a try/except around the import and let it fail.

Yeah, the incriminated function is only used for the orderbook stats, so we could add a note in --help about that particular option requiring Python 3.8+
If/when this is implemented in other places, we could add the same note there.

We can also replace the function with a (inspired) custom one:

def percentiles(data):
    n = len(data)
    if n < 2:
        raise ValueError('must have at least two data points')
    data = sorted(data)
    m = n - 1
    result = []
    for i in range(1, 100):
	j, delta = divmod(i * m, 100)
	interpolated = (data[j] * (100 - delta) + data[j + 1] * delta) / 100
	result.append(interpolated)
    return result

I didn't test it much but it should be the same.

so it's an oversight on our part

Yep, absolutely. Breaking compatibility with old Python versions wasn't ofc planned.