[explore] Filtering on int or numeric column: 'list' object has no attribute 'isdigit'

[rumbin]

Make sure these boxes are checked before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if any
• I have reproduced the issue with at least the latest released version of superset
• I have checked the issue tracker for the same issue and I haven't found one similar

Superset version

0.18.4, with ProstgreSQL as both superset management DB and data warehouse.

Expected results

Queries with applied filters on values of columns of type integer or numeric work.

Actual results

The query fails with the following Traceback:

Traceback (most recent call last):
  File "/opt/apps/superset/venv_py3_test/lib64/python3.4/site-packages/superset/viz.py", line 238, in get_payload
    df = self.get_df()
  File "/opt/apps/superset/venv_py3_test/lib64/python3.4/site-packages/superset/viz.py", line 78, in get_df
    self.results = self.datasource.query(query_obj)
  File "/opt/apps/superset/venv_py3_test/lib64/python3.4/site-packages/superset/connectors/sqla/models.py", line 550, in query
    sql = self.get_query_str(query_obj)
  File "/opt/apps/superset/venv_py3_test/lib64/python3.4/site-packages/superset/connectors/sqla/models.py", line 322, in get_query_str
    qry = self.get_sqla_query(**query_obj)
  File "/opt/apps/superset/venv_py3_test/lib64/python3.4/site-packages/superset/connectors/sqla/models.py", line 481, in get_sqla_query
    eq = utils.string_to_num(flt['val'])
  File "/opt/apps/superset/venv_py3_test/lib64/python3.4/site-packages/superset/utils.py", line 149, in string_to_num
    if s.isdigit():
AttributeError: 'list' object has no attribute 'isdigit'

This behavior was not present in superset==0.18.2.

I suspect #2908 and/or #2671 to be the culprit.

Steps to reproduce

Apply a filter on a numeric or integer column and run the query.

I don't know, however, to which extent this issue is specific to the ProstgreSQL DB backend.

[rumbin]

Possible fix (?)

In utils.py, function string_to_num, there is the following test included:

     if isinstance(s, (int, float)):
        return s

If we add just another test for numeric lists, everything seems to work again:

    if type(s) is list and all(isinstance(i, (int, float)) for i in s):
        return s

Or both tests combined:

    if isinstance(s, (int, float)) or (type(s) is list and all(isinstance(i, (int, float)) for i in s)):
        return s

As I do not know if these approaches are clean enough and which one to prefer, I would appreciate if a more experienced user could step in…

[rumbin]

I noticed that there may also be lists of strings, e.g. for timestamps (without timezone).
For this reason, my suggestion for the string_to_num function is as follows:

def string_to_num(s):
    """Converts a string to an int/float

    Returns ``None`` if it can't be converted

    >>> string_to_num('5')
    5
    >>> string_to_num('5.2')
    5.2
    >>> string_to_num(10)
    10
    >>> string_to_num(10.1)
    10.1
    >>> string_to_num('this is not a string') is None
    True
    """
    if isinstance(s, (int, float)) or (type(s) is list and all(isinstance(i, (int, float)) for i in s)):
        return s
    if type(s) is list:
        if all(i.isdigit() for i in s):
            return [int(i) for i in s]
        try:
            return [float(i) for i in s]
        except ValueError:
            return None
    else:
        if s.isdigit():
            return int(s)
        try:
            return float(s)
        except ValueError:
            return None

@mistercrunch Could you please step in, as I'm not sure if any of this makes sense? Are my corrections placed at a sensible position or should they rather be done in models.py?
Furthermore, I am generally confused, why in my case s always seems to be of type list , while the existing code just expects single values and why it doesn`t seem to do any harm in the further processing of the data.

[Mark-uiojxx]

I've also had this problem. Have you solved it? @rumbin
I replaced the code in utils.py like you said , and my problem solved.

[rumbin]

@Mark-uiojxx Could you please tell which database you are using? I would like to figure out if this issue only exists for PostgreSQL.

[Mark-uiojxx]

@rumbin MySQL.Sorry for late reply!

[rumbin]

In superset 0.18.5 the bug seems to be gone again. I tested it in a clean venv.
However, I do not know which commit actually fixed the issue.

@Mark-uiojxx Can I close the issue, or do you still encounter problems using the recent release?

[Mark-uiojxx]

@rumbin OK