Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

require numpy>1.26 #994

Merged
merged 2 commits into from
Aug 23, 2024
Merged

require numpy>1.26 #994

merged 2 commits into from
Aug 23, 2024

Conversation

rkingsbury
Copy link
Collaborator

@rkingsbury rkingsbury commented Aug 23, 2024

This seems necessary to prevent an unusual pandas error in open data (which surfaced in the Auto Dependency Upgrades workflow after allowing numpy>2

_________________________________ test_update __________________________________

self = Scope(scope=['Timestamp',
 'datetime',
 'True',
 'False',
 'list',
 'tuple',
 'inf',
 'Inf',
 '__name__',
 '__doc__',
..._index']
, resolvers=['ilevel_0',
 'index',
 'clevel_0',
 'columns',
 'last_updated',
 'task_id',
 'group_level_two']
)
key = 'np', is_local = False

    def resolve(self, key: str, is_local: bool):
        """
        Resolve a variable name in a possibly local context.
    
        Parameters
        ----------
        key : str
            A variable name
        is_local : bool
            Flag indicating whether the variable is local or not (prefixed with
            the '@' symbol)
    
        Returns
        -------
        value : object
            The value of a particular variable
        """
        try:
            # only look for locals in outer scope
            if is_local:
                return self.scope[key]
    
            # not a local variable so check in resolvers if we have them
            if self.has_resolvers:
>               return self.resolvers[key]

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/scope.py:231: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/collections/__init__.py:1006: in __getitem__
    return self.__missing__(key)            # support subclasses that define __missing__
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = DeepChainMap({'last_updated': 0   2024-08-23 18:30:08.900661
1   2024-08-23 18:30:08.900663
Name: last_updated, dtype:...ast_updated          last_updated
task_id                    task_id
group_level_two    group_level_two
dtype: object})
key = 'np'

    def __missing__(self, key):
>       raise KeyError(key)
E       KeyError: 'np'

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/collections/__init__.py:998: KeyError

During handling of the above exception, another exception occurred:

self = Scope(scope=['Timestamp',
 'datetime',
 'True',
 'False',
 'list',
 'tuple',
 'inf',
 'Inf',
 '__name__',
 '__doc__',
..._index']
, resolvers=['ilevel_0',
 'index',
 'clevel_0',
 'columns',
 'last_updated',
 'task_id',
 'group_level_two']
)
key = 'np', is_local = False

    def resolve(self, key: str, is_local: bool):
        """
        Resolve a variable name in a possibly local context.
    
        Parameters
        ----------
        key : str
            A variable name
        is_local : bool
            Flag indicating whether the variable is local or not (prefixed with
            the '@' symbol)
    
        Returns
        -------
        value : object
            The value of a particular variable
        """
        try:
            # only look for locals in outer scope
            if is_local:
                return self.scope[key]
    
            # not a local variable so check in resolvers if we have them
            if self.has_resolvers:
                return self.resolvers[key]
    
            # if we're here that means that we have no locals and we also have
            # no resolvers
            assert not is_local and not self.has_resolvers
            return self.scope[key]
        except KeyError:
            try:
                # last ditch effort we look in temporaries
                # these are created when parsing indexing expressions
                # e.g., df[df > 0]
>               return self.temps[key]
E               KeyError: 'np'

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/scope.py:242: KeyError

The above exception was the direct cause of the following exception:

s3store = <maggma.stores.open_data.OpenDataStore object at 0x7f18dfb08150>

    def test_update(s3store):
        assert len(s3store.index_data) == 2
>       s3store.update(
            pd.DataFrame(
                [
                    {
                        "task_id": "mp-199999",
                        "data": "asd",
                        "group": {"level_two": 4},
                        s3store.last_updated_field: datetime.utcnow(),
                    }
                ]
            )
        )

tests/stores/test_open_data.py:395: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/maggma/stores/open_data.py:531: in update
    sub_existing = existing_index.query(query_str)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/frame.py:4823: in query
    res = self.eval(expr, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/frame.py:4949: in eval
    return _eval(expr, inplace=inplace, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/eval.py:336: in eval
    parsed_expr = Expr(expr, engine=engine, parser=parser, env=env)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:809: in __init__
    self.terms = self.parse()
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:828: in parse
    return self._visitor.visit(self.expr)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:418: in visit_Module
    return self.visit(expr, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:421: in visit_Expr
    return self.visit(node.value, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:746: in visit_BoolOp
    return reduce(visitor, operands)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:739: in visitor
    lhs = self._try_visit_binop(x)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:735: in _try_visit_binop
    return self.visit(bop)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:719: in visit_Compare
    return self.visit(binop)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:532: in visit_BinOp
    op, op_class, left, right = self._maybe_transform_eq_ne(node)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:454: in _maybe_transform_eq_ne
    right = self.visit(node.right, side="right")
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:659: in visit_Call
    res = self.visit_Attribute(node.func)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:644: in visit_Attribute
    resolved = self.visit(value).value
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:412: in visit
    return visitor(node, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/expr.py:545: in visit_Name
    return self.term_type(node.id, self.env, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/ops.py:91: in __init__
    self._value = self._resolve_name()
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/ops.py:115: in _resolve_name
    res = self.env.resolve(local_name, is_local=is_local)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Scope(scope=['Timestamp',
 'datetime',
 'True',
 'False',
 'list',
 'tuple',
 'inf',
 'Inf',
 '__name__',
 '__doc__',
..._index']
, resolvers=['ilevel_0',
 'index',
 'clevel_0',
 'columns',
 'last_updated',
 'task_id',
 'group_level_two']
)
key = 'np', is_local = False

    def resolve(self, key: str, is_local: bool):
        """
        Resolve a variable name in a possibly local context.
    
        Parameters
        ----------
        key : str
            A variable name
        is_local : bool
            Flag indicating whether the variable is local or not (prefixed with
            the '@' symbol)
    
        Returns
        -------
        value : object
            The value of a particular variable
        """
        try:
            # only look for locals in outer scope
            if is_local:
                return self.scope[key]
    
            # not a local variable so check in resolvers if we have them
            if self.has_resolvers:
                return self.resolvers[key]
    
            # if we're here that means that we have no locals and we also have
            # no resolvers
            assert not is_local and not self.has_resolvers
            return self.scope[key]
        except KeyError:
            try:
                # last ditch effort we look in temporaries
                # these are created when parsing indexing expressions
                # e.g., df[df > 0]
                return self.temps[key]
            except KeyError as err:
>               raise UndefinedVariableError(key, is_local) from err
E               pandas.errors.UndefinedVariableError: name 'np' is not defined

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/pandas/core/computation/scope.py:244: UndefinedVariableError

@rkingsbury rkingsbury added the dependencies Pull requests that update a dependency file label Aug 23, 2024
Copy link

codecov bot commented Aug 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.83%. Comparing base (10689d6) to head (9f924ee).
Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #994   +/-   ##
=======================================
  Coverage   88.83%   88.83%           
=======================================
  Files          46       46           
  Lines        3985     3985           
=======================================
  Hits         3540     3540           
  Misses        445      445           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rkingsbury rkingsbury changed the title require numpy>1.26.4 require numpy>1.26 Aug 23, 2024
@rkingsbury rkingsbury merged commit e553944 into main Aug 23, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant