Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support deepcopy of Simulator objects #51

Open
dkirkby opened this issue Oct 19, 2016 · 3 comments
Open

Support deepcopy of Simulator objects #51

dkirkby opened this issue Oct 19, 2016 · 3 comments

Comments

@dkirkby
Copy link
Member

dkirkby commented Oct 19, 2016

The copy.deepcopy function does not work with a Simulator object:

import specsim.simulator
s = specsim.simulator.Simulator('test')
import copy
s2 = copy.deepcopy(s)

This fails when trying to copy an astropy time object:

/Users/david/anaconda/lib/python2.7/site-packages/astropy/time/core.pyc in _replicate(self, method, *args, **kwargs)
    723             if method is not None and val is not None:
    724                 if method == 'copy' or method == 'flatten' and val.size == 1:
--> 725                     val = val.copy()
    726 
    727                 elif val.size > 1:

AttributeError: 'float' object has no attribute 'copy'

This issue is to get this working and add unit tests. The use case for this is desisim unit tests which run many different configurations and currently have to call the relatively slow constructor each time. See desihub/desisim#178 for details.

@dkirkby
Copy link
Member Author

dkirkby commented Oct 19, 2016

After some testing, the deepcopy failure is isolated to the following two data members:

simulator.observation.observing_model, boresight_altaz

These are both instances of astropy.coordinates.AltAz, so the problem lies there...

@dkirkby
Copy link
Member Author

dkirkby commented Oct 20, 2016

This is already a known issue astropy/astropy#5225 and I think I have a workaround for the current version of astropy.

@sbailey
Copy link
Contributor

sbailey commented Dec 13, 2017

The desire for this feature came up again while debugging desihub/desisim#298 . If I could deepcopy specsim.simulator.Simulator objects, I could implement a better solution in desisim for caching Simulator objects (to avoid the expensive overhead of creating one) while being robust against the kind of bug I generated in desihub/desisim#298 (forgetting that I was reusing an object and having it change out from under me).

Checking again, the problem persists but the traceback is now different:

In [3]: import desisim.specsim

In [4]: desi = desisim.specsim.get_simulator('desi', num_fibers=10)

In [5]: import copy

In [6]: d2 = copy.deepcopy(desi)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-87e8012d3c48> in <module>()
----> 1 d2 = copy.deepcopy(desi)

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in deepcopy(x, memo, _nil)
    180                             raise Error(
    181                                 "un(deep)copyable object of type %s" % cls)
--> 182                 y = _reconstruct(x, rv, 1, memo)
    183 
    184     # If is its own copy, don't memoize.

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in _reconstruct(x, info, deep, memo)
    295     if state is not None:
    296         if deep:
--> 297             state = deepcopy(state, memo)
    298         if hasattr(y, '__setstate__'):
    299             y.__setstate__(state)

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in deepcopy(x, memo, _nil)
    153     copier = _deepcopy_dispatch.get(cls)
    154     if copier:
--> 155         y = copier(x, memo)
    156     else:
    157         try:

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in _deepcopy_dict(x, memo)
    241     memo[id(x)] = y
    242     for key, value in x.items():
--> 243         y[deepcopy(key, memo)] = deepcopy(value, memo)
    244     return y
    245 d[dict] = _deepcopy_dict

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in deepcopy(x, memo, _nil)
    180                             raise Error(
    181                                 "un(deep)copyable object of type %s" % cls)
--> 182                 y = _reconstruct(x, rv, 1, memo)
    183 
    184     # If is its own copy, don't memoize.

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in _reconstruct(x, info, deep, memo)
    295     if state is not None:
    296         if deep:
--> 297             state = deepcopy(state, memo)
    298         if hasattr(y, '__setstate__'):
    299             y.__setstate__(state)

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in deepcopy(x, memo, _nil)
    153     copier = _deepcopy_dispatch.get(cls)
    154     if copier:
--> 155         y = copier(x, memo)
    156     else:
    157         try:

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in _deepcopy_dict(x, memo)
    241     memo[id(x)] = y
    242     for key, value in x.items():
--> 243         y[deepcopy(key, memo)] = deepcopy(value, memo)
    244     return y
    245 d[dict] = _deepcopy_dict

/Users/sbailey/anaconda/envs/desi/lib/python3.5/copy.py in deepcopy(x, memo, _nil)
    172                     reductor = getattr(x, "__reduce_ex__", None)
    173                     if reductor:
--> 174                         rv = reductor(4)
    175                     else:
    176                         reductor = getattr(x, "__reduce__", None)

TypeError: can't pickle dict_keys objects

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants