In agasc supplement processing, only process observations that already happened.

[javierggt]

Description

To determine that an observation already happened, I use kadi.events.dwells. I only consider observations before the last dwell in kadi.events. This avoids the No level 0 data in weekly processing.

Interface impacts

None

Testing

Unit tests

• No unit tests
• Mac
• Linux
• Windows

Independent check of unit tests by Jean

• Linux

Functional tests

As a quick test that this PR does "something", I did the following, which succeeds on this branch, but fails in current master (and is the cause of the No level 0 data error):

from mica.archive import aca_l0
from agasc.supplement.magnitudes import star_obs_catalogs as soc
from cxotime import CxoTime
soc.load()
i = np.argmax(CxoTime(soc.STARS_OBS['obs_start']))
aca_l0.get_files(soc.STARS_OBS['obsid'][i])

[jeanconn]

I suppose the race condition I was thinking about (during our tagup) was if the CXCDS archive is still actively ingesting / processing level 0 image data at the time of the mica update, the kadi update and events may be more up-to-date than than the mica level 0 data. But this is unlikely with the current cron job processing timing (kadi events done 30 minutes before mica update).

[jeanconn]

I don't play much with the django but is something like:
kadi.events.dwells.all().latest('tstart').stop
faster? Not sure if there is a kadi method for this but for the one-off it seems fine.

[javierggt]

You are right. That seems to be faster and give the same result. I made the change.