2024.11

[javierggt]

ska3-flight 2024.11

This PR includes the following changes:

• starcheck: Update starcheck IR zone to (-5, +35).
• kadi: Support offset pitch for Safe mode and NSM events and improve Safe mode.
• proseco: Exclude COLOR1=0.7 stars from guide selection.
• ska_sun: Add functions get_att_for_sun_pitch_yaw() and get_nsm_attitude().
• ska_tdb: Updates to support remote access from Windows.
• aca_view: improvements to handle real-time telemetry.
• acispy: Updates to thermal model support, and improvements handling of vehicle loads in SimulateECSRun.

It also includes a collection of small fixes and improvements detailed below.

Interface Impacts:

Removed packages:

• The find_attitude package is being removed from ska3-flight. Operationally this is not an interface impact - the find_attitude package at the current version will remain installed in /proj/sot/ska3/flight, but removing it from the metapackage will allow us to transition to doing updates to this package (which is isolated in scope and users) via a single-package FSDS request.
• The chandra_cmd_states Python package is being removed from ska3-flight. This package will actually be removed - it has been replaced with functionality in kadi command states and has been deprecated (with an error if used without env var override) since ska3-flight 2023.10.

Other:

• ska_tdb. If cheta.remote_access.access_remotely is set to True when ska_tdb is loaded, the user will now be prompted to provide a hostname, username, and password (unless they have already been set in the cheta.remote_access package), this should not impact existing code, outside of the FOT MATLAB Tools.
• ska_sun: Add functions get_att_for_sun_pitch_yaw and get_nsm_attitude.
• ska_helpers. Added new utilities: logger context manager and random RA and Decs
• proseco. Added proseco.acq.get_acq_candidates_mask and proseco.guide.get_guide_candidates_mask.
• parse_cm. All backstop files written out by parse_cm will use the modern, .12f, output for the components of the quaternion in MP_TARGQUAT / AOUPTARQ. Previously these were written out as .8e .
• mica
  • Removed a small html file of the aiprops data.
  • Added two columns PIXTLM, BGDTYP to the output of get_slot_data. For L0 data that hasn't been reprocessed since DS10.8.3 and doesn't have those columns in the source data FITS file, these are filled with ORIG and FLAT, respectively. This also adds a new derived column IMG_VCDUCTR.
  • Removed useless outdir argument from mica.report.report.get_obs_temps.
• chandra_maneuver. NSM_attitude is deprecated.
• chandra_aca
  • Added one column IMG_VCDUCTR which correspond to the VCDUCTR of the first sub-image. This matters only for 6x6 and 8x8 images, which are updated over 2 and 4 ACA packets respectively.
  • The filtering mask in proseco requires two new AGASC columns compared to the previous chandra_aca.plot code: MAG_ACA and COLOR1.
  • The filtering in proseco is somewhat different from the previous code, so there will be changes in the color of plotted stars. In general the changes are for the better and accurately reflect what proseco is using in star selection.
• acispy. The accepted value to use for specifying the vehicle load option to the attitude argument in SimulateECSRun has changed. Previously, one gave as the value the name of the load to take the commands from. Now, to specify vehicle commands, simply specify "vehicle" as the argument.

Testing:

• HEAD.
• OSX-arm64.

skare3 dashboard and test result password at https://icxc.cfa.harvard.edu/aspect/skare3_dash_cred.txt

The latest release candidates will be installed in /proj/sot/ska3/test on HEAD, and all release candidates will be available for testing from the usual channels:

conda create -n ska3-flight-2024.11rc4 --override-channels \
  -c https://icxc.cfa.harvard.edu/aspect/ska3-conda/flight \
  -c https://icxc.cfa.harvard.edu/aspect/ska3-conda/test \
  ska3-flight==2024.11rc4

If this release includes an update to ska3-perl, the install process for Aspect will include that. Note: ska3-perl is generally not needed for non-Aspect users.

conda create -n ska3-flight-2024.11rc4 --override-channels \
  -c https://icxc.cfa.harvard.edu/aspect/ska3-conda/flight \
  -c https://icxc.cfa.harvard.edu/aspect/ska3-conda/test \
  ska3-flight==2024.11rc4 ska3-perl==2024.11rc4

Review

All operations critical or impacting PR's are independently and carefully reviewed. For other PR's the level of detail for review is calibrated to operations criticality. Some PR's that are confined to aspect-team-specific processing may have little to no independent review.

Deployment

ska3-flight 2024.11rc1 will be promoted to flight conda channel and installed on HEAD and GRETA Linux upon approval of FSDS Jira ticket.

Code changes

ska3-core changes (2024.7 -> 2024.11rc4)

New Packages

• libgcc: 14.1.0
• libgfortran: 14.1.0

Updated Packages

• ipython: 8.22.1 -> 8.29.0
• libgcc-ng: 13.2.0 -> 14.1.0
• libgfortran-ng: 13.2.0 -> 14.1.0
• libgfortran5: 13.2.0 -> 14.1.0
• scipy: 1.12.0 -> 1.14.1

ska3-flight changes (2024.7 -> 2024.11rc4)

Removed Packages

• chandra_cmd_states
• find_attitude

Updated Packages

• aca_view: 0.14.1 -> 0.15.0 (0.14.1 -> 0.14.2 -> 0.15.0)
  • PR 188 (Javier Gonzalez): Handle modern format 6
  • PR 184 (Javier Gonzalez): Never make slot data stale
  • PR 195 (Javier Gonzalez): fix null-quat check
  • PR 193 (Javier Gonzalez): avoid null quaternion
  • PR 189 (Javier Gonzalez): Ruff
• acispy: 2.6.0 -> 2.7.0 (2.6.0 -> 2.7.0)
  • PR 14 (John ZuHone): Updates to thermal model support
  • PR 13 (John ZuHone): Improve handling of vehicle loads in SimulateECSRun, other small changes
• agasc: 4.21.0 -> 4.21.2 (4.21.0 -> 4.21.1 -> 4.21.2)
  • PR 187 (Jean Connelly): Add a link to the old 1p7 readme from the docs index
  • PR 182 (Javier Gonzalez): Update readme for 1p8
  • PR 184 (Tom Aldcroft): Add type annotations and use numpydoc for key modules and functions
  • PR 191 (Jean Connelly): Handle bad stars from supplement in one test
  • PR 188 (Jean Connelly): Reduce number of bad stars in test
• chandra_aca: 4.45.1 -> 4.47.0 (4.45.1 -> 4.46.0 -> 4.47.0)
  • PR 172 (Jean Connelly): Add routine for modern dark scaling
  • PR 171 (Javier Gonzalez): Binomial uncertainty
  • PR 170 (Tom Aldcroft): Use proseco acq candidate mask for ACA plot (bad_acq_stars)
  • PR 180 (Javier Gonzalez): Fix END_INTEG_TIME
• chandra_limits: 0.9.1 -> 0.10.0 (0.9.1 -> 0.9.2 -> 0.10.0)
  • PR 17 (John ZuHone): Fix for when long ECS measurements are not in the obscat
  • PR 18 (John ZuHone): Fix ACIS focal plane limit colors and labels
  • PR 19 (John ZuHone): Try to catch incorrect reads of the local copy of the obscat
• chandra_maneuver: 4.2.0 -> 4.3.0 (4.2.0 -> 4.3.0)
  • PR 29 (Tom Aldcroft): Make NSM_attitude() a wrapper for ska_sun.get_nsm_attitude()
• cheta: 4.62.0 -> 4.62.1 (4.62.0 -> 4.62.1)
  • PR 264 (Tom Aldcroft): Update NOTES.fix_bad_ingest.rst
• cxotime: 3.8.0 -> 3.9.0 (3.8.0 -> 3.9.0)
  • PR 44 (Jean Connelly): Add a linspace class method
• fot-matlab: 2.4.0 -> 2.4.1 (2.4.0 -> 2.4.1)
  • PR 27 (James Kristoff): MATLAB-12089 - Updates to the structure of the model data
• kadi: 7.11.0 -> 7.13.0 (7.11.0 -> 7.12.0 -> 7.13.0)
  • PR 333 (Javier Gonzalez): Use single date for agasc1p8 promotion
  • PR 336 (Jean Connelly): Update to use get_nsm_attitude
  • PR 335 (Tom Aldcroft): Squelch logging warning message in one test
  • PR 341 (Javier Gonzalez): fix package data regex
  • PR 334 (Tom Aldcroft): Support offset pitch for Safe mode and NSM events and improve Safe mode
  • PR 339 (Tom Aldcroft): Improve validation: YAML data files and set KADI_COMMANDS_DEFAULT_STOP
• mica: 4.35.2 -> 4.37.0 (4.35.2 -> 4.36.0 -> 4.37.0)
  • PR 300 (Jean Connelly): Ruff
  • PR 299 (Jean Connelly): Use full agasc for lookups for guide/acq stats
  • PR 301 (Jean Connelly): Add a get_aca_images method for mica l0
  • PR 308 (Jean Connelly): Open up regex on file name and glob to work with CP data
  • PR 307 (Jean Connelly): Fix truncated V&V html column
  • PR 306 (Jean Connelly): Add color1 to agasc fetch in centroid_dashboard
  • PR 305 (Jean Connelly): Handle omitted star missing from miniagasc 1p8 in V&V
  • PR 304 (Jean Connelly): Fix a processing error in acq and guide stats
  • PR 303 (Jean Connelly): Fix V&V processing error with Sqsh
  • PR 302 (Jean Connelly): Remove aiprops report
• parse_cm: 3.15.0 -> 3.16.0 (3.15.0 -> 3.16.0)
  • PR 54 (Jean Connelly): Update backstop quaternion printed format and corresponding regress data
• proseco: 5.13.2 -> 5.15.0 (5.13.2 -> 5.14.0 -> 5.15.0)
  • PR 402 (Tom Aldcroft): Update README.md
  • PR 400 (Jean Connelly): Remove some agasc supplement bad star tests
  • PR 399 (Tom Aldcroft): Factor out guide and acq candidates mask
  • PR 401 (Jean Connelly): Exclude COLOR1=0.7 stars from guide selection
• ska3-core: 2024.7 -> 2024.11rc4
• ska_helpers: 0.16.0 -> 0.17.0 (0.16.0 -> 0.17.0)
  • PR 58 (Tom Aldcroft): New utils: logger context manager and random RA and Decs
• ska_sun: 3.14.0 -> 3.15.0 (3.14.0 -> 3.15.0)
  • PR 38 (Tom Aldcroft): Add functions get_att_for_sun_pitch_yaw() and get_nsm_attitude()
• ska_tdb: 4.0.0 -> 4.1.0 (4.0.0 -> 4.1.0)
  • PR 19 (James Kristoff): Updates to support remote access from Windows
• sparkles: 4.26.1 -> 4.27.1 (4.26.1 -> 4.26.2 -> 4.27.0 -> 4.27.1)
  • PR 211 (Jean Connelly): Use a different bad star for bad star test
  • PR 212 (Jean Connelly): Update sparkles tests for proseco no-color1 guide star change
  • PR 213 (Javier Gonzalez): set obsid report dir with a name consistent with proseco
• starcheck: 14.11.0 -> 14.12.0 (14.11.0 -> 14.12.0)
  • PR 447 (Jean Connelly): Update starcheck IR zone to (-5, +35)
  • PR 426 (Jean Connelly): Update acq model range warning for current grid model

ska3-flight-latest changes (2023.10 -> 2024.11rc4)

Removed Packages

• chandra_cmd_states
• find_attitude

Related Issues

Fixes #1393
Fixes #1394
Fixes #1396
Fixes #1397
Fixes #1403
Fixes #1404
Fixes #1405
Fixes #1406
Fixes #1407
Fixes #1409
Fixes #1410
Fixes #1411
Fixes #1413
Fixes #1414
Fixes #1416
Fixes #1419
Fixes #1420
Fixes #1421
Fixes #1424
Fixes #1425
Fixes #1428
Fixes #1429
Fixes #1430
Fixes #1431
Fixes #1432
Fixes #1433