Adds the ability to bundle adjust CSM images (#4537)

* copylab

* Initial work toward a working AbstractBundleObservation class

* Fix accidentl add of temporary changes from another project

* Trial 2 on fixing my accidental changes

* csm bundle building properly (#4409)

* Updates for linking and compiling

* Starting virtual functions

* Add virtuals to AbstractBundleObservation and get compiling again

* Remove function from parent

* Get jigsaw tests passing on csmbundle branch (#4413)

* Add virtuals to AbstractBundleObservation and get compiling again

* Remove function from parent

* Get existing jigsaw tests passing

* Remove unneeded functions from BundleObservation

* Clean up a bit more

* Clean up BundleObservation and AbstractBundleObservation more

* Remove unneeded BodyRotation

* Add target body solve back into bundle adjusment calc

* Remove debug output

* Removed unneeded memeber variables BundleObservation

Co-authored-by: Kristin Berry <kberry@gyro.wr.usgs.gov>

* Move some functions to pure virtual

* Csmobservation (#4419)

* Now compiling

* Updated CSMCamera tests and docs

* Remove duplicated member variables and accesor functions from BundleObservation and add Csm camera type to Camera enum

* Remove commented-out code

* Added required headers from removed ground map header (#4427)

* last missing header (#4428)

* Moved CSM settings into BOSS

* Generates Serial Numbers using CSM label information (#4437)

* Creates Serial Numbers using CSM label information

* Updated second SerialNumberList:add function to accommodate CSM label

* Add hooks for csm bundle settings to jigsaw (#4438)

* In progress

* Added tests and Pvl support

* Add missing 'y' to docs

Co-authored-by: Kristin Berry <kberry@usgs.gov>

* Moves partial calculation into CSM and ISIS observation classes. (#4436)

* Remove duplicated member variables and accesor functions from BundleObservation and add Csm camera type to Camera enum

* Remove commented-out code

* Imagecoeff migrated into BundleObservation class working

* Get base level working for all coeffs in ISIS

* Fix accidental move into if statement to get tests passing again

* Updated RHS for CsmBundleObservation to work

* Adds point3D support to CSM Camera observations and also adds ground Partials member function to CSMCamera

* Draft of sensor partials code

* Add docs, some fixes based on review requests

* move computation of weights and observation values to bundle observations

* Add observationvalue function to CsmBundleObservation and move the weighting that it is possible to to the observation classes

* Make CSMSolveSet an optional parameter to get jigsaw tests passing again

* cleanup

* Move observation weights into MLE if block

Co-authored-by: Kristin Berry <kberry@gyro.wr.usgs.gov>

* Implemented bundleOutputString for CSM (#4444)

* Added test for CSM bundleOutputString

* Passing unit test and updates to bundleOutputString

* Addressed PR feedback

* Csmcsv (#4446)

* CSM CSV output

* Small test clean-up

* typo fix

* obs header fix

* Remove param list from bo

* Added debug output

* Now reurning

* Fixed unit test

* xml save test update

* Clean up bundleresults unit test output

* Fixed qfile member

* Fixed old path

* Changed header to use param names

* Making separate csvs

* Added reverse on obs by instId

* Trying to fix multiple sensors

* Fixed images header

* More debug

* removed debug outs

* Added terrible imageindex search

* Added held config fix

* Updated for observation test

* Updated jigsaw test makefiles

* Updated with new tests

* Updates Bundle Observation Vector to support Csm observations. (#4457)

* Remove duplicated member variables and accesor functions from BundleObservation and add Csm camera type to Camera enum

* Remove commented-out code

* Imagecoeff migrated into BundleObservation class working

* Get base level working for all coeffs in ISIS

* Fix accidental move into if statement to get tests passing again

* Updated RHS for CsmBundleObservation to work

* Adds point3D support to CSM Camera observations and also adds ground Partials member function to CSMCamera

* Draft of sensor partials code

* Add docs, some fixes based on review requests

* move computation of weights and observation values to bundle observations

* Add observationvalue function to CsmBundleObservation and move the weighting that it is possible to to the observation classes

* Make CSMSolveSet an optional parameter to get jigsaw tests passing again

* cleanup

* Move observation weights into MLE if block

* Added isis vs. csm support to Bundle Observation Vector class to allow jigsaw to proceed past the setup stage for csm networks

* Comment cleanup

* Fix failing unit tests

* Add docs

* Updated based on feedback with potentially temporary fix to get unit tests passing

* Update tests a bit

* Aligned signature definitions and cleaned up

* remove unneeded functions

* More comment cleanup

Co-authored-by: Kristin Berry <kberry@gyro.wr.usgs.gov>

* Jigsaw Test Conversions (sccconfig, scconfigHeld, radar) (#4463)

* Converted jigsaw scconfig tests.

* Added jigsaw radar test

* Updated tests to reflect PR #4446 changes & removed makefile tests

* Updates to scconfig tests

* Gets csm bundle converging (#4464)

* Added cam type checks

* checking csm partials

* Debug partials

* debug corrections

* Fixed index

* Scaled ground partials

* Updated param comparison

* Removed debug out

* Fixes spacing issue in CSM bundleout.txt (#4465)

* Update csm bundlout to have correct spacing for paramter names

* Reverted unintential change

Co-authored-by: Kristin Berry <kberry@gyro.wr.usgs.gov>

* Removed unit-test specific changes I made to the code instead of the unit test

* Add latitudinal partials for CSM (#4468)

* Add latitudinal partials for CSM

* Added test

Co-authored-by: Jesse Mapel <jmapel@usgs.gov>

* Csm clean up (#4483)

* Added cms index list error

* Added CSMCameraTests for partials

* Observation clean-up

* Cleaned up vector

* Rename BundleObservation classes (#4518)

* rename BundleObservation classes

* omit changes to sip wrappers

* Adds more capabilities to TestCsmModel (#4527)

* Updated to get new isd/state form to test jigsaw parsing

* Add functional image-to-ground, ground-to-image, and isd-parsing to TestCsmModel

* Remove temporary csminit test changes

* Updated variable name for clarity

* add csm info to app docs for csminit and jigsaw

* Csmbundle - remove SIP wrappers and deprecated formatBundleOutputString (#4526)

* remove sip wrappers

* remove deprecated formatBundleOutputString

* remove more sip code and formatBundleOutputString

* Fix functional tests to work with updated TestCsmModel (#4534)

* Tests + test data for CSM jigsaw (#4530)

* Removed debug couts

* Added csmnetwork test fixture

* Added a test for CSM test network

* Adjusted trig functions + added deg/rad conversions

* Initial CSMNetwork data

* Uncommented code

* Removed network path and control network from CSMNetwork fixture

* Removed temporary path from csv comparison

* Adjusted spacing + responded to PR feedback

Co-authored-by: Austin Sanders <arsanders@ugs.gov>

* Added the ability to update CSM model state in jigsaw (#4529)

* Added the ability to update CSM model state in jigsaw

* Added missing function docs

* Updated CSM jigsaw test to check apply

* Wrangled up an F

* Added changelog entry

* General clean-up

* Changed Camera local photometric calculations to not use bodyrotation

* Changed csm camera to save look on non-intersection

* Fixed docs typos

Co-authored-by: Kristin Berry <kberry@gyro.wr.usgs.gov>
Co-authored-by: Kristin Berry <kberry@usgs.gov>
Co-authored-by: Amy Stamile <74275278+amystamile-usgs@users.noreply.github.com>
Co-authored-by: Tim Giroux <62255438+tgiroux@users.noreply.github.com>
Co-authored-by: Tim Giroux <tgiroux@contractor.usgs.gov>
Co-authored-by: AustinSanders <arsanders@usgs.gov>
Co-authored-by: Austin Sanders <arsanders@ugs.gov>

CHANGELOG.md

@@ -37,10 +37,13 @@ release.
 
 ### Added
 - Added a new dark current correction to hical that works with the higher temperatures recent images are captured at. Use the new config file, $ISISDATA/mro/calibration/hical.0023_darkrate.conf, to enable the new dark current correction over the old dark current correction. Runs of hical without the new dark current correction will also produce an extra line in the output log indicating that the ZeroDarkRate module is disabled. [#4324](https://github.com/USGS-Astrogeology/ISIS3/issues/4324)
+- Added the ability to bundle adjust CSM models in jigsaw. Use the new CSMSOLVESET, CSMSOLVETYPE, and CSMSOLVELIST arguments to specify what you solve for. [#4537](https://github.com/USGS-Astrogeology/ISIS3/pull/4537)
 
 ### Changed
 - Added the ability to export footprint information from a CaSSIS ISIS Cube label to the generated output PDS4 label in tgocassisrdrgen. [#4473](https://github.com/USGS-Astrogeology/ISIS3/issues/4473)
 - isisVarInit.py no longer writes a "cat" statement by default to the activate scripts which cause the ISIS version information to be written on conda activate.  This can be included in those scripts via a command line option to isisVarInit.py.  Also, a quiet option is provided to isisVarInit.py to suppress its own writing to standard out, if needed.
+- Changed how the images.csv file is output in jigsaw when there are multiple models. Now each sensor model will have its own images.csv file so that column headers can all be correct. For example, a solution involving LRONAC pairs and Apollo Metric images would generate three images.csv files: LRONAC Left, LRONAC Right, and Apollo Metric. [#4324](https://github.com/USGS-Astrogeology/ISIS3/issues/4324)
+- Changed the API of many Bundle utility classes as part of CSM support in jigsaw. [#4324](https://github.com/USGS-Astrogeology/ISIS3/issues/4324)
 
 ### Fixed
 - Fixed logging in FindFeatures where we were trying to get a non-existent Pvl group from the Pvl log. [#4375](https://github.com/USGS-Astrogeology/ISIS3/issues/4375)

isis/CMakeLists.txt

@@ -303,7 +303,6 @@ endif(APPLE)
 
 if(pybindings)
  find_package(Python REQUIRED)
- find_package(Sip    REQUIRED)
 endif()
 
 # Iterate through all variables and extract the libraries and include directories
@@ -356,70 +355,6 @@ list(REMOVE_DUPLICATES ALLLIBDIRS)
 list(REMOVE_DUPLICATES ALLLIBS)
 list(REMOVE_DUPLICATES ALLINCDIRS)
 
-#===============================================================================
-#===============================================================================
-
-# Set python bindings configuration and set target for generating C++ files
-if(pybindings)
-  message("Configuring Python Bindings")
-
-  if (NOT DEFINED PYINSTALL_DIR)
-    set(PYINSTALL_DIR ${PYTHON_SITE_PACKAGES_DIR})
-  endif()
-  message(STATUS "PYTHON BINDINGS INSTALL DIR: ${PYINSTALL_DIR}")
-
- # We need to get the locations for sip files, modules, etc.
- set(ISIS_SIP_DIR "${CMAKE_SOURCE_DIR}/sipfiles")
- set(ISIS_SIP_MODULE "${CMAKE_SOURCE_DIR}/sipfiles/master.sip")
- set(SIP_BUILD_FILE "isispy.sbf")
- set(ISIS_SIP_CODE_DIR ${CMAKE_BINARY_DIR}/sipsrc)
-
- # Create the output directory for the new .CPP files
- execute_process(COMMAND mkdir -p "${ISIS_SIP_CODE_DIR}")
-
- # get list of output files exepected from sip
-
- # get the PYQT configuration based flags from Python
- execute_process(COMMAND ${PYTHON_EXECUTABLE} -c
-                 "from PyQt5.QtCore import PYQT_CONFIGURATION as qtconfigdict;print(qtconfigdict['sip_flags'])"
-                 OUTPUT_VARIABLE PYQT_SIP_FLAGS)
-
- # CMAKE doesn't handle spaces from python well when piping that into the
- # command because of white space and a trailing new line,
- # so we turn it into a list
- message(STATUS "Getting SIP config...")
- message(STATUS "Attempting 'python -c \"from PyQt5.QtCore import PYQT_CONFIGURATION as qtconfigdict; print(qtconfigdict['sip_flags'])\"'")
-
- if (${PYQT_SIP_FLAGS} STREQUAL "")
-    message(FATAL_ERROR "print(qtconfigdict['sip_flags']) returned empty string, is sip installed? Python binding can be disabled with pybindings=OFF")
- endif()
- string(STRIP ${PYQT_SIP_FLAGS} PYQT_SIP_FLAGS)
- string(REPLACE " " ";" PYQT_SIP_FLAGS ${PYQT_SIP_FLAGS})
-
- message(STATUS "FLAGS: ${PYQT_SIP_FLAGS}")
- message(STATUS "Generating C++ code from sip files")
- message(STATUS "SIP BUILD FILE: ${SIP_BUILD_FILE}")
- message(STATUS "SIP MODULE: ${ISIS_SIP_MODULE}")
- message(STATUS "SIP GENERATED CODE DIR: ${ISIS_SIP_CODE_DIR}")
-
- execute_process(COMMAND ${SIP_BINARY_PATH} -e -o -c ${ISIS_SIP_CODE_DIR} -I ${SIP_DEFAULT_SIP_DIR}/PyQt5 ${PYQT_SIP_FLAGS} ${ISIS_SIP_MODULE})
-
- # add target so users can run the command after initial configuration
- add_custom_target(sipfiles
-                  COMMAND ${SIP_BINARY_PATH} -e -o -c ${ISIS_SIP_CODE_DIR} -I ${SIP_DEFAULT_SIP_DIR}/PyQt5 ${PYQT_SIP_FLAGS} ${ISIS_SIP_MODULE}
-                  COMMENT "Generating C++ code from sip files")
-
- file(GLOB SIP_GENERATED_SOURCE_FILES ${ISIS_SIP_CODE_DIR}/*.cpp)
- add_library(isispy MODULE ${SIP_GENERATED_SOURCE_FILES})
- target_link_libraries(isispy ${ALLLIBS})
- target_link_libraries(isispy isis)
- set_target_properties(isispy PROPERTIES LINK_DEPENDS isis INSTALL_RPATH ${CMAKE_INSTALL_PREFIX})
- add_dependencies(isispy sipfiles)
-
- install(TARGETS isispy DESTINATION ${PYINSTALL_DIR})
-endif()
-
-
 #===============================================================================
 #===============================================================================
 

isis/appdata/serialnumbers/CsmSerialNumber.trn

@@ -0,0 +1,30 @@
+Group = Keyword1
+  Auto
+  InputKey       = CSMPlatformId
+  InputGroup     = "IsisCube,CsmInfo"
+  InputPosition  = (IsisCube, CsmInfo)
+  OutputName     = Keyword1
+  OutputPosition = (Group, SerialNumberKeywords)
+  Translation    = (* , *)
+End_Group
+
+Group = Keyword2
+  Auto
+  InputKey       = CSMInstrumentId
+  InputGroup     = "IsisCube,CsmInfo"
+  InputPosition  = (IsisCube, CsmInfo)
+  OutputName     = Keyword2
+  OutputPosition = (Group, SerialNumberKeywords)
+  Translation    = (*, *)
+End_Group
+
+Group = Keyword3
+  Auto
+  InputKey       = ReferenceTime
+  InputGroup     = "IsisCube,CsmInfo"
+  InputPosition  = (IsisCube, CsmInfo)
+  OutputName     = Keyword3
+  OutputPosition = (Group, SerialNumberKeywords)
+  Translation    = (*, *)
+End_Group
+End