Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-41053: Add QA tasks to AuxTel/LATISS image processing pipeline #86

Merged
merged 1 commit into from
Oct 24, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 63 additions & 32 deletions pipelines/LATISS/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,36 +2,23 @@ description: DRP specialized for LATISS surveys
instrument: lsst.obs.lsst.Latiss
imports:
- location: $DRP_PIPE_DIR/pipelines/_ingredients/DRP-full.yaml
- location: $ANALYSIS_TOOLS_DIR/pipelines/coaddColumnValidate.yaml
- location: $ANALYSIS_TOOLS_DIR/pipelines/matchedVisitQualityCore.yaml
- location: $ANALYSIS_TOOLS_DIR/pipelines/coaddQualityCore.yaml
exclude:
- analyzeObjectTableCore
- location: $ANALYSIS_DRP_DIR/pipelines/analysis_drp_plots.yaml
include:
- isr
- characterizeImage
- calibrate
- writePreSourceTable
- transformPreSourceTable
- consolidatePreSourceTable
- consolidateVisitSummary
- isolatedStarAssociation
- fgcmBuildFromIsolatedStars
- fgcmFitCycle
- fgcmOutputProducts
- finalizeCharacterization
- updateVisitSummary
- makeCcdVisitTable
- makeVisitTable
- transformSourceTable
- consolidateSourceTable
- writeRecalibratedSourceTable
- makeWarp
- assembleCoadd
- detection
- mergeDetections
- deblend
- measure
- mergeMeasurements
- writeObjectTable
- transformObjectTable
- consolidateObjectTable
- forcedPhotCoadd
- plot_ri_gr_psf
- plot_ri_gr_cmodel
- plot_wFit_PSF
- plot_xFit_PSF
# The following are from analysis_drp
# TODO: remove once they have all been ported to analysis_tools (DM-39081)
- location: $ANALYSIS_DRP_DIR/pipelines/coaddQAPlots_tier2_3.yaml
include:
- plot_ri_gr_contour_psf
- plot_ri_gr_contour_cmodel

tasks:
fgcmFitCycle:
Expand All @@ -52,6 +39,27 @@ tasks:
photo_calib_provider: "global"
background_provider: "input_summary"

# Importing and defining analyzeObjectTableCore here to omit plots that
# require z and/or y band data.
analyzeObjectTableCore:
class: lsst.analysis.tools.tasks.ObjectTableTractAnalysisTask
config:
connections.outputName: objectTableCore
atools.shapeSizeFractionalDiff: ShapeSizeFractionalDiff
atools.e1Diff: E1Diff
atools.e2Diff: E2Diff
atools.skyFluxStatisticMetric: SkyFluxStatisticMetric
atools.skyFluxStatisticMetric.applyContext: CoaddContext
atools.wPerpPSFP: WPerpPSF
atools.wPerpCModel: WPerpCModel
atools.xPerpPSFP: XPerpPSF
atools.xPerpCModel: XPerpCModel
atools.skyObjectSky: SkyObjectSkyPlot
atools.skyObjectFlux: SkyObjectHistPlot
python: |
from lsst.analysis.tools.atools import *
from lsst.analysis.tools.contexts import *

subsets:
step1:
subset:
Expand Down Expand Up @@ -134,9 +142,32 @@ subsets:
description: |
Tract-level tasks. Allowed data query constraints: tract
Tasks that can be run together, but only after the 'step1', 'step2', and
'step3a' subsets (seperation into step3a and step3b is necessary for now
as otherwise failues in upstream tasks will cause these to fail. This is
hoped to be a temporary limitation).
'step3a' subsets (separation into step3a and step3b is necessary for now
as otherwise failures in upstream tasks will cause these to fail. This
is hoped to be a temporary limitation).

step3c:
subset:
- healSparsePropertyMaps
- analyzeObjectTableCore
- analyzeObjectTableSurveyCore
- catalogMatchTract
- refCatObjectTract
- validateObjectTableCore
- analyzeMatchedVisitCore
# The following are from analysis_drp
# TODO: remove once they have all been ported to analysis_tools (DM-39081)
- plot_ri_gr_psf
- plot_ri_gr_cmodel
- plot_wFit_PSF
- plot_xFit_PSF
description: |
Healsparse maps and "core" (most important), coadd-level plots/metrics from
analysis_tools (plus a few from analysis_drp to be deprecated once replicated
in analysis_tools).

Must be run after 'step3b'. Cannot be run with any dataquery constraints
if analyzeObjectTableSurveyCore is included.

step4:
subset:
Expand Down