Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-48089: Add fgcmcal to LSSTComCam DRP.yaml #191

Merged
merged 3 commits into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions bps/resources/LSSTComCam/DRP.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# This BPS configuration snippet adds requestMemory values for DRP
# and tasks.
#
# Use it by adding:
#
# includeConfigs:
# - ${DRP_PIPE_DIR}/bps/resources/LSSTComCam/DRP.yaml
#
# (with no outer indentation) to your BPS config file.

pipetask:
fgcmBuildFromIsolatedStars:
requestMemory: 16000
fgcmFitCycle:
requestCpus: 16
requestMemory: 32000
extraRunQuantumOptions: "-j 1 -n 16"
13 changes: 9 additions & 4 deletions pipelines/LSSTComCam/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@ description: DRP Pipeline for LSSTComCam
instrument: lsst.obs.lsst.LsstComCam
imports:
- location: $DRP_PIPE_DIR/pipelines/_ingredients/LSSTComCam/DRP.yaml
exclude:
- fgcmBuildFromIsolatedStars
- fgcmFitCycle
- fgcmOutputProducts
subsets:
step1:
subset:
Expand Down Expand Up @@ -47,6 +43,15 @@ subsets:
gbdesAstrometricFit produces solutions per-tract, per-visit
isolatedStarAssociation produces solutions per-tract.
TODO: Evaluate GBDES after first DRP
step2c:
subset:
- fgcmBuildFromIsolatedStars
- fgcmFitCycle
- fgcmOutputProducts
description: |
This subset is the FGCM run that runs in "global" mode, which means
that one should only use an `instrument` and a `skymap` connection
(the latter of which links with the correct isolated star tables).
step2d:
subset:
- finalizeCharacterization
Expand Down
7 changes: 7 additions & 0 deletions pipelines/LSSTComCam/nightly-validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ imports:
- fgcmOutputProducts
- gbdesAstrometricFit
tasks:
updateVisitSummary:
class: lsst.drp.tasks.update_visit_summary.UpdateVisitSummaryTask
config:
# Nightly validation should always keep these off.
wcs_provider: "input_summary"
photo_calib_provider: "input_summary"
background_provider: "input_summary"
analyzePreSourceTableCore:
class: lsst.analysis.tools.tasks.SourceTableVisitAnalysisTask
config:
Expand Down
6 changes: 3 additions & 3 deletions pipelines/_ingredients/LSSTComCam/DRP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ tasks:
class: lsst.fgcmcal.fgcmFitCycle.FgcmFitCycleTask
config:
doMultipleCycles: true
multipleCyclesFinalCycleNumber: 4
multipleCyclesFinalCycleNumber: 5
fgcmOutputProducts:
class: lsst.fgcmcal.fgcmOutputProducts.FgcmOutputProductsTask
config:
connections.cycleNumber: 4
connections.cycleNumber: 5
selectDeepCoaddVisits:
class: lsst.pipe.tasks.selectImages.BestSeeingSelectVisitsTask
config:
Expand All @@ -54,7 +54,7 @@ tasks:
# TODO: Evaluate GBDES, SkyCorr, FGCM after initial DRP
# Nightly validation should always keep these off.
wcs_provider: "input_summary"
photo_calib_provider: "input_summary"
photo_calib_provider: "global"
background_provider: "input_summary"
reprocessVisitImage:
class: lsst.drp.tasks.reprocess_visit_image.ReprocessVisitImageTask
Expand Down
3 changes: 2 additions & 1 deletion tests/test_pipelines.py
Original file line number Diff line number Diff line change
Expand Up @@ -638,6 +638,7 @@ def test_lsstcomcam_drp(self):
"#step1",
"#step2a",
"#step2b",
"#step2c",
"#step2d",
"#step2e",
"#step3a",
Expand All @@ -648,7 +649,7 @@ def test_lsstcomcam_drp(self):
"#step7",
],
initial_dataset_types=REFCATS,
expected_inputs=COMMON_INPUTS | LSSTCOMCAM_INPUTS,
expected_inputs=COMMON_INPUTS | LSSTCOMCAM_INPUTS | {"fgcmLookUpTable"},
expected_outputs=COMMON_OUTPUTS,
)
tester.run(butler, self)
Expand Down
Loading