Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variance calibration #2636

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/changes/2636.features.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Makes changes to the CameraCalibrator in ctapipe.calib.camera.calibrator that allows it to correctly variance images generated with the VarianceExtractor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please describe the actual change, not just say "there are changes". Also there is a verb missing here (calibrate?)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i added some explanation

18 changes: 15 additions & 3 deletions src/ctapipe/calib/camera/calibrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@
calibration_monitoring_id=r1.calibration_monitoring_id,
)

def _calibrate_dl1(self, event, tel_id):

Check failure on line 221 in src/ctapipe/calib/camera/calibrator.py

View check run for this annotation

CTAO-DPPS-SonarQube / ctapipe Sonarqube Results

src/ctapipe/calib/camera/calibrator.py#L221

Refactor this function to reduce its Cognitive Complexity from 30 to the 15 allowed.
waveforms = event.dl0.tel[tel_id].waveform
if self._check_dl0_empty(waveforms):
return
Expand Down Expand Up @@ -283,7 +283,11 @@
)

# correct non-integer remainder of the shift if given
if self.apply_peak_time_shift.tel[tel_id] and remaining_shift is not None:
if (
self.apply_peak_time_shift.tel[tel_id]
and remaining_shift is not None
and dl1.peak_time is not None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logically, I'd put the dl1.peak_time is not None check first

):
dl1.peak_time -= remaining_shift

# Calibrate extracted charge
Expand All @@ -292,13 +296,21 @@
and dl1_calib.absolute_factor is not None
):
if selected_gain_channel is None:
ctoennis marked this conversation as resolved.
Show resolved Hide resolved
dl1.image *= dl1_calib.relative_factor / dl1_calib.absolute_factor
if extractor.__class__.__name__ == "VarianceExtractor":
ctoennis marked this conversation as resolved.
Show resolved Hide resolved
dl1.image *= np.square(
dl1_calib.relative_factor / dl1_calib.absolute_factor
)
else:
dl1.image *= dl1_calib.relative_factor / dl1_calib.absolute_factor
else:
corr = (
dl1_calib.relative_factor[selected_gain_channel, pixel_index]
/ dl1_calib.absolute_factor[selected_gain_channel, pixel_index]
)
dl1.image *= corr
if extractor.__class__.__name__ == "VarianceExtractor":
ctoennis marked this conversation as resolved.
Show resolved Hide resolved
dl1.image *= np.square(corr)
else:
dl1.image *= corr

# handle invalid pixels
if self.invalid_pixel_handler is not None:
Expand Down
61 changes: 61 additions & 0 deletions src/ctapipe/calib/camera/tests/test_calibrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
GlobalPeakWindowSum,
LocalPeakWindowSum,
NeighborPeakWindowSum,
VarianceExtractor,
)
from ctapipe.image.reducer import NullDataVolumeReducer, TailCutsDataVolumeReducer

Expand Down Expand Up @@ -130,6 +131,66 @@
assert (event.dl1.tel[tel_id].image == 2).all()


def test_dl1_variance_calib(example_event, example_subarray):
# test the calibration of variance images
tel_id = list(example_event.r0.tel)[0]
calibrator = CameraCalibrator(
subarray=example_subarray,
image_extractor=VarianceExtractor(subarray=example_subarray),
apply_waveform_time_shift=False,
)
calibrator(example_event)
image = example_event.dl1.tel[tel_id].image
assert image is not None
assert image.shape == (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test with the LST (two gains)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to add a test for other extractor with the LST?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use a parametrized test here to test all telescope types. (@pytest.mark.parametrized). See for example the code in ctapipe/instrument/tests/test_telescope.py

1,
1764,
)


def test_calib_LST_camera(example_subarray):

Check warning on line 151 in src/ctapipe/calib/camera/tests/test_calibrator.py

View check run for this annotation

CTAO-DPPS-SonarQube / ctapipe Sonarqube Results

src/ctapipe/calib/camera/tests/test_calibrator.py#L151

Rename function "test_calib_LST_camera" to match the regular expression ^[a-z_][a-z0-9_]*$.
n_channels = 2
n_pixels = 1855 # number of pixels in LSTcam
n_samples = 100

random = np.random.default_rng(1)

tel_id = 1
y = random.normal(0, 6, (n_channels, n_pixels, n_samples))

absolute = random.uniform(100, 1000, (n_channels, n_pixels)).astype("float32")
y *= absolute[..., np.newaxis]

relative = random.normal(1, 0.01, (n_channels, n_pixels))
y /= relative[..., np.newaxis]

pedestal = random.uniform(-4, 4, (n_channels, n_pixels))
y += pedestal[..., np.newaxis]

event = ArrayEventContainer()
event.dl0.tel[tel_id].waveform = y
event.calibration.tel[tel_id].dl1.pedestal_offset = pedestal
event.calibration.tel[tel_id].dl1.absolute_factor = absolute
event.calibration.tel[tel_id].dl1.relative_factor = relative
event.dl0.tel[tel_id].selected_gain_channel = None
event.r1.tel[tel_id].selected_gain_channel = None

calibrator = CameraCalibrator(
subarray=example_subarray,
image_extractor=VarianceExtractor(subarray=example_subarray),
apply_waveform_time_shift=False,
)
calibrator(event)

image = event.dl1.tel[tel_id].image

assert image is not None
assert image.shape == (
2,
1855,
)


def test_dl1_charge_calib(example_subarray):
# copy because we mutate the camera, should not affect other tests
subarray = deepcopy(example_subarray)
Expand Down