Tutorial Jupyter Notebooks for Data Preview 0.2 and 0.3, created and maintained by the Rubin Observatory Community Science Team.
The prod branch will appear in IDF RSP users' home directories, in notebooks/tutorial-notebooks/.
Tutorial titles in bold have Spanish-language versions.
| Title | Brief Description |
|---|---|
| 01. Introduction to DP0.2 | Use the Jupyter Notebooks and Rubin python packages to access LSST data products. |
| 02a. Introduction to TAP | Explore the DP0.2 catalogs with the TAP service. |
| 02b. Catalog Queries with TAP | Execute complex ADQL queries with the TAP service. Visualize catalog data sets. |
| 02c. Image Queries with TAP | Retrieve and display images using the ObsTAP service. |
| 03a. Image Display and Manipulation | Learn how to display and manipulate images using the LSST Science Pipelines. |
| 03b. Image Display with Firefly | Use the Firefly interactive interface for image data. |
| 03c. Big deepCoadd Cutout | Use the GetTemplateTask to create a custom deepCoadd cutout that spans multiple patches and tracts. |
| 04a. Introduction to the Butler | Use the Butler to query DP0 images and catalogs. |
| 04b. Intermediate Butler Queries | Learn to discover data and apply query constraints with the Butler. |
| 05. Source Detection and Measurement | Access, display, and manipulate images; detect, deblend, and measure sources; and extract, plot, and use object footprints. |
| 06a. Interactive Image Visualization | Create interactive image visualizations with the HoloViews and Bokeh open-source python libraries. |
| 06b. Interactive Catalog Visualization | Create interactive catalog visualizations for large datasets with HoloViews, Bokeh, and Datashader. |
| 07a. DiaObject Samples | Use the DiaObject table parameters to identify a sample of time-variable objects of interest. |
| 07b. Variable Star Lightcurves | Use the DP0.2 catalogs to identify variable stars and plot their lightcurves. |
| 08. Truth Tables | Explore, retrieve, and compare data from the truth and measurement tables. |
| 09a. Custom Coadd | Create a custom "deepCoadd" using only a subset of the input visits. |
| 09b. Custom Coadd Sources | Detect and measure sources in a custom "deepCoadd" image. |
| 10. Deblender Data Products | Use the outputs of the multiband deblender to explore the footprints of parent and child objects. |
| 11. Working with User Packages | An example of how to install and set up user packages. |
| 12a. Point Spread Function Data Products | A demonstration of how to access calexp and deepCoadd PSF properties. |
| 12b. Point Spread Function Science Demo | Demonstration of the use of measured PSF properties in weak lensing analysis. |
| 13a. Using The Image Cutout Tool With DP0.2 | Demonstration of the use of the image cutout tool with a few science applications. |
| 14. Injecting Synthetic Sources Into Single-Visit Images | Inject artificial stars and galaxies into images. |
| 15. Survey Property Maps | Use the tools to visualize full-area survey property maps. |
| Title | Brief Description |
|---|---|
| 01. Introduction to DP0.3 | An overview of the contents of the DP0.3 moving object catalogs. |
| 02. Main Belt Asteroids | A brief exploration of the orbital properties of Main Belt asteroids in DP0.3 catalogs. |
| 03. Trans-Neptunian Objects | Explore the trans-Neptunian object populations in DP0.3. |
| 04a. Introduction to Phase Curves | Explore phase curves for DP0.3 solar system objects. |
| 04b. Advanced Phase Curve Modeling | Explicitly investigate the derivation of phase curves for Main Belt asteroids. |
| 05. Near-Earth Objects | Exploration of the orbital properties of near-Earth objects in the DP0.3 catalogs. |
| 06. User-Uploaded Catalogs | Use the TAP upload functionality for user-supplied tables and join them with DP0.3 catalogs. |
| 07. Interactive Catalog Visualization | Create interactive catalog visualizations for large datasets with HoloViews, Bokeh, and Datashader. |
The tutorials in this repo will only work on the Rubin Science Platform deployed at the Interim Data Facility for Data Preview 0.
The tutorials in this repo are subject to change, as the Rubin Science Platform and the LSST Science Pipelines are in active development.
DP0.2 tutorial notebooks 09a and 09b can only be used with uncached RSP Image "Weekly 2022_40".
Branches main and prod will usually match, except for short periods of time when updates from multiple development branches are being collected in main before "releasing" updated notebooks in a single PR to prod.
It is the prod branch which appears automatically in IDF RSP users' home directories.
Any user who obtains this repository with git clone can switch from main to prod using git checkout prod to ensure they are using the "released" versions.
The documentation for DP0 can be found at dp0-2.lsst.io and dp0-3.lsst.io.
To ask a question or report an issue (e.g., a bug), use either the Data Preview 0 category of the Rubin Community Forum, or submit a GitHub Issue on the Support repository.
Please don't submit GitHub Issues in this repository! Rubin staff only monitor the Support repository and the Community Forum.
These notebooks have drawn from these repositories:
- https://github.com/lsst-sqre/notebook-demo
- https://github.com/LSSTScienceCollaborations/StackClub
Many of the tutorial notebooks in this repository were originally developed by Stack Club members or Rubin staff, and have been altered and updated to be appropriate for DP0.
Want to contribute a tutorial? Contact Melissa Graham via direct message at Community.lsst.org.
The content of these notebooks are licensed under the Apache 2.0 License. That said, the use of the Vera C. Rubin logo is reserved. Thus, the content of these notebooks may be re-used, but the header and logo at the top of each notebook should not. Those re-using the contents of these notebooks should remove the original header and logo and provide their own, and not use the Rubin Observatory branding.
| Skills in DP0.2 Tutorial Notebooks |
|---|
| 01. Introduction to DP0.2 Level: Beginner Description: Use the Jupyter Notebooks and Rubin python packages to access LSST data products. Skills: Execute python code in a Jupyter Notebook. Use the TAP service to retrieve Object catalog data. Use the Butler to retrieve and display a deepCoadd image. Data Products: Object table; deepCoadd images Packages: lsst.rsp.get_tap_service, lsst.rsp.retrieve_query, lsst.daf.butler, lsst.afw.display, lsst.geom, pandas, matplotlib |
| 02a. Introduction to TAP Level: Beginner Description: Explore the DP0.2 catalogs with the TAP service. Skills: Use the TAP service. Make simple ADQL queries. Visualize retrieved datasets. Data Products: Catalog schema. Object table. Packages: lsst.rsp |
| 02b. Catalog Queries with TAP Level: Intermediate Description: Execute complex ADQL queries with the TAP service. Visualize catalog data sets. Skills: Use advanced ADQL and TAP functionality. Visualize retrieved datasets with bokeh and holoviews Data Products: Object, ForcedSource, CcdVisit tables. Packages: lsst.rsp, bokeh, pandas |
| 02c. Image Queries with TAP Level: Intermediate Description: Retrieve and display images using the ObsTAP service. Skills: Query for image data via the TAP service; retrieve image data with PyVO and Datalink; display and save images. Data Products: calexp, deepCoadd Packages: lsst.rsp, lsst.afw, pyvo.dal.adhoc, urllib.request |
| 03a. Image Display and Manipulation Level: Beginner Description: Learn how to display and manipulate images using the LSST Science Pipelines. Skills: Display and manipulate images, explore image mask planes, create cutouts and RGB images. Data Products: calexp and deepCoadd images; Object table Packages: lsst.afw.display, lsst.daf.butler, lsst.geom, lsst.afw.image |
| 03b. Image Display with Firefly Level: Beginner Description: Use the Firefly interactive interface for image data. Skills: Using Firefly as the display interface; visualizing images and their masks; overlaying sources on images. Data Products: calexp, deepCoadd_calexp images; src, deepCoadd_forced_src, deepCoadd_ref tables Packages: lsst.afw.display, lsst.daf.butler |
| 03c. Big deepCoadd Cutout Level: Intermediate Description: Create a large custom deepCoadd cutout. Skills: Identify tracts and patches and combine them into a large custom deepCoadd cutout. Data Products: deepCoadd images Packages: lsst.ip.diffim.GetTemplateTask |
| 04a. Introduction to the Butler Level: Beginner Description: Use the Butler to query DP0 images and catalogs. Skills: Query and retrieve images and catalog data with the Butler. Data Products: calexp, goodSeeingDiff_differenceExp, deepCoadd images; sourceTable, diaSourceTable, objectTable, forcedSourceTable, forcedSourceOnDiaObjectTable tables Packages: lsst.daf.butler |
| 04b. Intermediate Butler Queries Level: Intermediate Description: Learn to discover data and apply query constraints with the Butler. Skills: Use the Butler registry, dataIds, and spatial and temporal constraints. Data Products: calexps, deepCoadds, sources Packages: lsst.daf.butler, lsst.sphgeom |
| 05. Source Detection and Measurement Level: Intermediate Description: Access, display, and manipulate images; detect, deblend, and measure sources; and extract, plot, and use object footprints. Skills: Run source detection, deblending and measurement. Use source "footprints". Do forced photometry. Data Products: DP0.2 processed visit images and catalogs. Packages: lsst.pipe.tasks.characterizeImage, lsst.meas.algorithms.detection.SourceDetectionTask, lsst.meas.deblender.SourceDeblendTask, lsst.meas.base.SingleFrameMeasurementTask, lsst.meas.base.ForcedMeasurementTask |
| 06a. Interactive Image Visualization Level: Intermediate Description: Create interactive image visualizations with the HoloViews and Bokeh open-source python libraries. Skills: Visualize exposure images with HoloViews, interact with images with HoloViews Streams and Dynamic Map. Data Products: calexp, deepCoadd images; Source, Object tables Packages: bokeh, holoviews |
| 06b. Interactive Catalog Visualization Level: Intermediate Description: Create interactive catalog visualizations for large datasets with HoloViews, Bokeh, and Datashader. Skills: Create linked interactive plots for large datasets. Use Bokeh, HoloViews, and Datashader. Data Products: Object catalog Packages: bokeh, holoviews, datashader |
| 07a. DiaObject Samples Level: Intermediate Description: Use the DiaObject table parameters to identify a sample of time-variable objects of interest. Skills: Use the TAP service and the DP0.2 DiaObject and DiaSource tables. Data Products: DiaObject, DiaSource tables Packages: lsst.rsp, astropy.cosmology, astropy.stats, numpy, matplotlib |
| 07b. Variable Star Lightcurves Level: Intermediate Description: Use the DP0.2 catalogs to identify variable stars and plot their lightcurves. Skills: Use various TAP tables, including joining multiple tables. Extract time-series photometry. Measure periods and plot phased lightcurves. Data Products: Object, ForcedSource, CcdVisit, DiaObject, DiaSource tables Packages: numpy, matplotlib, astropy.units, astropy.coordinates, astropy.io.fits, astropy.timeseries.LombScargle, lsst.rsp.get_tap_service |
| 08. Truth Tables Level: Beginner Description: An introduction to using the truth data for the Dark Energy Science Collaboration's DC2 data set. Skills: Use the TAP service with table joins to retrieve truth data matched to the Object catalog. Data Products: TAP dp02_dc2_catalogs.Object, .MatchesTruth, and .TruthSummary tables Packages: lsst.rsp.get_tap_service, lsst.rsp.retrieve_query |
| 09a. Custom Coadd Level: Advanced Description: Create a custom "deepCoadd" using only a subset of the input visits. Skills: Use of pipetasks for image coaddition. Creating and writing to Butler collections. Properties of deepCoadds. Data Products: visitTable, deepCoadd images Packages: lsst.daf.butler, lsst.ctrl.mpexec, lsst.pipe.base |
| 09b. Custom Coadd Sources Level: Advanced Description: Detect and measure sources in a custom "deepCoadd" image. Skills: Use the Butler on user-generated collections. Run source detection and measurement. Data Products: user-generated deepCoadd; DP0.2 deepCoadd image and Object table Packages: lsst.afw, lsst.pipe, lsst.meas |
| 10. Deblender Data Products Level: Beginner Description: Use the outputs of the multiband deblender to explore the footprints of parent and child objects. Skills: Use of the catalog data products related to deblending objects. Data Products: TAP dp02_dc2_catalogs.Object, Butler objectTable and deepCoadd_deblendedFlux table Packages: lsst.afw.image, lsst.afw.detection, lsst.rsp, lsst.daf.butler, lsst.geom |
| 11. Working with User Packages Level: Beginner Description: An example of how to install and set up user packages. Skills: Installing sofware, building libraries, and modifying paths. Data Products: N/A Packages: os, bagpipes, PyMultiNest, MultiNest, PyCuba, Cuba |
| 12a. Point Spread Function Data Products Level: Intermediate Description: A demonstration of how to access calexp and deepCoadd PSF properties. Skills: Use of single-epoch and coadded PSF models. Data Products: DP0.2 calexp and deepCoadd images. Packages: lsst.daf.butler, lsst.geom, lsst.afw.display |
| 12b. Point Spread Function Science Demo Level: Advanced Description: Demonstration of the use of measured PSF properties in weak lensing analysis. Skills: Use of the catalog data products for PSF analysis. Data Products: DP0.2 Object and Source catalogs. Packages: lsst.analysis.tools.atools |
| 13a. Using The Image Cutout Tool With DP0.2 Level: Beginner Description: This notebook demonstrates how to use the Rubin Image Cutout Service. Skills: Run the Rubin Image Cutout Service for visual inspection of small cutouts of LSST images. Data Products: Images (deepCoadd, calexp), catalogs (objectTable, diaObject, truthTables, ivoa.ObsCore). Packages: PyVO, lsst.rsp.get_tap_service, lsst.pipe.tasks.registerImage, lsst.afw.display |
| 14. Injecting Synthetic Sources Into Single-Visit Images Level: Advanced Description: This tutorial demonstrates a method to inject artificial sources (stars and galaxies) into calexp images using the measured point-spread function of the given calexp image. Confirmation that the synthetic sources were correctly injected into the image is done by running a difference imaging task from the pipelines. Skills: Use the source_injection tools to inject synthetic sources into images. Create a difference image from a calexp with injected sources. Data Products: Butler calexp images and corresponding src catalogs, goodSeeingDiff_templateExp images, and injection_catalogs. Packages: lsst.source.injection |
| 15. Survey Property Maps Level: Intermediate Description: Use the tools to visualize full-area survey property maps. Skills: Load and visualize survey property maps using healsparse and skyproj. Data Products: Survey property maps. Packages: healsparse, skyproj, lsst.daf.butler |
| Skills in DP0.3 Tutorial Notebooks |
|---|
| 01. Introduction to DP0.3 Level: Beginner Description: An overview of the contents of the DP0.3 moving object catalogs. Skills: Use the TAP service and ADQL to access the DP0.3 tables. Data Products: TAP dp03_catalogs Packages: lsst.rsp.get_tap_service |
| 02. Properties of Main Belt Asteroids in DP0.3 Level: Beginner Description: An exploration of the orbital properties of Main Belt asteroids in the DP0.3 catalogs. Skills: Use the TAP service and ADQL to access the DP0.3 tables. Join information from multiple DP0.3 tables. Plot orbits of Solar System objects. Data Products: TAP dp03_catalogs Packages: lsst.rsp.get_tap_service |
| 03. Trans-Neptunian Objects (TNOs) Level: intermediate Description: Explore the trans-Neptunian object populations in DP0.3. Skills: Use of the DP0.3 catalogs to study TNO populations. Data Products: DP0.3 catalogs SSObject, DiaSource, and MPCORB (10-year catalogs). Packages: lsst.rsp |
| 04a. Introduction to Phase Curves Level: Intermediate Description: Explore phase curves for DP0.3 solar system objects. Skills: Use the TAP service and ADQL to access the DP0.3 tables. Join information from multiple DP0.3 tables. Plot phase curves. Data Products: TAP dp03_catalogs Packages: lsst.rsp.get_tap_service |
| 04b. Advanced Phase Curve Modeling Level: Advanced Description: Explicitly investigate the derivation of phase curves for Main Belt asteroids. Skills: Use the TAP service and ADQL to access the DP0.3 tables. Join information from multiple DP0.3 tables. Derive phase curves using three different models. Data Products: TAP dp03_catalogs with added columns Packages: lsst.rsp.get_tap_service |
| 05. Orbital properties of Near-Earth Objects (NEOs) in DP0.3 Level: Beginner Description: Exploration of the orbital properties of near-Earth objects in the DP0.3 catalogs. Skills: Use TAP queries to retrieve Solar System objects. Plot propoerties and orbits of near-Earth objects. Data Products: TAP DP0.3 MPCORB (10-year) and SSObject tables Packages: lsst.rsp.get_tap_service |
| 06. Using User-supplied Catalogs Level: Beginner Description: Use the TAP upload functionality for user-supplied tables and join them with DP0.3 catalogs. Skills: Use the TAP service to upload a table and join it to an LSST table with ADQL. Data Products: DP0.3 catalogs SSObject, DiaSource, and MPCORB (10-year catalogs). Packages: lsst.rsp.get_tap_service |
| 07. Interactive Catalog Visualization Level: Intermediate Description: Interactive Solar System catalog data visualizations with three open-source python libraries. Skills: Create linked interactive plots for large datasets, and output interactive plots to interactive HTML files. Use Bokeh, HoloViews, and Datashader. Data Products: Solar System Object catalogs Packages: bokeh, holoviews, datashader |
DP0.1 was deprecated and is no longer available.
Although they cannot be executed, the tutorial notebooks can still be found in this archived repository: tutorial-notebooks-dp01-archive.