From 33cfc71a3f1d4afbc0a6077627bc2504d89f93d6 Mon Sep 17 00:00:00 2001 From: "Douglas L. Tucker" Date: Mon, 23 Oct 2023 20:37:03 -0500 Subject: [PATCH] Update api-intro.rst Improved description of both the TOPCAT section and the pyvo section. --- data-access-analysis-tools/api-intro.rst | 85 ++++++++++++++++-------- 1 file changed, 59 insertions(+), 26 deletions(-) diff --git a/data-access-analysis-tools/api-intro.rst b/data-access-analysis-tools/api-intro.rst index a59217d5..b4a9e036 100644 --- a/data-access-analysis-tools/api-intro.rst +++ b/data-access-analysis-tools/api-intro.rst @@ -27,13 +27,11 @@ On the the main landing page at `data.lsst.cloud `_ the Check back soon for new information! -** Explain more about tokens and need to keep 'em safe!** - -**(Is this true? I don't know anything about the ObsTAP, SODA, or HiPS services.)** The API's services for DP0.2 include `TAP `_, `ObsTAP `_, `SODA `_ (image cutouts and mosaics), and `HiPS `_. -Currently, CST support for the RSP API services beyond TAP is limited, but more support will be available later. +Currently, CST support for the RSP API services beyond TAP is limited, but more support is becoming available soon. +For the time being, this page will focus on the API's TAP services. Longer term, Rubin Observatory will support `SCS `_ for simple catalog searches, `SIAv2 `_ for image searches, and `VOSpace `_ @@ -50,7 +48,7 @@ The primary means of accessing TAP is via the RSP's Portal and Notebook aspects. Use of the TAP service to query catalogs via the Portal is described in :doc:`/data-access-analysis-tools/portal-intro`. In the Notebook Aspect, a TAP service is instantiated in a python notebook and used to execute an `ADQL query `_ and return a result set. -A set of utilities are provided to get a TAP service instance. +Within the RSP's Notebook Aspect, a set of utilities are provided to get a TAP service instance. .. code-block:: python @@ -69,11 +67,18 @@ Several of the DP0 :ref:`DP0-2-Tutorials-Notebooks` demonstrate how to use the T Use of TOPCAT with the RSP TAP service ====================================== -One popular and useful non-RSP TAP utility is `TOPCAT `_. +One popular and useful non-RSP utility that supports the TAP API is +`TOPCAT `_. To access DP0.2 from a non-RSP TAP utility, one needs to generate an RSP access token. -How to generate and use an RSP access token is described by the `Rubin Science Platform APIs `_ webpage and +How to generate and use an RSP access token is described by the +`Rubin Science Platform APIs `_ webpage and by the `Science Platform Tokens `_ webpage. +**We note that tokens should be treated like passwords: they should not be shared with +others, and -- outside the RSP environment, where this is taken care of by default -- +the user should take special steps to keep tokens secure. This is a topic that is +described further in the material below.** + A TOPCAT-based Step-by-Step Guide --------------------------------- @@ -81,9 +86,9 @@ A TOPCAT-based Step-by-Step Guide 0. Create an RSP access token as described above. (See the `Creating user tokens webpage `_ for a step-by-step guide for creating an RSP access token. It is recommended that the token you create has the - following propoerties: a name that includes "TOPCAT" as a substring, a scope of `read:tap`, + following propoerties: a name that includes "TOPCAT" as a substring, a scope of ``read:tap``, and no expiration date.) It is highly recommended to cut-and-paste the token somewhere - for future reference. + secure for future reference. 1. Start up TOPCAT (see `TOPCAT homepage `_). @@ -99,7 +104,7 @@ A TOPCAT-based Step-by-Step Guide :name: API_TOPCAT_DLT_2 :alt: TBD -4. Fill in your security token under “User” in the Authentication window that pops up. Leave the “Password” blank. Click OK. +4. In the Authentication window that pops up, fill in `x-oauth-basic` for the "User" and your security token forthe "Password". Click "OK". .. figure:: /_static/API_TOPCAT_DLT_3.png :name: API_TOPCAT_DLT_3 @@ -111,14 +116,33 @@ A TOPCAT-based Step-by-Step Guide :name: API_TOPCAT_DLT_4 :alt: TBD +6. Now you can explore the Rubin DP0.2 data set via TOPCAT. +TOPCAT allows you to run ADQL queries, explore tables, and +make a variety of 2D and 3D plots via an interactive graphical +users interface (GUI). For an example, see this tutorial **TBD**. +For a broader view of TOPCAT capabilities, please see the +`TOPCAT webpage `_, +which includes plentiful documentation, many examples, and +various tutorials. + +.. figure:: /_static/API_TOPCAT_DLT_5.png + :name: API_TOPCAT_DLT_5 + :alt: TBD .. _Data-Access-Analysis-Tools-TAP-NB-NOIRLAB: -Use of the NOIRlab Data Lab with the RSP TAP service -==================================================== +Use of ``pyvo`` with the RSP TAP service +======================================== + +Another way to access the Rubin data from outside the RSP environment is via the +`pyvo `_ python module, an affiliated +package for `astropy `_. By this method, if ``pyvo`` +is installed, one can access the RSP TAP service directly from one's own laptop. +If not, one access the RSP TAP service from other freely accessible services +that have ``pyvo`` pre-installed (like, e.g., NOIRLab's +`Astro Data Lab `_ Jupyter Notebook server). + -One may wish to access the rubin data from outside the RSP environment. -One way to do this is via the ``pyvo`` python module. As with the TOPCAT example above, to do this one needs to generate an RSP access token. For simplicity, the token generated above for the TOPCAT example can be used here as well. Here, as an example, we make use of the NOIRlab Astro Data Lab. @@ -126,29 +150,38 @@ Here, as an example, we make use of the NOIRlab Astro Data Lab. ** See K.-T. Lim's reply and Michael Wood-Vasey's ``test_rsp_tap_service.py`` from https://community.lsst.org/t/will-there-be-external-tap-access-to-rsp-dp0-2-tables/6660/7 ** -A NOIRlab Astro Data Lab-based Step-by-Step Guide -------------------------------------------------- +A ``pyvo``-based Step-by-Step Guide +----------------------------------- -1. Go to the NOIRLAB DataLab ( https://datalab.noirlab.edu/ ) and launch a Jupyter Notebook. +0. As with the TOPCAT example above, one needs an RSP access token. +Either generate one as described above in :ref:`Data-Access-Analysis-Tools-TAP-NB`, +or just use a previously generated (but unexpired) RSP access token. +Ideally, copy the RSP access token in a file that is only read/write +by the file owner and that is accessible to the python session that +will be accessed in the Step 1 below. For example, in UNIX/Linux:: + + emacs /Users//.rsp-tap.token # Copy RSP token into this file + chmod 600 /Users//.rsp-tap.token # Make .rsp-tap.token read/write to only the file owner + +1. Start up a python session. This could be a standalone python session +running on (say) a laptop, or a Jupyter notebook running elsewhere but +displayed on a one's own browser. 2. At the very minimum, import the ``pyvo`` python module:: import pyvo -3. Define the data.lsst.cloud TAP server URL and your security token:: +3. Define the data.lsst.cloud TAP server URL and read in your security token:: + + RSP_TAP_SERVICE = 'https://data.lsst.cloud/api/tap' + token_file = '/Users//.rsp-tap.token' + with open(token_file, 'r') as f: + token_str = f.readline() - RSP_TAP_SERVICE = "https://data.lsst.cloud/api/tap" - #tap_url = 'https://data.lsst.cloud/api/tap' - # having the token in your code is poor security. - # See Michael Wood-Vasey's solution below. - #token = 'yy-xxxxxx' # insert your own RSP TAP server security token here. 4. Set up appropriate authorization to access the RSP TAP server:: cred = pyvo.auth.CredentialStore() - token_file = "read_tap.token" - with open(token_file, "r") as f: - token_str = f.readline() cred.set_password("x-oauth-basic", token_str) credential = cred.get("ivo://ivoa.net/sso#BasicAA") rsp_tap = pyvo.dal.TAPService(RSP_TAP_SERVICE, credential)