Skip to content

Commit

Permalink
Merge pull request #56 from RENCI-NRIG/xs100-edits
Browse files Browse the repository at this point in the history
Xs100 edits
  • Loading branch information
kenygren authored Jun 5, 2024
2 parents b823c05 + a013140 commit 9412b6b
Showing 1 changed file with 12 additions and 11 deletions.
23 changes: 12 additions & 11 deletions theme4/XS100/data-collection.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Most experiments start well in advance of the awarded beamtime - and so does the

While CHESS provides state-of-the-art hardware, software, computing resources and trainings, **users are responsible for the integrity of their experiment** through thoughtful planning, experimental execution, and **data handling and analysis**. This includes maintaining best practices in experimental logs, metadata tracking, and recording of researcher decisions. Due to the nature of synchrotron experiments, the data integrity and intepretabilty - even within one research group - will be dependent on the practices adpoted by the research group.

### Beamtime Notes and Experimental Logbook:
## Beamtime Notes and Experimental Logbook:

It is always the responsibility of the experimenter to take **detailed** beamtime notes and a log of the data. Although work is ongoing to integrate metadata and capture requisite information in the data itself, automate workflows, visualizations, etc., it is imperative that experimenter notes are taken and ideally a copy is kept with the data on the CHESS system.

Expand All @@ -40,7 +40,7 @@ Whenever possible, we encourage users to leverage the metadata services, beamlin

Comment any code produced at the beamtime. *If this code was used to make decisions about the experiment, it should be saved and referred to in your experimental log.*

### Station Computer : Beamline Control Central
## Station Computer : Beamline Control Central

Every experimental station has a **station computer** that acts as controls central. The **station computer** typically runs a number of processes and is responsible for orchestrating data collection, motor motions, synchronized triggers, metadata logging, and more.

Expand All @@ -54,19 +54,19 @@ Every experimental station has a **station computer** that acts as controls cent

- The station computer has many special permissions, for instance it is able to *write* to the the CHESS DAQ (**raw** directory). When saving files such as beamtime notes, it is important to save these in the directories prescribed by your beamline scientist. (link to later section on CHESS file system and directories)

<img src="./xs100-figures/AnnotatedStationComputer.png" alt="annotatedstationcomputer" width="1000"/>
<img src="./xs100-figures/AnnotatedStationComputer.png" alt="annotatedstationcomputer" width="700"/>

This is an example of a station computer screen shot with many processes. There are 4 desktops on the station computer, each with windows spanning 4 screens. This image is of the first desktop and shows a main controls terminal (SPEC), controls screens (MEDM screens), a data reduction GUI (HEXRD). There will typically be even more processes running than this.

Every beamline will have a unique version of this computer - some techniques even may be executed exclusively through a GUI.

#### Controls Hardware, Software, and Signal Monitoring
###1 Controls Hardware, Software, and Signal Monitoring

This section will discuss the hardware connections, motor configurations, and overall connectivity of signals being tracked in the beamline.

<img src="./xs100-figures/xcite-overview-controls.png" alt="controls" width="500"/>

#### Controls Software
## Controls Software

There are many controls languages and strategies across the lab. The two most common cases are **SPEC** and **EPICS** which will briefly be introduced here. Python-based controls are also very common.

Expand Down Expand Up @@ -134,7 +134,7 @@ Data Collection, Storage, and Analysis can be customized to comply with data agr
- Securing the experimental station with an entry password
- Disconnecting streaming video to the experimental station

### Detectors and Data Handling
## Data Handling and Analysis
<img src="./xs100-figures/xcite-overview-expstation.png" alt="expstation" width="500"/>

If you wish to move any data from the CHESS filesystem to another location, the preferred way of doing so is through Globus. Please see here (https://wiki.classe.cornell.edu/Computing/GlobusDataTransfer) for directions on ways to transfer data from the CHESS filesystem.
Expand All @@ -149,7 +149,7 @@ The data is still saved, but to transfer or perform analysis on the files you wi
<img src="./xs100-figures/xcite-overview-datastorage.png" alt="datastorage" width="500"/>


#### Bring Your Own Device (BYOD)
### Bring Your Own Device (BYOD)

Users may need to bring their own devices to be beamline - either physically in the lab or remotely connected to the CHESS networks

Expand All @@ -163,7 +163,7 @@ Because the CHESS-DAQ filesystems are a critical resource for data collection, *

*Read access* to the CHESS-DAQ filesystem may be obtained by registering your device for the LNS Protected network using [this request form](https://wiki.classe.cornell.edu/Computing/LaptopRegistration).

#### MetaData Handling
### MetaData Handling

Ideally, all the data necessary to **fully reproduce your results** are recorded and disseminated in a manner that others can interpret after your experiment. Ideally the provenance remains unbroken from experiment planning.

Expand All @@ -174,15 +174,15 @@ The **metadata service** (https://wiki.classe.cornell.edu/bin/viewauth/CHESS/Pri
<img src="./xs100-figures/MetadataService.png" width="700"/>


#### On-the-fly Data Processing & Visualization
### On-the-fly Data Processing & Visualization

Need to be looking at your data as it is coming off for data fidelity. At some beamlines, you will need to interact manually with a GUI to render your images and make sure the data quality is what you are expecting. At some beamlines, the new NSDF (National Data Science Fabric) Dashboard has been deployed for some datatypes which allows interactive visualization of data as it is coming off the beamline. An example can be found here: https://services.nationalsciencedatafabric.org/chess/.

Many beamlines have an initial “data reduction” procedure that reduces the size of the data through compression or on-the-fly analysis to have smaller file sizes that are more manageable to analyze at your home institution.

Some of these processes are software procedures performed on the data after it is written to the raw directory, while other beamlines utilize inline data processing/compression prior to writing their files to the raw directory. Overtime we will build out station specific training on these, for now it is the responsibility of the user to work with their beamline scientist and read any requisite materials they provide.

#### Data Reduction and Analysis
### Data Reduction and Analysis

<img src="./xs100-figures/xcite-overview-computefarm.png" alt="computefarm" width="500"/>

Expand All @@ -209,4 +209,5 @@ CHAP pipelines can be executed from a Linux command line or from the Galaxy scie

**Technique/Beamline Specific Software**


CHESS has many common X-ray software packages available on the system.
Speak with your beamline scientist for the preferred software package for your experiment.

0 comments on commit 9412b6b

Please sign in to comment.