Skip to content

Commit

Permalink
Simonyi LOTO Procedures
Browse files Browse the repository at this point in the history
  • Loading branch information
Kshitija authored and isotuela committed Aug 29, 2024
1 parent 34022b8 commit ad3037d
Show file tree
Hide file tree
Showing 14 changed files with 141 additions and 30 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ from the *LOVE/MTQueue* or the MTRotator EUI.
.. warning::

This check cannot be accomplished if MTRotator is in ``FAULT`` on the CSC and/or with interlocks
activated on the :guilabel:`GIS` or the MTRotator EUI. Please refer to `MTRotator Recovery
Procedure <https://obs-ops.lsst.io/Simonyi/Troubleshooting/MTCS/HexRot/MTRotator-Recovery/MTRotator-Recovery.html>`_
activated on the :guilabel:`GIS` or the MTRotator EUI. Please refer to :ref:`MTRotator Recovery
Procedure <MTRotator-Recovery>`
to clear these warnings before proceeding.


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,8 +217,8 @@ Hard Reboot

When the MTRotator is in the ``OFFLINE Publish only`` state in the :guilabel:`runRotEui` you need to go to the
:guilabel:`Offline Substate Commands` and then send the :guilabel:`System Ready` command. The
DDS can then be enabled from ``OFFLINE`` state. Refer to `MTRotator Recovery
procedure <https://obs-ops.lsst.io/Simonyi/Troubleshooting/MTCS/HexRot/MTRotator-Recovery/MTRotator-Recovery.html>`_ for more detials.
DDS can then be enabled from ``OFFLINE`` state. Refer to :ref:`MTRotator Recovery
procedure <MTRotator-Recovery>` for more details.



Expand Down
162 changes: 136 additions & 26 deletions Simonyi/Non-Standard-Operations/MTCS/TMA/Simonyi-LOTO-procedure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,22 @@
- If a section within the template is not needed, comment out the section title and label reference. Include a comment explaining why this is not required.
- If a file cannot include a title (surrounded by ampersands (#)), comment out the title from the template and include a comment explaining why this is implemented (in addition to applying the ``title`` directive).
.. raw:: html

<style> .red {color:red} </style>
.. role:: red


.. raw:: html

<style> .gold {color:gold} </style>
.. role:: gold

.. Include one Primary Author and list of Contributors (comma separated) between the asterisks (*):
.. |author| replace:: *replace*
.. |author| replace:: Karla Aubel
.. If there are no contributors, write "none" between the asterisks. Do not remove the substitution.
.. |contributors| replace:: *Safety Team*
.. |contributors| replace:: Kshitija Kelkar, *safety team*

.. This is the label that can be used as for cross referencing this procedure.
.. Recommended format is "Directory Name"-"Title Name" -- Spaces should be replaced by hyphens.
Expand All @@ -20,8 +32,7 @@
.. To reference a label that isn't associated with an reST object such as a title or figure, you must include the link an explicit title using the syntax :ref:`link text <label-name>`.
.. An error will alert you of identical labels during the build process.
.. warning::
DRAFT
######################
Simonyi LOTO Procedure
Expand All @@ -34,9 +45,12 @@ Overview

.. This section should provide a brief, top-level description of the procedure's purpose and utilization. Consider including the expected user and when the procedure will be performed.
AuxTel must be under LOTO (lock-out tag-out) any time physical work is undertaken on the mount or the dome.
When working with environmental sensors, on the truss, mirrors, or anywhere in the dome where unexpected movement could cause a safety hazard, the telescope must be locked out.
Trained summit support staff may use this procedure if they meet all of the preconiditon requirements.
This document describes how to prevent any telescope movement as a safety measure through **Lock-Out-Tag-Out** (**LOTO**)
procedure. To perform this procedure, LOTO training must have been received. Observers during operations may be
expected to LOTO the **Telescope Main Assembly** (**TMA**) due to unforeseen work done at night. The TMA LOTO is done directly
in the **Oil Supply System** (**OSS**).



.. _Simonyi-LOTO-procedure-Precondition:

Expand All @@ -47,19 +61,10 @@ Precondition
.. It is preferred to include them as a bulleted or enumerated list.
.. If there is a different procedure that is critical before execution, carefully consider if it should be linked within this section or as part of the Procedure section below (or both).
.. warning::

.. _Simonyi-LOTO-procedure-Post-Condition:

Post-Condition
==============

.. This section should provide a simple overview of conditions or results after executing the procedure; for example, state of equipment or resulting data products.
.. It is preferred to include them as a bulleted or enumerated list.
.. Please provide screenshots of the software status or relevant display windows to confirm.
.. Do not include actions in this section. Any action by the user should be included in the end of the Procedure section below. For example: Do not include "Verify the telescope azimuth is 0 degrees with the appropriate command." Instead, include this statement as the final step of the procedure, and include "Telescope is at 0 degrees." in the Post-condition section.
The drives are powered off and the AuxTel Control cabinet is locked from the outside.
The Simonyi Telescope is under LOTO and is not able to move by any controls.
To perform this procedure formal LOTO training must have been received; refer also to
the `TMA LOTO Procedure video. <https://drive.google.com/file/d/1rOvu1ITDCm0HNepfvoBWMzhGZkMPkKnH/view>`_

.. _Simonyi-LOTO-procedure-Procedure-Steps:

Expand All @@ -71,19 +76,124 @@ Procedure Steps
.. For highly complicated procedures, consider breaking them into separate procedure. Some options are a high-level procedure with links, separating into smaller procedures or utilizing the reST ``include`` directive <https://docutils.sourceforge.io/docs/ref/rst/directives.html#include>.
Applying LOTO on OSS
--------------------

.. _Simonyi-LOTO-procedure-Troubleshooting:
.. note::

Troubleshooting
===============
**LOTO box**

Keys and locks are inside the LOTO Box. You can find LOTO box (with a tag "Electronic team" on the box) on
Level 1 in the OSS room. If you cannot find or open that LOTO box, you can take a LOTO box from the meeting room
on Level 2.

.. image:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-7.png
:width: 30%
.. image:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-8.jpeg
:width: 30%






#. **Announcement**: Before/right after applying LOTO on the OSS, announce "LOTO on the OSS" on *#summit-announce* and/or *#summit-simonyi*.

#. **Turning off the OSS**: Turn :guilabel:`OFF` the OSS in the EUI panel (indicated by the :gold:`circle` in the following Figure)

.. figure:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-1.png
:name: tma-loto-1

Figure 1: The :guilabel:`OSS GENERAL VIEW` in the TMA EUI can be accessed by entering the :guilabel:`MONITOR & CONTROL` > :guilabel:`MAIN AXES` tab in the left vertical menu panel of the EUI.

#. **Wait until the OSS is off**: the :guilabel:`status` turns `Idle` and :guilabel:`OSS` flag goes from green to gray.
It takes around 5-10 minutes to complete turning off the OSS.

.. figure:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-2.png
:name: tma-loto-2

Figure 2: The OSS :guilabel:`status` indicator (Zoom view of the top left sdection of the :guilabel:`OSS GENERAL VIEW` shown in Figure 1)



#. **Applying LOTO on the OSS Pump**: Go to the OSS room from Level 1 (first floor). Apply LOTOs on the OSS pump switches (highlighted by the :red:`box` in
the following Figure). Close valves (Position "—-") and put locks on the holes to apply three LOTOs on the safety
switches of the OSS pumps.

.. figure:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-3.png
:name: tma-loto-3

Figure 3: Location of the OSS Pump Switches on Level 1
.. image:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-4.png
:width: 30%
.. image:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-5.png
:width: 30%
.. image:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-6.png
:width: 30%

Figure 4: OSS Pump Valves in closed position and locked.

Unlocking LOTO on OSS
---------------------

#. Go to the the OSS room and remove the locks from the pump switches. Find the key for each lock from the LOTO box
and match the number on the key and the lock.

#. Put the LOTO locks with the keys back into the LOTO box.

#. Open the switches (Position "|") on the pump (See the Figure 5 below).
.. figure:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-9.png
:name: tma-loto-9

Figure 5

#. Turn on the OSS from TMA EUI (**Optional**).

#. Announce the status of the OSS on *#summit-announce* and/or *#summit-simonyi*.

.. note::

**LOTO at night**

During night-time operations, only **ONE** pump needs to be locked
to consider the TMA fully under LOTO. This helps to move the process faster by taking into consideration the
reduced on-summit personnel (~ 2 persons operation at night) and mitigating situations like when one cannot find the LOTO boxes, risk of losing
multiple locks + keys, leaving lights on longer etc.




.. _Simonyi-LOTO-procedure-Post-Condition:

Post-Condition
==============

.. This section should provide a simple overview of conditions or results after executing the procedure; for example, state of equipment or resulting data products.
.. It is preferred to include them as a bulleted or enumerated list.
.. Please provide screenshots of the software status or relevant display windows to confirm.
.. Do not include actions in this section. Any action by the user should be included in the end of the Procedure section below. For example: Do not include "Verify the telescope azimuth is 0 degrees with the appropriate command." Instead, include this statement as the final step of the procedure, and include "Telescope is at 0 degrees." in the Post-condition section.
#. **After applying LOTO on OSS**
Telescope is unable to move. OSS is under LOTO shows :guilabel:`status` as `Idle` and the three pumps in :red:`red`, down to the left in the image.

.. figure:: /Simonyi/Non-Standard-Operations/MTCS/TMA/_static/tma-loto-10.png
:name: tma-loto-10

Figure 6: The :guilabel:`OSS GENERAL VIEW` when OSS is under LOTO

#. **After unlocking LOTO on OSS**

.. This section should include troubleshooting information. Information in this section should be strictly related to this procedure.
- OSS idle but not turned on (:red:`Need Figure`)

.. If there is no content for this section, remove the indentation on the following line instead of deleting this sub-section.
- OSS turned on after removing LOTO (:red:`Need Figure`)

No troubleshooting information is applicable to this procedure.
In the IMPOSSIBLE event that the telescope moves after turning off all three breakers, stop work immediately and notify the day crew, including Jacques Sebag, Mario Rivera, and the electronics team.
.. _Simonyi-LOTO-procedure-Contingency:

Contingency
===========

Announce the status of locking and unlocking the LOTO in *#summit-simonyi* and/or *#summit-announce* and/or activate
the `Out of hours support <https://obs-ops.lsst.io/Safety/out-of-hours-support.html#safety-out-of-hours-support>`_.

This procedure was last modified |today|.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions documenteer.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ ignore = [
'https://rucio.cern.ch/',
'https://confluence.slac.stanford.edu/',
'https://tea-pdu01.cp.lsst.org/',
'https://1password.com/downloads'
]

[sphinx]
Expand Down

0 comments on commit ad3037d

Please sign in to comment.