SLEAPyExample.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7e3285a8",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "SLEAPyFaces Example\n",
    "==================="
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63d38a16",
   "metadata": {},
   "source": [
    "This notebook is an example of how to use the SLEAPyFaces package to analyze SLEAP data. It is intended to be run in a Jupyter notebook, but can be run in a Python script as well. The notebook is available on GitHub at [annie444/SLEAPyFaces](https://github.com/annie444/SLEAPyFaces)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2e18a91",
   "metadata": {},
   "source": [
    "There are three main base types in SLEAPyFaces: \n",
    "1. `Experiment` - a single experiment, which contains a single SLEAP file, a single video file, and a single events file. \n",
    "2. `Project` - a collection of experiments, which can be used to compare multiple experiments.\n",
    "3. `Projects` - a collection of projects, which can be used to compare multiple projects. (and their inherited experiments)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c20a81d1",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "# Setup "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d000b91",
   "metadata": {},
   "source": [
    "For a single `Experiment` analysis, you will need to have the following files in the same directory: \n",
    "1. A SLEAP file (`.h5`), which contains the SLEAP tracking data. \n",
    "2. A video file (`.mp4`), which contains the video data. \n",
    "3. An events file (`.csv`), which contains the events data. \n",
    "4. An experiment setup file (`.json`), which contains the experiment setup/metadata. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "ename": "",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31mJupyter cannot be started. Error attempting to locate Jupyter: Select an Interpreter to start Jupyter\n",
      "\u001b[1;31mRun the following command to install 'jupyter and notebook' into the Python environment. \n",
      "\u001b[1;31mCommand: 'python -m pip install jupyter notebook -U\n",
      "\u001b[1;31mor\n",
      "\u001b[1;31mconda install jupyter notebook -U'\n",
      "\u001b[1;31mClick <a href='https://aka.ms/installJupyterForVSCode'>here</a> for more info."
     ]
    }
   ],
   "source": [
    "from sleapyfaces.base import Experiment\n",
    "\n",
    "expr = Experiment(\n",
    "    name=\"SLEAPyExampleExperiment\", # first we give our experiment a name \n",
    "    base=\"/Volumes/specialk_cs/2p/raw/CSE011/20211105/\", # then we assign the base directory, in this case it is on `specialk_cs/2p/raw/CSE011/20211105`\n",
    "    ExperimentEventsFile=(\"*_events.csv\", True), # then we assign the events file, which is a csv file\n",
    "    ExperimentSetupFile=(\"*.json\", True), # then we assign the experiment setup file, which is a json file \n",
    "    SLEAPFile=(\"*.h5\", True), # then we assign the SLEAP file, which is a h5 file \n",
    "    VideoFile=(\"*.mp4\", True), # then we assign the video file, which is a mp4 file \n",
    ") \n",
    "\n",
    "# For any of the above files, you can also just pass the filename (not in parentases). \n",
    "# However, byt passing a tuple with the 'naming convention' and a boolean, you can use wildcards `*` to find the file based on a consistent naming scheme. \n",
    "# This is less important for individual experiments, but is very useful for `Project` and `Projects` analysis. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64a374e6",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "For a single `Project` analysis, you will need to have multiple folders containing all the necessary `Experiment` files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b8e23f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sleapyfaces.base import Project\n",
    "\n",
    "proj = Project(\n",
    "    ExperimentEventsFile=(\"*_events.csv\", True), # we assign the events file (which is a csv file)\n",
    "    ExperimentSetupFile=(\"*.json\", True), # we assign the experiment setup file (which is a json file)\n",
    "    SLEAPFile=(\"*.h5\", True), # we assign the SLEAP file (which is a h5 file)\n",
    "    VideoFile=(\"*.mp4\", True), # we assign the video file (which is a mp4 file)\n",
    "    base=\"/Volumes/specialk_cs/2p/raw/CSE011/\", # we assign the base directory, in this case it is on `specialk_cs/2p/raw/CSE011` \n",
    "    # NOTE THIS IS DIFFERENT FROM THE ABOVE EXAMPLE\n",
    "    name=\"SLEAPyExampleProject\", # we give our project a name\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0ff9833",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "For a single `Projects` analysis, you will need to have multiple folders containing all the necessary `Project` files. \n",
    "However, the `Projects` class will automatically search the direct subdirectories for `Experiment` files.\n",
    "This results in an `Experiment` for each subdirectory. If the subdirectoy names contain numbers, they will also automatically be sorted and labeled in ascending order. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba1c945c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sleapyfaces import Projects\n",
    "\n",
    "projs = Projects(\n",
    "    ExperimentEventsFile=(\"*_events.csv\", True), # we assign the events file (which is a csv file)\n",
    "    ExperimentSetupFile=(\"*.json\", True), # we assign the experiment setup file (which is a json file)\n",
    "    SLEAPFile=(\"*.h5\", True), # we assign the SLEAP file (which is a h5 file)\n",
    "    VideoFile=(\"*.mp4\", True), # we assign the video file (which is a mp4 file)\n",
    "    base=\"/Volumes/specialk_cs/2p/raw/\", # we assign the base directory, in this case it is on `specialk_cs/2p/raw`\n",
    "    file_structure={\n",
    "        \"CSE011\": \"CSE011\",\n",
    "        \"CSE014\": \"CSE014\",\n",
    "        \"CSE016\": \"CSE016\",\n",
    "        \"CSC011\": \"CSC011\",\n",
    "        \"CSE020\": \"CSE020\",\n",
    "    }) # we assign the file structure, which is a dictionary of the subdirectories and their names"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e7d59a1",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "# Data Manipulation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b208a6a0",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "> Moving forward I am going to use all three objects to show how they can be used in different scenarios. \n",
    "\n",
    "So far we've only imported all of the data and stored them in pandas dataframes. Now we have to do some data manipulation to get the data into the correct format for analysis. \n",
    "The first thing in this process is adding the correct annotation columns to the data and initializing it all together. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3817c641",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "expr.buildColumns() # `.buildColumns()` is a method on all base obejcets, and it can be empty. There is no need to add annotations if there aren't any to add, but this is still a necessary method to call as it initializes the data into a main dataframe.\n",
    "\n",
    "proj.buildColumns([\"Mouse_Type\"], [\"Resilient\"]) # If you choose to annotate the data, the `.buildColumns()` method takes a list of column names and a list of their corresponding values. The values will be added to the column names for each experiment in the project and can be referenced as needed. \n",
    "\n",
    "# for the `Projects` object we're wanting to label different projects with a different `Mouse_Type`\n",
    "for name, project in projs.data.items(): # so we can loop through all of the project data items, which give the name (from the `file_structure`) and the `Project` object for the corresponding project \n",
    "    if name in [\"CSE011\", \"CSE014\", \"CSE016\"]: # This allows us to run a fast comparison of the project names to the names we want to label \n",
    "        project.buildColumns([\"Mouse_Type\"], [\"Resilient\"]) # and then we can build the columns for each project, accessing the `Project` object directly only once per project \n",
    "    elif name in [\"CSE020\"]:\n",
    "        project.buildColumns([\"Mouse_Type\"], [\"Susceptible\"])\n",
    "    else:\n",
    "        project.buildColumns([\"Mouse_Type\"], [\"Control\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad6465b8",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "> Now that we have build our data, it's now time to align the video data with the event data and the setup data to extract the idividual trials. \n",
    "> While aligning timeseries data is a computationally expensive operation, it is necessary to get the data into the correct format for analysis. \n",
    "> In lieu of this, the following command utilized `multiprocessing` which is known to be tempermental, so don't mess with the code when executing the following lines. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4436906",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "From here on out I'm just going to use the `Experiemnt` object, as the next steps are the same across all objects. This is to minimize the amount of time lost to computations in this example. \n",
    "However, all of the information here in can be extrapolated to the `Project` and `Projects` objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4dc11f66",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# In order to build the trials, we have to pass in two lists. \n",
    "\n",
    "# The first list is the column names from the `ExperimentEventsFile` that dictate the start of a trial from the events (or DAQ) data. \n",
    "cols = [\"Speaker_on\", \"LED_on\"]\n",
    "\n",
    "# The second is a list of boolean values that let the code know if there are multiple events for each column in the trial period. \n",
    "# e.g. For Austin's data, the LED will blink repeatedly during a trial, but we don't want to initialize each blink as a new trial, so we set that list to True for trial reduction. \n",
    "# Conversely, the speaker will only turn on once per trial, so we set that list to False for trial reduction. \n",
    "reduce = [False, True]\n",
    "\n",
    "# Now we can build the trials. \n",
    "expr.buildTrials(cols, reduce)\n",
    "\n",
    "# NOTE: These lists can be passed directly, however, for the sake of this example they were seperated out. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e92f654",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "# Normalization "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d35fcaf4",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "> Now that we have our data in the correct format, we can start to analyzing it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8aafdcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "expr.normalize() # This will normalize the data for each trial. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "318f61b3",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "# Visualizing the Data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d3ecb91",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "expr.visualize(dimensions=3, normalized=True, filename=\".ignore/3DExample.html\") # This will visualize the data in 3D and output the visualization as an interactive plot. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7d49083",
   "metadata": {
    "lines_to_next_cell": 2
   },
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "sleapyfaces",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0 | packaged by conda-forge | (main, Jan 14 2023, 12:26:40) [Clang 14.0.6 ]"
  },
  "vscode": {
   "interpreter": {
    "hash": "b0ea88426352522f750fe5516728f15656e2289bf25c218d2f74ad9754db936f"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}