Skip to content
Hrishikesh Paul edited this page Apr 5, 2021 · 11 revisions

Build Status license


Introduction


Anatomical Structures, Cell Types, plus Biomarkers (ASCT+B) Master tables aim to capture the nested part_of structure of anatomical human body organ systems from gross anatomical structure scale to subcellular biomarker scale. Functional tissue units (FTUs) for an organ system should be identified as well as the typology of cells and biomarkers used to uniquely identify cell types within that organ system and FTUs (e.g., gene, protein, proteoforms, lipid or metabolic markers). Ontology terms and unique identifiers are matched to AS, CT, and B wherever possible for semantic search capability within MC-IU products: Registration User Interface (RUI), Exploration User Interface (EUI), ASCT+B Reporter. The tables are authored and reviewed by an international team of anatomists, pathologists, physicians, and other experts.

The CCF ASCT+B Reporter is a visualization tool for displaying anatomical structures, cell types, and biomarker (ASCT+B) authored by domain experts for different human organs. The tables are used to develop a common coordinate framework (CCF) of the healthy human body, see also Hubmap Consortium website. The Reporter includes a partonomy tree that presents relationships between various anatomical structures and substructures, that is combined with their respective cell types and biomarkers via a bimodal network. The reporter also presents an indented list tree for a more traditional look. Along with visualizing, the reporter has a report generator that enlists various meta data for the visualized ASCT table, which is downloadable. There is also an in-house debug logger that lists any issues related to the data provided in the table. The Reporter is also accompanied by a backend server, ASCT+B Data Miner, to retrieve the data from Google Sheets and convert it to a machine readble format.


Snapshots



Production and Development

This is the production or the main version of the Reporter. It can be found here. This will contain the latest official releases and will be updated twice a month with minor patch updates.

The staging version of the Reporter can be found here which is built off the develop branch. This application is mainly for staging, where power users and developers can see the results on a production environment.


Contributing

If you'd like to contribute, follow the steps below,

$ git clone https://github.com/hubmapconsortium/ccf-asct-reporter
$ cd ccf-asct-reporter
$ git checkout -b <new_branch_name>

Commit the changes to the new feature branch and make a pull request!


License

MIT

About


The ASCT+B Reporter includes a partonomy tree to display the relationships between various anatomical stuctures and their substructures - which is clumped with a bimodal network that depicts the relationships between these anatomical structures, their cell types and associated biomarkers. The interactive visualization empowers you to click on a particulat node, and single out their relationships to closely view how each entity is related. The nodes in the visualization can also be sorted/sized based on their connections to further gain insights on their relationships. Along with the visualization, the Reporter is packed with many useful features like calculating statistics, comparing, searching and even temporarily uploading your own data into the visualization.


Technologies


Architecture

The ASCT+B Reporter is built using Angular 11 accompanied by a powerful state management system called NGXS that helps to maintain a predictable state container that makes it easier to understand how the data flow works in the application. It makes the application modular, and helps in debugging and testing. Below shows all the components of the ASCT+B Reporter and how they're interlinked to create a powerful visualization tool.


Details

Below are a list of ASCT+B tables supported by the Reporter

Data Format


Existing ontologies (Uberon, FMA, CL, HGNC, etc.) have thousands of terms and form a complex knowledge graph. They were designed for different purposes and frequently cover developmental and disease terms. For developing a reference atlas of the healthy human adult body within HuBMAP, we need a unified view of human anatomical structures (AS), cell types (CT), and biomarkers (B). In close collaboration with other consortia, we agreed to focus on part_of relationships so that the network graph can be simplified to a hierarchy. This makes it easier to develop user interfaces that enable investigators to quickly drill down, in an intuitive way, from whole-body, to organ, to organ parts, to cell types, and eventually to specific biomarkers assays associated with those cells. Plus, the ASCT+B tables capture the located_in relationships of CTs and AS and the characterizes relationships between Bs and CTs. The ASCT+B tables are not creating new ontologies. Rather, they help construct a uniform and simplified view of AS, CT, and Bs and their interlinkages relevant for the design of a healthy Human Reference Atlas with cross-walks to multiple existing ontologies that are revised and extended in the process.

The template for the ASCT+B Tables can be found here.


“About Table” Information (first 10 rows)

The first 10 rows describes the following content,

  • Name of the organ
  • Author information name and ORCID (lead author first, co-authors following) on row 3
  • Reviewer name(s) on row 4
  • Date Started
  • Date Last Modified
  • Version Number
  • List of Major Publications
  • Coverage of Table organ

Header Information (11th row)

The data is formatted in a specific way to keep it standardized across all organs and make it easier to convert the data in to a machine readable format. The 11th row is the most important row where each column has to have text in the following pattern(s)

ENTITY/NUMBER/DATA_TYPE

  • ENTITY values: AS, CT, BP, BG, REF (required)
  • NUMBER can be non-negative numbers starting from 1 (required)
  • DATA_TYPE values: ID, LABEL, DOI (optional)

Note
  • For the data to be visualized, the data must conform to this structure.
  • There should be only one item per cell. Multiple items in a single cell separated by commas or any delimiter will be considered and visualized as a single entity.
  • Headers must be on row 11. The positioning of the columns do not matter (i.e AS/1/LABEL does have to be beside AS/1. It can be anywhere on the table as long as the column header is on row 11).
Screenshot of the ASCT+B Table Template

Visualization


This section goes over what the visualization comprises of, how it works and how the user the navigate through it.


Getting Started
  1. Go to the ASCT+B Reporter via this link.

  2. Click on the Go to Visualization button and select the organ sheet that you would like to view.




The visualization in the ASCT+B Reporter consists of a tree structure that respresents the relationships between Anatomical Structures (AS) and its substructures (colored in red). This tree is clumped with a bimodal network that represents the relationships between these AS and their Cell Types (CT, colored in blue) and their Biomarkers (B, colored in green).

ASCT+B Reporter Visualization

Partonomy Tree

The partonomy tree visualizes anatomical structures (AS) and substructures. They are color coded in red. It makes use of Vega's Tree Layout to visualize relationships between anatomical structures and its substructures. The tree consists for a root node called Body. This node is a pseudo node, and is added to keep the visualization error free. Since it is a tree structure, having multiple parents for a node violated the algorithm. Therefore, to prevent any errors why comparing or linking your own sheet, the Body node acts as a singular parent.

On hovering over any AS nodes which is not on the last layer (leaf nodes) will reveal a tooltip that will show the structure's ontology link. Please note that this part of the visualization is not interactive. This part of the visualization cannot be sorted. Ontology links are shown below the name of the node by default. To know how to switch them off click here.


Defining Uniqueness of Nodes

To create a error free visualization while also trying to capture the requirements of the data, the Reporter takes a few steps to define unique nodes. The algorithm uses a combination of,

  1. name of the structure
  2. ontology link of the structure
  3. rdfs label of the structure
  4. a custom comparator value that keeps tract of the current node's parent(s).

The comparator is needed because there are many instances where the node has the same name, ontology link and rdfs label, but have different parents. So if the comparator was not present, such nodes would get merged and would be show linked to the parent node that appears first in the visualization creating process.


Peculiar Behavior

Sometimes, the anatomical structures count can be lesser than the absolute count. This is due to the fact that this algorithm is a tree, and having multiple parent nodes is not allowed. So such nodes are clubbed together, if they occur. For example consider a table like the one given on the right,


Below is how the table would be visualized as,



Here, A3’s parent is A2 (in row 13), so even though there’s no structure after A2 in row 12, the A3 in row 2 will be added to A2. Which is why the number of connections are lesser as a lot of the nodes are getting matched together, giving a lower number. The reason it is this way is because the visualization is a tree and a node cannot have multiple parents. Therefore, the A2 in row 12 and A2 in row 13 cannot appear as 2 different nodes (unless they’re different with diff uberon or label) because then A3 would have 2 parents – which is not allowed by the visualization (Vega).


Last Layer of the Partonomy Tree

The nodes in the last layer of the partonomy tree are technically not a part of the partonomy tree. These nodes behave differently than all the other AS nodes (they can be horved and clicked on). The x and y coordinates from the last level of the partonomy tree are used to build the first level of the bimodal network, that is then placed on top of the layer layer of the partonomy tree. This is a perfect overlap because the coordinates for all the nodes are the same. By being placed in a custom built visualization, it follows the algorithm that makes it interactive. Hovering and clicking on these nodes will highlight the respective CT and B nodes that the node is connected to.


Bimodal Network

The bimodal network links the anatomical structures to the cell types, and then the cell types to the biomarkers. This section will be describing how that is done.


Cell Type Layer (CT nodes)

This is the second layer of the biomdal network that depicts the relationships between the Anatomical Structures and their typology of cells. These nodes are colored in blue. Hovering and clicking on these nodes will highlight the respective AS and B nodes that the node is connected to. These connections are made by using the last layer of the anatomical structures from the main data. These nodes are case sensitive. They are built by using 2 configuration variables bimodal_distance_x (which is the horizontal distance between the last layer of the AS nodes and the CT nodes) and bimodal_distance_y (which is the vertical distance between each CT node). These both can be configured on the fly (please see Graph Controls). Uniqueness of these nodes are defines by the combination of the name, ontology link and the rdfs label. Ontology links are shown below the name of the node by default. To know how to switch them off click here.

These nodes have additional functions. To more about them click here.


Biomarker Layer (B nodes)

This is the third layer of the bimodal network that shows teh relationships between Cell Types and their Biomarkers. These nodes are colored in green. Hovering and clicking on these nodes will highlight the respective CT and B nodes that the node is connected to. They are built by using 2 configuration variables bimodal_distance_x(which is the horizontal distance between the CT nodes and the B nodes) and bimodal_distance_y (which is the vertical distance between each B node). These both can be configured on the fly (please see Graph Controls). These nodes are case sensitive. Uniqueness of these nodes are defines by the combination of the name, ontology link and the rdfs label. Ontology links are shown below the name of the node by default. To know how to switch them off click here.

These nodes have additional functions. To more about them click here.


To sum it up, the image below depicts how the visualization is built,




Interaction

The last layer of the Partonomy Tree (AS nodes), Cell Type nodes and Biomarker nodes are interactive, i.e you can click/hover over them.

Hover

Hovering over these nodes will highlight the paths and the nodes that the hovered over node is linked to. The nodes that are not a part of the relations, are greyed. When hovered on a node, a tooltip pops up that has the follow data,

  • Name
  • Degree
  • Indegree
  • Outdegree
  • Ontoloty ID
  • rdfs:label

The image below shows a screenshot of a hovered over node,



Click

Clicking on a node will cause the highlights between the nodes to persist. The nodes that are not related have their opacities further recuded. The node text should also be bolded. The image below shows a screenshot of a clicked over node,



Paths can be clicked on to show the DOIs. Clicking on path will open a bottom sheet which will have the DOI References listed.


Clicking on the name of a node will cause a bottom sheet to pop up that shows the Description, Ontology ID and the IRI of the particular node. This data is fetched from this API. Note that this only works for AS nodes that have Ontology IDs. The image below shows a screenshot of the bottom sheet.



Visualization Functions


To make the visualization more interactive, there are certain functions like sorting, sizing and filtering that can be done on various nodes. These functionalities only apply to the Cell Types nodes and the Biomarker nodes. The figure below shows the visualization functions that can be accessed via the Control Pane on the left.

Vis Functions Full


Terminologies
  • Indegree: Number of incoming links into a node
  • Outdegree: Number of outgoing links from the node
  • Degree: Sum of the indegree and outdegree

Sorting

CT and B nodes can be sorted Alphabetically or by their Degree. By default, these nodes are sorted Alphabecially. Selecting any of the options will cause the visualization to re-render.


Sizing

CT and B nodes can be sized based on their Indegree, Outdegree and Degree. By default, all the nodes are sized based on the default value of 300. Since the outdegree is the same and the indegree for the B nodes, you can just sort B nodes by their Degree. Selecting any of the options will cause the visualization to re-render.


Filtering

The Reporter currently supports 2 types of Biomarkers,

  • Protein (denoted by BP in the ASCT+B Tables)
  • Gene (denoted by BG in the ASCT+B Tables)

By using the third select menu in the Biomarker Functions, B nodes can be filtered based on their type. Selecting any of the options will cause the visualization to re-render.

Visualization Controls


Due to the ever increasing data, the visualization can sometimes get messy and hard to read. The visualization Controls can be used to solve this problem by being able to resize the visualization and change the default configurations on the fly. These configurations can also be downloading in a JSON format. These controls can be found on the Control Pane on the left (figure below),

Vis Controls

Ontology IDs

By default this option is switched on. Toggling this options will show/hide the ontology names of nodes (if present) that can be found below the name of the node.

Show all AS

This is an options that is only visible in the All Organ Visualization. This is switched off by default and the visualization only renders the organ instead of all its anatomical structures and substructures. Toggling this to be on will show all the Anatomical structures and their substructures in the all organs visualization.

Tree Width

Sets the width of the partonomy tree (AS nodes).

Tree Height

Sets the height of the partonomy tree (AS nodes).

Bimodal Distance X

Sets the horizontal width of the bimodal nodes (distance between AS and CT nodes/CT and B nodes).

Bimodal Distance Y

Sets the vertical width of the bimodal nodes (distance between AS and CT nodes/CT and B nodes).

Compare


You can now compare your own data with that found in the ASCT+B Master tables, which is visualized in the ASCT+B Reporter. In order to do this, you will need to have your data in the ASCT+B table Google Sheets template with public access that can be uploaded into the Reporter. NOTE: .xls files are not supported. The Compare feature is particularly useful if you have data from your own research that you would like to compare to the data in the ASCT+B Master Tables.


Usage

  1. To access the Compare feature, go to the toolbar at the top and click on Compare (shown in the image below)

    Compare Icon Toolbar

  2. A drawer will pop up from the side that will show a few instructions and the various input fields.

    • Title: Optional. This is the title by which the uploaded sheet can be identified. By default it is given a title in the format Sheet <number>. This number corresponds to the index of the sheet in the compare feature. If you have added sheets, the default title of the sheets would be Sheet 1 and Sheet 2 respectively.
    • Description: Optional. This field holds a small description about what your data is about. It is empty by default.
    • Google Sheets Link: Required. This field will hold the sheet link to your data. Your data has to be in the specified format that can be found here.
    • Color picker: Optional. This is a swatch that will allow you to pick the color. A random color is assigned my default.

    Compare Dialog

  3. Additional sheets can also be added in the same way by clicking on the + Add button at the bottom.

  4. Clicking on Compare will then render the new data on the visualization after the Reporter have successfully received the data. (sometimes this can take a few seconds). The results of a compared data is shown below (with the updated legend),

    Compare Results Compare Legend

  5. Reports are generated for each separately that lists various statistics (common counts, uncommon counts etc). More information about the report can be found here.

Characteristics

  • All links (paths, relationships) that are common between the master data and the uploaded data are highlighted in the selected color.
  • All nodes that are common between the master table and the uploaded data are highlighted in the selected color.
  • All the new nodes have been added to the visualization with a dashed stoke around them in the selected color.

Note

In order for the compare feature to work properly, make sure,

  • The sheet on Google Sheets has public access. The permissions on Google should look something like this,

    Compare Results

  • The data should be in the format specified here. Its completely fine to not include headers that are not required for your data. For example, if you have just cell types in your data you can just have CT/1 in your sheet, and that would work fine with the Reporter. Therefore, the headers matter.


To try it out, go to the Kidney Visualization and try to compare this sheet to replicate the above demonstration.

Playground


The latest feature, Playground, allows you to play around with a sample visualization, as well as upload your own data that can be visualized. This also comes with an in-house table that can be edited to see the changes in the visualization in real time. Uploading your sheet/data is temporary and once the page is refreshed the link is terminated.

Usage

  1. To active playground mode, click on the Playground button on the toolbar.

    Playground Icon Toolbar

  2. This should take you to an example visualization with just a handful of nodes.

    Playground

  3. There are 3 tabs,

    • Visualization: This is the default view which shows the visualization (as shown in the image above).

    • Table: This is a tabular view of the data. This is not a full featured table, just a basic view to do some quick edits. Alternatively, you can edit your sheet on Google Sheets and then head over to the Reporter, click on the Refresh button and get the latest data.

      Playground

    • Upload: This is where you can upload your own sheet. In the input field, paste your Google Sheet URL from your browser and hit the Link button to visualize your data.

      Playground

Note

In order for the playground feature to work properly, make sure,

  • The sheet on Google Sheets has public access. The permissions on Google should look something like this,

    Compare Results

  • The data should be in the format specified here.


Indented List


This visualization is a traditional, hierarchical structure that displays anatomical structures and its sub-structures. This makes use the of Angular Material Tree.


Usage

  1. Click on the Indented List icon button on the nav-bar.

    Indented List nav bar

  2. This will show the Indented list.

    Indented List

  3. Clicking on a name will bring up the bottom sheet with the additional information.

    Indented List bottom sheet

Report


The report computes and generates various statistics from the data (current organ sheet). These can be exported in XL format. The numbers in the report are affected by the applied filters (eg. count for gene biomarkers, protein biomarkers etc).

Usage

  1. Click on the Report button in the nav-bar to generate and show the report.

    Report Icon Nav

  2. The generated report will pop up from the right.

    Report

  3. To download the Report in XL format, click on the download button at the top. If there are multiple compare sheets, these would be different sheets in the excel workbook.

    Report Download


Report Tabs


Main Sheet

This tab will contain the statistics of the Main Master table of that selected organ. The main sections of this tab are,

  • Unique Entities: Unique counts of Anatomical Structures, Cell Types and Biomarkers.
  • Entity Links: Count of the linkages between various entities, namely,
    • part_of
    • located_in
    • characterizes
  • Ontology Links: Shows the count of structures that have Ontology IDs and that do not have the same.
  • Alphabetical listing of AS, CT & B: This section has expandable panels that name provide the names of the nodes for which the counts were generated above.

Compare Sheet

Each compare sheet has its own report. In order for data to be visible in this tab, make sure to use the compare feature and upload your data. These sheets can be individually downloaded as well as deleted. Once the data is there, this tab would look something like this,

Report Download

Indented List


This visualization is a traditional, hierarchical structure that displays anatomical structures and its sub-structures. This makes use the of Angular Material Tree.


Usage

  1. Click on the Indented List icon button on the nav-bar.

    Indented List nav bar

  2. This will show the Indented list.

    Indented List

  3. Clicking on a name will bring up the bottom sheet with the additional information.

    Indented List bottom sheet

Search


This useful feature allows for anatomical structures, cell types and biomarkers to be searched and pinpointed in the visualization. The input auto-complete filters the entities based on the name. Additionally, you can also select multiple structures to search for.


Usage

  1. Click on the search bar at at the top.

    Search nav bar

  2. Type in the name in the input field. The options will keep changing as letters are typed (auto complete).

    Search typing

  3. Clicking on the available options will highlight those entities in the visualization with a light blue box around the name.

    Search typing

Export


The ASCT+B Reporter visualization can be saved out in PNG (Portable Network Graphics) and SVG (Scalable Vector Graphics File) format. The Vega Specification can also be exported in JSON format. The data that is currently being supplied to the visualization gets saved in this JSON too.

This can be done by clicking on the download button on the navigation toolbar on the top (beside the refresh icon) and selecting the suitable image format.

Export