feat: include an essentiality summary for each release

[haowang-bioinfo]

Inspired by this PR from yeast-GEM, it could be nice to include and update an essentiality summary for each Human-GEM release.

[JonathanRob]

I can see the benefit of such a test, but there are key differences between yeast and human that make this difficult to implement in practice.

The three major challenges that come to mind are:

1. Unicellular vs. multicellular organism
2. Ability to grow on minimal medium
3. Feasibility of experimental procedures

Unicellular vs. multicellular organism: Yeast are unicellular, so an essential gene can be defined relatively easily as a gene that is necessary for a single cell to remain viable. Not only do humans consist of many cells, but these are different cell types as well. So then how to define an essential gene? Essential for any cell type? Some types? All types? Essential for an entire tissue, or organism, or just the single cells? There is no right or wrong answer, so it becomes very subjective and dependent on context.

Ability to grow on minimal medium: Yeast can grow using defined, minimal media containing a single carbon source (with some other salts, vitamins, etc.). This allows for more controlled tests of gene essentiality, where one can define genes as essential when they are required for yeast viability when grown on that media. Mammalian cells are comparatively difficult to grow, and require complex growth media with amino acids, essential fatty acids, etc., which adds complexity to the problem. Both yeast and human cells will share the complication that gene essentiality is relative to the media they're grown in. So the set of essential genes for yeast grown in minimal media with glucose will be different than when yeast is grown in media supplemented with amino acids, for example.

Feasibility of experimental procedures: We cannot perform true organism-level gene essentiality tests on humans (for obvious reasons), which is otherwise relatively simple for yeast. We therefore need to rely on cell line assays, and this requires the assumption that essential genes identified for a cell line will translate to in vivo.

So at least these points would need to be considered when designing a "standard test" for gene essentiality. And keep in mind that using a specific dataset/condition/cell line can lead to "overfitting" the model on that dataset, so the more general, the better.

[haowang-bioinfo]

@JonathanRob very good and thorough considerations!

1. Unicellular vs. multicellular organism
2. Ability to grow on minimal medium
3. Feasibility of experimental procedures

As you correctly pointed out that it's impossible to perform true organism-level gene essentiality tests on humans, a feasible way might start to collect essential genes for specific tissues and cell lines with high-quality omics data, as well as associated medium conditions. What do you think?

[feiranl]

Agree, maybe also to ensure the metabolic tasks can fill with each release and also the energy and redox production are realistic? For now, it can be run manually before each release. If we find a good way to automate that, then we can incorporate that into the check function

[mihai-sysbio]

We can already run actions with Python code and Matlab code. As long as tests are nicely scripted, they can be easily turned into actions.

[haowang-bioinfo]

maybe also to ensure the metabolic tasks can fill with each release and also the energy and redox production are realistic

this might be an urgent thing because the model may easily get broken due to high amount of changes or even accidental mistakes, it would be better to ensure these in each PR

[mihai-sysbio]

How should the output be shared? As a comment on the PR?

[haowang-bioinfo]

output can be in evaluated by such a function

[haowang-bioinfo]

A practical implementation probably is to begin with the Hart2015 datasets which are:

1. homogenous cell lines
2. able to grow on defined medium (McCoy's 5A, DMEM, or Neurocult NS-A Basal media)
3. with fitness genes detected by high-resolution CRISPR knockout and screening procedures

[mihai-sysbio]

I didn't look any deeper, but here are the genes with no symbols

ENSG00000100101
ENSG00000255730
ENSG00000259916
ENSG00000272916
ENSG00000284844
ENSG00000285043
ENSG00000285269

[haowang-bioinfo]

here are the genes with no symbols

are these genes in the latest genes.tsv or the model?

[mihai-sysbio]

These are the IDs that have no symbols in the TSV file.

[JonathanRob]

Thanks for checking @mihai-sysbio. Just a reminder @haowang-bioinfo that Ensembl IDs and gene symbols are never guaranteed to be 1:1. Sometimes they are 1:many, many:1, 1:0, 0:1, etc.

[haowang-bioinfo]

Just a reminder @haowang-bioinfo that Ensembl IDs and gene symbols are never guaranteed to be 1:1. Sometimes they are 1:many, many:1, 1:0, 0:1, etc.

@JonathanRob sometimes yes. Currently the symbols in genes.tsv are extracted from Ensembl by the built-in python script. So far they are congruent with Uniprot mapping. In the latest model and genes.tsv, all symbols are uniquely assigned as 1:1 pair

[feiranl]

Also check wether there are duplicate rxns for each release to avoid the error identified in #561 #535, and it would also be nice if we can check wether the added rxns are mass balanced and charge balanced. I can work on that.

[mihai-sysbio]

Since there is already access to Memote in the Docker image, I would recommend reusing as many of the Memote tests as possible (as previously mentioned here), for example test_reaction_charge_balance and test_reaction_mass_balance.

[haowang-bioinfo]

might be a good idea to reuse the memote tests, and probably begin with mass and charge balance checks?

[haowang-bioinfo]

it would also be nice if we can check wether the added rxns are mass balanced and charge balanced. I can work on that.

@feiranl agree and feel free to go ahead with your checks, regardless of memote tests

[feiranl]

I suggest performing a balance check for the model after each update. Regarding the output and failure criteria, my suggestion is to generate a default file documenting the currently unbalanced reactions and store it in a test folder. For the update, the document is read and a new test is run, if new unbalanced reactions occur, it would indicate a check failure.

[haowang-bioinfo]

some thoughts IMHO:

• memote tests and in-house tests can go ahead together, then maybe decide which to use later
• output of such checks should be boolean by default, in the case of unbalanced/duplicated rxns that probably can be directly reported as error message (instead of a standalone file)

[haowang-bioinfo]

close for now, feel free to reopen

[mihai-sysbio]

While PRs #563 and #564 resolve the main discussion items, I feel some more actionable conclusions were needed before closing the discussion, for example:

• is there an intention to include some memote tests?
• how is the gene essentiality going to be reported?
• is there a desire to run the gene essentiality automatically?

[haowang-bioinfo]

IMHO:

is there an intention to include some memote tests?

open to this

how is the gene essentiality going to be reported?

now is reported when pushing new release to main branch

is there a desire to run the gene essentiality automatically?

no, not yet - because computational load is too heavy

[mihai-sysbio]

how is the gene essentiality going to be reported?

now is reported when pushing new release to main branch

Are you thinking of posting this directly in the PR discussion? My concern is that it is hard to find.

is there a desire to run the gene essentiality automatically?

no, not yet - because computational load is too heavy

Even if it's heavy, if it's only run every few months then it's still good. There is an attempt to achieve this in PR #675.

[haowang-bioinfo]

how is the gene essentiality going to be reported?

now is reported when pushing new release to main branch

Are you thinking of posting this directly in the PR discussion? My concern is that it is hard to find.

any suggestions?

is there a desire to run the gene essentiality automatically?

no, not yet - because computational load is too heavy

Even if it's heavy, if it's only run every few months then it's still good. There is an attempt to achieve this in PR #675.

then try and see

[mihai-sysbio]

any suggestions?

How about generating a chart (image) to embed directly in the Readme?