changes.txt

Mar 08 2024
  jbrowse2: added rn8

Mar 04 2024
  carpe-variants: fixes of positions for deletions; processing in 10 simultaneous threads

Mar 01 2024
  carpe-variants: fixes of positions for non-snvs,
                  export variant type in column 3 of gff3 file

Feb 23 2024
  ensembl-prep: fixed code to handle scaffold-only assemblies
  ensembl: fixed output dir for tracks
  clinvar: added generation of pathogenic track

Feb 22 2024
  ensembl: updated source to build 111

Feb 19 2024
  agr gff3: fixed export of merged gene positions

Dec 15 2023
Dec 14 2023
  jbrowse2: gff3 dir name must not have spaces in the name -- it causes problems with some tools

Dec 05, 2023
  jbrowse2: gff3 dir name must be same as assembly RefSeq name

Nov 28, 2023
  jbrowse2: fixed script to create dir for uth files

Nov 27, 2023
  jbrowse2: added handling of squirrel HiC_Itri_2 assembly
  Gff3ColumnWriter: simplified code

Oct 06, 2023
  agr: generated files are sorted abd bgzipped

Sep 22, 2023
  jbrowse2: fixed generation of aliases file

Sep 11, 2023
  jbrowse2: added generation of file with aliases for every assembly

Sep 08, 2023
  jbrowse2: added support for species other than rat/mouse/human

Sep 05, 2023
  jbrowse2: added generation of phenotypic variants file

Aug 29, 2023
  jbrowse2: updated config for mouse and human

Aug 28, 2023
  jbrowse2: config tune ups

Aug 21, 2023
  jbrowse2: variants

Aug 11, 2023
  jbrowse2: variants tune up

Aug 8, 2023
  agr gff3: updated header

Aug 3, 2023
  jbrowse2: reenabled generation of data for rn6, rn5 and rn3.4 assemblies

Jul 31, 2023
  jbrowse2: added generation of '#sequence-region' lines for genes, qtls, sslps, protein domains and ontologies

Jul 07, 2023
  jbrowse2: temporarily restricting output only to rn7 assembly
            tuned up output of gff3 files

Jun 26, 2023
  jbrowse2: added assemblies for dog, pig, green monkey and naked mole rat

Jun 22, 2023
  jbrowse2: added mouse assemblies mm39, mm10 and mm9 to the processing
  
Jun 20, 2023
  mutant & congenic strains, protein domains: JBrowse2 files for rn6, rn5 and rn4
  ontologies, genes, qtls, markers: JBrowse2 files for rn6, rn5 and rn4
  bam conf generator: helper to generate JBrowse2 config info for a set of bam files

Jun 15, 2023
  agr gff3: updated assembly names for rat and human in gff3 headers

Jun 09, 2023
  ontologies, genes, qtls, markers: JBrowse2 files for rn7

Jun 06, 2023
  ensembl-prep: updated config to download gff3 files from Ensembl release 109 (instead of 106, as it was previously)

May 12, 2023
  genes: generates '_genes_only' and _genes_and_transcripts' files
  genes: updated loading of genes tracks into JBrowse

May 01, 2023
  genes, strains, qtls: removed generation of RatMine files (due to RatMine discontinuation)
  Gff3ColumnWriter: implemented compression in 'gz' (gzip) format or 'bgzf' (bgzip) format
     by default, the compression will be in 'bgzf' format

Jan 30, 2023
  markers: removed generation of RatMine files (due to RatMine discontinuation)

Jan 13, 2023
  updated build.gradle to be gradle 6.x + compatible

Oct 31, 2022
  genes: added sorting of lines, to be tabix-ready (gff3 file will be instantly ready to be used by tabix)
  Gff3ColumnWriter: better reporting of features with start pos > stop pos

Oct 27, 2022
  agr: added handling of tr biotype 'protein_coding_CDS_not_defined'

Oct 18, 2022
  fixed typo in config for loading rn7 gene tracks

Oct 17, 2022
  fixed typo in JBrowse loading script

Sep 21, 2022
  load positions for UTH assemblies from gff3 file

Sep 12, 2022
  new tool to prep UTH gff3 files

Sep 12, 2022
  gff3 tracks for CHEBI are generated only for rat, mouse and human (there is very little data available for other species)
  JBrowse: loads CHEBI tracks only for rat, mouse and human

Sep 09, 2022
  jbrowse scripts: changed JBrowse-1.16.3 into JBrowse-1.16.11 to avoid perl problems on new machine

Aug 08, 2022
  agr: fix in tr biotype processing

Jul 07, 2022
  agr: tr biotype: fix for genes of type 'gene'
  agr: updated header for generated gff3 files

May 23, 2022
  fixed typo in EnsemblPrep

May 19, 2022
  updated Ensembl JBrowse tracks from build 101 to 106; new assemblies: mRatBN7.2.106, GRCm39.106, ROS_Cfam.1.0.106
  added generation of JBrowse tracks loading script
  genes: added extract of dog assembly ROS_Cfam_1.0

Feb 23, 2022
  strain specific variants: added generation and loading of strain specific variant and damaging variants for rn7

Feb 04, 2022
  strains: added verbose gff3 header

Feb 03, 2022
  strains: simplified code for handling strains

Jan 19, 2022
  agr: switched to assembly rn7 for rat
  updated log4j to avoid zero day exploit

Nov 30, 2021
  added generation of ontology/CHEBI tracks for GRCm39 assembly
  sped up generation of ontology files

Nov 02, 2021
  agr: fixed

Sep 24, 2021
  ontology extractor: fixed names for generated ontology files, for vervet and molerat

Sep 21, 2021
  NCBI prep: added code to prep fna files for scaffold assemblies for loading into RGD JBrowse
  JBrowse tracks: added support for green monkey Vero_WHO_p1.0 assembly

Jul 14, 2021
  AGR gene extractor: fixed code to not emit duplicate gene entries

Jun 17, 2021
  improved code for handling species

Jun 16, 2021
  rn7.2: added generation and loading of disease/gene-drug tracks
  vervet: chlSab2 assembly: added generation and loading of gene/disease/gene-drug tracks
  molerat: hetGla2 assembly: added generation and loading of gene/disease/gene-drug tracks

Jun 15, 2021
  improved species handling

Jun 11, 2021
  added gff3 qtl extractor and JBrowse track loader for mouse assembly GRCm39

May 18, 2021
  agr extractor: date_produced field aligned with DQM requirements
  damaging variants: fixed sql and code to work with new variant table structure

Apr 29, 2021
  rat strains: added code to generate gff3 files and tracks for mutant and congenic strains on assembly 7.2
  rat qtls: added code to generate gff3 files and tracks for qtls on assembly 7.2

Apr 26, 2021
  rat markers: added export for assembly mRatBN7.2

Jan 28, 2021
  added NcbiPrep tool: to prepare NCBI fasta files for loading into JBrowse
  rat genes: added export for assembly mRatBN7.2

Jan 19, 2021
  genes: removed code referencing UniGene IDs (UniGene IDs removed from RGD database)

Dec 4, 2020
  JBrowse tracks: added gene tracks for GRCm39 assembly

Nov 06, 2020
  genes: added export of GRCm39 assembly

Sep 18, 2020
  qtls: code refactor (1st step for parallelization)

Sep 16, 2020
  bonobo genes: added export for new reference assembly 'Mhudiblu_PPA_v0'

Sep 04, 2020
  agr: fixed export of CDS

Aug 28, 2020
  agr: added export of Ensembl data and additional attributes protein_id, transcript_id

Aug 21-25
  ensembl: added ensembl gff3 assemblies to be downloaded and then uploaded to Jbrowse

Jul 08, 2020
  agr genes: added export of genes with multiple loci

Apr 6, 2020
  genes: tuned up logging: debug messages no longer go to main summary; they go to detail log instead
  genes: fixed generation of unique ids

Mar 30, 2020
  agr genes: added handling of gene type 'biological-region'

Mar 09, 2020
  protein domains: improved logging of exceptions
  CreateGff4Eva and eva.sh were added for Eva variants

Feb 17, 2020
  protein domains: simplified gff3 generation and loading for all species

Feb 14, 2020
  agr: updated to latest 1.0.1.0 AGR spec

Feb 06, 2020
  gene files: code redone to process gene assemblies as specified in the app property file

Jan 31, 2020
  fasta preprocessor updated to preprocess scaffold reference sequence for 'naked mole rat'

Jan 24, 2020
  gene files: added generation of files for vervet (green monkey)

Jan 17, 2020
  added loading of promoter gff3 files into JBrowse

Jan 08, 2020
  generation of gff3 files for all promoters in RGD

Nov 26, 2019
  qtls: condensed summary emails into one

Nov 14, 2019
  agr: fixed code

Nov 11, 2019
  agr: fixed script

Nov 04, 2019
  protein domains: only generates gff3 file if there is any data for given assembly

Oct 30, 2019
  protein domains: extended to handle non-primary assemblies

Oct 28, 2019
  ontologies: fix NullPointerException for genes without names

Oct 25, 2019
  gene files: generate only one file for all chromosomes

Oct 18, 2019
  disease and chebi ontology tracks:
    generates only one file for all chromosomes
    DO terms that should processed must be in 'RGD_JBrowse_slim'
    uses merged-gene-descriptions if available
    fixed detection of obsolete ontology files
    added generation of disease/chebi tracks for chnonchilla, bonobo, squirrel, pig and dog

Oct 1, 2019
  AGR gff3 files:
    - gff3 fields 'ID:' and 'Parent:' are now set to RGD IDS for genes
    - 'protein_id' field set to protein RefSeq acc is emitted for protein coding transcripts
    - UniGene ids removed from emitted xdb ids

Sep 25, 2019
  genes module: added 'biological-region' to list of handled gene types (occurs for mouse genes)

Sep 24, 2019
  added gff3 generator for protein domains

ver 1.4.18, Jun 6, 2019
  gff3 for genes generator: optimized logging and parallelized code

ver 1.4.17, Mar 5, 2019
  JBrowse track loader: added track loading for RNAcentral

ver 1.4.16, Jan 31, 2019
  added loading of PIG tracks into JBrowse

ver 1.4.15, Jan 30, 2019
  added generation of gene tracks for PIG

ver 1.4.14, Jan 16, 2019
  jbrowse track loader: updated url for popup menu content (it is absolute now, instead of relative,
    to support JBrowse installation on the cloud)

ver 1.4.13, Nov 29, 2018
  ensembl prep module: added handling of scaffold assemblies like chinchilla

ver 1.4.12, Nov 21, 2018
  added a module to prepare Ensembl gff3 files for loading into JBrowse

ver 1.4.11, Nov 19, 2018
  AGR gff3 files: refactored generation of IDs, to produce smaller file

ver 1.4.10, Nov 15, 2018
  updated scripts to generate all files in directory /home/rgddata/pipelines/RGDGff3Pipeline/data;
  previously it was: /home/rgddata/pipelines/RGDGff3Pipeline/dist/log/RGDGFF3/Output

ver 1.4.9, Nov 14, 2018
  AGR gff3 files: for transcripts, added generation of 'biotype' attribute
  (to be used by Ensembl VEP tool: https://useast.ensembl.org/info/docs/tools/vep/script/vep_cache.html#gfftypes)

ver 1.4.8, Nov 6, 2018
  tuned up loading of Ensembl tracks for Rnor6 genes into JBrowse; per RGDD-1523
  scripts fixups: replaced all occurrences of server KYLE with REED

ver 1.4.7, Oct 23, 2018
  generation of fasta file with primer pair sequences for markers: updated to use the new sequence data model

ver 1.4.6, Sep 28, 2018
  moved from subversion to github

ver 1.4.5, Jan 4, 2018
  Genes: discontinued generation of gene tracks in gff format

ver 1.4.4, Sep 29, 2017
  GeneAgr: tuned up generation of Note field, so the genes will look better in AGR browser

ver 1.4.3, Sep 20, 2017
  added generation of gff3 files for AGR

ver 1.4.2, Jul 21, 2017
  initial version for DbSnp tracks for human build 38 and 37

ver 1.4.1, Jul 19, 2017
  export of additional attributes for ClinVar tracks

ver 1.4.0, Jun 21, 2017
  moved from ANT to GRADLE

ver 1.3.30, May 8, 2017
  gene extractor: added CHINCHILLA to list of supported species

ver 1.3.29, Mar 8, 2017
  gene extractor: added SQUIRREL to list of supported species

ver 1.3.28, Feb 24, 2017
  gene extractor: added BONOBO to list of supported species

ver 1.3.27, Feb 23, 2017
  gene extractor: added DOG to list of supported species

ver 1.3.26, Feb 22, 2017
  gene extractor: HGNC IDs and MGI IDs are exported as Aliases for genes, if available

ver 1.3.25, Feb 15, 2017
  gene extractor: discontinued export of HPRD IDs as xdb ids; NCBI-annot-status field is exported if available
  generated .gff files are now copied to data_release/GFF directory while .gff3 files are copied to data_release/GFF3 as usual

ver 1.3.24, Sep 16, 2016
  strain tracks: enhanced list of mutant strain types -- per RGDD-1272

ver 1.3.23, Apr 29, 2016
  moved GBrowse loading scripts to graveyard code branch -- per RGDD-1202

ver 1.3.22, Mar 2, 2016
  updated generation of gff3 for RATMINE: for genes with multiple loci, only first locus is emitted

ver 1.3.21, Feb 26, 2016
  updated generation of gff3 for RATMINE
  fixed gff3 format issues

ver 1.3.20, Nov 6, 2015
  added generation of gff3 files for MT chromosome (genes and sslps)

ver 1.3.19, Aug 8, 2015
  fixed generation of CHEBI gff3 files: reenabled export of annotation qualifiers
    which was inadvertently disabled during unification of generation code for ontologies

ver 1.3.18, Jul 23, 2015
  genes: fixed bug in generating CDS features: start pos > stop pos

ver 1.3.17, Jun 26, 2015
  unified code for generation of disease and drug-gene interaction files

ver 1.3.16, Jun 1, 2015
  congenic strains module: fixed encoding of attributes in gff3 file

ver 1.3.15, May 29, 2015
  congenic strains module: changed attribute name from 'source' to 'origin'
   'source' is a reserved keyword in gff3 specification and having an attribute named 'source'
   messed up data in JBrowse

ver 1.3.14, May 28, 2015
  congenic strains module: code refactored to simplify it;
    multicongenics are now exported with object type set to 'multicongenic' instead of 'congenic'
    for multicongenics new attributes are exported: 'loc2','loc3', etc with the positions of other loci,
      f.e. loc2=Chr2:224991853..224991955

ver 1.3.13, May 26, 2015
  congenic strains module: in the notes field, every double-quote character is now replaced by a single-quote
  justification: (per Aurash)
  
    The source of the error was found within the “Note” field: JBrowse typically hates double-quotes, as this truncates
  individual feature descriptions (we’ve seen and dealt with this previously), but in this particular case, JBrowse went
  absolutely haywire in passing the IFrame when it saw both a double-quote and a backslash (literal “/”) within
  the Note field of this one very troublesome feature.
    Anyway, the fix:
  substitute all double-quotes into single strong quotes within the “Note” field for mutant strains tracks.

v 1.3.12, May 13, 2015
  congenic strains module now generates two files: one for congenics, second for mutants

v 1.3.11, Feb 2, 2015
  gene/qtl/strains/sslp/promoters/ontology tracks are exported with all chromosomes per one file;
  ontology tracks are split by object type: for genes/sslps/qtls/strains

v 1.3.10, Jan 29, 2015
  for Disease (RDO) and GeneDrug (CHEBI) tracks: an email is sent if new tracks / obsolete tracks are detected

v 1.3.9, Jan 26, 2015
  .gff3 files are generated compressed, with .gff3.gz extension

v 1.3.8, Jan 19, 2015
  refactored generation of ontology tracks for RDO and CHEBI: code simplified
    and gff3 files are generated only for terms having annotations

v 1.3.7, Sep 26, 2014
  fixed generation for genes having transcripts mapped to multiple assemblies

v 1.3.6, Sep 5, 2014
  fixed generation for genes having positions and transcripts without positions

v 1.3.5, Aug 8, 2014
  generation of DO files optimized

v 1.3.4, Aug 7, 2014
  sslps: refactored: download data directly from db, not from sslp ftp files
  fixed generation of 'Alias' field

v 1.3.3, Aug 5, 2014
  files: genes: xdbIds: exdb ids for KeggPathway, MGC clone, Image clone and PharmGkb ids
     are no longer in the generated gff3 files

v 1.3.2, Jun 27, 2014
  no more duplicates are generated for genes having loci on multiple genes

v 1.3.1, Jun 25, 2014
  an artificial feature line is generated for every gene without features (mostly pseudogenes and ncRNAs)
    otherwise these genes are not shown properly in JBrowse

v 1.3, Mar 31, 2014
  gff3 files are now more compatible with gff3 format -- after fixes the generated files do pass
   online gff3 validation at http://modencode.oicr.on.ca/cgi-bin/validate_gff3_online

v 1.2, Mar 27, 2014
  fixed generation of gff3 files for genes (it was generating multiple duplicated sections for genes/transcripts
  having multiple loci on a chromosome; also code simplified and refactored to speed up processing

v 1.1, Feb 28, 2014
  simplified processing of chromosomes, no more hard-coded chromosome 'conventions' that are unnatural
  (f.e. passing 21 to denote chr X in rat, or passing 21 to denote chr Y in mouse)
  now the chromosomes to be processed are passed in 'natural' format, f.e. you can specify for rat
  'chr:1-20,X' what is intuitive for a human to understand