vg 1.35.0 - Ghizzano

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.35.0

Buildable Source Tarball: vg-v1.35.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• Multipath alignments from vg mpmap are more decomposable in tails near soft clip
• Giraffe does not attempt rescue if it would be too slow.
• vg gamcompare can use distances from a distance index instead of path position annotations.
• Fix bug in vg chunk -T where input path ranges were incorrectly used to find subgraph size.
• Fix GAF parsing bug that caused memory explosion for long paths.
• Fix vg augment -B to not initialize packer object unnecessarily.
• surject joins deconstruct in supporting vg subpath naming, but path lengths must come from --ref-paths in order to be valid in the output header.
• deconstruct AT INFO field fixed to have Number=R instead of A
• vg mpmap has improved sensitivity for detecting spliced alignments and ability to use genome-derived splicing statistics for non-humans
• vg surject -S no longer crashes if it decides that a deleted/spliced segment is too long to realign.
• vg autoindex can handle VCFs that list contigs for which there are no variants
• Giraffe now shuffles tied top read mapping candidates like mpmap and map, to avoid arbitrary biases
• vg sim now produces more realistic RNA-seq fragment length distributions.
• libbdsg dependency should no longer sometimes fail to build on a clean checkout due to missing mio/mmap.hpp\
• Tail alignments can no longer skip read sequence completely when alignment would be too large
• Giraffe progress reports now include fragment length distribution information
• Giraffe can now find .giraffe.gbz GBZ files automatically.
• vg pack gets new -s option to ignore first and last N bases of reads. -s 5 works well in practice for SV genotyping.
• vg call default parameters re-tuned to work a bit better on complex graphs. An extra filter for completely unsupported breakpoints is added to increase precision.
• libvgio now uses a simpler CMake build
• vg call -e now works again
• CI testing now uses Toil 5.4.0
• vg rna now supports using GBWT haploytypes as reference paths (--use-hap-ref option).
• vg rna can now create a spliced pangenome graph and pantranscriptome with chopped nodes (--max-node-length option).
• Spliced pangenome graphs and pantranscriptomes can be sorted and have their ids compacted in vg rna (currently only works for inputs in the PackedGraph format).
• The transcript haplotype info table from vg rna no longer contains the "Reference" column (is part of the "Haplotypes" column now).
• The names for the options --use-all-paths and --add-non-ref-paths in vg rna have been changed (single-letter option the same) and the option --gbwt-id-interval has been removed (not relevant anymore).

New and Updated Submodules

• libbdsg and libvgio have been updated.

Make sure to git submodule sync and git submodule update --init --recursive if building from source.

vg 1.34.0 - Arguello

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.34.0

Buildable Source Tarball: vg-v1.34.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• Support for the GBZ format (GBWT + GBWTGraph):
  • vg autoindex: --workflow giraffe outputs GBZ instead of GBWT and GBWTGraph.
  • vg convert: GBZ -> other graph types.
  • vg gbwt: GBWT + GBWTGraph <-> GBZ.
  • vg giraffe: GBZ as the default input format for GBWT + GBWTGraph.
  • vg index: Graph format for distance index construction.
  • vg minimizer: An alternative to GBWT + graph.
  • vg snarls: Graph format with options that do not need embedded paths.
• vg convert outputs HashGraph instead of VG by default.
• GBWT / GBWTGraph / GCSA2 structures are serialized as bare files for better compatibility with external tools.
• vg convert -w can now handle separator tokens in contig names
• Docker base images are now pulled through Google's cache
• vg giraffe now uses IndexRegistry instead of IndexManager and demands that passed file names already exist
• Wiki now includes a tutorial under test for Giraffe
• vg benchmark evaluates the minimum distance index
• Improved build out of the box on M1 Macs with Homebrew
• vg deconstruct now writes "Allele Traversals" (AT field in INFO) for each site. The traversals map the alleles to paths in the graph.
• IntegratedSnarlFinder properly generates single-node chains for bridge edges in the Cactus graph.
• Allow seeding path traversals from XG path set even when we are using GBWT input for vg deconstruct.
• vg index, vg minimizer, and vg snarls can load GBZ without a distinguishing option
• vg mpmap now uses the RNA mapping preset (--nt-type rna) by default instead of DNA (--nt-type dna)
• Fixed a bug that sometimes caused incomplete multipath alignments from vg mpmap
• IntegratedSnarlFinder properly tracks whether cycles and bridge edge paths have been reached regardless of orientation
• Minimizer indexes no longer incorrectly cache distances for looping chains, fixing a duplicate-alignment, low-MAPQ bug in Giraffe
• DYNAMIC now has hopscotch-map as a sub-submodule
• vg autoindex can create GBZ files for vg giraffe using a GFA with W-lines as input
• The speed of vg mpmap is more robust to low-quality RNA-seq reads
• Multipath alignments can be viewed in dot format with vg view -Kd
• vg clip added to remove alt alleles in bed regions or low-path-depth alleles from the graph

New and Updated Submodules

DYNAMIC, gbwt, gbwtgraph, libvgio, libbdsg, gssw, and sdsl-lite have been updated.

Make sure to git submodule sync and git submodule update --init --recursive if building from source.

vg 1.33.0 - Moscona

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.33.0

Buildable Source Tarball: vg-v1.33.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• vg call -a fixed to not include redundant base at beginning of alleles
• "path end" field corrected in GAF output
• VG now supports ARM Macs.
• vg recalibrate has been removed (deprecated since 1.30.0).
• vg find kmer output also lists samples/phases for the matching haplotypes.
• vg's GAF parser now reads cg cigar strings when no cs cigars found so, for example, vg pack can now properly load a GAF from GraphAligner.
• vg gbwt -R now removes multiple samples correctly.
• vg deconstruct can more easily use reference paths (or subpaths) from the GBWT
• vg mcmc now accepts non-vg::VG graphs as input
• vg deconstruct fixed to not write sites with no alt alleles when genotyping with a gbwt

New and Updated Submodules

The DYNAMIC, gbwt, gbwtgraph, gcsa2, libhandlegraph, libvgio, and vcflibsubmodules have been updated. Thehtslibandtabixppsubmodules have been added. Thevowpal_wabbit` submodule has been removed.

Make sure to git submodule sync and git submodule update --init --recursive if building from source.

vg 1.32.0 - Sedlo

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.32.0

Buildable Source Tarball: vg-v1.32.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• CI now tests with sequenceTubeMap
• vg map and vg mpmap now annotate properly-paired read mappings
• vg stats -a reports proper pairing statistics
• Faster graph to GFA conversion with vg convert.
• vg surject now gives a helpful warning if you try to use an unsurjectable GAF, rather than just crashing
• GBWT index can now be used to find alt alleles in vg deconstruct with -g
• Single nodes at the end of bridge paths in the Cactus graph won't show up twice in the snarl decomposition
• Giraffe self-profiling is improved and a standardized speed test script is included
• vg construct can accept multiple --region arguments for different contigs
• Convert GBWTGraph to GFA with vg convert.
• VCF files with * alleles are now accepted by vg construct, and only the particular variant records with * alleles are rejected and warned about
• Uncompressed GFA files can now be directly input (or streamed via stdin) as graphs into vg commands. (previously vg convert -g was required beforehand)
• Karger-stein min cut alogirtm and alt proposal distribution added to vg mcmc genotyper. Scripts for performance analysis are in scripts/MCMC_makefile.sh, scripts/mcmc_dist.sh.
• Count the number of nodes with self-loops with vg stats.
• vg srpe has been removed
• vg mod options --kill-labels and --strong-connect have been removed.
• vg stats -L counts self-loops.
• vg stats -e lists nondeterministic edge sets.
• vg find option --mapping enables finding nodes that map to the selected ids.
• vg is now able to build bundled htslib on Autoconf 2.70+

New and Updated Submodules

The fermi-lite, gbwt, gbwtgraph, 'gcsa2', gfakluge, jemalloc, libvgio, and sdsl-lite submodules have been updated.

Make sure to git submodule sync and git submodule update --init --recursive if building from source. The jemalloc update is also known to not build over top of a previous built version; you may need to rm -Rf deps/jemalloc and git submodule update --init --recursive to fix it in your copy.

vg 1.31.0 - Caffaraccia

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.31.0

Buildable Source Tarball: vg-v1.31.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• Large variants are now always handled as flat, and not aligned to themselves, in vg construct.
• rGFA files (such as those output by minigraph) can now be imported into vg with vg convert -g
• vg autoindex can now construct indexes for the vg mpmap/rpvg transcriptomic pipeline and the vg giraffe tool
• Cigar overlaps replaced with *'s and trailing tabs removed to make GFA output more compliant with 1.0 spec.
• GBWT and GBWTGraph construction from GFA input.
• vg convert can convert GBWTGraph to other graph formats.
• Fixed typo in the IntegratedSnarlFinder test case description
• XG indexes now take somewhat less space (especially those with many paths).
• vg convert can output vg paths (that adhere to certain naming conventions) to GFA W-lines with the -w option.
• Build should fail less on GCC 10.2.1-1.
• vg convert can now output GFA, with selected reference paths written to "rank 0" rGFA tags
• vg combine no longer loads graphs as, and writes graph to, protobuf. Also fixed to join id spaces if necessary.
• vg now uses the vgteam fork of SDSL.
• CI now moved to Gitlab and Github Actions.
• vg autoindex can now simplify plans with joint recipes
• IndexRegistry uses per-instance temp directories
• vg Docker build is now optimized for layer caching
• vg can now build on the arm64 architecture and pass unit (but not CLI) tests
• Updated to latest vgteam version of the DYNAMIC library
• Converting GFA with P-lines and W-lines into GBWTGraph stores the P-lines under sample name _gbwt_ref.
• vg convert from GBWTGraph to another graph type stores the threads for a specified sample (default _gbwt_ref) as paths.

New and Updated Submodules

• DYNAMIC, fermi-lite, GBWT, GBWTGraph, GCSA2, gssw, libbdsg, mmmultimap, sdsl-lite, ssw, and xg have been updated.
• mio and atomic_queue have been added.

Make sure to git submodule sync and git submodule update --init --recursive if building from source.

vg 1.30.0 - Carentino

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.30.0

Buildable Source Tarball: vg-v1.30.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• vg paths now has -C option to list which paths are cyclic and which aren't
• fixed vg view bug which caused outputting to GFA to use too much memory
• Giraffe can now --show-work to explain itself.
• Giraffe funnel plots no longer invent fake candidates
• Giraffe now maps and caps MAPQs properly with syncmers
• vg mpmap produces more complete multipath alignments around indels in low complexity sequences.
• Update libvgio dependency (fixed cmake handlegraph dependency)
• The following subcommands are now deprecated: combine, compare, explode, join, kmers, locify, msga, recalibrate, sift, sort, srpe, translate.
• The following subcommands have been deleted: bugs, crash.
• Crash reports now direct users directly to the new issue URL
• rocksdb is no longer a dependency.
• vg index no longer will produce RocksDB indexes of GAMs or graphs.
• vg find can no longer be used to query from RocksDB indexes.
• vg genotype no longer accepts RocksDB GAM indexes, and instead accepts only GAMs.
• vg gamsort can no longer sort using the RocksDB order.
• vg compare has been removed.
• vg locify has been removed.
• The following subcommands are now deprecated: concat, explode, join, kmers, msga, recalibrate, sift, sort, srpe, translate.
• vg prune is now more memory efficient.
• Giraffe mapping quality cap is now more numerically stable and fails assertions less
• The roc plot script now runs

New and Updated Submodules

A submodule for the wiki has been added in doc/wiki.

Thelibbdsg, libhandlegraph, and libvgio submodules have been updated.

The rocksdb submodule has been removed.

Make sure to git submodule sync and git submodule update --init --recursive if building from source.

vg 1.29.0 - Sospiro

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.29.0

Buildable Source Tarball: vg-v1.29.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• XG indexing now uses the temporary directory also for SDSL temp files.
• New GBWT construction interface in vg gbwt.
• vg mpmap now can perform multiple spliced alignments on one read and spliced alignments on multiple mappings.
• Update to latest vcflib ("Configure for default, libcurl-using HTSLib config") to get non-x86 support
• Cross building for other architectures is now easier due to the use of $PKG_CONFIG for specifying pkg-config binary
• SAM/BAM/CRAM TLENs should now match BWA's for identical alignments
• Remove multiple samples with vg gbwt -R.
• Mac builds on Catalina no longer fail with a "missing separator" error.
• CIGAR strings in SAM/BAM/CRAM should now consistently coalesce runs of adjacent insertions and deletions.
• Readme no longer encourages using the unsupported GNU GCC build on Mac.
• Spliced multipath alignments are strand consistent and have a more complete representation of variation near splice sites.
• vg filter can now work on GAMP files with -M, and reads with a given subsequence can be filtered with -a and -A.
• vg mod -n should now work properly when self loops happen at the ends of merged nodes
• CIGAR strings for spliced surjection from GAM files now have correct intron lengths
• CI now tests macOS 10.15
• vg now uses (and needs!) system Boost for Vowpal Wabbit instead of bundling a subset
• Parallel GBWT merging algorithm in vg gbwt.

New and Updated Submodules

The gbwt, mmmultimap, vcflib, vowpal_wabbit, and xg submodules have been updated.

The source repository for the vowpal_wabbit submodule has moved.

The boost-subset submodule has been removed.

Make sure to git submodule sync and git submodule update --init --recursive if building from source.

New dependencies

Instead of shipping a subset of Boost, vg now requires Boost to be installed on the system already in order to build.

vg v1.28.0 - Acquafredda

Changes in this release:

• vg surject now supports GAF input with -G.
• vg surject can now take GAMP files from vg mpmap as input with -m
• vg surject is now substantially faster
• vg mpmap can now toggle between output formats (including single path and linear formats) with a single option. Some command line arguments have changed semantic meaning.
• vg mpmap's spliced alignment is now significantly more accurate.
• Divided handles with reversing self-edges no longer drop the edge.
• New path cover GBWT defaults: n = 64 for local haplotypes and 16 otherwise.
• Giraffe now properly sets paired end read cross-references
• vg deconstruct includes new option -a to process every snarl (instead of just top-level snarls)
• vg mpmap -m now correctly removes the full length bonus from reported scores
• -lpthread and -lm can be linked dynamically again
• Base quality-adjusted scoring now adjusts the full length bonus.
• Updated with less-patched Raptor dependency
• vg convert calls with -g as the last option will now convert to the correct format, not always Protobuf
• Now vg outputs raw handle graph files readable by libbdsg without the need for vg view -x/--extract-tag.
• vg msga handles terrible alignments better.
• vg giraffe can now output to SAM/BAM/CRAM directly
• Dependency retrieval no longer upgrades already-installed packages
• vg giraffe, vg map, vg mpmap, and vg surject now take a .dict file to order HTSlib sequence headers via --ref-paths.
• GaplessExtender no longer matches non-ACGT characters.
• SAM header lines in dict files are now tested
• vg add now supports overlapping reference paths, as in Cactus graphs

New and Updated Submodules

• libvgio, raptor, dozeu, and vcflib have been updated.
• htslib has been removed. We now use the HTSlib from vcflib.

vg 1.27.1 - Deliceto

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.27.1

Buildable Source Tarball: vg-v1.27.1.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• No longer crashes when converting GFAs that lack a header line
• GAF support in vg paths
• GAF support in vg augment
• vg mpmap can now produce spliced alignments when using the rna preset
• vg gampcompare now handles spliced GAMP files correctly
• Build an r-index with vg gbwt.
• vg snarls -e will output a traversal for every embedded path through each snarl, regardless of snarl complexity
• Refactored vg gbwt.
• Fixed a rare GaplessExtender bug.
• Use the same temp directory for all submodules in vg index.
• Faster parallel GAF input streaming, especially in vg pack.

New and Updated Submodules

The gbwt, gfakluge, and libvgio submodules have been updated.

Make sure to git submodule update --init --recursive if building from source.

vg 1.27.0 Deliceto

Don't forget to mark the static binary executable:

chmod +x vg

Docker Image: quay.io/vgteam/vg:v1.27.0

Buildable Source Tarball: vg-v1.27.0.tar.gz

Includes source for vg and all submodules. Use this instead of Github's "Source Code" downloads; those will not build as they do not include code for bundled dependencies that the vg build process needs.

This release includes:

• mpmap's agglomerate option fixed for paired-end reads