Releases · catmaid/CATMAID

21 Dec 13:01

tomka

2021.12.21

b74f51f

2021.12.21 Latest

Latest

2021.12.21

Contributors: Chris Barnes, Albert Cardona, Andrew Champion, Stephan Gerhard, Sanja Jasek, Tom Kazimiers

Notes

The version requires PostgreSQL 12. If you also want to upgrade PostGIS,
update PostGIS first and run ALTER EXTENSION postgis UPDATE; in every
existing database in the cluster that should be upgraded. For docker-compose
setups this database update is performed automatically if DB_UPDATE=true is
set for the db container (watch the Docker output). CATMAID's documentation
Docker has more information. If a replication setup is in use, the database
configuration changes for Postgres 12. CATMAID's replication documentation
explains what needs to be done. The next CATMAID release will require Postgres
13, which is also supported by this version.

In Postgres 12, JIT compilation is available to the planner by default. We
found that being more conservative with the use of it helped a few common
queries (like the tracing data field of view). We set the required minimum
cost of JIT use to 1,000,000 in postgresql.conf::

jit_above_cost = 1000000
If R extensions are used, make sure to use R 3.6. On Ubuntu this can be made
available by first installing the official R PPA repository:

sudo gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-key E084DAB9
sudo gpg -a --export E084DAB9 | sudo apt-key add -
echo "deb https://cloud.r-project.org/bin/linux/ubuntu xenial-cran35/" | sudo tee -a /etc/apt/sources.list

And second update the package index and update the local R setup:

sudo apt-get update
sudo apt-get install r-base r-base-dev mesa-common-dev libglu1-mesa-dev
libssl-dev libssh2-1-dev libcurl4-openssl-dev cmtk

This also requires updating all installed R packages. In all likelihood this
requires executing "manage.py catmaid_setup_nblast_environment".
The default downsample factors for a stack without explicit factors defined,
changed slightly: if you relied on the automatic creation of downsample
factors for image stacks, make sure the changed behavior still works for you
or enforce the previous configuration with an explicit specification of
downsample factors. Previously as many zoom levels as needed were created to
map the image size in a quadratic fashion per level onto a 1024x1024px tile.
Instead of hard-coding the 1024px, the minimum tile size defined in the mirror
set is used. Without any mirrors, no extra zoom levels will be assumed.
The recommended python version is now 3.8, although support for 3.6 is maintained.
The next CATMAID release will require at least Python 3.8 and will drop support for
Python 3.6.
A virtualenv update is required.

Features and enhancements

Notes widget:

This new and simple widget can be used to take notes either only for the
current project, shared across all projects or globally visible for all users.
The last option is only available to admin users for writing. Use the "Open
Widget" dialog to open it using the "Notes" keyword.

Publication widget:

This new widget makes management of publication related annotations easier. It
lists all annotations that annotated with one of the publication annotations
defined in the Settings Widget in the Publication section. Each of them
represents a publication for the widget. The export options for each one of
them can now be configured through the table.

Project management widget:

This new widget allows superusers and users with can_administer role in a
project to edit user and group permissions for this project. This interface is
easier to find and manage then the admin view and allows for more granular
admin permissions. It is also useful to share own spaces (own copies of
projects) with other users.
The widget allows to define for both user and groups the following
permissions: can browse (read), can annotate (write), can administer, can
import, can queue, compute taks, can write through API, can fork
Clicking any checkbox will store the value right away.
If a user has "can_administer" permissions in a project, they can edit the
project title and project description (comment) in the second tab.
If a user has "can_administer" and "delete_project" permissions in a project,
the project management widget allows to delete the current project. This is
mainly useful for personal spaces, in which users typically have
delete_project permissions.
The "User data permissions" tab allows to configure user level edit
permissions (who is allowed to edit who's data).
The "Project token" tab allows project admins to view existing project tokens
("invitation codes") available for a project. For each generated token, the
table allows to copy both the token itself as well as an invitation link using
this token to the clipboard. If a user opens an invitation link, they will
automatically apply the token. Anonymous users are asked to log in first.
As admin user in a project, it is also possible from to manually change
skeleton IDs from the "ID updates" tab. This is only allowed if the new ID
isn't used already. Also, if the new ID is a new maximum ID number, all new
IDs will be automatically be larger than this ID. This functionality is also
available as the back-end managment command catmaid_change_skeleton_id.

Skeleton ID change management command:

Just like the Project Management functionality explained in the last
paragraph, the management command catmaid_change_skeleton_id can be used to
change skeleton IDs (and optionally neuron IDs) from the command line.

Link widget and deep links:

This new widget allows users to create persistent deep links into the
dataset using an alias, allowing for friendlier URLs of the form
<catmaid-url>/<project-id>/links/<alias>. The stored parameters are the same
as for regular deep links, with most of them being optional.
Like with fully parameterized URLs, features like layout, widget settings and
loaded skeletons can be used.
Links can be marked private, making them only accessible (and listable) by the
creator of the link.
An optional message can be stored with the link and is shown to the user when
the links is opened.
The context menu for creating different types of links moved from entries
in the Layouts menu to its own menu that is shown when hovering the mouse
cursor over the "URL to this view" link. The first menu entry is new and opens
a link creation dialog, similar to the Link Widget's 'Add link' dialog.

Neuron renaming:

There is now a new neuron renaming dialog, which allows a search/replace
renaming workflow for many neurons at the same time. This can be launched
through the "Rename" button in both the Neuron Search widget and the neuron
lists of the Neuron Navigator (e.g. neurons with a particular annotation).
The dialog supports simple find/replace, but also regular expressions
including capture groups and presents live previews.

Neuron search:

Neuron search: add option to show meta data columns (cable length, number of
nodes, creation time and last edition time) for neurons.
Removal tools for annotations are now opt-in, i.e. not shown by default. To
enable them during annotation display, click the "Show removal tools"
checkbox.
The optional annotation column has now the option to only show annotations
with a particular meta-annotation. This meta-annotation can be configured in
the input field in the table column header.
The CSV export comes now with a header row to make clear the first column is a
neuron ID.
The CSV export includes now the skeleton ID as well and will export metadata
like cable length, if metadata display is enabled in the UI.
Expanded elements (e.g. annotations) are now sorted correctly within their
expansion group. This also fixes the out-of-order expansion display.
Escape underscore and percent sign properly. They were treated as wildcards
before.

Connectivity Matrix:

The new buttons "Save JSON" and "Open JSON" can be used to save the row and
column skeletons, including groups and widget settings.
The loading speed has been improved significantly.
With the new "Link type" selector in the Main tab it is now possible to show
the connectivity matrix for other link types (like desmosomes or gap
junctions). The selected link type is also respected by the CSV export.

Layouts:

New layout menu item: "Copy URL to view with layout". This will create a URL
to the current view like the link button on the right hand side of the second
toolbar, but with a layout spec included. It will open all widgets in the new
CATMAID like in the current one. This includes skeletons, their colors and
widget configurations by default. A second menu entry with the "(no
skeletons)" suffix allows to create link with only the layout and no
skeletons. And a third menu entry with the suffix ("no widget settings)"
allows to copy only the layout plus skeletons, without any widget
configuration.
New layout menu item: "Copy current layout spec". This will copy the layout
specification of the current view to the clipboard. This can be useful when
constructing URLs to specific views.
Widgets in layouts can now optionally specify a list of skeletons to load.
This is done by adding the skeletons parameter:

"skeletons": [16035815, {"id": 16035701, "color": "rgb(0.5,1,0.2)"}]

The list of skeletons can consist of skeleton IDs and/or optionally objects
that can also specify a color in terms of common CSS definitions.
The Selection Table can now receive an options parameter from the layout
initialization. If it contains a list of skeleton IDs like ...

Assets 2

16 Feb 05:31

tomka

2020.02.15

4e39b4a

2020.02.15

Contributors: Chris Barnes, Andrew Champion, Stephan Gerhard, Pat Gunn, Tom Kazimiers

Notes

Python 3.5 is not supported anymore. Use Python 3.6, 3.7 or 3.8.
Postgres 11 and PostGIS 2.5 is required, Postgres 12 and PostGIS 3 is
recommended. If Postgres needs to be updated, update directly to Postgres 12.
If both needs to be updated, update PostGIS first and run
ALTER EXTENSION postgis UPDATE; in every database. For docker-compose
setups this database update is performed automatically. If a replication setup is in
use, the database configuration changes for Postgres 12. CATMAID's replication
documentation explains what needs to be done.
The next version of CATMAID will require Postgres 12 and PostGIS 3.
A virtualenv update is required. Before you start it, please remove some
packages that are not needed anymore first:

pip uninstall asgi-ipc asgi-rabbitmq
If ASGI was set up before, make sure to install channels_rabbitmq or
channels_redis (depending on what yous use). The older asgi_rabbitmq and
asgi_redis packages aren't supported anymore. This also requires an update of
the CHANNELS_LAYERS in settings.py. The channels_rabbitmq documentation for an
example: https://github.com/CJWorkbench/channels_rabbitmq/. This variable
isn't defined by default anymore. Therefore you likely have to replace any
CHANNELS_LAYERS[…] = … with something like CHANNELS_LAYERS = { … }. The
new format is (use custom credentials on any production system!):
```
CHANNEL_LAYERS = {
  "default": {
      "BACKEND": "channels_rabbitmq.core.RabbitmqChannelLayer",
      "CONFIG": {
          "host": "amqp://guest:guest@127.0.0.1/asgi",
      },
  },
}
```
Also, if supervisord is in use to manage CATMAID process groups, the main
Daphne process needs an adjustment: instead of calling daphne with the
mysite.asgi:channel_layer parameter, use mysite.asgi:application. A
complete supervisord entry would then look something like this:
```
[program:catmaid-daphne]
directory = /home/catmaid/catmaid/django/projects/
command = /home/catmaid/catmaid/django/env/bin/daphne --unix-socket=/var/run/daphne/catmaid.sock mysite.asgi:application
user = www-data
stdout_logfile = /var/log/daphne/catmaid-server.log
redirect_stderr = true
```
As last step, the supervisor entry for the daphne worker process has to be
removed. New types of workers can be added, but are not needed in most cases.
The channels documentation has more information on this.
GDAL v2 or newer is now needed. If your Ubuntu version doesn't support this
yet, there is an official PPA:
```
sudo add-apt-repository ppa:ubuntugis/ppa`
sudo apt-get update`
sudo apt-get install gdal-bin
```
The management command gdal-bincatmaid_populate_summary_tables is now known as
catmaid_refresh_node_statistics.
When enabling and disabling both history tracking and spatial update events on
startup, advisory locks are now used. This should make the startup of parallel
CATMAID workers more robust.
The application of migrations 88-91 and 98-99 might take a while to complete,
because they rewrite potentially big database table (treenode_edge, treenode,
class_instance, and more). Therefore, make also sure that there is enough
space available at the database storage location (25% of database data
directory should be plenty). If no replication is used, setting the following
Postgres options can speed up the process: wal_level = minimal,
archive_mode = off and max_wal_senders = 0.

Due to this database update data consistency and correctness was improved,
because additional foreign key relationships have been added that were missing
before.
Both configuration.py.example and create_configuration.py support now the
option catmaid_default_enabled_tools, which defines the list of front-end
tools that are enabled by default for new users. The following options are
supported: cropping, tagging, textlabel, tracing, ontology and roi. By default
the tracing tool is now enabled for new users, because it seems in many setups
users want to have this enabled by default.
Back-end errors result now in actual HTTP error status codes. Third-party
clients need possibly some adjustments to handle API errors. In case of an
error, status 400 is returned if an input data or parameter problem, 401 for
permission problems and 500 otherwise.
On startup, CATMAID will now test if the output directory is accessible as
well as if its expected folder layout is present. If expected subfolders are
missing, they will now be created.
Python code is now linted by flake8. The config is fairly relaxed
(see .travis.flake8), but contributions which do not conform with the selected
rules will fail CI checks.

Features and enhancements

Permissions:

If data is imported by a user, e.g. from a remote CATMAID instance, this user
has all permissions over the imported data. If data was imported or created by
someone else, the regular rules apply: permissions are granted if the user is
creator, superuser or has permission over the data's creator. If user A has
permissions over user B's data, user A will also have permission to edit user
B's imported skeletons. This is done in order to be able to maintain the
original creator and editor data of the imported data, should it be available.

Node filters:

Binary split: a new toggle labeled "inclusive" allows to decide whether to
include the split not in an upstream sub-arbor (default: true).

CLI importer:

Performance improvement: only the skeleton summary entries of the imported
skeletons are now created.
Only update treenode edges and skeleton summary if treenodes were actually
imported (rather than e.g. only volumes).
Add --auto-name-unknown-users option, to automatically generate names for
users that are not available from an import, but referenced in it (by ID).

CLI exporter:

The new --annotation-annotation option allows to specify a meta-annotation
that all exported annotations need to share transitively. If for instance all
all top-level annotations that should be exported are annotated with
"exportable" and "--annotation-annotation exportable" is used, only
annotations from those annotation hierarchies marked as "exportable" will be
exported. Without this option, all annotations are exported.
If users should be exported, also export users of volumes.

Project statistics:

Requests to the back-end our now performed in parallel, resulting in faster
loading times.
It is now possible to configure how many largest neurons should be displayed
as well as an optional name pattern.
The user/time action table will now also list imported nodes and imported
cable length. These values are subtracted from the corresponding regular
counters. If "Include imports" is unchecked, the same information that was
displayed before this commit is shown, i.e. the regular node count and cable
length field include the imported data.

Tracing tool:

The tool bar button to refresh caches has been replaced with a button to open
a context menu titled "More tools". The context menu has an entry to refresh
cashes and a list of tracing data in other projects.
Tracing data from other CATMAID instances (or other local projects) can now be
added to the active stack viewer by selecting the respective remote project
from the "More tools" menu item "Remote data".
To open a remote CATMAID instance at the same location as the current view,
find the remote project in the "Remote data" section of the "More tools" menu,
and click on "Open remote view".
The marker type used fore remote tracing data can be configured through the
Settings Widget in the Tracing Overlay section, option "Remote node marker". A
ring is used by default.
Merges will now by default respect the special annotation "stable". If a
neuron is marked as such it will always win a merge and if both merge partners
are marked "stable", the merge is canceled. This can be disabled by setting
the project or instance setting "Stable join annotation" in the Tracing
section of the Settings Widget to an empty string. The merge dialog will show
a warning if stable merge annotations have any bearing on the merge direction.
Fast merge mode can now optionally be applied only to neurons/fragments with
a particular name pattern (regular expression). This can be configured in the
Fast Merge section of the Tracing section in the Settings Widget.
Alt + G will now select the closest node in the active skeleton. If no
skeleton is selected, the globally closest node is select. Regular G works as
before in the current section.
Skeleton splitting: if a skeleton sampler was in use, a split could so far
only be successfully executed if the split node is within a sampler domain.
This is fixed now and sampled skeletons can now be split upstream or
downstream of a domain as well. If split upstream, the domain will be deleted,
because it is part of the split-off fragment.
The default interaction mode (skeleton tracing, synapse dropping, select,
move, import) can now be configured in the Tracing section of the Settings
Widget. This interaction mode will be enabled when opening the Tracing tool.

Tracing overlay:

The remote mirror CATMAID instance configuration is now taken from the central
configuration, which can be edited in the "Other CATMAID instances" section in
the Settings Widget. The "Read-only mirror index" settings is kept but refers
to list elements in the central configuration list.

-...

Assets 2

21 Jun 04:12

tomka

2019.06.20

abe05d2

2019.06.20

Contributors: Chris Barnes, Albert Cardona, Andrew Champion, Stephan Gerhard, Pat Gunn, Tom Kazimiers, William Patton

Notes

A virtualenv update is required.
All *.pyc files in the folders django/applications/catmaid/migrations/, django/applications/performancetests/migrations/ and django/applications/pgcompat/migrations/ need to be deleted.
Python 3.7 is now supported.
Heads-up: The next CATMAID version will require Postgres 11, PostGIS 2.5 and Python 3.6 or 3.7.
Should migration 0057 fail due a permission error, the Postgres extension "pg_trgm" has to be installed manually into the CATMAID database using a Postgres superuser: sudo -u postgres psql -d <catmaid-db> -c 'CREATE EXTENSION pg_trgm;'
CATMAID's version information changes back to a plain git describe format. This results overall in a simpler setup and makes live easier for some third-party front-ends, because the commit counter is included again. The version display is now the same git describe format for both regular setups and Docker containers.
Tile loading is clamped to (0,0) again, i.e. there are no negative tile coordinates anymore by default. If you need them, set the respective stack's metadata field to {"clamp": false}.
To write to CATMAID through its API using an API token, users need to have now dedicated "API write" permission, called "Can annotate project using API token" in the admin UI. To restore the previous behavior (regular annotate permission allows API write access) the settings.py variable REQUIRE_EXTRA_TOKEN_PERMISSIONS can be set to False. This is done as a safety measure to prevent accidental changes through automation.
If R based NBLAST is used, make sure to execute to update all dependencies: manage.py catmaid_setup_nblast_environment.
The main documentation on catmaid.org has now a place for widget specific documentation as well. Only a few widgets have been updated yet, but more will follow.

Features and enhancements

Tracing tool:

Add a new Tracing Tool icon button to compute the distance between two nodes on a skeleton. Respects virtual nodes.
The globally nearest node can now be selected and brought into view using Alt + G, as opposed to selecting the nearest node in the current section using the G key without modifier.
Using the H shortcut now without an active skeleton, the most recently edited node of the current user (Alt: anyone) in the last active skeleton will be selected. Using Shift will look in all skeletons.
Using Shift + Alt + Click with a connector selected, will will now create a presynaptic node. This is consistent with the already existing Shift + Alt + Click behavior with a treenode selected, which creates a presynaptic connector.

Graph widget:

GraphML files can now be imported, positions and colors are respected. This is useful if layouting is done in e.g. Gephi and coloring should be done in CATMAID. The help page explains a possible workflow for this.
The button "Group equally colored" in the "Main" tab will group all skeletons with the same color into a single group. The user is asked for group names for each color.

3D viewer:

A new synapse coloring mode has been added: polyadicity. The number of partner nodes for each connector is color coded (for synaptic connectors, this is the number of postsynapses). The colors and ranges can be configured through the "Polyadicity colors" button in the "Shading parameters" tab. This is basically a configurable version of an absolute "N with partner" coloring.
Branches with leaf nodes tagged "not a branch" can now be collapsed using the 'Collapse "not a branch"' checkbox in the Skeleton Filters tab.
The visibility of radius information can now be controlled using the "Show radius" checkbox in the "Views settings" tab.
History animations can now be exported in full length without requiring to guess the number of frames for the export. The animation export dialog will show an additional checkbox ("Complete history") if a history animation should be exported. If complete history is enabled, CATMAID will export the complete history of the exported skeletons.
Animations can now be exported as stream directly to a file, which allows for much larger exports (32GB maximum at the moment).
Fractional rotations are now allowed in the animation export.
The default time per rotation in the animation export is set to 15 seconds now, slowing down the default by a factor of 3, which makes it easier to look at.
Stack Z slices can now be animated. Configurable are the change frequency and the change step in terms of sections. This is available for animation exports as well.
Stack Z slices can now be thresholded to replace a background color with another color. If enabled and the sum of all channels is in a configurable range [a,b] it will be replaced with another color.
The rotation time for animations can now specified in seconds rather than angular distance.
The background color is now fully adjustable through the "backgorund" button in the "View settings" tab.
Basic support for Virtual Reality is now available on Windows platforms using a Mixed Reality / SteamVR.3D setup and Firefox >= v65. To enable, check the VR checkbox in the "View" tab and click the "Enter" button right next to it.

Skeleton history widget:

A basic view of the change of a set of skeleton IDs over time based on all nodes that are part of a given skeleton ID or that have been in the past.
Skeleton history can also be used with past skeleton IDs to see into what skeleton they changed (if any).
All past and present treenodes with a passed in skeleton ID are tracked through the complete history and their path of skeleton ID changes is recorded along with the number of treenodes following a given skeleton path.
The widget shows a graph from origin skeletons to the final skeleton IDs in every available path, summing the treenode counts for each contributing path.
Existing skeletons are colored in yellow, past skeletons are colored in cyan. Selected skeletons are colored green.
Ctrl+Click on skeleton will select it and go to the closest location in it. Shift+Click allows selecting multiple skeletons. All selected skeletons are available through the Skeleton Source interface.

Node and skeleton filters:

Filter rules support now an "invert" option during creation, which allows to create filters that include everything but whatever is matched by a particular filter strategy. This can be useful e.g. during neuron review to only look at segments that have been created by people other than oneself or connectivity everywhere excluding a particular compartment.

Measurement table:

Node filters are now supported. Like in other widgets, the respective panel can be opened through the funnel icon in the widget title bar. Measurements are displayed for all independent fragments of a skeleton after a set of node filter rules has been applied.
With the help of the "Sum fragments" toggle all fragments that result from a node filter application can be aggregated into one row by summing individual values.

Remote CATMAID instances

Some tools in CATMAID gained support to communicate with other CATMAID instances, e.g. the Landmark Widget (see below). To enable this functionality, remote CATMAID instances can now be added to the user settings.
The Settings Widget contains now a section labeled "Other CATMAID instances". It allows to manage a list of other CATMAID servers based on their URL, an API key and optional HTTP authentication. This information is stored under an alias.

Landmarks:

Clicking on the name of a landmark group in the Landmarks tab will now open the group member edit dialog that was previously accessible by clicking any landmark name. Clicking a landmark name will now show a new dialog, which allows to specify of which landmark groups the clicked landmark is a member of.
The color of each active display transformation can be established separately in the respective table row. Color changes are now also visible in the 3D Viewer.
It is now possible to load skeletons and landmarks from other CATMAID instances. Point matches are done on the basis of matching landmark names. To enable the UI for this, check the "Source other projects" checkbox in the Display tab.
With the "other projects" UI enabled and a remote CATMAID instance configured in the Setting Widget (see above), it is now possible to select a remote CATMAID instance from the "Source remote" dropdown menu. Alternatively, the local instance can be selected if another project from the same instance should be used.
Next the source project in the selected instance needs to be specified. This list is updated when a different remote instance is selected.
Skeletons from a remote instance are collected through annotations. The respective annotation has to be entered in the "Source skeleton annotation". With the help of the "Preview" button it is possible to load the matching skeletons from their remote CATMAID to inspect if the correct ones are selected.
As a last step for the remote data configuration, the source landmark group has to be defined. This list is updated if the source project changes. Landmarks from this group are mapped to the selected target group. The matching is done by name, i.e. no landmarks can have the same names in a group.
Adding such a transformation adds it to the list at the bottom of the widget, just like with regular transformations and they can be used in the same way. The can be shown in 3D Viewers, superimposed on the Tracing Layer and used in NBLAST queries from the Neuron Similarity Widget (see below).
The new checkbox 'Multiple mappings' displays additional user interface elements...

Assets 2

09 Nov 07:30

tomka

2018.11.09

0e2b5fe

2018.11.09

Contributors: Andrew Champion, Chris Barnes, Tom Kazimiers, William Patton, Eric Trautman

Notes

Python 3 is now required for the back-end. We recommend the use of Python 3.6.
CATMAID's version information is now presented in a different form. It follows
the pattern <base-version>[-dev]-<commit>. The <base-version> is baked
into the source code on a release. The -dev part will only be present if
CATMAID's dev branch is used for deployment. It won't be present for
master branch based setups. The <commit> part is the 10 digit version of
the Git commit ID. This version representation is now also consistent with
what is display in Docker images. In the rare event that no commit information
can be found, <commit> will fallback to "unknown". This version will now
also logged during start-up of the back-end.

Features and enhancements

Connectivity matrix:

The new checkbox labeled "Fractions" in the "Main" tab makes it now possible
to display connector fraction instead of an absolute link number in each cell.
The number of connections from one source row to a target column is divided by
the number of total posynaptic connections that are made to the target
skeleton (column). This makes columns better comparable to each other.
The auto-connectivity matrix of a large set of skeletons can now be exported
as CSV without displaying it using the "Auto-connectivity CSV" button. This
makes it possible to export larger connectivity matrices in a usable format.
The aggregation method for the connectivity count in groups can now be
selected. Available are: sum (default), min, max and average.

3D viewer:

Treenodes that are linked to connector nodes can now be scaled independently
from other node handles using the "Link node scaling" option.
Volume picking is now optional and disabled by default, i.e Shift + Click will
go through volumes. To enable volume surface location selection, the
"Pickable" checkbox needs to be checked.
If Reconstruction Sampler domain shading or interval shading is used, a list
of valid domains and intervals can now be specified in the "Shadings
parameter" tab.
The X/Y/Z rainbow coloring modes are now also available normalized to each
individual skeleton.
Add custom connector coloring. The pre and post colors can be adjusted in the
"Shading parameters" tab.
The X/Y/Z/ axes can now be displayed in the lower right corner using the
"Axes" checkbox in the "View settings" tab.

Reconstruction sampler:

Improve performance of interval length computation
Sampled skeletons can now be split after the users confirm this is their
intention. The split-off part of the skeleton will not contain any sampler
information anymore. Intervals on the split-off part are removed, split-point
crossing intervals are shortened and domain end points will be removed and
recreated as needed.
Sampled skeletons can now be merged into. All samplers that reference the
merged-in fragment are deleted. If the merged fragment is merged outside
of a domain, nothing special is happening---it is a regular merge. If the
merge treenode is in a sampler domain, there are currently three options,
"Branch", "Domain end" and "New domain": 1. Branch: add the new fragment to the
skeleton without changing domain end nodes or intervals. This is only allowed
if the merge target is not the start or end of an interval. 2. Domain end: add
a new domain end node right where the merged in fragment starts. This keeps
the new fragment isolated from the sampled domain. 3. New domain: a new
domain is created for the merged in fragment. This also adds the domain end
node from (2).
Domain completion is now shown in percent along with interval coverage of the
domain in "Interval" step.
Merge decisions can now be limited when the 'merge' or 'merge-or-create' leaf
handling mode is selected. This means a percentage can be specified which
defined below which ratio of extra cable versus interval length the extra
cable should be merged into the previous interval (if possible) rather than a
new interval is created.
Ignored lead segments can now be inspected in more detail using the
"Uncovered domain parts" button the "Interval" step. This will open a dialog
window with a histogram on all ignored leaf segments in the current domain.
Clicking on individual bins will open a treenode table containing the
respective start nodes of the ignored leaf segments. From this dialog, it is
also possible to list all downstream/upstream partners linked to nodes in
ignored segments of a domain.
For samplers using the 'ignore' leaf handing mode, it is now possible to
update this to 'short-interval' mode including the generation of missing
intervals for existing domains. A visual confirmation dialog is shown. To use
this, press the "Set short-interval leaf mode" button in the Domain tab.
The Synapse tab now also shows all leaf nodes of an interval. This makes it
easier to find the places where an interval needs to be continued.

Tracing layer:

Tracign layer: cycle open end in reverse using Shift + Alt + R.
Alt + Click now opens consistently the link type context menu, regardless of
whether a treenode or connector node is currently selected.
The layer options now allow to select a user who's tracing data won't be
fetched from the server. The main motivation is to hide data imported by a
dedicated import user by default and not even fetch it from the server.
Similar to image data mirrors, it is now possible to configure read-only
tracing data mirror servers from which the tracing data will load all data
except for the active node, which will be read from the main server. This is
particularly useful if connecting to the main server from a remote location.
To make this work reliably, it is expected that physical replication is setup
on the database level that mirrors the main server constantly. A separate
CATMAID instance needs to be setup on the mirror server as well. To configure
this, the "Read-only CATMAID mirrors" option in the settings widget can be
used together with the "Read-only mirror index".

Neuron similarity:

The new Neuron Similarity Widget makes it possible to compare neurons to each
other, to neurons transformed based on landmarks as well as to arbitrary point
clouds. Point clouds can for instance be created from light microscopy data.
It creates a similarity ranking based on NBLAST. To open the widget, use Ctrl
- Space or the Open Widget button and then search for "Neuron similarity".
To compare two different objects, NBLAST will compare a query object pairwise
with potential target objects. It iterates over each point of the query
object, find the closest point in a target object and computes a score based
on the distance of these points and their orientation to each other.
This scoring is done based on a scoring matrix, which needs to be created
before any comparisons can be made. Scoring matrices are typically reused and
don't need to be recomputed every time. The "Configurations" tab allows to
create new similarity matrices and lists existing ones. For a new scoring
matrix, probabilities for distance and orientation are computed for both a set
of of similar neurons and a representative sample of random neurons. Both are
combined into a single matrix in which a value of zero makes a particular a
pair of points equally likely to be random or to be a match. Values above zero
make a match more likely. Computed similarity matrices can be visualized by
clicking the "View" link in the Scoring column of the respective similarity
configuration.
With a similarity matrix computed, similarity queries ca be performed from the
"Neuron similarity" tab. In its most basic form, this compares neurons to
other neurons. It is also possible to select transformed neurons and point
clouds as query type or target type in a search. This however requires
additional setup (see below). Query and target skeletons can be selected by
selecting a skeleton source for each. A similarity matrix has to be selected
as well, but all other options have reasonable defaults. A click on "Compute
similarity" queues a new similarity request, which is computed asynchronously.
Once the task is complete its table entry will switch its status to
"complete".
Once completed, the similarity query results can be viewed by clicking on
"View" in the "Scoring" column. This will open a new result window (or
dialog, if selected in the "View" option), which shows the similarity ranking.
To query with or against transformed skeletons, a landamark based "display
transformation" has to be created. To do so, open the Landmark Widget, and
create a transformation in its "Display" tab. Transformations created this way
are selectable from the Similarity Widget, if "transformed skeletons" is
selected for either query or target. Depending on available landmark groups,
this could be for instance a skeleton transformation to its contralateral
location.
The Point cloud tab allows to import individual point clouds, along with an
optional transformation and representative images. It also provides a list of
all point clouds that are visible to the current user. A group visibility
option during import allows to restrict visibility of imported point clouds to
selected groups (which need to be added from the admin interface).
The "Point cloud import" tab allows to import many point clouds at the same
time, optionally transformed and with linked representative images.

Volumes:

The general widget controls are now distributed across tabs.
...

Assets 2

19 Jul 04:39

tomka

2018.07.19

16afda4

2018.07.19

Contributors: Albert Cardona, Andrew Champion, Pat Gunn, Tom Kazimiers, Will Patton, Eric Trautman

Notes

Both the standalone Docker image and the Docker-compose setup can now be updated after a Postgres version change. This makes it possible again to use CATMAID versions after 2018.02.16 with Docker. The documentation has more information.
This is the last CATMAID version with support for Python 2.7. Starting from next version, only Python 3 will be supported.

Features and enhancements

Volume widget:

Add a "List connectors" link to each volume, to show all connectors in a volume bounding box.
The new "Min skeleton nodes" and "Min skeleton length" options allow to further constrain volume based skeleton selections ("List skeletons").
The Connector List widget that is shown when clicking on "List connectors", now supports connector filtering. The volume of the link's row is now automatically set as filter in the new Connector List widget. This means connector links in this table are now shown only if they intersect exactly with the volume (and not only with the bounding box like before).
State saving is now supported.

3D Viewer:

The line width of skeletons can be adjusted again on platforms other than Linux. This Requires "Volumetric lines" in the "View settings" tab to be enabled (it is by default). Unchecking this option brings back the previous line rendering behavior.
The new "Focus skeleton" button in the "View" tab will look at the active skeleton's center of mass from the current camea location.
Volumes can now be smoothed by subdivision. The volume option panel available from the View Settings tab now contains a "Subdivide" checkbox.
A volume's bounding box can now be displayed using the "BB" checkbox that is available for visible volumes in the volume option panel.
New coloring option: X/Y/Z rainbow lookup table coloring for active stack dimensions.
The active node respects now a node radius by default and is scaled to 1.5x its size. This behavior can be disabled using the "Radius adaptive active node" checkbox in the "View settings" tab.
Catalog export: use global neuron name for sorting and display by default.
Catalog export: support for multiple neurons per panel has been added (separate from pinned neurons). The export dialog contains now a "Skeletons per panel" input field. Essentially, the displayed skeletons can bow be iterated in batches.
Catalog export: individual skeleton panels can now also be exported as PNG instead of SVG, which reduces the file size, export time, parsing time. Plus it allows for an exact copy of what is shown in the 3D Viewer.
Catalog export: in orthographic mode it is now possible to export a scale bar on either none, the first or all exported panels.
If the estimated size of the tiles to load for a Z plane exceeds 100 MB, users are asked for confirmation.

Connectivity matrix:

Connector node filters can now be applied using the funnel icon in the widget title bar.
The new "Groups" tab allows to group rows, columns or both by their displayed name.
State saving is now supported.

Reconstruction sampler:

Different leaf handling strategies are now available to be selected for a sampler. The behavior so far (and current default) is to just ignore leaf segments that are shorter than the interval length minus the error margin. Alternatively, it is now possible to merge the leaf segment into the last interval, to create new shorter intervals for the leaf segments or, combining both, it is possible to try to merge it into the last interval and if that's not possible (e.g. on a small twig with no previous interval on the same segment), then create a new short interval. This option is available in the Sampler tab.
Both the domain table and the interval table now show the cable length of each domain and interval in nanometers, respectively. Additionally, the interval tab also show the aggregated cable length of all completed intervals.

Tracing layer:

The new option "Update tracing data while panning" allows to configure weather the tracing data on the layer will be updated when the view is panned around.
A set of new options allows now to configure a "tracing window", which will restrict tracing data loading by allowing it only in view centered rectangle. Width and height can be configured independently. This is useful for remote review and tracing.
A new option to show only the N largest skeletons in a field of view is now available for the layer settings (and the API).

Miscellaneous:

The behavior of the Ctrl modifier on section navigation with , and . can now be inverted using the "Animate section change by default" option in the Settings Widget.
Connectivity widget: annotations can now be used for additional filtering per partner table.
Split/merge dialog: the node count of the individual parts is now shown when hovering over their cable length information.
Review widget: the user who created the last node of each segment is now displayed in the review table. This allows to focus review on segments not created by oneself.
A skeleton cable length limit can be set so that a warning is displayed if a change to the skeleton morphology results in a cable length larger than the limit. This is available in the Warnings section of the Settings Widget.
Node filters: if a neuron name is provided for a rule, the rule is now valid for all neurons with neuron names that include the provided name and not only exact matches.
Neuron name display: neighboring duplicate name components are now removed by the default. This setting can be adjusted from the Annotation sections in the Settings Widget.
Boolean parameters for API endpoints are no case-insensitive, allowing the use of regular boolean values in requests from Python.
CLI exporter: the new --excluded-annotation parameter can be used to exclude neurons from the export based on annotations.
CLI exporter: placeholder nodes are now exported as completely new skeletons that are not linked to their original skeleton, unless it is part of the set of exported skeletons or the --original-placeholder-context flag is provided.

Bug fixes

Measurements table: column headers in CSV export are now quoted.
Export management command: the default output filename can be used again.
Export management command: class instances and links of skeletons and neuronsa are now exported alongside treenodes.
Neuron name service: missing naming components don't lead to removal of all whitespace between neighbors anymore.
Initial skeleton coloring of merge dialog when merging from smaller into larger skeletons is fixed.
3D viewer: refreshing the active skeleton does not refresh all skeletons anymore.
3D viewer: connector restrictions like "show only shared connectors" now respect the pre/post visibility toggles in the Selection Table.
3D viewer: the initial text scaling for label text is now correctly set again.
Reconstruction sampler: during interval creation preview, only intervals from the currently active domain are now shown.
Graph Widget: when subscribed to other widgets, their skeletons are not removed anymore from the Graph Widget when the other widget is closed.
Neuron Search: annotation data range can be used again.
The Strahler number computation no correctly increases the Strahler number when two more children have the local maximum number rather than requiring all children to share the same number.
Radius editing: using undo (Ctrl + Z) after editing the radius of a node works message.

API changes

Additions

POST /{project_id}/treenodes/compact-detail:
Retrieve treenode information for multiple nodes in the format of regular node
queries, accepts a parameter "treenode_ids".
GET /{project_id}/treenodes/{treenode_id}/compact-detail:
Retrieve treenode information for a single node.
DELETE /{project_id}/landmarks/{landmark_id}/groups/{group_id}/:
Delete landmark location links to the same locations from a landmark and
group.
PUT /{project_id}/landmarks/groups/links/:
Add new links between groups, e.g. "adjacent_to".
DELETE /{project_id}/landmarks/groups/links/{link_id}/:
Delete a single landmark group link.
GET /{project_id}/landmarks/groups/{landmarkgroup_id}/transitively-linked:
Get a list of landmark groups that are linked to the referenced group using a
passed in relation, respects reciprocal relations.
POST /{project_id}/landmarks/groups/materialize:
Create pairs of landmark groups along with their landmarks based on a simple
description.
GET /{project_id}/skeletons/cable-length:
Get the cable length for multiple skeletons using the skeleton_ids parameter.
GET /{project_id}/skeletons/{skeleton_id}/cable-length:
Get the cable length for a single skeleton.
GET /{project_id}/skeletons/in-bounding-box:
Get IDs of all skeltons that intersect with the passed in bounding box.

Modifications

POST /{project_id}/annotations/query-targets:
Accepts now a "annotation_reference" parameter which can either be 'id'
(default) or 'name'. If it is set to 'name', all annotation references in
annotated_with, not_annotated_with and sub_annotated_with are interpreted as
annotation names instead of IDs.
POST /{project_id}/skeleton/split:
Returns now also the split location as fields x, y and z.
POST|GET /{project_id}/node/list offers a new optional parameter
"with_relation_map", whi...

Assets 2

16 Apr 04:39

tomka

2018.04.15

477b88f

2018.04.15

Contributors: Albert Cardona, Andrew Champion, Chris Barnes, Rob Court, Tom Kazimiers

A recording of a presentation discussing the changes in this release can be found here.

Notes

Requires a virtualenv update.
A new management command "catmaid_find_node_provider_config" is available,
which can be used to compare different node providers on existing data, which
is useful to configure the NODE_PROVIDERS setting.
The Docker images now support the options CM_HOST and CM_PORT to configure
where uwsgi is listening.
When using the Javascript console, node positions returned from tracing
overlays are now in project space coordinates (physical nm), not stack space
voxel coordinates. Transformation to project space is no longer necessary.
Postgres 10+ is now required.

Features and enhancements

Layouts:

A new "Layouts" menu is shown in the top bar when a project is opened. It
allows to save the current window layout under a name, it provides an option
to close all widgets and will show all available layouts as menu entries.
Layouts store window arrangement, window sizes, tabs and subscriptions.
Saved layouts can also be manually configured from the "Custom layouts"
setting in the Settings Widget.
Tabbed windows are now supported in layout specs by using "t([a, b, c])" where
a, b, c or any other number of elements can be children of the tabbed window.

Landmarks:

Support transformation of nodes that cross space between landmark groups and
even reach into target groups. This allows to e.g. transform skeletons that
cross the midline. Doing this is enabled by default, but can be disabled
through the "Interpolate between groups" option.
Virtual transformed skeletons are now also shown on a separate layer in all
open Stack Viewers. Nodes of those skeletons can currently not selected. To
disable the Landmark Layer, uncheck the "Show landmark layers" checkbox in the
Display tab's button panel.
The new "Edit landmark" tab provides a simpler interface to add new landmark
locations to landmark groups. If the option "Update existing landmark
locations" is selected, new landmark locations will replace existing ones
shared between the provided landmark and the selected group. The lower
section of this tab allows to edit links between landmark groups, which can
be used for rule based display transformations.
As an alternative to selecting a target landmark group explicitly to create a
display transformation, it is now possible to instead select a target relation
in the Display tab. Doing so will automatically create all display
transformations from the source group to all landmark groups transitively
linked to the source group using the selected relation. The reciprocity of
relations is respected.
The new "Create groups" tab provides an option to create landmark groups along
with required landmarks from the bounding boxes of two volumes. This allow to
quickly create simple landmark group mappings.
The color and extra scaling for nodes on Landmark Layers can be adjusted from
the widget.
All 3D Viewers are now enabled by default as transformation display target.
State saving is now supported.

3D Viewer:

A scale bar can now be displayed when in orthographic mode. The scale bar
can disabled via a checkbox in the View Settings tab.
PNG and SVG exports offer now a filename input field.
Loaded volumes are now stored along with their styling in the widget state.
Landmark groups are now show with landmark name labels. This can be disabled
from the landmark menu.
Text scaling can be adjusted from the View tab.
The width and height of animation exports are now restricted to even numbers.
This is required by the H264 codec we refer to in our documentation.

Graph widget:

The old "Graph" tab was split into two: "Nodes" and "Edges", each with the
corresponding functionality.
New feature: color edges with the same color as the source node, the target
node, or the general color specified in the "Properties" menu (from the "Main"
tab). See the "Edges" tab.
New feature: change the arrow shape to a circle, diamond, tee, etc. The new
"Set" button in the "Edges" tab applies the change to selected nodes.
New feature: the new "Selections" tab can record sets of nodes, stored as a
named selection. Then these can be selected or deselected. The "Select all"
button selects all nodes from all created selections.
Basic state saving is now supported.

Reconstruction Sampler:

Intervals are now displayed with only two colors by default, it makes
distinguishing many intervals easier. The previous multi-color mode can be
reactivated from the preview window.

Tracing general:

The settings widget allows now to configure a "fast split mode" and a "fast
merge mode" to allow particular groups of skeletons to be split and merged
without confirmation. Similarly to Visibility Groups, these Fast Split/Merge
Groups can be defined in terms of a universal match (all skeletons), a
required meta-annotation or a creator ID. In fast split mode, all annotations
from the split skeletons are copied over to the split off part. In fast merge
mode, all annotations are taken over from a skeleton merged in without
confirmation.
The P shortcut (peek) will now show the closest skeleton to the cursor in
all open 3D Viewers. To show the active skeleton use Shift + P.
A move/navigation mode can now be used using the new (third) button in the
tracing tool bar. If enabled, no mouse based node actions will be performed
anymore. Left mouse button clicks/movements are handled like right mouse
button clicks/movements.
The Split Neuron Dialog has now "select all" checkboxes for annotations.

Statistics widget:

Statistics widget: The new option "All" in the time unit selection control
allow to aggregate user data for the whole time range.
The number of newly created treenodes is now displayed alongside the cable
length in the contribution table.
Each user can now be included in an aggregate statistics row at the end of the
table by checking the checkbox in front of the username.
Add extra Refresh button to top bar.

Volume widget:

The new link "List skeletons" in each volume table row allows to open a new
Selection Table containing all skeletons the bounding box of the respective
volume intersects. Due to large numbers of skeletons in bigger volumes, this
is currently mainly useful for smaller volumes.
When creating box volumes, the new button "Define cube at
current location" allows to conveniently create a cube with a configurable
edge length at the current location.

Miscellaneous:

Basic Search: allow search for treenode IDs and connector IDs.
Project administration: selected projects can now be exported as JSON or YAML
file using the respective action from the drop-down menu.
In addition to the '/apis' endpoint, the API documentation is now also
available as part of the general documentation.
Connectivity Widget: partner header indexes can now optionally be replaced
with the neuron name and rotated by 90 degrees.
The Keyboard/Mouse Help Widget has now a text filter, which allows to show
only items containing a particular text.
The Tracing Overlay has a new setting "Allow lazy node updates", available in
the Settings Widget. If enabled (default), stack viewers can skip node updates
if a change didn't occur in the Viewer's field of view. This is useful when
e.g. the Connector Viewer is open while tracing, because not all viewers get
update due to the addition of a node.
Selection Table: Skeletons can now be imported from CSV files.
The status bar shows now both stack space and project space coordinates of the
mouse cursor.
The skeleton projection layer works now with orthogonal views.
Detailed review colors are now enabled by default. To get the old behavior
back, adjust your settings (admins can do this for the whole project or
server).
Stacks can now be created whose planar axes have anisotropic resolution.
The stack viewer will display these stacks correctly by scaling tiles
separately along each axis. The tracing overlay is also compatible with these
stacks in all orthoviews.
Added H2N5 tile source type.
Due to the new skeleton summary tables, some APIs are much faster now:
1. Obtaining review info skeletons with /{project_id}/skeletons/review-status,
2. API and connectivity information through /{project_id}/skeletons/connectivity,
3. Listing skeletons with a minimum node count with /{project_ids}/skeletons/
4. Getting skeleton node count with /{project_id}/skeleton/{skeleton_id}/node_count.
In consequence the following widgets became faster too: Connectivity Widget,
Selecting skeleton counts in the Statistics Widget, Review count based
coloring (e.g. in Graph Widget). Opening the Merge Dialog is also faster due
to this change.

Bug fixes

Reconstruction Sampler: 3D visualizations for interval preview and color mode
were sometimes wrong and showed additional intervals. This is fixed now.
Actual interval boundaries were not affected and are correct.
Reconstruction Sampler: Individual intervals are not silently deleted anymore
if referenced start or end node is deleted. As consequence interval start and
end nodes can't be deleted anymore.
Measurement table: no error is shown anymore after merging two listed
skeletons.
Radio button drop-downs and checkbox drop-downs now hide on a mouse click
outside of the contr...

Assets 2

16 Feb 23:54

tomka

2018.02.16

6f33ead

2018.02.16

Contributors: Albert Cardona, Andrew Champion, Chris Barnes, Tom Kazimiers

Notes

Three new OS package dependencies have been added (due to a Django framework
upgrade), make sure they are installed:

sudo apt-get install binutils libproj-dev gdal-bin
Python 3.6 is now supported. Make sure to update your settings.py by replacing
the line

COOKIE_SUFFIX = hashlib.md5(CATMAID_URL).hexdigest()

with the following line:

COOKIE_SUFFIX = hashlib.md5(CATMAID_URL.encode('utf-8')).hexdigest()
A virtualenv upgrade is required. To correctly install one updated dependency,
the django-rest-swagger Python package has to be removed first from from the
virtualenv, before the virtualenv is updated:

pip uninstall django-rest-swagger
pip install -r requirements.txt
requirements files now inherit from each other: -test.txt includes
the production requirements, -dev.txt includes test (and therefore
production) and doc.
CATMAID extensions no longer require users to manually edit their
INSTALLED_APPS in settings.py. Remove if they are already in use.
The NODE_PROVIDER settings variable (settings.py) is replaced with the
NODE_PROVIDERS variable. The new variable takes a list of node provider names,
which are iterated as long as no result nodes are found. Replace the former
single string value with a list with this name as single element, e.g. if
the current setting reads NODE_PROVIDER = 'postgis2d', replace it with
NODE_PROVIDERS = ['postgis2d'].
Three new types of cache backed node providers have been added: cached_json,
cached_json_text, cached_msgpack. Tests suggest that cached_msgpack is the
fastest.

Features and enhancements

Reconstruction Sampler:

3D previews now allow to use a white background and toggle meta spheres.
To match a given interval length, the Reconstruction Sampler will now create
new nodes during interval creation. This allows better sampling on long
straight lines without intermediate nodes. To disable this behavior uncheck
the "Create bounding nodes" setting. Newly created nodes are automatically
labeled with the "sampler-created" tag.
By adjusting the new interval "max error" setting, it is possible to only
create bounding nodes (if enabled) if the closest existing node is farther
away from the ideal location than the max error value. This allows better
reuse of existing nodes.
Created interval boundary nodes are also automatically deleted on sampler
removal, if possible. If a created node has not been altered in any way during
its life time and it still forms a straight line with its parent and single
child, it can be removed. Otherwise the node is kept.
A new "Partner" step has been added. After a connector is selected at random,
users are now asked to add all partner nodes to the synapse and let the widget
choose a partner site for them.
Sections that should be interpolated (e.g. because of large tracing data
shifts due to image data shifts) are now respected and not over-sampled
anymore.
Completed intervals can now be optionally excluded from random selection.

Node Search:

Tagged connector nodes are now shown in search results.
Annotation search results allow now to open the Neuron Navigator page for the
respective annotation.

Graph Widget:

It is now possible to show links of all connector types. By default, only
synaptic connectors are available, but this can be changed using the "Link
types" drop down control in the Graph tab. There also the edge color for
particular link types can be set.
The edge color button in the option dialog has been replaced with a link type
drop down which allows to set color and visibility per connection type.

3D Viewer:

Stored interpolatable sections are now loaded by default. Additional project
space sections can be configured from the Skeleton filters tab.
Custom tags can now also apply to predefined labels like TODO or
uncertain_end.

Miscellaneous:

The maximum number of frames per second rendered by a stack viewer window can
now be adjusted through the Settings Widget > Stack view > Max frames per
second.
The Open Widget dialog is now only opened with Ctrl + Space when no other keys
are pressed. This allows to use key combinations like Ctrl + Shift + Space + >
to browse smoothly through the image stack while hiding the tracing layer.
The active node can now be moved in Z by holding the Alt key and
using ,/.. The stack viewer follows the node.
Filters: a Reconstruction Sampler Domain node filer is now available. The
required node ID can be optained from the Reconstruction Sampler Widget.
Neuron History: a lower and upper bound for the time window in which changes
are respected and counted can now be set.
On CATMAID front-pages (project overview) the keys 0-9 can now be used to open
the n-th stackgroup or stack, stackgroups have precedence.
Sections in which nodes should be interpolated for various displays can now be
persisted in the database using the "interpolatable sections" data model from
the admin interface.

Bug fixes

Reconstruction sampler: Strahler shading color updates work now reliably.
Reconstruction sampler: connectors in neighboring intervals are now not
included anymore in connector listing.
3D Viewer: SVG export styles are correctly applied again, the exported data
isn't transparent anymore.
3D Viewer: custom label matches are now always given precedence when multiple
labels are matched on a node.

Assets 2

07 Dec 06:01

tomka

2017.12.07

63e464f

2017.12.07

Contributors: Albert Cardona, Andrew Champion, Chris Barnes, Tom Kazimiers

Notes

PostgreSQL 9.6 and Postgis 2.4 are now required.
A virtualenv update is required.
New settings field: CROPPING_VERIFY_CERTIFICATES. This controls whether SSL
certificates should be verified during cropping and defaults to True.
The NODE_LIST_MAXIMUM_COUNT setting can now be set to None, which disables
node limiting. If node count limiting isn't really needed in most cases, this
can slighly improve query performance.
The file UPDATE.md will now list all administration related changes for each
release, including required manual tasks. These changes also continue to be
part of this change log.
Requires running of: manage.py catmaid_update_project_configuration
Tracing data is now by default transmitted in a binary form, please make
therefore sure your web-server applies GZIP not only to the "application/json"
content type, but also to "application/octet-stream". For Nginx this would be
the gzip_types setting.

Features and enhancements

Node filters:

A new node filter called "In skeleton source" has been added. This allows to
add a filter which only allows nodes of a particular set of skeletons,
accessed through a skeleton source. This can be useful to look at connectivity
of a particular set of neurons. It can of course be combined with other
filters like volume intersection.
New filter: Pruned arbor, which keeps the arbor proximal to the (tagged) cut
points, discarding distal subarbors.

Connector table:

Node filters are now supported.
Attachment connectors are now supported.

Landmarks:

The new "Landmark Widget" allows to create landmarks, form groups of them and
use these groups to virtually transform skeletons from one landmark group into
another one. This can be used e.g. to find homologues neurons. Virtual
skeletons can currently displayed in the 3D Viewer.
Defining landmarks and their groups is done in two stages: 1. define abstract
landmarks and group those abstract landmarks. 2. link locations to individual
landmarks and their presence in a group.
Abstract landmarks represent e.g. a feature that can be found at multiple
places in the data set, like the entry point of a particular neuron on the
left and right side of a Drosophila brain. The left and right side of the
brain would then be represented as groups, each one having expected abstract
landmarks as members.
Creating new abstract landmarks and groups can be done through the respective
"Name" fields and "Add" buttons in the Landmarks tab. To link an abstract
landmark to a group, click on either an existing landmark group member in the
landmark groups table or on the "(none)" placeholder. The newly opened dialog
will allow to associate landmarks with groups. Landmarks can be member of
multiple groups.
Linking locations to landmarks and the groups they are member of is done by
using the right click context menu on either an existing location in the ...

Assets 2

03 Oct 03:39

tomka

2017.10.02

e66585f

2017.10.02

Contributors: Chris Barnes, Dylan Simon, Albert Cardona, Andrew Champion, Tom Kazimiers

Notes

A virtualenv update is required.
This release adds optional statics summary tables, which can increase the
performance of project/user statistics significantly, but statistics will also
be correct without them. The additional table keeps aggregated information
about various user actions. To initialize this table, the following
manangement command has to be run after the migration:

./manage.py catmaid_populate_summary_tables

To maintain good performance, this command has to be run regularly, e.g.
through a cron job or Celery every night. Because summary updates are
incremental by default, they don't take much time to update.
CATMAID's Docker images changed: The existing catmaid/catmaid image is now
only a base image that is used for a simple standalone demo image, available
as catmaid/catmaid-standalone. Additionally, the base image is used in a new
docker-compose setup, which can be used if persistent data is required. The
documentation has been updared with all the details.
The Docker default port is now 8000 for both regular setups and docker-compose
setups.
The data view "legacy project list" has been renamed to "simple project list".
It now supports separate project title and stack title filters, which can be
preconfigured using the data view config options "projectFilterTerm" and
"stackFilterTerm". For both simple terms and regular expressions can be used.
The filter input boxes can optionally be hidden by setting the "filter" config
property to false.
Running periodic tasks is now easier and a default setup for cleaning up
cropping data at 23:30 and update tracing project statistics at 23:45 every
night is now available. All it needs is to run a Celery worker and a Celery
beat scheduler. The documentation has more details.
The cropping output file file name prefix and file extension can now be
specified from settings.py. The defaults are:

CROPPING_OUTPUT_FILE_EXTENSION = "tiff"
CROPPING_OUTPUT_FILE_PREFIX = "crop_"
CATMAID can now make use of a ASGI server to utilize WebSocket connections.
Such a server (e.g. Daphne) needs to handle URLs that start with /channels/.
Currently only messages can be updated this way, which already removes many
requests for setups with many users. This allows for instance an immediate
feedback after a cropping a sub-stack finished without requiring the client to
check for new messages every minute.

Features and enhancements

Project/user statistics:

The widget should now be much faster.
Import actions are not counted anymore by default. The "Include imports"
checkbox can be used to change this.
State saving is now supported.

Neuron history:

Events by different users are now collected in separate bouts to attribute
parallel user activity. To restore the previous behavior (users are ignored)
the "Merge parallel events" checkbox can be checked.
State saving is now supported
Individual neurons can be removed with the help of an "X" icon in the firs
column.
A "total time" column has been added, which aggregates time across all active
bouts formed by both tracing and review events. Since events are binnned in
bouts, the total time is not just the sum of both tracing time and review
time.

Node filters:

A Reconstruction Sampler interval can now be used as a node filter. This
allows e.g. reviewing only an interval or look only at the connectivity of
the interval.

Graph Widget:

Edge labels can now take different forms and can be configured in the
properties dialog. There are two new label options available: "Fractions of
outbound connections" and "Fractions of inbound connections". Instead of an
absolute number they display the relative fraction. This works for both
regular connections and connections involving groups.

Reconstruction sampler:

Intervals for a particular interval length can now be previewed before
creating a new sampler. To do so, use the "Preview intervals" button in the
Sampler tab.
Interval reviews can now be initiated directly from the interval step table.
It will open a new review widget with pre-set interval filter and added
skeleton. The same is possible from the synapse workflow step using the
"Review interval" button.

Layouts:

The Stack View section of the Settings Widget allows now the configuration of
a list of default layouts that can be applied to newly opened stacks and stack
groups. Layouts are useful for having a reasonable default configuration of a
newly opened CATMAID workspace including stacks and stack groups.
Layouts mimic nested function calls and are constructed from v(a,b) and h(a,b)
elements for vertical and horizontal splits, o(a) for optional windows, where
a and b can each be other v() or h() nodes, one of [XY, XZ, ZY, F1, X3D] or any
quoted widget handle (e.g. "neuron-search", see Ctrl + Space). At the moment,
in o(a), "a" can't be XY, XZ or ZY. To reference the 3D Viewer use X3D.
By default only one layout is available: organize XY, XZ and ZY in four
quadrants using one additional window (if non available the help page). This
is its specification: h(v(XY, XZ), v(ZY, o(F1))). With h() and v(), horizontal
and vertical splits are declared, respectively. With o(F1) a help window will
be opened as fourth window if not already another window exists.
Useful for organizing orthogonal views in a custom way and to create default
workspaces. For instance, to always open a Neuron Search widget right to the
a single XY view stack, the layout can be used: h(XY, "neuron-search").

CATMAID extensions:

As well as supporting standalone 3rd party static files, CATMAID now supports
fully-featured extensions, which can include database models, API endpoints,
and tests as well as static files.
CATMAID extensions are Django apps which follow a particular layout, and can
be installed with pip
The goal is for CATMAID extensions to be reusable and interoperable
between versions and installations of CATMAID, reducing the need to fork it
and keep the fork updated in parallel.
More details can be found in the docs.

3D viewer:

Volumes can now be filtered in the volume drop-down menu with the help of a
built-in text box.

Miscellaneous:

The Open Widget dialog now displays a table that includes more information on
widgets. Instead of the previously used auto-completion this table is now
filtered.
If multiple tile layers are used, the stack viewer layer settings (blue white
box in lower left corner) allow to set which stacks should be respected if
broken sections are skipped. This can be done through the "Skip broken
sections of stacks" drop-down menu. Whether layers added after the first layer
should be respected by default when checking for broken sections can be set in
the Settings Widget in the "Stack view" section.
The dialog to add a new annotation displays now the existing annotations as
well.
Tabbed windows: changing window aliases are now reflected in tab headers.
The performance of node creation and deletion operations has been improved by
preventing full node updates after these operations.
Widgets with state saving support now also support removing previously saved
states through a button in the widget controls available through the window
icon in the widget title bar.
The Selection Tool (the first icon in the top bar) has been removed, because
it didn't provide any functionality. It is replaced by an icon to show the
"Open Widget" dialog, which can otherwise be shown using Ctrl + Space.
When splitting a skeleton on a virtual node, the virtual node will now only be
instantiated if the user presses OK in the dialog, canceling won't cause a
change of the virtual node.
The default for hiding tile layers if the nearest section is broken (instead
of showing the next available) can now be configured from the Settings Widget
in its Stack View section.
plotly.js, a d3-based plotting library, is now available within CATMAID,
making it much easier to generate common plots.

Bug fixes

Review widget: moving in reverse direction from a virtual node doesn't show
error anymore.
Review widget: the Shift + Q key combination to select the next unreviewed
node in downstream direction is respected again.
Review widget: Shift + W and Shift + Q work now correctly if the first
unreviewed node is a virtual node.
Key combinations involving the Alt key were not respected on Mac OS. This is
fixed now.
Reconstruction sampler: the list of connectors in the selected interval of the
synapse workflow step is now complete when refreshed.
3D viewer: the error shown when changing skeleton properties with an active
connector restriction is fixed.
The neuron history widget now calculates both tracing time and review time
correctly.
'Peek skeleton' (P key) works again.
The Split Skeleton Dialog updates the embedded 3D view again after all
skeletons are loaded.
Former partner nodes of deleted connector nodes can now be deleted without an
additional tracing layer update.
The Skeleton Projection Layer can now use the skeleton source colors and other
colors source again.
The simple front end data view's project filter works again
Skeleton source subscriptions: opting in and out of applying color changes in
source skeletons to existing target skeletons works again.
Dragging nodes in stacks with smaller resolutions (larger nm/px) doesn't
require large drag distances anymore, ...

Assets 2

28 Jul 04:52

tomka

2017.07.28

f2c587c

2017.07.28

Contributors: Chris Barnes, Albert Cardona, Tom Kazimiers, Daniel Witvliet

Notes

Prepared statements can now also be used together with connection pooling.
A virtualenv update is required.

The following lines have to be removed from settings.py,

import djcelery
djcelery.setup_loader()
INSTALLED_APPs += ("kombu.transport.django")
BROKER_URL = 'django://'

Two new treenode and connector node providers have been added: postgis2dblurry
and postgis3dblurry. They works like the regular postgis2d and postgis3d node
providers except that they allow more false positives, because edges are only
tested for bounding box intersection with the query bounding box. Depending on
the dataset, this can help performance but might require a larger node limit.

Features and enhancements

Synapse Fractions:

New button "Append as group". Multiple neurons will be shown in a single column.
New UI functions to set the synapse confidence.
Default to Upstream (fractions for input neurons).
Shift-click to toggle selected state of partner neurons or groups, and then
push 'J' to create a new partner group.
X axis labels can now be rotated, from the Options tab.
Node filters can now be applied to filter valid connector links.

3D Viewer:

New skeleton shading modes:
- "Axon and dendrite": like the coloring mode "Axon and dendrite", but using
  shading like in the "Split at active node" mode.
- "Single Strahler number": color branches of a specific Strahler number
  in full color, and darken everything else. The Strahler number is specified
  in the tab named "Shading parameters".
- "Strahler threshold": color branches of a Strahler number equal or higher
  than the number specifed in the Stahler number field ("Shading paramters"
  tab) in full color, darken all others. When inverted, show in full color
  branches with a Strahler number strictly lower than the specified one.
The Z plane display now supports stack viewer layers. If multiple layers are
shown in a stack viewer, its Z plane will render all visible layers on top of
each other.

Review widget:

Large neurons are handled better, rendering tables with many segments is
now faster.
The set of displayed columns can now be constrained with the help of the new
"Visible reviewers" setting in the widget. Available options are "All", "Team"
and "Self".
A new option to automatically scroll the active segment into view has been
added ("Scroll to active segment"). With this enabled (default), when a node
of a segment of the reviewed skeleton is selected in the tracing layer, the
review widget will scroll the segment into view.
State saving is now supported.

Node filters:

A Strahler number filter has been added to only show nodes with a Strahler
value below/same as/above a user defined number.

Neuron Search:

It is now possible to use the "not" operator with neuron name and annotation
search criteria.
Both neuron name and type columns can now be sorted.
Filters are now supported. Only neurons will be shown of which at least one
node is in the node filter result set.

Neuron History:

This new widget can be opened using Ctrl+Space together with the key
"neuron-history" or through a button in the Neuron Navigator.
For all neurons added to this widget, time related information is presented.
Currently, the following is calculated: Tracing time, review time, cable
length before review, cable length after review, connectors before review and
connectors after review.
The components that contribute to the tracing time can be adjusted with the
"Tracing time" drop down menu.

Reconstruction Sampler:

This new widget can be opened using Ctrl + Space and the keyword
"reconstruction-sampler". It allows to target reconstruction effort based on
the spatial sampling of a skeleton. This is mainly useful for large neurons
that can't be quickly traced to completion. The widget is organized as a
workflow that prevents skipping steps.
To keep track of reconstruction progress, a so called sampler is created
for a skeleton of interest. This skeleton is typically the backbone of a
larger neuron. The sampler keeps track of some global properties for the
sampling. Once created, a sampler can be "opened" either by clicking "Open" or
by double clicking the respective table row. Also note that with a sampler
attached to a skeleton, the skeleton can not be deleted. If a skeleton should
be deleted, delete its samplers first.
Once opened, a sampler allows creating so called sampler domains, which are
regions on the skeletons which should in principle be considered for sampling.
Topological and tag based definitions of sampler domains are possible. Created
samplers can be opened through a click on "Open" or a double click on the
table row.
Once opened, sampler domains can be further divided in so called sampler
intervals. To do so the "Create intervals" buttons has to be pressed. These
have initially all the same length (defined in sampler) and no branches.
Intervals are meant to picked at random through the respective button.
With an interval selected, the goal is now to reconstruct it to completion
with all branches and connectors. CATMAID will show a warning when moved out
of the interval. The workflow page will show both input and output connectors.
As soon as the interval is reconstructed completely, a synapse can be picked
at random from which the next backbone can be reconstructed. Once the backbone
is found and reconstructed, the sampling can start over. Alternatively,
another interval in the original skeleton can be selected at random.

Miscellaneous:

Split/Merge dialog: the node count for both respectively remaining/new and
remaining/old are now on top of the annotation lists.
The General Settings section of the Settings Widget now provides an option to
disable asking for confirmation when the active project is closed.
Tile layers can now be configured to not automatically switch to the next
accessible mirror if the present one is inaccessible. This is useful for some
custom mirror use cases and can be set through the "Change mirror on
inaccessible data" checkbox in the layer settings.
Empty neuron name pattern components are now by default trimmed automatically.
Empty components at the pattern endings won't leave any white space behind.
Empty components between other components are allowed one space maximum (if
there were spaces before). The difference is visible e.g. in the Graph Widget.
URL-to-view links now allow selected connector nodes.
Adding a custom mirror will now by default disable automatic mirror switching
of the respective tile layer. Custom mirrors often come only with a subset of
the data, which currently triggers a mirror switch by default if some tile is
not accessible.
The tracing layer is now faster with creating skeleton nodes, connectors and
partner nodes by avoiding unneeded node updates.
Export widget: exporting neurons as NRRD files is now supported. The NAT R
package is used for this. Check documentation for setup.
The tracing tool has now a button to refresh CATMAID's caches for neuron names
and annotations. This can be used to update neuron names with components that
were changed by other users. Additionally, such a cache update is performed
automatically once every hour.
URLs to a particular view work now also with a location and a skeleton ID only
rather than requiring a node ID always.
Using "Shift + x" and "Ctrl + x" will now activate a checkbox selection mode
and the cursor turns into a crosshair. In this mode one can draw a rectangle
everywhere on the screen and all checkboxes that are behind the rectangle will
be toggled using "Ctrl + x" (turned on if off and vise versa) or checked with
"Shift + x". Either releasing the drawing mouse click or a second "Shift + x"
or "Ctrl + x" will deactivate the tool again.
All skeleton source widgets (typically those with a skeleton source drop-down
menu) now support copy and paste of skeleton models. Pressing "Alt + Ctrl + c"
in an active widget will copy its skeletons (along with colors) into a
clipboard. Pressing "Alt + Ctrl + v" in another widget will then paste those
models into the now active widget.
When using the Z key to create new nodes, existing nodes are not as easily
selected from a distance anymore. Before, the radius around a click event was
too large and the closest node around the click was selected. Other nodes will
now only be selected if the mouse cursor is close to their circle graphic on
screen.

Bug fixes

Graph widget: re-layout works again.
The Docker image can be used again without requiring a manual restart of the
uWSGI server.
Zooming in using the "=" key and using Shift + Z to join and create nodes both
work again.
Links inside the simple Search Widget can be clicked again.
Back-end calls with large parameter lists can be used again (e.g. review
status in the Connectivity Widget).
Highlighted tags in the Neuron Dendrogram widget are now correctly displayed
next to the nodes they belong to. Before, each tagged node had all selected
tags shown next to it, regardless of whether it would be tagged with them.
3D viewer: the textured Z section rendering in the 3D Viewer now respects the
mirror setting from the tile layer.
3D viewer: a memory leak in displaying Z sections with images has been fixed.
This allows long continued Z section browsing without crashing.
3D viewer: the accurateness of object picking on neurons and location...

Assets 2

Releases: catmaid/CATMAID

2021.12.21

2021.12.21

Notes

Features and enhancements

2020.02.15

2020.02.15

Notes

Features and enhancements

Permissions:

Node filters:

CLI importer:

CLI exporter:

Project statistics:

Tracing tool:

Tracing overlay:

2019.06.20

2019.06.20

Notes

Features and enhancements

2018.11.09

2018.11.09

Notes

Features and enhancements

2018.07.19

2018.07.19

Notes

Features and enhancements

Volume widget:

3D Viewer:

Connectivity matrix:

Reconstruction sampler:

Tracing layer:

Miscellaneous:

Bug fixes

API changes

Additions

Modifications

2018.04.15

2018.04.15

Notes

Features and enhancements

Layouts:

Landmarks:

3D Viewer:

Graph widget:

Reconstruction Sampler:

Tracing general:

Statistics widget:

Volume widget:

Miscellaneous:

Bug fixes

2018.02.16

2018.02.16

Notes

Features and enhancements

Bug fixes

2017.12.07

2017.12.07

Notes

Features and enhancements

2017.10.02

2017.10.02

Notes

Features and enhancements

Bug fixes

2017.07.28

2017.07.28

Notes

Features and enhancements

Bug fixes