New update core

[Durman]

Addressed problem description

#3077 #3058

Previous attempts:
#2380 #2393

After a year from previous attempt I'm trying to handle this task again. I have made some code and it looks like the approach is correct. Actual result is a node tree is never recalculate everything (only if base node for all other was not changed). Also there are some extra logic which prevents from useless updates.

In general process can described in next steps:

• get signals from Blender
• analyze them with using tree reconstruction
• translate them into Sverchok events
• correct tree reconstruction according this events
• get outdated nodes from tree reconstruction
• try update given nodes
• colorize nodes after all

I splited all code into several modules:

• tree_recnstruction.py

This data structure is coping data structure of existing Sverchok node trees
It helps to analyze changes which happens in a node tree
Also it is used for walking on tree for searching nodes which should be recalculated
It is always consistent to Blender trees
So it can be used for searching connections between nodes
The only problem is it has not any information about sockets for simplicity
They can be added in future

• hashed_tree_data.py

This module keeps node and links instances of bpy.types.NodeTree objects
They are available by their IDs
It used for converting nodes of Reconstruction tree to Blender nodes
But also can be used for any other purposes when collection of node_tree.links and node_tree.nodes are unchanged
It is known that keeping such links can lead to Blender crash
So all this data is refreshing if collection of nodes or links was changed
This module can be adopted to getting links of a socket for constant time

• events.py

It used for getting signals of Blender, analyze them, keeping tree reconstruction, updating and colorizing nodes.

If someone would like to test keep in mind next things:

• Many things are broken now
• node statics and colorize panels can help to understand what is going on
• also extra info can be get from console in debug mode
• not all node properties works correctly
• keep in mind what does not still work according list below

Panels:

Preflight checklist

• Code changes complete.
  • handle reroutes
  • handle frame nodes
  • handle show in viewport node tree property
  • handle freeze whole tree node tree property
  • handle update current tree operator
  • handle update all trees operator
  • handle macros efficiently
  • handle post load signal
  • support monads
  • support monads loops
  • support color presets
  • handle animation event
  • handle animate node tree property
  • handle undo event
  • handle property update events with costume update function
  • handle nodes_dict
  • open examples
  • all nodes should calculate the data for all output nodes
• Code documentation complete.
• Documentation for users complete.
• Manual testing done.
  • open existing layouts
  • open existing layouts with deprecated nodes
  • export/import json
• performance tests
• Unit-tests implemented.
• cleaning outdated code
• Ready for merge.

[vicdoval]

Does this override all the last changes to the update system?

[Durman]

Yes and even more. It overrides existing color system, update core of monads, refreshing IDs, recreating node trees, handle property updates and I think it is not the full list. I have tried to impact as less as possible but in such matters it is not as simple.

In this way this PR is quite revolutionary. It undo many efforts of other developers.

[zeffii]

i welcome any effort to introduce less updates of the node tree. When you feel it's ready, then it's ready for testing by as many people as possible. but i think it's unwise to merge it into master without extensive testing (not just "it passes the automated tests i wrote" , tests are only as good as the mortals that write them). Users will find ways to break even the most hermetic code, often without even trying very hard.

You will get no resistance from me if there's a way to have the current update system and this implementation live side-by-side , hot swap-able, letting users go into "experimental update system" mode. That way you get many more people testing it harder, yet we can still switch to the current update system if we need to.

big changes need big testing, and expect big skepticism.

[Durman]

You will get no resistance from me if there's a way to have the current update system and this implementation live side-by-side, hot swap-able, letting users go into "experimental update system" mode.

Than more I'm working on this than less possible it seems to me to maintain two update system simultaneously. Anyway it is far from ready to merge.

[Durman]