[v1.4.0] Many new features and improvements

Adoption of pygame-ce for the graphical user interface

As previously announced on GitHub with appropriate arguments/evidence in favor of the change (and after giving users enough time to voice their opinions), we now use pygame-ce (pygame community edition) for Nodezator's GUI, instead of the original pygame library instance.

Since pygame-ce is, for the most part, a drop-in replacement for the original pygame, the change was rather swift.

The only additional measure users need to take is to make sure the regular pygame library is not installed anymore in the Python instance they use to run Nodezator and that pygame-ce is installed. This can be done with these commands:

pip uninstall pygame
pip install pygame-ce --upgrade

Since both libraries use the same name for import statements (the pygame name), if users don't uninstall the original pygame library, it may shadow the pygame-ce library. If this happens, Nodezator will only show a dialog informing the need to uninstall regular pygame and make sure pygame-ce is installed. This is needed because Nodezator is only useable with pygame-ce due to exclusive imports.

Different modes for nodes

All nodes that represent callables now exist in 01 of 03 modes: expanded, collapsed and callable mode. Together, all of these modes contribute for your graph to be more powerful, versatile and compact.

The expanded mode is the default one, in which the node represents a call to its underlying callable, and thus receives inputs and returns outputs, leaving all of them visible so users can easily interact with them. In collapsed mode the node also represents a call, but it is more compact, only showing inputs/outputs that are connected to other nodes. In callable mode, however, the node represents a reference to its callable, rather than a call, and can thus pass this reference to other nodes, enabling users to perform seemingly complex tasks with simplicity, power and versatility.

Such modes are presented in more detail in this section of the manual.

Shadowed widgets are hidden

As we know, input sockets that have a connection use the incoming data from the connection instead of the widget's value when executed. In other words, the widget is not relevant anymore is just adds visual noise to the graph. Because of that, in order to make the graph more readable and free of visual noise, widgets that are shadowed by connections are now hidden, regardless of whether the node is in expanded or collapsed mode. If the connection is severed, the widget becomes visible again.

New viewer node capabilities

Viewer nodes can now display visualizations on the graph/canvas (for instance, from their outputs).

Original strawberry basket image by NickyPe can be found here.

The new viewer nodes supersede the old ones. The old viewer nodes still work fine, but the new ones have more features.

The matplotlib example node pack has a new viewer node called view_figure2 that uses such new features. The Pillow example node pack has several new viewer nodes that use the new features as well, all in the visualization category. Nodezator now also has some general viewer nodes available by default (we'll talk more about this a bit further).

You can learn all you need to know about all viewer node functionality in the manual. There is a simple and some complex ways to define viewer nodes (you should probably start with the simple one). Also, complex viewer nodes aren't better than the simpler ones. The complex ones exist to meet specific visualization needs. The way to define simpler viewer nodes is described in a single chapter. The way to define complex nodes is described across several chapters towards the end of the manual, since they are a more advanced topic. All these new chapters on viewer nodes are listed below:

• Introductory chapter on viewer nodes and the simpler way to define them
  • Defining viewer nodes
• Chapters on more complex viewer nodes
  • Defining a custom visualization loop for viewer nodes
  • Defining a more complex viewer node
  • More advanced viewer node features - Part 1
  • More advanced viewer node features - Part 2

New chapters/appendix in the manual

New chapters were added to the manual. Besides the ones related to viewer nodes mentioned in the previous section, there were also 02 new chapters. One on conditional execution and another on looping in Nodezator. There is also a new appendix with recipes for common tasks.

All these new chapters/appendix are listed below:

• Conditional execution (branching) in Nodezator
• Looping/iterating in Nodezator
• Appendix: Recipes

New categories of app-defined nodes

There are 02 new categories of app-defined nodes (nodes available by default in Nodezator):

• General viewer nodes
• pygame-ce nodes

Because this version introduces new capabilities for viewer nodes, specially the ability to display visualizations on the graph itself beside the viewer nodes, we also introduced new categories of app-defined/default nodes to help in such tasks and other related ones. This way users won't need to write their own nodes for the most basic visualization tasks. As a consequence this also prevents a lot of code repetition that would exist across node packs defined by users if they had to write nodes for such basic actions.

The vast majority of the new nodes are from the pygame-ce nodes category. The image below represents a tiny sample of the many nodes introduced with both categories.

Original pomegranate tree image by AselvadaAna can be found here.

You can learn more about general viewer nodes and pygame-ce nodes in this section and the subsequent one from the manual.

New app-defined nodes in preexisting categories

Besides the many nodes in the new app-defined categories, there are also some new app-defined nodes in preexisting categories:

• New standard library nodes added:
  • operator.attrgetter
  • operator.itemgetter
  • operator.methodcaller
  • 02 different signatures of pathlib.Path
• New snippet nodes (under the Encapsulations submenu of the popup menu):
  • for_item_in_obj_pass
  • get_constant_returner
  • perform_attr_call
  • return_untouched
• The ternary operator a if c else b

Changes in appearance

Inputs closer to the bottom, outputs closer to the top

Although all the changes in this version are backward-compatible, your graphs will look a bit different when opened in the new version. This is so because the position of inputs and outputs in the nodes were changed. Before, the inputs were closer to the top of the node and the outputs were closer to the bottom. Now this was inverted and the inputs are closer to the bottom and the outputs closer to the top, as can be seen in the image below:

This change was made because the empty space that sometimes remains below the outputs of a node (between the last output socket and the bottom of the node) is a proper/nice spot for placing the visualizations shown beside viewer nodes on the graph. This positioning of the sockets is also adopted by other existing node-editing applications, like Blender3D (by which Nodezator is heavily inspired).

The image below shows the same graph opened in the previous version (at the top) and in the new one (at the bottom):

Though they are the same graph with the same nodes and same connections, they appear slightly different due to the position of the sockets (and, as a consequence, the position of the lines that represent their connections). There's not much difference in this small graph, but in larger graphs the differences may be even more pronounced.

Operation nodes are more compact

Operation nodes had their size reduced, specially the big characters in their center. Here's an image showing the old and new appearance of the operation nodes appear, with a print() node for scale:

As can be seen in the image, although the size was reduced, the characters are still big, so they can be easily spotted on the graph. It is just that they are not needlessly big anymore. The color of the big characters was also changed in order to add more contrast. This is particularly helpful to keep the difference between the ~a and -a nodes noticeable.

Local Pages Browser

The Local Pages Browser, previously named HTSL Browser (the viewer we use to display stylized hypertext content like the manual, controls, etc.) had its colors changed, its width increased and some padding added around the contents shown on it. Reading from it should be a bit more pleasing now. The image below shows what it looked like previously (on the left) and how it looks now (on the right):

As can be seen in the previous image, vertical padding around nested lists was increased as well and links are not underlined anymore. Several other small changes that don't appear in the image were made as well. A functional change was also made: as can be seen in the image below, blocks of preformatted text/code now have a button that can be used to copy the contents to the clipboard.

Fixed and improved doctests

The failing doctests were fixed and files used exclusively for doctests were converted from Python to markdown files. Now it is more pleasant to visualize them on GitHub or in markdwon viewer apps, since markdown can display both stylized hypertext (headings, paragraphs, blockquotes, links, images, lists, tables and even some HTML tags) and code blocks that can be executed as doctests.

On GitHub, even the font used for rendering in markdown seems to be more easy on the eye. The code blocks are also syntax highlighted, which helps reading the code as well. See the comparison below between a doctest written in a Python file (at the top) and the same doctest written as a markdown file (at the bottom):

Previewing Python code to be exported from graph

This new version adds the ability to view the Python code representing the graph without needing to export it to a file.

Previously, the user had to save the exported code to a file before being able to see it. And even so, the user had to open it in another app.

Now the user can see the code instantly, by clicking in the View as Python command in the menubar or pressing Shift+Ctrl+P.

A single step to export graph to Python

Exporting the graph to a Python file can be done in a single step now. When the command to export the graph as Python is executed (by pressing Ctrl+P or clicking File > Export as Python in the menubar), the file manager is presented and the user can pick the location and save the exported file right away.

Before there were more steps: Once the command was triggered, a form was presented to the user and only after filling out the form and confirming the user could export the file.

This form had a button used to set the location to save the exported file with the file manager. There was also an additional optional control that was never explained in the manual, which served to prefix import statements in the exported file with package/module names, in case the user wanted the callables in the node pack to be exported from a different folder structure. For instance, the user could cause an import statement that would be imported like from nodepack.category01... to be actually exported like from parentfolder.nodepack.category01... by adding parentfolder as a prefix (users could add as much as desired).

This is an action that many users will never need to perform and even when they need, they can quickly do that by themselves either manually or with help from a text editor or other utility. In other words, that actual convenience of this optional control was questionable and since it wasn't explained in the manual, it wasn't even clear whether it was being used or not. Because of that, this option, along with the form, was removed altogether.

If future feedback from users indicates that the optional control was indeed useful we'll promptly bring it back along with the form. Additionally, we'd like to point out that we seldom remove stuff from the app without consulting our users. It was done in this instance solely because of the given reasons (the obscurity and questionable usefulness of the control).

Of course, regardless of this, if other options are added to the exporting feature in the future that require the creation of additional controls, the form will also be brought back.

Bug fixes

Several bugs were fixed as well.

For instance, bugs which resulted in annoying behaviours in the IntFloatEntry widget (the one we use to hold and edit integers/floats). There was one which made the mouse pointer to become invisible sometimes. Another bug was causing the value in the widget to "jump" too much when changing the value by dragging the mouse. Now the behaviour of this widget doesn't suffer from these problems anymore.

Ongoing work on the playback feature

Because of the advanced state of the ongoing development work on the playback feature, in order to avoid future merge conflicts, the commits for this new Nodezator 1.4.0 version were rebased on top of the pre-existing commits for the playback feature. This means all changes from the ongoing work on the playback feature were published with the changes from the implementation of the new features.

The playback feature is responsible for recording and playing input in Nodezator and will be used as the basis for other features like automated GUI testing and a demonstration mode. Although the playback feature is already partially working, it isn't ready to be released, so the feature isn't available to users yet. In other words, the related changes only affect the backend. They represent a lot of the foundation needed to support the playback feature.

The only visible difference caused by these extra changes is that a new Playback submenu is now visible in the menubar. However, its functionality isn't available to users, so attempting to click on the underlying commands will just show a message notifying the user that the feature is not implemented or that it is a work in progress.