Non-empty JSON type custom field which evaluates to boolean False is set to None

[jwbensley]

Deployment Type

Self-hosted

NetBox Version

v3.5.9

Python Version

3.10

Steps to Reproduce

We have a custom field on devices which is a "JSON" type custom field. We store a list of static routes on that device. By default this is just an empty list (JSON array):

$ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pynetbox
>>> nb = pynetbox.api('http://localhost:8080',token='0123456789abcdef0123456789abcdef01234567')
>>> dev = nb.dcim.devices.get(4)
>>> dev.custom_fields
{'static_routes': []}

Adding data is no problem:

>>> dev.custom_fields["static_routes"] = [1,2,3]
>>> dev.custom_fields
{'static_routes': [1, 2, 3]}
>>> dev.save()
True
>>> dev = nb.dcim.devices.get(4)
>>> dev.custom_fields
{'static_routes': [1, 2, 3]}

We can see the data in the NetBox GUI too:

Deleting data is no problem:

>>> dev.custom_fields["static_routes"].pop(0)
1
>>> dev.custom_fields["static_routes"].pop(0)
2
>>> dev.custom_fields["static_routes"].pop(0)
3
>>> dev.custom_fields["static_routes"]
[]
>>> dev.save()
True
>>> dev = nb.dcim.devices.get(4)
>>> dev.custom_fields["static_routes"]
[]

Again, in the GUI we see the data is gone:

What is important to note here is that on the CLI we see that the value of the custom field is an empty list, the custom field is not empty, even though it looks empty in the GUI.

This is the first bug I guess, the GUI should show an empty JSON array? I'm not really bothered about this personally because I am only using the API. But this is probably a hint to my second/main bug I want to report.

When I edit the device in the GUI I see the custom field value is an empty JSON array:

Here is the configuration of the custom field, the default value is an empty list, so we can just append/pop values from the list, and the import point is that there is always a list to append to/pop from:

Here comes the main bug...

So, we have an empty array stored in the JSON custom field and adding/removing data from that list works as expected.

BUG If the list is empty, and I then edit "something" about the device in the GUI, in this case I will add a tag to the device, the custom field is updated and the empty list is remove, meaning the custom field is now empty/blank/null.

1. Initially no tag and empty JSON array (although you can't see the empty array in the GUI, due to the possible GUI bug I mentioned above but, I showed it via the CLI there is an empty list and that it's the custom field default value):
2. Add a tag to the device (sorry for the tiny font, trying to show in a single screenshot the tag AND the empty JSON array):
3. Click Save
4. The GUI now shows the tag on the device, but due to the GUI bug, what you can't see is that the empty array in the JSON custom field is now removed and the field is empty:
5. This can be seen better via the CLI:

>>> dev = nb.dcim.devices.get(4)
>>> dev.custom_fields["static_routes"]
>>>

There is a third related bug here too. If I edit the device via the GUI and try to set the custom JSON field value to [] and click Save, nothing is saved. If I edit the device again afterwards I see the custom field is empty. I am unable to restore the empty list.

Expected Behavior

What I expect is happening here is that the JSON custom field is being checked using a boolean check, something like if custom_field: ... and [] is evaluating to False.

When updating a device, JSON fields which are not empty should not be set to None/null. Adding a tag should not remove the empty list. Now the code which appends to that empty list is broken because the list doesn't exist anymore.

I should also be able to set the value of a JSON field to [] when editing a device via the GUI..

Observed Behavior

When updating a device, JSON fields which evaluate to False in Python are set to None.

I am also unable to set a JSON custom field to [] when editing a device via the GUI.

[jeremystretch]

You seem to have included a lot of unrelated information in your report, and the reproduction steps rely upon the pynetbox API client. Please rewrite the reproduction steps above to show how the suspect behavior can be reproduced using only the NetBox UI, or (if relevant) using raw API requests.

This is the first bug I guess

If your intent is to report multiple bugs, please open a separate issue for each one.

[jwbensley]

You seem to have included a lot of unrelated information in your report -> what would that be, it all seems relevant to me?

and the reproduction steps rely upon the pynetbox API client -> because as I mentioned, there seems to be a UI bug..

OK, 2nd attempt using only the GUI.

1. We have a custom field which is of type JSON and it has a default value of an empty JSON array [], this is applied to devices:
2. When I edit a device, you can see the default value of the customer field is an empty array:
3. Here I change something (it can be anything) about this device, on the edit device page in this example, I add a tag to the device, and slick Save:
4. Now when I edit the device again, the custom field's empty JSON array [] is gone, the field has been changed to now contain JSON null:

This also happens if the custom field value is an empty JSON object {}.

It seems that when editing an device, if the custom field value evaluates to boolean False, then when saving my changes, the custom field is updated from [] or {} to null.

[abhi1693]

I tested the reproduction steps on v3.7.0 and I can confirm, this bug still exists.