IBX-5388: Fixed performance issues of content updates after field changes

[Nattfarinn]

Question	Answer
JIRA issue	IBX-5388
Type	improvement
Target Ibexa version	v3.3
BC breaks	no (or soft)

When new field was added we iterated over each content and applied field changes with default values to each existing version. When we removed the field we iterated over each content and removed data from every version as well. This cause major performance issue as all these changes happened in runtime each time you tried to publish Content Type and grow exponentially with increasing number of content items leading to memory issues, transaction nesting issues and exceeded execution times.

This PR introduces two keystones:

• on field removed: there is no need to update content items, we can just ignore additional data within persistence layer during content load,
• on field added: there is no need to update content items with default field values, as we can add these fields in runtime to object without touching persistence (like "virtual" fields).

This makes publishing Content Type process much faster and doesn't make content loading process slower (loading times differences are unnoticeable). Change itself is almost transparent to the user and developer and these virtual fields does not need any special handling for day-to-day operations. Once the content is published, copied or anyhow saved, virtual fields become persisted and will be indistinguishable from any other field. But there are some behavior changes, but with low impact:

• virtual fields are not stored: this means fields do not have an id (it remains null),
• virtual fields are dynamic: this means default value is not static anymore. Changes to default values within Content Type field definitions will be immediately reflected by all content items that still rely on virtual fields.

Depends on: https://github.com/ezsystems/ezplatform-version-comparison/pull/85

Checklist:

• Provided PR description.
• Tested the solution manually.
• Provided automated test coverage.
• Checked that target branch is set correctly (master for features, the oldest supported for bugs).
• Ran PHP CS Fixer for new PHP code (use $ composer fix-cs).
• Asked for a review (ping @ezsystems/engineering-team).

[alongosz]

The direction here makes perfect sense. We need to test this extensively of course:

Code-wise remarks:

[sonarcloud]

Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
No data about Coverage
0.5% Duplication on New Code

See analysis details on SonarCloud

[Steveb-p]

LGTM. All my comments are optional notes that you can ignore as you see fit, since they are mostly opinion-based.

One thing I would very much like to add, but would require effort (probably a lot), is an integration test for it. It would be extremely valuable in my opinion, as we are dealing with a behavior that is very specific and depends on interaction between multiple components.

[micszo]

Following issue found in image asset.

Steps:

0. image CT (default), image asset CT (ezstring, ezimageasset)
1. create image
2. create image asset, select from repository
3. edit image CT, add single relation field or keywords
4. edit image asset
5. open image editor
6. flip image
7. save as / apply changes to all

Actual result: In UI error occurs "Internal sever error". Exception occurs in log:

php.CRITICAL: Uncaught Error: reset(): Argument #1 ($array) must be of type array, null given {"exception":"[object] (TypeError(code: 0): reset(): Argument #1 ($array) must be of type array, null given at /Users/michalszoltysek/Projects/workspace/ibexa_website_4/vendor/ezsystems/ezplatform-kernel/eZ/Publish/Core/FieldType/Relation/SearchField.php:32)"} []

or

php.CRITICAL: Uncaught Error: implode(): Argument #1 ($pieces) must be of type array, string given {"exception":"[object] (TypeError(code: 0): implode(): Argument #1 ($pieces) must be of type array, string given at /Users/michalszoltysek/Projects/workspace/ibexa_website_4/vendor/ezsystems/ezplatform-kernel/eZ/Publish/Core/FieldType/Keyword/SearchField.php:29)"} []

[micszo]

Retest status.

• ezobjectrelation: OK
• ezkeyword: NOK - the same exception.

[pspanja]

For what it's worth, there was a way to process Content update asynchronously, removed as deprecated in the PR: ezsystems/ezpublish-kernel#2976

IMHO, this should be properly handled by asynchronous processing and not by introducing inconsistencies to the Repository API.

[micszo]

QA Approved on Ibexa Commerce 3.3.38-dev.

[sonarcloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.3% Duplication on New Code

See analysis details on SonarCloud