Fix diff_last aggregate calculation

[crnjan]

In order to calculate diff_last include boundary=true parameter when requesting datapoints to read datapoint before requested interval (before datestart), if available. Use that one to calculate diff for first value.

fixes #912 - partly, only applies for diff_last

Signed-off-by: Boris Krivonog boris.krivonog@inova.si

[relativeci]

Job #709: Bundle Size — 15.97MiB (~+0.01%).

cf993c8(current) vs 62f0bb4 main#708(baseline)

Metrics (no changes)

	Current Job #709	Baseline Job #708
Initial JS	1.73MiB	1.73MiB
Initial CSS	608.52KiB	608.52KiB
Cache Invalidation	90.48%	90.48%
Chunks	218	218
Assets	688	688
Modules	2011	2011
Duplicate Modules	108	108
Duplicate Code	1.8%	1.8%
Packages	133	133
Duplicate Packages	15	15

Total size by type (2 changes)

	Current Job #709	Baseline Job #708
CSS	856.56KiB	856.56KiB
Fonts	1.08MiB	1.08MiB
HTML	1.23KiB	1.23KiB
IMG	140.74KiB	140.74KiB
JS	9.04MiB (~-0.01%)	9.04MiB
Media	295.6KiB	295.6KiB
Other	4.59MiB (~+0.01%)	4.59MiB

---

View job #709 report View main branch activity

[ghys]

When testing this PR with an existing chart I get (the server is 3.4.0 release build):

And the response is:
"{\"error\":{\"message\":\"Invalid start/end time in fetch request: 1675205999 \\u003e 1673910752\",\"http-code\":500,\"exception\":{\"class\":\"java.lang.IllegalArgumentException\",\"message\":\"Invalid start/end time in fetch request: 1675205999 \\u003e 1673910752\",\"localized-message\":\"Invalid start/end time in fetch request: 1675205999 \\u003e 1673910752\"}}}"
In the server logs:

[ERROR] [.internal.JSONResponseExceptionMapper] - Unexpected exception occurred while processing REST request.
java.lang.IllegalArgumentException: Invalid start/end time in fetch request: 1675205999 > 1673910931
        at org.rrd4j.core.FetchRequest.<init>(FetchRequest.java:38) ~[?:?]
        at org.rrd4j.core.RrdDb.createFetchRequest(RrdDb.java:943) ~[?:?]
        at org.openhab.persistence.rrd4j.internal.RRD4jPersistenceService.query(RRD4jPersistenceService.java:318) ~[?:?]
        at org.openhab.core.io.rest.core.internal.persistence.PersistenceResource.createDTO(PersistenceResource.java:352) ~[?:?]

It seems the rrd4j persistence service doesn't like the boundary parameter... which is a bummer since it's the one most would have.

Against main without the boundary parameter:

[cweitkamp]

The REST resource does not set an endDate on the FilterCriteriato query the value after given end time.

https://github.com/openhab/openhab-core/blob/8f9dafd1d68a92248cd39414145175952d61cd46/bundles/org.openhab.core.io.rest.core/src/main/java/org/openhab/core/io/rest/core/internal/persistence/PersistenceResource.java#L346-L351

RRD4J persistence service uses current timestamp for endDate if not set. This probably is the culprit for the IAE thrown. We should place an issue in openhab-addons to address this.

https://github.com/openhab/openhab-addons/blob/80094b5e9d293dfb73c81377325da13172d6baaf/bundles/org.openhab.persistence.rrd4j/src/main/java/org/openhab/persistence/rrd4j/internal/RRD4jPersistenceService.java#L283-L284.

[cweitkamp]

One temporarily solution for RRD4J would be to catch the thrown IAE and return an empty List.

https://github.com/openhab/openhab-addons/blob/80094b5e9d293dfb73c81377325da13172d6baaf/bundles/org.openhab.persistence.rrd4j/src/main/java/org/openhab/persistence/rrd4j/internal/RRD4jPersistenceService.java#L333

[cweitkamp]

I created openhab/openhab-addons#14238

[cweitkamp]

@ghys May I ask you an additional question? What does happen if you call the PersistenceExtensions.minimumBetween() or PersistenceExtensions.maximumBetween() methods from rules with parameters begin > end? Do you see the IAE too?

[ghys]

@cweitkamp it seems so:

logInfo('interval', FGWP102WallPlug_ElectricmeterkWh.minimumBetween(now, now.minusDays(30)).getState().toString())

00:30:38.894 [ERROR] [.internal.handler.ScriptActionHandler] - Script execution of rule with UID 'test_rrd4j_interval' failed: Invalid start/end time in fetch request: 1674171038 > 1671579038

logInfo('interval', FGWP102WallPlug_ElectricmeterkWh.minimumBetween(now.minusDays(30), now).getState().toString())

00:31:40.941 [INFO ] [rg.openhab.core.model.script.interval] - 525.25 kWh

[ghys]

Closing as superseded by #1649. The discussion may continue here or be moved to #1649.

[cweitkamp]

@ghys Thanks for testing. I will take care of that and add checks in OHC PersistenceExtension.

The fix for RRD4J has been merged. It takes care of both issues (start > end) and additional boundary value after end of requested time range.

[ghys]

Excellent, then let's merge #1649.

[m-jarzebowski]

Shouldn't the fix be backported to 3.4.x?